NAT incompatibility with NETGEAR N150 WNR1000 and WNR2000

André Guimarães, 2012-02-07 (Updated)

André Guimarães, 2012-01-16

Switzernet

This document describes a problem detected with some customers after migrating to Astrad. This problem prevents them from receiving calls and all outgoing calls are dropped after 30 seconds.

Description

After the migration of Astrad3, Astrad13 and Astrad14 some customers have complained about being unable to receive calls and all calls made dropped after 20 to 30 seconds. These customers had the same router model (Netgear N150 WNR1000 or WNR2000) and as an ISP provider Cablecom (meanwhile some Swisscom users were also detected).

When a customer makes a call we receive the INVITE and reply with the appropriate messages and with sound when the other side answers. The phone receives these messages and the customers is able to maintain a dialog with the other person during some questions until Astrad disconnects the call due to a missing critical response to the INVITE handshake. As it does not receive the corresponding ACK to the 200 OK it disconnects after approximately 30 seconds.

What is SIP ALG

SIP ALG (Application-level gateway) is a security component commonly found in router or firewall devices. This feature allows VoIP traffic to pass both from the private to public side of the firewall and vice-versa when using NAPT (Network Address and Port Translation). It inspects and modifies the content of SIP packets to allow SIP traffic to pass through the firewall.

This theoretically could help solve NAT related problems, but, as many of the existing implementations of this feature have problems, in practice it breaks the SIP protocol. In consequence, calls are dropped or aren’t even started, sound doesn’t pass between both parties freely or another connection problems which make communication impossible. This feature is usually enabled by default and in some cases cannot be disabled. It also prevents some kind of server side solution to NAT problems.

Problem

The problem seems to be due to a modification of the SIP packet made by the router which prevents messages sent by the client to reach our servers. If the customer connects the phone directly to the modem there is no problem. If the router is in between communication is impossible. The reason why it works with the more ancient Switzernet servers appears to be due to a SIP header called Via.

As previous versions were SIP proxies all messages sent add double Via headers. The current version uses a B2B SIP server which only sends one Via.

Apparently the SIP ALG implementation used by NetGear WNR1000 and WNR2000 only works when those two Via headers exist. This router is known to have SIP ALG implementation problems ^[1]. This feature cannot be disabled in some firmwares but in others the options to disable it is available.

By removing the router and connecting the phone directly to the modem the problem is solved. We also have hundreds of Cablecom customers registered in Astrads that don’t have this issue. This shows that the problem is due to the router and not due to the ISP.

It seems that the SIP messages sent by Porta-SIP are able to surpass the problem. Both Astrad v10 (production) and Astradv11 (testing) are unable to call customers behind this router and drop calls initiated on their side after some seconds.

The SIP signaling is correct on Astrad side and all messages are sent to the correct IP and port but replies to those messages are lost.

When calling a customer with this issue Astrad sends several SIP INVITEs but does not receive any reply. The phone receives these replies and starts to ring, but the expected messages (100 Trying and 180 ringing) are probably being sent elsewhere due to the router’s SIP ALG header manipulation.

SIP traces from the customer’s side will be needed to further understand why the messages are lost.

Affected customers

A script was created to send two NOTIFY messages to each customer. One with only one Via and another with two Vias. The phones of the customers affected by the problem described in this document would only answer to the second message. This method was used to detect how many and which customers are affected by this problem. Customer’s username and IP address have been masked but can be accessed here by Switzernet staff to protect customer’s privacy. The problem affected approximately 1% of Switzernet migrated customers (5 servers migrated during December).

These were the customers detected. Customer’s information about the ISP provider, Switzernet SIP server, last migration date and router information when available are shown bellow.

Number	ISP	SIP server	Router	Date
412X55XXX57	Cablecom	dk1.youroute.net	NETGEAR N150 WNR 1000	2011/11/30
412X55XXX69	Cablecom	dk1.youroute.net	NETGEAR N150 WNR 1000	2011/11/30
412X55XXX78	Cablecom	dk1.youroute.net	NETGEAR WNR 2000	2011/11/30
412X55XXX52	Swisscom	astrad17.switzernet.com	To be confirmed	2011/11/30
412X55XXX86	Swisscom	astrad17.switzernet.com	To be confirmed	2011/11/30
412X55XXX91	Cablecom	astrad15.switzernet.com	To be confirmed	2011/12/05
412X55XXX59	Cablecom	astrad3.switzernet.com	To be confirmed	2011/12/05
412X55XXX24	Cablecom	fr8.youroute.net	To be confirmed	2011/12/07
412X55XXX21	Cablecom	astrad3.switzernet.com	To be confirmed	2011/12/05
412X55XXX67	Cablecom	astrad10.switzernet.com	To be confirmed	2011/05/13
412X55XXX98	Swisscom	astrad17.switzernet.com	To be confirmed	2011/11/30
413X55XXX85	Cablecom	astrad17.switzernet.com	To be confirmed	2011/11/30

Other servers

List of customers detected on Porta-SIP server that might have the problem if migrated. Additional information about telephone used and ISP is also shown.

Possible solutions

For the moment customers affected can register on server sip100.youroute.net which is a temporary server which uses the previous version of SIP server Porta-SIP.

There is also another test server which uses port 5070. As SIP ALG only analyzes traffic with destination to the 5060 port customers registered in this port are not affected by the problem.

Another test server redirects the customers with problems’ new calls to another server where the requests are modified to be compatible with the router requests. Calls going to the customer are also modified to the needed format before being sent to the customer’s phone.

Also in some firmwares and other routers it is possible to simple disable this feature. If a customer is having this problem, he should disable SIP ALG and SPI (Stateful Packet Inspection) if possible. For instance with a firmware downgrade to 1.1.3.9 in Netgear’s WNR2000 the problem is solved. It appears the problem is also solved by upgrading to the latest firmware but until this moment we could not confirm this information.

We are actively working on a more transparent solution to solve this problem. The aim is to have a solution that doesn’t require the customer to change anything on his side.

Test results

In this section the result for each test made is shown. The corresponding trace is available to download.

021550XXXX is a customer without the problem.

021550YYYY is a customer with Netgear WNR1000 router.

Calls were made from the 021550XXXX to 021550YYYY in different situations.

As the customers are now registered in a Porta-SIP to be able to make calls, the DNS configuration was changed to allow easy swapping between the Porta-SIP and

Registrations

Registration from 021550YYYY in Porta-SIP [Download]

Registration from 021550YYYY in Astrad v11 [Download]

No difference in the registration. It appears that the phone is receiving Astrad messages.

Call to 021550YYYY in Porta-SIP [Download]

All signaling is correct. Customer can receive call and speak with the other party.

Call to 021550YYYY in Astrad v11 without modifications [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

Call to 021550YYYY in Astrad v11 with additional Record-route to Astrad IP [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

/etc/asterisk/extensions.conf:

exten => _[*0-9]!,n,Set(rr=<sip:91.121.122.64\;lr>)

exten => _[*0-9]!,n,SIPAddHeader(Record-route:${rr})

Call to 021550YYYY in Astrad v11 with Astrad telling it is running on port 5061, messages sent by 5060 [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

/etc/asterisk/sip.conf:

bindport=5060

externaddr=SERVERIP:5061

localnet=127.0.0.1/32

Call to 021550YYYY in Astrad v11 with Astrad telling it is running on port 5061, messages sent by 5060, with additional Record-route to Astrad IP port 5061 [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

/etc/asterisk/extensions.conf:

exten => _[*0-9]!,n,Set(rr=<sip:91.121.122.64:5061\;lr>)

exten => _[*0-9]!,n,SIPAddHeader(Record-route:${rr})

/etc/asterisk/sip.conf:

bindport=5060

externaddr=SERVERIP:5061

localnet=127.0.0.1/32

Firewall:

iptables -t nat -A PREROUTING -i eth0 -p udp --dport 5061 -j REDIRECT --to-ports 5060

Call to 021550YYYY in Astrad v11 with Astrad telling it is running on port 5061, messages sent by 5060, with additional Record-route to Astrad IP port 5061, additional Via to Astrad IP port 5060 [Download]

Replies to INVITE are received on port 5060 but, due the additional Via inserted, Asterisk ignores all replies to the INVITE sent.

/etc/asterisk/extensions.conf:

exten => _[*0-9]!,n,Set(rr=<sip:91.121.122.64:5061\;lr>)

exten => _[*0-9]!,n,Set(sabranch=${MD5(${H323_ID_HEX})})

exten => _[*0-9]!,n,Set(savia=SIP/2.0/UDP 91.121.122.64\;branch=${sabranch}\;rport=5061)

exten => _[*0-9]!,n,SIPAddHeader(Record-route:${rr})

exten => _[*0-9]!,n,SIPAddHeader(Via:${savia})

/etc/asterisk/sip.conf:

bindport=5060

externaddr=SERVERIP:5061

localnet=127.0.0.1/32

Firewall:

iptables -t nat -A PREROUTING -i eth0 -p udp --dport 5061 -j REDIRECT --to-ports 5060

Call to 021550YYYY in Astrad v11 with Astrad additional Via to Astrad IP port 5060 [Download]

/etc/asterisk/extensions.conf:

exten => _[*0-9]!,n,Set(sabranch=${MD5(${H323_ID_HEX})})

exten => _[*0-9]!,n,Set(savia=SIP/2.0/UDP 91.121.122.64\;branch=${sabranch}\;rport=5061)

exten => _[*0-9]!,n,SIPAddHeader(Via:${savia})

Replies to INVITE are received on port 5060 but, due the additional Via inserted, Asterisk ignores all replies to the INVITE sent.

Call to 021550YYYY in Astrad v11 without rport parameter [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

Changed nat option for the customer in the database from nat=yes to nat=comedia to prevent Asterisk from sending rport.

Call to 021550YYYY in Astrad v11 with additional Via to random IP and random port [Download]

No reply to the INVITE is received. Repeated INVITEs are sent until timeout. Customer tries to answer the call but Astrad doesn’t receive that information.

/etc/asterisk/extensions.conf:

exten => _[*0-9]!,n,Set(sabranch=${MD5(${H323_ID_HEX})})

exten => _[*0-9]!,n,Set(savia=SIP/2.0/UDP 91.121.122.64\;branch=${sabranch}\;rport=5061)

exten => _[*0-9]!,n,SIPAddHeader(Via:${savia})

Messages from Porta-SIP sent by an Astrad using ncat [Download]

Replies are received to INVITE, CANCEL or NOTIFY only with the additional Via.

Test results with one of the affected router models

For further testing a Netgear router WNR1000v2 was bought. This is one of the routers with the problem. On this version it is possible to solve the problem by disabling SPI firewall and SIP ALG as shown in the image bellow. This solves the problem completely:

These two options are enabled by default. With these options all messages sent from Astrad servers are dropped and calls made by the customer are dropped after 20/30 seconds.

Calls to the customer

Using a packet sniffer after the router, we can see no messages arrive to the telephone. The router firewall drops all the messages. If an additional Via is added the messages start arriving to the phone. All sip methods sent by an Astrad are affected with this problem (INVITE, CANCEL, BYE, NOTIFY, etc). Additional tests proved that it suffices to add something that resembles a Via. If an extra header in the format of “X-Via: …” is added the result is the same as adding the complete Via. The problem is that the phones removes that field from the reply and the final handshake messages are unable to pass the firewall. Also Astrad is only capable of inserting the additional header in INVITEs. It is not possible to add this new field to other messages and replies.

#!/bin/bash

PORT=5060

IPADDRSERVER="91.121.122.64"

FROM="41215XXXXXX"

TEST=1

PORT2=5060

TESTATTEMPTS=1

MAKEALLTESTS=0;

CUSTNUMBER=$1

CUSTIPADDR=$2

CUSTPORT=$3

if [ "$CUSTNUMBER" == "" ]

then exit

if [ "$CUSTIPADDR" == "" ]

then exit

if [ "$CUSTPORT" == "" ]

then CUSTPORT=5060

echo "Executing test "$TEST" on "$CUSTNUMBER"@"$CUSTIPADDR":"$CUSTPORT

CALLID1=`date +%s%d|md5sum|awk '{print $1}'`

CALLID2=`date +%s%m|md5sum|awk '{print $1}'`

CALLID3=`date +%s%Y|md5sum|awk '{print $1}'`

echo -ne "INVITE sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite11.txt

echo -ne "Record-Route: <sip:$IPADDRSERVER;ftag=$CALLID3;lr>\r\n" >> invite11.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite11.txt #With only this line uncommented the message don’t arrive to the phone

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite11.txt #Test1: With this line uncommented the call arrives to the phone and the phone replies with the additional Via

#echo -ne "X-Via: SIP/2.0/UDP $IPADDRSERVER\r\n" >> invite11.txt #Test2: With this line uncommented the call arrives to the phone. The phone removes this field on the reply

#echo -ne "X-Astrad-Via: SIP/2.0/UDP $IPADDRSERVER:$PORT\r\n" >> invite11.txt #Test3: With this line uncommented the call arrives to the phone. The phone removes this field on the reply

echo -ne "Max-Forwards: 16\r\n" >> invite11.txt

echo -ne "From: $FROM <sip:$FROM@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite11.txt

echo -ne "To: <sip:$CUSTNUMBER@$IPADDRSERVER:$CUSTPORT>\r\n" >> invite11.txt

echo -ne "Call-ID: $CALLID2@176.31.102.152\r\n" >> invite11.txt

echo -ne "CSeq: 200 INVITE\r\n" >> invite11.txt

echo -ne "Contact: Anonymous <sip:$IPADDRSERVER:5061>\r\n" >> invite11.txt

echo -ne "Expires: 300\r\n" >> invite11.txt

echo -ne "User-Agent: Sippy\r\n" >> invite11.txt

echo -ne "cisco-GUID: 1683660840-1260417872-566774332-589833878\r\n" >> invite11.txt

echo -ne "h323-conf-id: 1683660840-1260417872-566774332-589833878\r\n" >> invite11.txt

echo -ne "Content-Length: 389\r\n" >> invite11.txt

echo -ne "Content-Type: application/sdp\r\n" >> invite11.txt

echo -ne "\r\n" >> invite11.txt

echo -ne "v=0\r\n" >> invite11.txt

echo -ne "o=Sippy 141326892 0 IN IP4 $IPADDRSERVER\r\n" >> invite11.txt

echo -ne "s=session\r\n" >> invite11.txt

echo -ne "t=0 0\r\n" >> invite11.txt

echo -ne "m=audio 27398 RTP/AVP 0 8 18 4111 101\r\n" >> invite11.txt

echo -ne "c=IN IP4 176.31.102.152\r\n" >> invite11.txt

echo -ne "a=rtpmap:0 PCMU/8000\r\n" >> invite11.txt

echo -ne "a=rtpmap:8 PCMA/8000\r\n" >> invite11.txt

echo -ne "a=rtpmap:18 G729/8000\r\n" >> invite11.txt

echo -ne "a=fmtp:18 annexb=no\r\n" >> invite11.txt

echo -ne "a=rtpmap:4 G723/8000\r\n" >> invite11.txt

echo -ne "a=fmtp:4 annexa=no\r\n" >> invite11.txt

echo -ne "a=rtpmap:111 G726-32/8000\r\n" >> invite11.txt

echo -ne "a=rtpmap:101 telephone-event/8000\r\n" >> invite11.txt

echo -ne "a=fmtp:101 0-16\r\n" >> invite11.txt

echo -ne "a=silenceSupp:off - - - -\r\n" >> invite11.txt

echo -ne "a=ptime:20\r\n" >> invite11.txt

echo -ne "a=sendrecv\r\n" >> invite11.txt

FILE1="invite"$TEST"1.txt"

cat $FILE1|nc -u -w 1 -p $PORT2 $CUSTIPADDR $CUSTPORT >> result1.txt

Calls from customer

The problem with the calls made from the customer is that the router replaces the content of the Contact field of every message sent from the Astrad with the IP and port of the phone. This means that if the phone as to reply to the the reply to close the handshake it has no means to determine to which IP and port it should send the message. After several retransmissions without receiving the ACK for the 200 OK the call is dropped by the Astrad. Using the following script it was determined that the router only changes the field contact if this field has the format: “Contact: <sip:NUMBER@IP>”. If the “NUMBER@” is removed the router doesn’t modify the packet. The lines in bold should be commented or uncommented to test each scenario.

#!/usr/bin/perl

#udpserver.pl

use IO::Socket::INET;

# flush after every write

$| = 1;

my ($socket,$received_data);

my ($peeraddress,$peerport);

# we call IO::Socket::INET->new() to create the UDP Socket and bound

# to specific port number mentioned in LocalPort and there is no need to provide

# LocalAddr explicitly as in TCPServer.

$socket = new IO::Socket::INET (

PeerHost => '212.147.8.99:62061',

LocalAddr => '91.121.122.64:5060',

#LocalPort => '5060',

Proto => 'udp',

) or die "ERROR in Socket Creation : $!\n";

my $state = 0;

my $branch = '';

my $tag = '';

my $callid = '';

my $via = '';

my $from = '';

while(1)

{

# read operation on the socket

$socket->recv($recieved_data,1024);

#get the peerhost and peerport at which the recent data received.

$peer_address = $socket->peerhost();

$peer_port = $socket->peerport();

#print "\n($peer_address , $peer_port) said : $recieved_data";

#Convert data to array

print "\nArray:\n";

@message = split(/\r\n/, $recieved_data);

my $i = 0;

CYCLE2: foreach (@message) {

my $line = $_;

if ($line =~ m/^INVITE/ && $state == 0) {

print "Initial INVITE!!!\n";

$state = 1;

next CYCLE2;

}

if ($line =~ m/^ACK/ && $state == 1) {

print "ACK!!!\n";

$state = 2;

next CYCLE2;

}

if ($line =~ m/^INVITE/ && $state == 2) {

print "Second INVITE!!!\n";

$state = 3;

next CYCLE2;

}

if ($state == 1) {

if ($state == 1 && $line =~ m/branch=(.*);/) {

$branch = $1;

$via = $line;

print "Branch is: ".$branch."\n";

next CYCLE2;

}

if ($state == 1 && $line =~ m/tag=(.*)$/) {

$tag = $1;

print "Tag is: ".$tag."\n";

next CYCLE2;

}

if ($state == 1 && $line =~ m/^Call-ID: (.*)$/) {

$callid = $line;

print "Callid is: ".$callid."\n";

last CYCLE2;

}

if ($state == 3) {

if ($line =~ m/^Via/) {

$via = $line;

next CYCLE2;

}

if ($line =~ m/^From/) {

$from = $line;

last CYCLE2;

}

print "$i:$line\n";

$i++;

}

print "\nEndArray\n";

#send the data to the client at which the read/write operations done recently.

#$data = "data from server\n";

#print $socket "$data";

if ($state == 1) {

print "Should send reply1 now:\n";

my $send = "SIP/2.0 401 Unauthorized\r\n".

"$via=$peer_port\r\n".

"From: \"41215502419\"<sip:41215XXXXXX@astrad.switzernet.com>;tag=$tag\r\n".

"To: \"VTX\"<sip:0215XXXXXX\@astrad.switzernet.com>;tag=as64f69da1\r\n".

"$callid\r\n".

"CSeq: 1 INVITE\r\n".

"Server: Astradv11\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

"WWW-Authenticate: Digest algorithm=MD5, realm=\"switzernet\", nonce=\"5f453916\"\r\n".

"Content-Length: 0\r\n\r\n";

print $socket "$send";

}

if ($state == 3) {

print "Should send reply2 now:\n";

my $send = "SIP/2.0 100 Trying\r\n".

"$via=$peer_port\r\n".

"$from\r\n".

"To: \"VTX\"<sip:0215XXXXXX\@astrad.switzernet.com>\r\n".

"$callid\r\n".

"CSeq: 2 INVITE\r\n".

"Server:Astradv11\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

"Contact: <sip:0215XXXXXX\@91.121.122.64:5060>\r\n".

# "Contact: <sip:91.121.122.64:5060>\r\n".

"Content-Length: 0\r\n\r\n";

print $socket "$send";

print "Should send reply3 now:\n";

$send = "SIP/2.0 183 Session Progress\r\n".

"$via=$peer_port\r\n".

"$from\r\n".

"To: \"VTX\"<sip:0215XXXXXX\@astrad.switzernet.com>;tag=as64f69da1\r\n".

"$callid\r\n".

"CSeq: 2 INVITE\r\n".

"Server:Astradv11\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

# "Contact: <sip:021XXXXXX\@91.121.122.64:5060>\r\n".

"Contact: <sip:91.121.122.64:5060>\r\n".

"Content-Length: 386\r\n\r\n".

"v=0\r\n".

"o=root 907590463 907590463 IN IP4 91.121.122.64\r\n".

"s=astrad\r\n".

"c=IN IP4 91.121.122.64\r\n".

"t=0 0\r\n".

"m=audio 17874 RTP/AVP 8 0 3 97 98 101\r\n".

"a=rtpmap:8 PCMA/8000\r\n".

"a=rtpmap:0 PCMU/8000\r\n".

"a=rtpmap:3 GSM/8000\r\n".

"a=rtpmap:97 speex/8000\r\n".

"a=rtpmap:98 iLBC/8000\r\n".

"a=fmtp:98 mode=30\r\n".

"a=rtpmap:101 telephone-event/8000\r\n".

"a=fmtp:101 0-16\r\n".

"a=silenceSupp:off - - - -\r\n".

"a=ptime:20\r\n".

"a=sendrecv\r\n\r\n";

print $socket "$send";

print "Should send reply3 now:\n";

$send = "SIP/2.0 200 OK\r\n".

"$via=$peer_port;received=$peer_address\r\n".

"$from\r\n".

"To: \"VTX\"<sip:0215661380\@astrad.switzernet.com>;tag=as64f69da1\r\n".

"$callid\r\n".

"CSeq: 2 INVITE\r\n".

"Server: Astradv11\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

# "Contact: <sip:021XXXXXX\@91.121.122.64:5060>\r\n".

"Contact: <sip:91.121.122.64:5060>\r\n".

"Content-Type: application/sdp\r\n".

"Content-Length: 386\r\n\r\n".

"v=0\r\n".

"o=root 907590463 907590463 IN IP4 91.121.122.64\r\n".

"s=astrad\r\n".

"c=IN IP4 91.121.122.64\r\n".

"t=0 0\r\n".

"m=audio 17874 RTP/AVP 8 0 3 97 98 101\r\n".

"a=rtpmap:8 PCMA/8000\r\n".

"a=rtpmap:0 PCMU/8000\r\n".

"a=rtpmap:3 GSM/8000\r\n".

"a=rtpmap:97 speex/8000\r\n".

"a=rtpmap:98 iLBC/8000\r\n".

"a=fmtp:98 mode=30\r\n".

"a=rtpmap:101 telephone-event/8000\r\n".

"a=fmtp:101 0-16\r\n".

"a=silenceSupp:off - - - -\r\n".

"a=ptime:20\r\n".

"a=sendrecv\r\n\r\n";

print $socket "$send";

$socket->close();

exit;

}

Customer detection

Based on the last test we are able to detect which customers have this problem by sending several NOTIFY packets with one or two Vias. The customer’s phones behind problematic routers will only reply to the double Via message enabling us to detect which customers have the problem.

Based on this a script was created to send 5 Notify packets (to account for UDP losses) with only one Via. If at least one is received the customer does not have the problem described in this document. If not an additional 5 Notify packets with two Vias are sent. If at least one is replied the customer is marked as having the problem. If not the phone has probably changed IP address or is unregistered at the moment so it will be added for another try later. It is also possible that in the last group there are phones or other equipments that don’t reply to the Notify message.

The script generate_sip_message.sh receives 3 arguments where port is optional. The first is customer number, the second is customer IP address and the last one is port. Its code is the following

#!/bin/bash

PORT=5060

IPADDRSERVER="XXX.XXX.XXX.XXX"

FROM="41YY55YYYYY"

TEST=3

PORT2=5060

CUSTNUMBER=$1

CUSTIPADDR=$2

CUSTPORT=$3

if [ "$CUSTNUMBER" == "" ]

then exit

if [ "$CUSTIPADDR" == "" ]

then exit

if [ "$CUSTPORT" == "" ]

then CUSTPORT=5060

echo "Executing test "$TEST" on "$CUSTNUMBER"@"$CUSTIPADDR":"$CUSTPORT

CALLID1=`date +%s%d|md5sum|awk '{print $1}'`

CALLID2=`date +%s%m|md5sum|awk '{print $1}'`

CALLID3=`date +%s%Y|md5sum|awk '{print $1}'`

if [ "$TEST" == "1" ]

then

echo -ne "INVITE sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite1.txt

echo -ne "Record-Route: <sip:$IPADDRSERVER;ftag=$CALLID3;lr>\r\n" >> invite1.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite1.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER:5061;branch=z9hG4bKc47$CALLID2;rport=5061\r\n" >> invite1.txt

echo -ne "Max-Forwards: 16\r\n" >> invite1.txt

echo -ne "From: $FROM <sip:$FROM@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite1.txt

echo -ne "To: <sip:$CUSTNUMBER@$IPADDRSERVER:$CUSTPORT>\r\n" >> invite1.txt

echo -ne "Call-ID: $CALLID2@176.31.102.152\r\n" >> invite1.txt

echo -ne "CSeq: 200 INVITE\r\n" >> invite1.txt

echo -ne "Contact: Anonymous <sip:$IPADDRSERVER:5061>\r\n" >> invite1.txt

echo -ne "Expires: 300\r\n" >> invite1.txt

echo -ne "User-Agent: Sippy\r\n" >> invite1.txt

echo -ne "cisco-GUID: 1683660840-1260417872-566774332-589833878\r\n" >> invite1.txt

echo -ne "h323-conf-id: 1683660840-1260417872-566774332-589833878\r\n" >> invite1.txt

echo -ne "Content-Length: 389\r\n" >> invite1.txt

echo -ne "Content-Type: application/sdp\r\n" >> invite1.txt

echo -ne "\r\n" >> invite1.txt

echo -ne "v=0\r\n" >> invite1.txt

echo -ne "o=Sippy 141326892 0 IN IP4 $IPADDRSERVER\r\n" >> invite1.txt

echo -ne "s=session\r\n" >> invite1.txt

echo -ne "t=0 0\r\n" >> invite1.txt

echo -ne "m=audio 27398 RTP/AVP 0 8 18 4111 101\r\n" >> invite1.txt

echo -ne "c=IN IP4 176.31.102.152\r\n" >> invite1.txt

echo -ne "a=rtpmap:0 PCMU/8000\r\n" >> invite1.txt

echo -ne "a=rtpmap:8 PCMA/8000\r\n" >> invite1.txt

echo -ne "a=rtpmap:18 G729/8000\r\n" >> invite1.txt

echo -ne "a=fmtp:18 annexb=no\r\n" >> invite1.txt

echo -ne "a=rtpmap:4 G723/8000\r\n" >> invite1.txt

echo -ne "a=fmtp:4 annexa=no\r\n" >> invite1.txt

echo -ne "a=rtpmap:111 G726-32/8000\r\n" >> invite1.txt

echo -ne "a=rtpmap:101 telephone-event/8000\r\n" >> invite1.txt

echo -ne "a=fmtp:101 0-16\r\n" >> invite1.txt

echo -ne "a=silenceSupp:off - - - -\r\n" >> invite1.txt

echo -ne "a=ptime:20\r\n" >> invite1.txt

echo -ne "a=sendrecv\r\n" >> invite1.txt

if [ "$TEST" == "2" ]

then

echo -ne "CANCEL sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite21.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite21.txt

echo -ne "Max-Forwards: 16\r\n" >> invite21.txt

echo -ne "From: $FROM <sip:$FROM@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite21.txt

echo -ne "To: <sip:$CUSTNUMBER@$IPADDRSERVER>\r\n" >> invite21.txt

echo -ne "Call-ID: $CALLID2@176.31.102.152\r\n" >> invite21.txt

echo -ne "CSeq: 200 CANCEL\r\n" >> invite21.txt

echo -ne "Expires: 300\r\n" >> invite21.txt

echo -ne "User-Agent: Sippy\r\n" >> invite21.txt

echo -ne "\r\n" >> invite21.txt

echo -ne "CANCEL sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite22.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite22.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER:5061;rport=5061;branch=z9hG4bKc47$CALLID2\r\n" >> invite22.txt

echo -ne "Max-Forwards: 16\r\n" >> invite22.txt

echo -ne "From: $FROM <sip:$FROM@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite22.txt

echo -ne "To: <sip:$CUSTNUMBER@$IPADDRSERVER>\r\n" >> invite22.txt

echo -ne "Call-ID: $CALLID2@176.31.102.152\r\n" >> invite22.txt

echo -ne "CSeq: 200 CANCEL\r\n" >> invite22.txt

echo -ne "Expires: 300\r\n" >> invite22.txt

echo -ne "User-Agent: Sippy\r\n" >> invite22.txt

echo -ne "\r\n" >> invite22.txt

if [ "$TEST" == "3" ]

then

echo -ne "NOTIFY sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite31.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite31.txt

#echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER:5061;rport=5061;branch=z9hG4bKc47$CALLID2\r\n" >> invite31.txt

echo -ne "Max-Forwards: 70\r\n" >> invite31.txt

echo -ne "From: "asterisk" <sip:asterisk@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite31.txt

echo -ne "To: <sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT>\r\n" >> invite31.txt

echo -ne "Contact: <sip:asterisk@$IPADDRSERVER>\r\n" >> invite31.txt

echo -ne "Call-ID: astradnot$CALLID2@$IPADDRSERVER\r\n" >> invite31.txt

echo -ne "CSeq: 102 NOTIFY\r\n" >> invite31.txt

echo -ne "User-Agent: Astradv12\r\n" >> invite31.txt

echo -ne "Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n" >> invite31.txt

echo -ne "Supported: replaces, timer\r\n" >> invite31.txt

echo -ne "Subscription-State: terminated\r\n" >> invite31.txt

echo -ne "Event: keep-alive\r\n" >> invite31.txt

echo -ne "Content-Length: 0\r\n" >> invite31.txt

echo -ne "\r\n" >> invite31.txt

echo -ne "NOTIFY sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n" > invite32.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER;branch=z9hG4bK22fa.$CALLID1.0\r\n" >> invite32.txt

echo -ne "Via: SIP/2.0/UDP $IPADDRSERVER:5061;rport=5061;branch=z9hG4bKc47$CALLID2\r\n" >> invite32.txt

echo -ne "Max-Forwards: 70\r\n" >> invite32.txt

echo -ne "From: "asterisk" <sip:asterisk@$IPADDRSERVER>;tag=$CALLID3\r\n" >> invite32.txt

echo -ne "To: <sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT>\r\n" >> invite32.txt

echo -ne "Contact: <sip:asterisk@$IPADDRSERVER>\r\n" >> invite32.txt

echo -ne "Call-ID: astradnot$CALLID2@$IPADDRSERVER\r\n" >> invite32.txt

echo -ne "CSeq: 102 NOTIFY\r\n" >> invite32.txt

echo -ne "User-Agent: Astradv12\r\n" >> invite32.txt

echo -ne "Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n" >> invite32.txt

echo -ne "Supported: replaces, timer\r\n" >> invite32.txt

echo -ne "Subscription-State: terminated\r\n" >> invite32.txt

echo -ne "Event: keep-alive\r\n" >> invite32.txt

echo -ne "Content-Length: 0\r\n" >> invite32.txt

echo -ne "\r\n" >> invite32.txt

FILE1="invite"$TEST"1.txt"

FILE2="invite"$TEST"2.txt"

echo "" > result1.txt

echo "" > result2.txt

for i in 1 2 3 4 5

echo -ne "."

cat $FILE1|nc -u -w 1 -p $PORT2 $CUSTIPADDR $CUSTPORT >> result1.txt

LINES1=`cat result1.txt|grep astradnot|wc -l`

if [ $LINES1 -gt 0 ]

then

echo "R1"

echo "$CUSTNUMBER" >> replied.txt

exit

done

echo "F1"

for i in 1 2 3 4 5

echo -ne "."

cat $FILE2|nc -u -w 1 -p $PORT2 $CUSTIPADDR $CUSTPORT >> result2.txt

LINES2=`cat result2.txt|grep astradnot|wc -l`

if [ $LINES2 -gt 0 ]

then

break

done

if [ $LINES2 -eq 0 ]

then

echo "F2"

echo "Phone didn't reply to any message"

echo "$CUSTNUMBER" >> didntreplied.txt

exit

else

echo "R2"

echo "Phone replied to message with 2 vias"

echo "$CUSTNUMBER" >> netgearscustomers.txt

After the Invite test a Cancel test should be sent or the destination phone will ring until someone answers it.

This script is called from another one that parses a SQL file with all the required user information for that server.

#!/bin/bash

cat dump.sql |awk -F: '{print "bash generate_sip_message.sh \""$1"\" \""$2"\" \""$3"\""}' > temp.sh

bash temp.sh

Afterwards the test is run again to contact the remaining customers who haven’t answered the first time. The SQL used to generate the dump.sql’s for each server is the following:

#Parses totest.txt which contains the list of all customers not registered yet. This list should not contain any of the phones in replied.txt and netgearcustomers.txt

rm /tmp/*.sql

customerslist=`cat totest.txt`

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.117.76' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad3.sql';" > export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.16.79' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad4.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.178.108' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad5.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.142.9' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad6.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.147.45' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad7.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.143.56' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad8.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.205.108' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad9.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.167.75' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad10.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.70.119' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad11.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.138.5' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad12.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.172.156' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad14.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.151.58' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad15.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.121.115' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad16.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '176.31.247.50' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad17.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '213.251.169.218' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad18.sql';" >> export_cust.sql

echo "select CONCAT(username,':', SUBSTRING_INDEX(SUBSTRING_INDEX( contact , ';', 1 ), '@',-1 )) as contact from (select * from \`porta-sip\`.location where domain = '91.121.99.16' and username in ($customerslist) order by expires desc) a where expires > NOW() group by username into outfile '/tmp/dumpastrad19.sql';" >> export_cust.sql

The script should be run from the server where the customer is registered or else the packets will not reach the phone due to modem/router/firewall restrictions.

After obtaining the customers with problems from file netgearscustomers.txt a nslookup was run on each of their registered IPs to obtain the provider:

nslookup IP|grep "name "

Other servers

A modified version of the previous script was used to detect how many customers with the problem are in the last Porta-SIP server. These customers could be affected if the server is migrated to a newer version. The kernel in Porta-SIP does not support binding ports that are already being used unlike Debian so for detecting these customers it is required that Openser is stopped while the SIP messages are being sent. All replies are collected using a ngrep script. First the script sends only NOTIFYs with only in Via to filter out the customers without problems. Then it is run again sending the Double Via message. The phones that didn’t reply to the first message but reply to this second have the problem. Script code:

#!/usr/bin/perl

#udpserver.pl

use IO::Socket::INET;

#use Time::HiRes::usleep();

# flush after every write

$| = 1;

#Variables

my $CUSTIPADDR = @ARGV[1];

my $CUSTPORT = @ARGV[2];

my $CUSTNUMBER = @ARGV[0];

my $MESSAGEREPETITIONS = 4;

my $messagetype = 1;

my $IPADDRSERVER = '';

my $LOCALPORT = '5060';

#Code and internal variables

$CUSTPORT = '5060' if ($CUSTPORT eq '');

if ($IPADDRSERVER eq '') {

$IPADDRSERVER = `ifconfig | grep -oE "inet [^ ]+"`;

$IPADDRSERVER =~ m/([0-9.]+)/;

$IPADDRSERVER = $1;

}

if ($LOCALPORT eq '') {

$LOCALPORT = 5060;

}

my $CALLID1 = `date +%s%d%s|md5`;

$CALLID1 =~ s/\r\n//mis;

$CALLID1 =~ m/([0-9a-zA-Z]+)/;

$CALLID1 = $1;

my $CALLID2 = `date +%s%m%s|md5`;

$CALLID2 =~ s/\r\n//mis;

$CALLID2 =~ m/([0-9a-zA-Z]+)/;

$CALLID2 = $1;

my $CALLID3 = `date +%s%Y%s|md5`;

$CALLID3 =~ s/\r\n//mis;

$CALLID3 =~ m/([0-9a-zA-Z]+)/;

$CALLID3 = $1;

my $viaport = '';

my $send;

$viaport = ':'.$LOCALPORT if ($LOCALPORT != 5060);

if ($messagetype == 1) {

$send = "NOTIFY sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n".

"Via: SIP/2.0/UDP $IPADDRSERVER$viaport;branch=z9hG4bK2$CALLID1\r\n".

"Max-Forwards: 70\r\n".

"From: \"asterisk\" <sip:asterisk@$IPADDRSERVER>;tag=$CALLID3\r\n".

"To: <sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT>\r\n".

"Contact: <sip:$IPADDRSERVER>\r\n".

"Call-ID: astradnot".$CUSTNUMBER."not$CALLID2@$IPADDRSERVER\r\n".

"CSeq: 102 NOTIFY\r\n".

"User-Agent: Astradv12\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

"Subscription-State: terminated\r\n".

"Event: keep-alive\r\n".

"Content-Length: 0\r\n".

"\r\n";

} else {

$send = "NOTIFY sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT SIP/2.0\r\n".

"Via: SIP/2.0/UDP $IPADDRSERVER$viaport;branch=z9hG4bK2$CALLID1\r\n".

"Via: SIP/2.0/UDP $IPADDRSERVER:5061;rport=5061;branch=z9hG4bK3$CALLID2\r\n".

"Max-Forwards: 70\r\n".

"From: \"asterisk\" <sip:asterisk@$IPADDRSERVER>;tag=$CALLID3\r\n".

"To: <sip:$CUSTNUMBER@$CUSTIPADDR:$CUSTPORT>\r\n".

"Contact: <sip:$IPADDRSERVER>\r\n".

"Call-ID: astradnot".$CUSTNUMBER."not$CALLID2@$IPADDRSERVER\r\n".

"CSeq: 102 NOTIFY\r\n".

"User-Agent: Astradv12\r\n".

"Allow: INVITE, ACK, CANCEL, OPTIONS, BYE, REFER, SUBSCRIBE, NOTIFY, INFO, PUBLISH\r\n".

"Supported: replaces, timer\r\n".

"Subscription-State: terminated\r\n".

"Event: keep-alive\r\n".

"Content-Length: 0\r\n".

"\r\n";

}

#print $send;

for(my $i = 0; $i < $MESSAGEREPETITIONS; $i++) {

print "$IPADDRSERVER:$LOCALPORT > $CUSTIPADDR:$CUSTPORT\n";

$socket = new IO::Socket::INET (

PeerHost => "$CUSTIPADDR:$CUSTPORT",

LocalAddr => "$IPADDRSERVER:$LOCALPORT",

Proto => 'udp',

Reuse => 1,

) or die "ERROR in Socket Creation : $!\n";

print $socket "$send";

$socket->close();

# usleep ( 100000 ) if ($MESSAGEREPETITIONS > 1);

}

To read the replies:

ngrep -pql -W byline "astradnot" port 5060 or port 5070 > ngrep.txt &

After the script finishes the ngrep should be stopped and the resulting file should be treated:

The number in green (in the script and in the previous commands) should be the number of send attempts defined in the script. In the example above 4 send attempts of one Via NOTIFYs where made. In the trace file we should see at least this number of messages. If we see at least one more then the phone replied to the message

References

Possible solutions tested

http://ftp.switzernet.com/3/public/120131-outgoing-packets-interception/

Description

What is SIP ALG

Problem

Affected customers

Other servers

Possible solutions

Test results

Test results with one of the affected router models

Calls to the customer

Calls from customer

Customer detection

Other servers

References

SIP ALG:

Netgear SIP ALG references [1]:

Netgear FTP/Manuals

Customers affected (internal link)

Netgear SIP ALG references ^[1]: