UPnP doesn't always work

[yurivict]

On my system UPnP doesn't work. UPnP is checked, some port (18XXX) is specified, but UPnP page on the router never shows bitmessage. It does show ZeroNet and Skype, just not bitmessage.

FreeBSD 10.3

(Before I was running with manually opened port and it worked fine.)

[PeterSurda]

I don't have a FreeBSD system behind a UPnP firewall. I can do some tcpdump but even if I find the cause I probably can't test it.

[yurivict]

I use wireshark display filter: udp.dstport == 1900 or udp.srcport == 1900
When bitmessage starts, this exchange happens:

BM->router

0000   4d 2d 53 45 41 52 43 48 20 2a 20 48 54 54 50 2f  M-SEARCH * HTTP/
0010   31 2e 31 0d 0a 48 4f 53 54 3a 20 32 33 39 2e 32  1.1..HOST: 239.2
0020   35 35 2e 32 35 35 2e 32 35 30 3a 31 39 30 30 0d  55.255.250:1900.
0030   0a 4d 41 4e 3a 20 22 73 73 64 70 3a 64 69 73 63  .MAN: "ssdp:disc
0040   6f 76 65 72 22 0d 0a 4d 58 3a 20 32 0d 0a 53 54  over"..MX: 2..ST
0050   3a 20 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75 70  : urn:schemas-up
0060   6e 70 2d 6f 72 67 3a 64 65 76 69 63 65 3a 49 6e  np-org:device:In
0070   74 65 72 6e 65 74 47 61 74 65 77 61 79 44 65 76  ternetGatewayDev
0080   69 63 65 3a 31 0d 0a 0d 0a                       ice:1....

router->MB

0000   48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d  HTTP/1.1 200 OK.
0010   0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20  .Cache-Control: 
0020   6d 61 78 2d 61 67 65 3d 33 30 30 0d 0a 44 61 74  max-age=300..Dat
0030   65 3a 20 54 68 75 2c 20 32 36 20 4d 61 79 20 32  e: Thu, 26 May 2
0040   30 31 36 20 31 39 3a 34 31 3a 31 33 20 47 4d 54  016 19:41:13 GMT
0050   0d 0a 45 78 74 3a 20 0d 0a 4c 6f 63 61 74 69 6f  ..Ext: ..Locatio
0060   6e 3a 20 68 74 74 70 3a 2f 2f 31 39 32 2e 31 36  n: http://192.16
0070   38 2e 35 2e 31 3a 31 37 38 30 2f 49 6e 74 65 72  8.5.1:1780/Inter
0080   6e 65 74 47 61 74 65 77 61 79 44 65 76 69 63 65  netGatewayDevice
0090   2e 78 6d 6c 0d 0a 53 65 72 76 65 72 3a 20 50 4f  .xml..Server: PO
00a0   53 49 58 20 55 50 6e 50 2f 31 2e 30 20 44 44 2d  SIX UPnP/1.0 DD-
00b0   57 52 54 20 4c 69 6e 75 78 2f 56 32 34 0d 0a 53  WRT Linux/V24..S
00c0   54 3a 20 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75  T: urn:schemas-u
00d0   70 6e 70 2d 6f 72 67 3a 64 65 76 69 63 65 3a 49  pnp-org:device:I
00e0   6e 74 65 72 6e 65 74 47 61 74 65 77 61 79 44 65  nternetGatewayDe
00f0   76 69 63 65 3a 31 0d 0a 55 53 4e 3a 20 75 75 69  vice:1..USN: uui
0100   64 3a 44 38 31 39 36 44 46 41 2d 37 38 46 36 2d  d:D8196DFA-78F6-
0110   37 38 46 39 2d 34 35 36 38 2d 35 44 45 39 37 38  78F9-4568-5DE978
0120   33 37 41 36 41 37 3a 3a 75 72 6e 3a 73 63 68 65  37A6A7::urn:sche
0130   6d 61 73 2d 75 70 6e 70 2d 6f 72 67 3a 64 65 76  mas-upnp-org:dev
0140   69 63 65 3a 49 6e 74 65 72 6e 65 74 47 61 74 65  ice:InternetGate
0150   77 61 79 44 65 76 69 63 65 3a 31 0d 0a 0d 0a     wayDevice:1....

But this doesn't establish the UPnP forward for some reason.

Also, Network Settings screen asks for port even when UPnP is checked. But the port I specified doesn't appear in the above exchange. I don't think BM should ask for port when UPnP is checked.

[yurivict]

For comparison, ZeroNet establishes UPnP fine with this exchange:

ZN->router:

0000   4d 2d 53 45 41 52 43 48 20 2a 20 48 54 54 50 2f  M-SEARCH * HTTP/
0010   31 2e 31 0d 0a 48 4f 53 54 3a 20 32 33 39 2e 32  1.1..HOST: 239.2
0020   35 35 2e 32 35 35 2e 32 35 30 3a 31 39 30 30 0d  55.255.250:1900.
0030   0a 4d 41 4e 3a 20 22 73 73 64 70 3a 64 69 73 63  .MAN: "ssdp:disc
0040   6f 76 65 72 22 0d 0a 4d 58 3a 20 32 0d 0a 53 54  over"..MX: 2..ST
0050   3a 20 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75 70  : urn:schemas-up
0060   6e 70 2d 6f 72 67 3a 64 65 76 69 63 65 3a 49 6e  np-org:device:In
0070   74 65 72 6e 65 74 47 61 74 65 77 61 79 44 65 76  ternetGatewayDev
0080   69 63 65 3a 31 0d 0a 0d 0a                       ice:1....

router->ZN:

0000   48 54 54 50 2f 31 2e 31 20 32 30 30 20 4f 4b 0d  HTTP/1.1 200 OK.
0010   0a 43 61 63 68 65 2d 43 6f 6e 74 72 6f 6c 3a 20  .Cache-Control: 
0020   6d 61 78 2d 61 67 65 3d 33 30 30 0d 0a 44 61 74  max-age=300..Dat
0030   65 3a 20 54 68 75 2c 20 32 36 20 4d 61 79 20 32  e: Thu, 26 May 2
0040   30 31 36 20 31 39 3a 35 36 3a 35 30 20 47 4d 54  016 19:56:50 GMT
0050   0d 0a 45 78 74 3a 20 0d 0a 4c 6f 63 61 74 69 6f  ..Ext: ..Locatio
0060   6e 3a 20 68 74 74 70 3a 2f 2f 31 39 32 2e 31 36  n: http://192.16
0070   38 2e 35 2e 31 3a 31 37 38 30 2f 49 6e 74 65 72  8.5.1:1780/Inter
0080   6e 65 74 47 61 74 65 77 61 79 44 65 76 69 63 65  netGatewayDevice
0090   2e 78 6d 6c 0d 0a 53 65 72 76 65 72 3a 20 50 4f  .xml..Server: PO
00a0   53 49 58 20 55 50 6e 50 2f 31 2e 30 20 44 44 2d  SIX UPnP/1.0 DD-
00b0   57 52 54 20 4c 69 6e 75 78 2f 56 32 34 0d 0a 53  WRT Linux/V24..S
00c0   54 3a 20 75 72 6e 3a 73 63 68 65 6d 61 73 2d 75  T: urn:schemas-u
00d0   70 6e 70 2d 6f 72 67 3a 64 65 76 69 63 65 3a 49  pnp-org:device:I
00e0   6e 74 65 72 6e 65 74 47 61 74 65 77 61 79 44 65  nternetGatewayDe
00f0   76 69 63 65 3a 31 0d 0a 55 53 4e 3a 20 75 75 69  vice:1..USN: uui
0100   64 3a 44 38 31 39 36 44 46 41 2d 37 38 46 36 2d  d:D8196DFA-78F6-
0110   37 38 46 39 2d 34 35 36 38 2d 35 44 45 39 37 38  78F9-4568-5DE978
0120   33 37 41 36 41 37 3a 3a 75 72 6e 3a 73 63 68 65  37A6A7::urn:sche
0130   6d 61 73 2d 75 70 6e 70 2d 6f 72 67 3a 64 65 76  mas-upnp-org:dev
0140   69 63 65 3a 49 6e 74 65 72 6e 65 74 47 61 74 65  ice:InternetGate
0150   77 61 79 44 65 76 69 63 65 3a 31 0d 0a 0d 0a     wayDevice:1....

[yurivict]

Two requests are actually identical. I don't understand what the problem is. This needs investigation.

[PeterSurda]

The UPnP port mapping typically has multiple phases, for simplicity I'll describe the two most important ones:

• broadcast to find the gateways
• contacting the gateway to establish the mapping

Try dumping host 192.168.5.1 tcp port 1780, that will show you the second phase.

[bmng-dev]

A lot of the UPnP specification is optional resulting in a variety of behaviors across devices. It is better to use an existing UPnP or port mapping library which handles the nuances of different implementations.

[yurivict]

ZeroNet appears to handle UPnP reliably, except they don't delete it on the graceful shutdown.

[PeterSurda]

I used zeronet's UPnP to fix the bugs in PyBitmessage's UPnP, but it's possible I missed something. Anyway, I switched my FreeBSD VM into bridged mode, and UPnP works fine. @yurivict can you tell me something more about your setup, like what UPnP server runs on your router, if BM logs an error when trying to obtain port mapping, etc.

Maybe compare the second phase (port 1780) between PyBitmessage and ZeroNet?

[yurivict]

I have a standard DD-WRT router software.

BM actually prints a lot of messages like this:

2016-06-20 02:59:26,713 - ERROR - UPnP error: <s:Envelope xmlns:s="http://schemas.xmlsoap.org/soap/envelope/" s:encodingStyle="http://schemas.xmlsoap.org/soap/encoding/"><s:Body><s:Fault><faultcode>s:Client</faultcode><faultstring>UPnPError</faultstring><detail><UPnPError xmlns="urn:schemas-upnp-org:control-1-0"><errorCode>402</errorCode><errorDescription>Invalid Argument</errorDescription></UPnPError></detail></s:Fault></s:Body></s:Envelope>

[PeterSurda]

That may mean the TCP port is unavailable (which is normal), or it could be a bug. I have OpenWRT which is more-or-less the same as DD-WRT, so it's probably the same server. Can you do a dump of what BM is sending to the router?

[yurivict]

Ok, I will do a UPnP dump with the latest version when I will have some time.

[bmng-dev]

I believe the problem is the hardcoding of WANIPConnection as the service. @yurivict most likely has a device advertising the WANPPPConnection.