RFC Violation in Subscribe POST request

[mwareman]

I have been trying to setup MobiLinc (on Android) to work thru a reverse proxy - since I only have 1 IP address at home and multiple services I need to access - all thru a single IP and SSL port (443). I have a wildcard certificate on my server and use Apache to reverse proxy to my different services internally. This works fine with most things - but Mobilinc has a curiosity when talking to my ISY.

In MobiLinc - I add the hostname of the proxy, the port and credentials - and the proxy does it's thing - proxying the requests to my ISY. It connects and syncs fine. However, I am not getting status when devices and scenes change state. Everything shows as 'Off' - even when not.

I did notice in the apache log file - when MobiLinc is first started I get this error:

Code: Select all

[Sun Mar 10 09:26:07 2013] [error] [client 172.20.1.151] client sent HTTP/1.1 request without hostname (see RFC2616 section 14.23): /services

I don't get this after initial opening - and I see many other 'POST' requests to /services that work fine - just one initially when opening the application.

Wondering whats is going on - I used tcpdump to look at the request causing this:

Code: Select all

09:28:18.240625 IP 172.20.1.151.45070 > 172.20.1.21.80: Flags [P.], seq 1:415, ack 1, win 229, options [nop,nop,TS val 13052198 ecr 390573769], length 414
        0x0000:  0027 19d9 ed7e 5046 5d81 1a11 0800 4500  .'...~PF].....E.
        0x0010:  01d2 52ca 4000 4006 8b87 ac14 0197 ac14  ..R.@.@.........
        0x0020:  0115 b00e 0050 f92e 7bee 0f4c d072 8018  .....P..{..L.r..
        0x0030:  00e5 3b4d 0000 0101 080a 00c7 2926 1747  ..;M........)&.G
        0x0040:  aec9 504f 5354 202f 7365 7276 6963 6573  ..POST./services
        0x0050:  2048 5454 502f 312e 310d 0a41 7574 686f  .HTTP/1.1..Autho
        0x0060:  7269 7a61 7469 6f6e 3a20 4261 7369 6320  rization:.Basic.
        0x0070:  0000 0000 0000 0000 0000 0000 0000 0000  xxxxxxxxxxxxxxxx
        0x0080:  0000 0000 0000 003d 0d0a 436f 6e74 656e  xxxxxxx=..Conten
        0x0090:  742d 4c65 6e67 7468 3a20 3139 320d 0a43  t-Length:.192..C
        0x00a0:  6f6e 7465 6e74 2d54 7970 653a 2074 6578  ontent-Type:.tex
        0x00b0:  742f 786d 6c3b 2063 6861 7273 6574 3d22  t/xml;.charset="
        0x00c0:  7574 662d 3822 0d0a 4e54 3a75 706e 703a  utf-8"..NT:upnp:
        0x00d0:  6576 656e 7453 4f41 5041 4354 494f 4e3a  eventSOAPACTION:
        0x00e0:  2275 726e 3a75 6469 2d63 6f6d 3a73 6572  "urn:udi-com:ser
        0x00f0:  7669 6365 3a58 5f49 6e73 7465 6f6e 5f4c  vice:X_Insteon_L
        0x0100:  6967 6874 696e 675f 5365 7276 6963 653a  ighting_Service:
        0x0110:  3123 5375 6273 6372 6962 6522 0d0a 0d0a  1#Subscribe"....
        0x0120:  3c73 3a45 6e76 656c 6f70 653e 3c73 3a42  <s:Envelope><s:B
        0x0130:  6f64 793e 3c75 3a53 7562 7363 7269 6265  ody><u:Subscribe
        0x0140:  2078 6d6c 6e73 3a75 3d22 7572 6e3a 7564  .xmlns:u="urn:ud
        0x0150:  692d 636f 6d3a 7365 7276 6963 653a 585f  i-com:service:X_
        0x0160:  496e 7374 656f 6e5f 4c69 6768 7469 6e67  Insteon_Lighting
        0x0170:  5f53 6572 7669 6365 3a31 223e 3c72 6570  _Service:1"><rep
        0x0180:  6f72 7455 524c 3e52 4555 5345 5f53 4f43  ortURL>REUSE_SOC
        0x0190:  4b45 543c 2f72 6570 6f72 7455 524c 3e3c  KET</reportURL><
        0x01a0:  6475 7261 7469 6f6e 3e69 6e66 696e 6974  duration>infinit
        0x01b0:  653c 2f64 7572 6174 696f 6e3e 3c2f 753a  e</duration></u:
        0x01c0:  5375 6273 6372 6962 653e 3c2f 733a 426f  Subscribe></s:Bo
        0x01d0:  6479 3e3c 2f73 3a45 6e76 656c 6f70 653e  dy></s:Envelope>

(I have zerod the authorization header...). I noticed the lack of a 'Host:' header in the request. This is a RFC violation. Apache returns a '400 Bad Request' in response to this - because it's declaring HTTP/1.1 and is not providing a 'Host:' header.

The second request looks like this:

Code: Select all

09:28:18.241391 IP 172.20.1.151.38544 > 172.20.1.21.80: Flags [P.], seq 1:487, ack 1, win 229, options [nop,nop,TS val 13052198 ecr 390573769], length 486
        0x0000:  0027 19d9 ed7e 5046 5d81 1a11 0800 4500  .'...~PF].....E.
        0x0010:  021a 3547 4000 4006 a8c2 ac14 0197 ac14  ..5G@.@.........
        0x0020:  0115 9690 0050 e613 1c61 8ae6 2e19 8018  .....P...a......
        0x0030:  00e5 8792 0000 0101 080a 00c7 2926 1747  ............)&.G
        0x0040:  aec9 504f 5354 202f 7365 7276 6963 6573  ..POST./services
        0x0050:  2048 5454 502f 312e 310d 0a43 6f6e 7465  .HTTP/1.1..Conte
        0x0060:  6e74 2d74 7970 653a 2074 6578 742f 786d  nt-type:.text/xm
        0x0070:  6c3b 2063 6861 7273 6574 3d75 7466 2d38  l;.charset=utf-8
        0x0080:  0d0a 736f 6170 6163 7469 6f6e 3a20 7572  ..soapaction:.ur
        0x0090:  6e3a 7564 692d 636f 6d3a 7365 7276 6963  n:udi-com:servic
        0x00a0:  653a 585f 496e 7374 656f 6e5f 4c69 6768  e:X_Insteon_Ligh
        0x00b0:  7469 6e67 5f53 6572 7669 6365 3a31 2347  ting_Service:1#G
        0x00c0:  6574 416c 6c44 3244 0d0a 4175 7468 6f72  etAllD2D..Author
        0x00d0:  697a 6174 696f 6e3a 2042 6173 6963 2000  ization:.Basic.x
        0x00e0:  0000 0000 0000 0000 0000 0000 0000 0000  xxxxxxxxxxxxxxxx
        0x00f0:  0000 0000 0000 3d0d 0a55 7365 722d 4167  xxxxxx=..User-Ag
        0x0100:  656e 743a 2044 616c 7669 6b2f 312e 362e  ent:.Dalvik/1.6.
        0x0110:  3020 284c 696e 7578 3b20 553b 2041 6e64  0.(Linux;.U;.And
        0x0120:  726f 6964 2034 2e32 2e32 3b20 4e65 7875  roid.4.2.2;.Nexu
        0x0130:  7320 3720 4275 696c 642f 4a44 5133 3929  s.7.Build/JDQ39)
        0x0140:  0d0a 486f 7374 3a20 0000 0000 0000 2e00  ..Host:.xxxxxx.x
        0x0150:  0000 0000 0000 0000 0000 0000 2e63 6f6d  xxxxxxxxxxxx.com
        0x0160:  0d0a 436f 6e6e 6563 7469 6f6e 3a20 4b65  ..Connection:.Ke
        0x0170:  6570 2d41 6c69 7665 0d0a 4163 6365 7074  ep-Alive..Accept
        0x0180:  2d45 6e63 6f64 696e 673a 2067 7a69 700d  -Encoding:.gzip.
        0x0190:  0a43 6f6e 7465 6e74 2d4c 656e 6774 683a  .Content-Length:
        0x01a0:  2031 3238 0d0a 0d0a 3c73 3a45 6e76 656c  .128....<s:Envel
        0x01b0:  6f70 653e 3c73 3a42 6f64 793e 3c75 3a47  ope><s:Body><u:G
        0x01c0:  6574 416c 6c44 3244 2078 6d6c 6e73 3a75  etAllD2D.xmlns:u
        0x01d0:  3d22 7572 6e3a 7564 692d 636f 6d3a 7365  ="urn:udi-com:se
        0x01e0:  7276 6963 653a 585f 496e 7374 656f 6e5f  rvice:X_Insteon_
        0x01f0:  4c69 6768 7469 6e67 5f53 6572 7669 6365  Lighting_Service
        0x0200:  3a31 223e 3c2f 753a 4765 7441 6c6c 4432  :1"></u:GetAllD2
        0x0210:  443e 3c2f 733a 426f 6479 3e3c 2f73 3a45  D></s:Body></s:E
        0x0220:  6e76 656c 6f70 653e                      nvelope>

Notice that the second (and, in fact, subsequent) requests include the 'Host:' header as they should (again - zeroed out for privacy).

The first request appears to be the 'Subscribe' request. This likely explains why the application does not show or update status.

According to the RFC - All HTTP/1.1 requests *must* include the 'Host:' header. The subscribe request seems to be failing due to the RFC violation in the request.

Can this please be fixed?

Thanks,

Michael.

[AdminWes]

Hi Michael,

Have you tried using our MobiLinc Connect service instead? It's designed to work in restricted network scenarios like your environment all without having to do port forwarding, etc.

It's free to try for 30 days. To get started select MobiLinc Connect as the Host Type in the MobiLinc->Lighting Controller settings.

We likely cannot makes changes to the network interface as what we implemented was specific to talk directly to the ISY controller. It's uncertain as to what side-effects MobiLinc would have if we made changes to our network interface.

Wes

[mwareman]

I have. Honestly - even if you made it 'free forever' - and I know I should trust you - but I prefer to keep things entirely within my own control thanks. I want to be able to control my device when my Internet circuit is down (from within my own house) - without having to switch profiles. Having a split-dns and a reverse-proxy allows me to do this - and uncovered this bug.

Asking me to subscribe to a for-cost service to work around an RFC non-compliance bug seems a little silly to me.

As I said - I'm merely asking that you add a 'Host:' header to the 'SUBSCRIBE' request to bring the application into RFC compliance. That way - it will work with RFC compliant reverse proxies. I don't see this as 'major changes to the network interface'. I don't see how this will break anything.

Michael.

[AdminWes]

Michael,

We'll take a look at it, but I can't guarantee anything. I understand your point about RFC compliance, however our product is not a general use network appliance. It was designed to talk directly to the ISY or to our MobiLinc Connect service. RFC or not, our network interface needs to be compliant to what the ISY will accept for communication.

My point is adding this sounds trivial, but may introduce many issues we have to then debug and add workarounds (if even possible) to support your use case.

Wes

[mwareman]

Thanks Wes. I appreciate the response. Just one thing:

RFC or not, our network interface needs to be compliant to what the ISY will accept for communication.

Seems a strange statement, IMO. It's HTTP/1.1 - and the IETF has set a standard for formatting HTTP/1.1. To create headers non-compliant defeats the whole purpose of declaring that the request is a HTTP/1.1 request in the first place.

Certainly, if I manually craft a SUBSCRIBE request to the ISY WITH a Host: header - it appears to accept it fine. Seems like such a simple thing to fix - and allow the application to work thru SSL proxies.

There is another use-case you should be aware of.

You may be intending to talk directly to the ISY (and so feel it's OK to ignore the RFC for HTTP/1.1 compliance) - but when used on many corporate networks (and, increasingly, ISPs) that employ WPAD (Web Proxy Auto Discovery) or other forced proxy scenarios - your application could be accessing the ISY API THRU a proxy that is enforcing protocol compliance. Sending non-RFC compliant requests will get the request denied. Not declaring HTTP/1.1 in the header (using, say, HTTP/1.0 which does not require the Host: header) would allow it thru - as would adding the Host: header to bring the request into compliance. That wouldn't solve the reverse proxy scenario - but I could work around that by proxying based on the user agent string.

I have shown two common scenarios so far that protocol non-compliance is causing right now. If the intention was to do custom protocol talking between clients and the ISY - then HTTP/1.1 should not be declared.

So you know, I have purchased the app on BOTH IOS (HD version for the iPad) and Android, and added the camera module on IOS. I have invested a lot (in the app world) in your application. So far - I've only been able to use it on my LAN going directly to the ISY. I would *really* like to be able to use it externally as well.

Thanks!

Michael.

[AdminWes]

Michael,

As I stated before, we're looking into this for Android. For iOS, it does set the HOST field on the subscribe request. Based on your description, the iOS version should meet the RFC and work in your use-case scenario.

Have you tried your setup with our iOS app? If you can confirm that this works, it will make our update to MobiLinc/Android a little smoother.

Wes

[mwareman]

Wes,

I get a different 'Connection Error' on IOS - but I have confirmed the 'SUBSCRIBE' request. This is what it looks like on IOS:

Code: Select all

POST /services HTTP/1.1
Authorization: Basic xxxxxxxxxxxx
Host: lights.domain.com:80
Content-Length: 193
Content-Type: text/xml; charset="utf-8"
NT:upnp:event
SOAPACTION:"urn:udi-com:service:X_Insteon_Lighting_Service:1#Subscribe"

<s:Envelope><s:Body><u:Subscribe xmlns:u="urn:udi-com:service:X_Insteon_Lighting_Service:1"><reportURL>REUSE_SOCKET</reportURL><duration>infinite</duration></u:Subscribe></s:Body></s:Envelope>

I'll create a thread for the other connect issue I have with IOS in the correct forum.

Thanks!

Michael.

[AdminWes]

Ok, thanks.

Wes