A weird issue with the site: https://myip.net.il/

[elico]

I own the site:
https://myip.net.il/

It's a simple httpd (apache) server and it seems to work fine on http via redwood but not on intercepted traffic.
I am using the latest version of Redwood built today from sources.
I have tested the issue and for now what I got to the point is that I have from the browser a har file:
myip.net.il_Archive [21-07-27 20-56-16].har

From the server side I see a full response being sent to the client but for some reason RedWood think that the response is malformed.
To see the issue just try to access the site URL at:
https://myip.net.il/

behind Redwood with ssl-bump interception.

[andybalholm]

For me, the main part of the page works fine, but the image fails. However, if I fetch the image in a new tab, it works. It is an issue with reusing the connection. Probably if I would add malformed HTTP response "" to the list of errors that trigger a "redial" that would take care of it.

[andybalholm]

But you should try to figure out why your server has the issues with reusing the connection.

[elico]

It the most simple apache server... I will try to verify the issue next week. Thanks. בתאריך יום ד׳, 28 ביולי 2021, 0:58, מאת Andy Balholm ‏< ***@***.***>:

…

But you should try to figure out why your server has the issues with reusing the connection. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#78 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AANUHMECWKJ6OFWHSJJEOOTTZ4TYNANCNFSM5BC3J7GA> .

[elico]

@andybalholm I have verified what was the issue and I believe this should be fixed some how.
The basic issue was that the Content-Length header was off by 1 byte '\n'.
Due to this Redwood was reading only the Content-Length and was leaving 1 byte at the buff(er.
Then when trying to read the next response it was actually reading the last byte from the last response and it was considered the "status line" for the new request..

I don't know if it's a golang or Redwood specific issue.

[elico]

@andybalholm The main question is:
Is this issue specific to Redwood or is it a GoLang http library bug?

I have also tested this issue with couple other proxy services, both commercial and open source.
All of the commercial products handled this issue pretty easy.
Squid and couple proxies(both reverse and forward) handled this the same way without any other issue.
Any browser accessing directly the service receives the html and the image in the same connection without any errors at all.

[andybalholm]

There is no bug anywhere but your server. Apparently that bug is common enough that most browsers and proxies have added workarounds, although I don't think we've run into it before.

We could add a workaround at

redwood/transport.go

Line 172 in 2efa4ab

resp, err = ReadResponse(ct.br, req)

,
checking for this error and calling ReadResponse again.

[elico]

@andybalholm Did we got into a conclusion what to do with this error?

[andybalholm]

I think fixing your server to send the correct Content-Length was the right thing to do.