100 (Continue) HTTP response should be handled gracefully regardless of whether it was requested/expected

[stephentalley]

I'm using OkHttp 3.8.1 on Android to POST to a poorly-behaved server.

Even though the request does not include an "Expect: 100-continue" header, the web server returns a 100 Continue response. Rather than continuing to send the request body (or just ignoring the 100 response since the body had already been sent), then getting the actual (200) response, OkHttp stops there and sends back the 100 Continue response in my okhttp.Callback.

The w3c definition of the 100 status code states:

The client SHOULD continue by sending the remainder of the request or, if the request has already been completed, ignore this response.

and:

A client MUST be prepared to accept one or more 1xx status responses prior to a regular response, even if the client does not expect a 100 Continue status message.

I have verified the correct behavior on three other HTTP clients (iOS, Windows, and curl(1)), who correctly ignore/continue when receiving an unexpected 100 Continue response.

In contrast, it looks like OkHttp only handles the 100 response iff it was explicitly requested via an "Expect: 100-continue" header. From CallServerInterceptor.java:

// If there's a "Expect: 100-continue" header on the request, wait for a "HTTP/1.1 100
// Continue" response before transmitting the request body. If we don't get that, return
// what we did get (such as a 4xx response) without ever transmitting the request body.
if ("100-continue".equalsIgnoreCase(request.header("Expect"))) {
    httpCodec.flushRequest();
    realChain.eventListener().responseHeadersStart(realChain.call());
    responseBuilder = httpCodec.readResponseHeaders(true);
}

Incidentally, the obvious workaround (including "Expect: 100-continue" in the request to trigger OkHttp's logic) didn't work for me because this particular server is further broken in that it complains about a mangled Expect: header. 😑 It doesn't appear any other workaround is possible, though I'd love to be proven wrong.

See: https://stackoverflow.com/a/46382361/920849

[stephentalley]

The key takeaway is that a server can choose to return 100 when the client doesn't request it, or return a non-100 response when even when the client does request it.

[yschimke]

@stephentalley Thanks for the detailed writeup. It sounds like a legitimate bug. Given the rarity of it and it being triggered by a badly behaving server, it may not be at the top of the pile to work on. So if it's really impacting you we would love a PR and I'll happily review quickly.

[yschimke]

@stephentalley I think the appropriate spec is now https://tools.ietf.org/html/rfc7231#section-6.2.1.

The client ought to continue sending the request and discard the 100 response.

Since when is "ought" a rfc2119 keyword :(

[rfc2822]

I have also encountered this problem, and I don't want to add the Expect header because we send requests to a lot of different servers and I don't want to risk breaking working things.

What do you mean by "badly behaving server"? As I understand it, any server may send 100 Continue, and it should be ignored.

Do you think it's possible to write a patch/test without much okhttp experience?

[yschimke]

The server is arguably doing the wrong thing by the spec. The spec says the client should be tolerant even if it didn't request this (Postel's law).

I say arguably, because as the spec mentions, ther are clients out there that don't support this and servers will break by implementing.

Note: The Expect header field was added after the original
publication of HTTP/1.1 [RFC2068] as both the means to request an
interim 100 (Continue) response and the general mechanism for
indicating must-understand extensions. However, the extension
mechanism has not been used by clients and the must-understand
requirements have not been implemented by many servers, rendering
the extension mechanism useless. This specification has removed
the extension mechanism in order to simplify the definition and
processing of 100-continue.

But it's now a bug based on the latest spec, so OkHttp should support this even when not sending Expect.

If you need this fixed ASAP, then definitely read through the code, try making a failing test and PR, ask questions here and I'll (and others) help if you get stuck.

[johan12345]

We are also encountering this issue in our Android app with one specific server (see opacapp/opacclient#497) since we switched from Apache HttpClient to OkHttp. I don't have any experience with the OkHttp codebase either, but maybe I can try to look into it.

[johan12345]

I have sent a pull request (#3766) that includes a failing test and a commit that seems to fix the problem. However, there is still one issue where I am stuck -> see the description of the PR.