Response body not closed in redirect handler when downloading content from sharepoint

[milapshahot]

Exception

Caused by: java.lang.IllegalStateException: null
at okhttp3.internal.connection.Transmitter.prepareToConnect(Transmitter.java:129)
at okhttp3.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:79)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.RedirectHandler.intercept(RedirectHandler.java:123)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.RetryHandler.intercept(RetryHandler.java:140)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.AuthenticationHandler.intercept(AuthenticationHandler.java:31)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)
at com.microsoft.graph.httpcore.TelemetryHandler.intercept(TelemetryHandler.java:37)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:142)
at okhttp3.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:117)

Cause

• Sharepoint API 'sometime' return back response for 302 redirect call without "Content-Length" and with "Transfer-Encoding: chunked"
• Above cause, responseBody close call skip as content-length would be set as -1.
• Subsequent request fail with above exception

Expected behavior

• WHEN RESPONSE HAS NO BODY BYTES, USE 'CONTENT-LENGTH: 0' RATHER THAN 'TRANSFER-ENCODING: CHUNKED' for 302 re-direct call

SDK Version

<microsoft-graph.version>2.5.0</microsoft-graph.version>
<microsoft-graph-auth.version>0.2.0</microsoft-graph-auth.version>

AB#8012

[baywet]

Hi @milapshahot
Thanks for reaching out.
Can you share the lines of code calling the SDK you wrote to get that exception please?

[milapshahot]

Thanks for response. Please find below sample hava code and actual request / response which is causing above issue Code (java 11): // Build Graph Service Client var client = GraphServiceClient .builder() .authenticationProvider(authProvider) .httpProvider(httpProvider) .buildClient(); // Download the file from siteId/DriveId/ItemId try(var inputStream = client .sites(siteId) .drives() .byId(driveId) .items(itemId) .content() .buildRequest() .get()) { // Exception would be thrown here... } Request: Request{method=GET, url=https://graph.microsoft.com/v1.0/sites/...,<>/drives/<>/items/<>/content, tags={class com.microsoft.graph.httpcore.middlewareoption.RedirectOptions=com.microsoft.graph.httpcore.middlewareoption.RedirectOptions@303e3593, class com.microsoft.graph.httpcore.middlewareoption.RetryOptions=com.microsoft.graph.httpcore.middlewareoption.RetryOptions@4ef27d66, class com.microsoft.graph.httpcore.middlewareoption.TelemetryOptions=com.microsoft.graph.httpcore.middlewareoption.TelemetryOptions@314b8f2d}} Response: Response{protocol=http/1.1, code=302, message=Found, url=https://graph.microsoft.com/v1.0/sites/<>.sharepoint.com,<>/drives/<>/items/<>/content Cache-Control: no-cache Transfer-Encoding: chunked Location: https://<>.sharepoint.com/_layouts/15/download.aspx?UniqueId=24e7c35d-fad4-4578-a9b4-67dfd33f5c54&Translate=false&tempauth=<>&ApiVersion=2.1 Strict-Transport-Security: max-age=31536000 request-id: 079e969a-2ffd-416d-b7af-12c1dae36b3d client-request-id: 304784e8-17ff-447b-a145-0b23bcb75347 x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Central US","Slice":"E","Ring":"2","ScaleUnit":"000","RoleInstance":"CH01EPF00003EE2"}} Note: Response header is having `Transfer-Encoding: chunked` for 302 (redirect) and hence it response body remain open causing subsequent download request to fail. Can you share the lines of code calling the SDK you wrote to get that exception please? Proprietary/Internal Proprietary/Internal

[baywet]

Thanks for the additional details.
I'm not able to reproduce the issue so far, when I execute
GET https://graph.microsoft.com/v1.0/sites/baywet.sharepoint.com,feb5ca4c-5076-4547-b7a6-ba449bb5b732,e7e9fdf9-21e0-4d0c-9680-d4ec71dac07c/drive/items/01Q6JGVZQ3WTAAVMTOJNEK4AQXTJDKTGKD/content

The response is

HTTP/1.1 302 Found
Content-Length: 0
Content-Type: text/plain
Location: https://baywet.sharepoint.com/_layouts/15/download.aspx?UniqueId=0ac0b41b-6eb2-484b-ae02-179a46a99943&Translate=false&tempauth=[REDACTED]&ApiVersion=2.1

Note the location URL pointing to SharePoint and not graph.microsoft.com, which is the expected behavior.
Is there anything special with that file? Was it created using SharePoint's copy API?

Please also upgrade auth to 0.3.0 and SDK to 2.6.0 to rule these options out.

[milapshahot]

Thanks for suggestion.

Above http response contains "Content-Length" as 0 and hence you were not able to reproduce.

In example posted above, issue occurs if there is missing 'Content-Length' 0 in response for 302

Response{protocol=http/1.1, code=302, message=Found, 
Cache-Control: no-cache
Transfer-Encoding: chunked
Location: https://<>.sharepoint.com/_layouts/15/download.aspx?UniqueId=24e7c35d-fad4-4578-a9b4-67dfd33f5c54&Translate=false&tempauth=<>&ApiVersion=2.1
Strict-Transport-Security: max-age=31536000
request-id: 079e969a-2ffd-416d-b7af-12c1dae36b3d
client-request-id: 304784e8-17ff-447b-a145-0b23bcb75347
x-ms-ags-diagnostic: {"ServerInfo":{"DataCenter":"North Central US","Slice":"E","Ring":"2","ScaleUnit":"000","RoleInstance":"CH01EPF00003EE2"}}

Above cause, "response body" to remain open and causing exception as stated in original reply since 'RedirectHandler' assume okhttp to close the response body before proceeding to next interceptor

[baywet]

Thanks for the additional details.
I understand the Content-Length response header is not set in your case, I'm not able to get to this step.
Can you share more information about the file? Size, how was it created, any DLP in place that might lock the file, etc...
Also, any HTTP proxy between your application and Microsoft Graph?
Thanks!

[baywet]

I went back and did some more testing. This case is similar to microsoftgraph/msgraph-sdk-java#635 in the sense that the service behavior changes depending on which "version of the service" your request lands on. The asp.net core version of the service has enabled chunked encoding by default which the "older" version of the service has not.
See the different results in fiddler for the following request.

GET /v1.0/sites/baywet.sharepoint.com,feb5ca4c-5076-4547-b7a6-ba449bb5b732,e7e9fdf9-21e0-4d0c-9680-d4ec71dac07c/drive/items/01Q6JGVZQ3WTAAVMTOJNEK4AQXTJDKTGKD/content

(new instance)

(old instance)

Forcing the instance using the IP with the hosts configuration file allowed me to reproduce the issue systematically. And implement a fix for it.
I'll now transfer the issue in the core repository as the faulty code is in the core repo.

[baywet]

To follow up on the fix. I just authored #139 which fixes this issue and will close it once merged. It'll go out with core version 1.0.8.
The following stack overflow discussion put me on the path of understanding that responses are kept open with chunk encoding for interceptors. And that a chain should not contain 2 open responses so it needs to be closed because the newer request for the redirect can be executed.