Use HttpClient rather than WebClient

[panesofglass]

The HttpClient from System.Net.Http returns much richer types than those used by WebClient and HttpWebRequest. Would you be amenable to switching to an HttpClient-based implementation? HttpClient is now supported on Windows Phone and Windows 8, as well.

[ovatsus]

HttpClient is .NET 4.5 only, which limits it's applicability greatly. There's lot of people that can have VS2012 and .NET 4.5 on their dev machines, but can't upgrade the solutions to .NET 4.5 because it dropped support for Windows XP. Currently, FSharp.Net.Http works on .NET 4.0, Silverlight 5, Windows 8 and Mono (and I'll push support for Silverlight 4 and Windows Phone 7 soon)
The FSharp.Data 1.1.1 documentation hasn't been published yet, but none of the examples is using WebClient any more as it was in version 1.0.13, which is the published version of the docs. I think FSharp.Net.Http is taking a great shape, what features of HttpClient do you miss when using it? We could try to enhance it further. I have some local changes I've made that I haven't pushed yet that add support for setting the Referer, set GET or POST method automatically, allow to send and retrieve cookies, and allow to get the the response headers and the response uri in addition to the response body.

[panesofglass]

HttpClient is available for 4.0, as well. I've been using it with Frank for a long time.

I specifically like the rich headers and full support for all HTTP methods. It's the best client for HTTP I've seen to date. Also, rather than just reading everything in as a string directly, you can read the HttpContent in various ways (or not at all), as well as inspect headers, etc. It just offers a lot more control. I would love to see an F# wrapper that makes using this much simpler. I had planned to expand some of my earlier attempts, specifically around pattern matching responses. Would that be of interest? Something like (made-up, record-like syntax, though Active Patterns are probably more reasonable):

match response with
| { Status = 200; Accept = "application/json" } -> // do something with JSON
| { Status = 401 } -> // go login
| { Status = 404 } -> // not found
| _ -> // not what you expected

I've talked with many of the HTTP guys on the Web API Advisors about this, and pattern matching is the area where you would see the most benefit with F#. My first forays into this can be found at WebApiKoans.

[ovatsus]

The problem is that to support .Net 4.0 and other platforms, you have to introduce a dependency on another NuGet package. This isn't to say an F# wrapper for it wouldn't be interesting, but for FSharp.Data I would prefer to extend FSharp.Net.Http with the extra functionality, but still keeping it based on WebRequest. Actually, the extensions I've made are similar to your example, I'll try to find time today or tomorrow to post them online.

[panesofglass]

You already have a dependency on System.Net. System.Net.Http is a BCL library that wasn't in the box in .NET 4.0. We're going to have a hard time keeping to in-the-box only dependencies going forward, as more and more MS libraries are released via NuGet.

I live in both the F# and Web API worlds. There's a lot of good stuff happening with Web API, especially in the form of message handlers for security, etc. I know mine is only one opinion, but I would only recommend use of HttpClient for any future work. Perhaps there's some way to leverage some of the other pieces coming out of System.Net.Http without using that assembly directly, but I cannot imagine how that would work at this time.

I think I understand you are moving forward with HttpWebRequest, so I'll go back to my idea to create a new library to use HttpClient. Thanks for the discussion.

[ovatsus]

Don't take my comments as a final decision, it was also just my opinion :), let's see what @tpetricek has to say about this

[panesofglass]

The other option where we could leverage reuse of common components is OWIN. I don't know of as many items available for that platform, but you should be able to adapt message handlers to the OWIN spec. Ultimately, HttpClient just wraps a common interface, and it uses an adapter around HttpWebRequest. So it is certainly possible that this library could do something similar.

[tpetricek]

I would not particularly mind having a dependency on another NuGet package as long as it is something standard (like a BCL library coming from Microsoft) if it makes our HTTP functionality better.

1. Does the HttpClient give us something that is impossible/hard to do with the current approach?
2. Are there some platforms where the new API would not work? This is probably the key limiting factor...

The multi-platform support is kind of painful. Perhaps we should not worry if the FSharp.Net.Http class provides only limited API on some platforms - but F# Data does not aim to be "F# Net", so perhaps this would better fit somewhere else....

I suppose this would not solve the problem with forward slashes in URL? (We currently use a horrible Reflection workaround from StackOverflow).

[ovatsus]

There's a new HttpClient nuget package from MS now (http://nuget.org/packages/Microsoft.Net.Http/2.1.10), which supports portable class libraries, so I thought about trying to use it instead of WebRequest directly, to see if it would fix some problems we have, but then I noticed that the license says it can't be used on non windows platforms. So although the dll itself probably works fine on Mono, you're not legally allowed to use it on a Mac or Linux...
See http://haacked.com/archive/2013/06/24/platform-limitations-harm-net.aspx

[ovatsus]

There have been some new developments on this. The Portable HTTP Client will now work on mono:
http://blogs.msdn.com/b/dotnet/archive/2013/10/14/portable-class-library-pcl-now-available-on-all-platforms.aspx

We should revisit this. WebClient has trouble with headers on Windows 8, and we should check if HttpClient has the slashes problem fixed

[ovatsus]

I had a look at this, and the portable HttpClient is a mess. It has a dependency on Microsoft.Bcl.Build, which has some powershell scripts that change your project files, and if we used it, every project that directly or indirectly referenced FSHarp.Data would have to add a reference to Microsoft.Bcl.Build explicitely.
Also, the handling of cookies seems to be broken for Windows Phone. We're better off keeping using WebRequest directly

[paulyoung]

@panesofglass I'm curious if you ever wrote or found an F# wrapper for HttpClient.

I'm still weighing things up but wondering if something compatible with ModernHTTPClient could be the way to go.

[paulyoung]

This looks promising: https://github.com/relentless/Http.fs

/cc @relentless