AsyncHTTPClient support for linux by JonasProgrammer · Pull Request #143 · huggingface/AnyLanguageModel

JonasProgrammer · 2026-03-05T14:13:39Z

As discussed in #127, the current workaround is not really feasible for async applications.

This PR includes an AsyncHTTPClient trait that does the following:

conditionally depend on AsyncHTTPClient
conditionally include the AsyncHTTPClient trait for EventSource
use typealiases to distinguish the session types

When untraited, the library should work as-is. With the trait active, the interface is changed to consume an HTTPClient as the session type.
I tried to change the business logic as little as possible, mostly implementing URLSession's extensions for HttpClient.

KotlinFactory · 2026-03-10T11:30:30Z

Any updates on this perhaps?

JonasProgrammer · 2026-03-10T11:35:40Z

Btw, as it stands now, the increased deadline fixes some of the timeouts, but not all of them. The existing HttpClient.shared instance, which is the default session with the enabled trait, has a read time out of only 90s and cannot be reconfigured.

Internally, we use modified timeouts along with another shared singleton session, but this is not ideal (AsyncHTTPClient does not expose the means of creating a proper singleton that cannot be shut down as they do with their own .shared singleton), thus not included in the PR.

If you have any suggestions regarding default timeouts etc., I'd be happy to improve the PR in that direction.

Copilot

Pull request overview

Adds an optional AsyncHTTPClient-based transport path (via SwiftPM traits) to avoid Linux URLSession/FoundationNetworking concurrency issues, while keeping the existing URLSession-based behavior when the trait is not enabled.

Changes:

Introduces SessionType + makeDefaultSession() to abstract over URLSession vs AsyncHTTPClient.HTTPClient.
Updates the network session types in model implementations to use SessionType.
Adds HTTPClient convenience extensions for JSON fetch + streaming/SSE, and wires conditional dependencies/traits in Package.swift.

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
Sources/AnyLanguageModel/Transport.swift	Adds conditional `SessionType` + default session factory for URLSession vs AsyncHTTPClient.
Sources/AnyLanguageModel/Models/OpenResponsesLanguageModel.swift	Switches stored session type and init default to `SessionType`.
Sources/AnyLanguageModel/Models/OpenAILanguageModel.swift	Switches stored session type and init default to `SessionType`.
Sources/AnyLanguageModel/Models/OllamaLanguageModel.swift	Switches stored session type and init default to `SessionType`.
Sources/AnyLanguageModel/Models/GeminiLanguageModel.swift	Switches stored session type and init defaults to `SessionType` (both initializers).
Sources/AnyLanguageModel/Models/AnthropicLanguageModel.swift	Switches stored session type and init default to `SessionType`.
Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift	Adds AsyncHTTPClient-backed `fetch`, newline-stream decoding, and SSE decoding helpers.
Package.swift	Adds AsyncHTTPClient trait + conditional dependencies, and forwards the trait to EventSource.
Package.resolved	Locks AsyncHTTPClient + transitive NIO dependencies and bumps EventSource resolution.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-03-23T11:21:00Z

Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift

+                    let asyncBytes = AsyncStream<UInt8> { byteContinuation in
+                        SwiftTask {
+                            do {
+                                for try await buffer in response.body {
+                                    for byte in buffer.readableBytesView {


In fetchEventStream, the AsyncStream<UInt8> is produced by launching an inner Task (the one iterating response.body), but that task is not stored or cancelled when the outer AsyncThrowingStream terminates. This can leave a request/body-consumption task running after the consumer cancels, keeping the connection and resources alive. Consider keeping a reference to that inner task and cancelling it (and finishing the byte stream) in an onTermination handler (or otherwise wiring cancellation through so cancelling the stream reliably stops reading the body).

Copilot · 2026-03-23T11:21:00Z

Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift

+#if canImport(FoundationNetworking)
+import FoundationNetworking
+#endif
+import JSONSchema


FoundationNetworking and JSONSchema appear to be unused in this file, which will trigger unused-import warnings (and may fail CI if warnings are treated as errors). Please remove unused imports (or add uses if they’re intentional).

Suggested change

#if canImport(FoundationNetworking)

import FoundationNetworking

#endif

import JSONSchema

Copilot · 2026-03-23T11:21:00Z

Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift

+enum HTTPClientError: Error, CustomStringConvertible {
+    case invalidResponse
+    case httpError(statusCode: Int, detail: String)
+    case decodingError(detail: String)
+
+    var description: String {
+        switch self {
+        case .invalidResponse:
+            return "Invalid response"


New HTTPClientError mirrors URLSessionError, but there are no tests covering its description formatting or basic error cases under the AsyncHTTPClient trait. Since the repo already tests URLSessionError descriptions, it would be good to add equivalent tests guarded by #if canImport(AsyncHTTPClient) to prevent regressions when the trait is enabled.

Copilot · 2026-03-23T11:21:01Z

Sources/AnyLanguageModel/Models/OpenAILanguageModel.swift

+    private let urlSession: SessionType

    /// Creates an OpenAI language model.
    ///


With the introduction of SessionType (URLSession vs AsyncHTTPClient.HTTPClient), the private property name urlSession is now misleading when the AsyncHTTPClient trait is enabled, and it makes the initializer docs (“URL session”) inaccurate. Consider renaming the stored property to something neutral like session (and updating the parameter docs accordingly) to avoid confusion.

mattt · 2026-03-23T11:51:44Z

Thanks so much for your work on this, @JonasProgrammer @KotlinFactory

I have some follow-up changes, but it doesn't look like I can push directly to this branch, so I'll make that post-merge.

mattt · 2026-03-23T11:35:21Z

Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift

@@ -0,0 +1,213 @@
+#if canImport(AsyncHTTPClient)
+// AsyncHTTPClient.HTTPHandler introduces a Task type that clashes
+typealias SwiftTask = Task


It's really too bad that Task is shadowed by AsyncHTTPClient and requires an internal typealias. Worse still that the alternative is referring to its fully-qualified _Concurrency.Task.

mattt · 2026-03-23T11:36:43Z

Sources/AnyLanguageModel/Extensions/HTTPClient+Extensions.swift

+    }
+}
+
+enum HTTPClientError: Error, CustomStringConvertible {


This should probably conform to LocalizedError. I also generally like to make thrown error types public, unless this is wrapped elsewhere.

mattt · 2026-03-23T11:40:07Z

Sources/AnyLanguageModel/Models/AnthropicLanguageModel.swift

    public let model: String

-    private let urlSession: URLSession
+    private let urlSession: SessionType


bikeshed: I think this should be spelled HTTPSession

… race (#134) * Serialize Linux URLSession request paths to mitigate _MultiHandle race * Incorporate feedback from review * Replace withLock instance method with top-level withLinuxRequestLock helper * Fix Linux compiler bug around generic returning lock helper * More workarounds for Linux compiler bugs * Incorporate feedback from review * AsyncHTTPClient support for linux (#143) * deps: add trait-based import of AsyncHTTPClient * feat: implement transparent AsyncHTTPClient wrapper * deps: conditionally include trait in EventSource * Increase HTTP request timeout from 60 to 180 seconds --------- Co-authored-by: Leonhard Solbach <49833472+KotlinFactory@users.noreply.github.com> * Incorporate feedback from review * swift format -i -r . --------- Co-authored-by: Jonas Stoehr <jonass@dev.jsje.de> Co-authored-by: Leonhard Solbach <49833472+KotlinFactory@users.noreply.github.com>

JonasProgrammer and others added 4 commits February 24, 2026 14:41

deps: add trait-based import of AsyncHTTPClient

f9df98b

feat: implement transparent AsyncHTTPClient wrapper

e033248

deps: conditionally include trait in EventSource

02c0ba9

Increase HTTP request timeout from 60 to 180 seconds

431be86

KotlinFactory mentioned this pull request Mar 15, 2026

Use-after-free crash on Linux: URLSession._MultiHandle thread-safety bug #127

Closed

JonasProgrammer changed the title ~~AysyncHTTPClient support for linux~~ AsyncHTTPClient support for linux Mar 16, 2026

mattt requested review from Copilot and mattt March 23, 2026 11:16

Copilot started reviewing on behalf of mattt March 23, 2026 11:16 View session

Copilot AI reviewed Mar 23, 2026

View reviewed changes

mattt merged commit 056dccd into huggingface:mattt/gate-linux-urlsession Mar 23, 2026
4 checks passed

mattt approved these changes Mar 23, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

AsyncHTTPClient support for linux#143

AsyncHTTPClient support for linux#143
mattt merged 4 commits intohuggingface:mattt/gate-linux-urlsessionfrom
SolbachLeads:bugfix/linux-url-session-2

JonasProgrammer commented Mar 5, 2026 •

edited

Loading

Uh oh!

KotlinFactory commented Mar 10, 2026

Uh oh!

JonasProgrammer commented Mar 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

Copilot AI Mar 23, 2026

Uh oh!

mattt commented Mar 23, 2026

Uh oh!

Uh oh!

mattt Mar 23, 2026

Uh oh!

mattt Mar 23, 2026

Uh oh!

mattt Mar 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

JonasProgrammer commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KotlinFactory commented Mar 10, 2026

Uh oh!

JonasProgrammer commented Mar 10, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

mattt commented Mar 23, 2026

Uh oh!

Uh oh!

mattt Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

mattt Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

mattt Mar 23, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

JonasProgrammer commented Mar 5, 2026 •

edited

Loading