Parse port numbers from MCP server URLs in CORS proxy by eapache · Pull Request #20208 · ggml-org/llama.cpp

eapache · 2026-03-07T19:18:08Z

Fixes #20205.

When specifying an MCP server on a custom port (e.g. http://127.0.0.1:8123/mcp), the CORS proxy was not parsing the port number, leading to a malformed host (with the trailing port still attached) and falling back to port 80/443.

This is my first PR here and it's been a while since I touched C/C++ so likely I've done something wrong with the formatting or data types or something 😅

The only AI used in making this PR was Copilot's autocomplete, which I trusted to pick the right stdlib methods for stoi and throwing a runtime error as my C++ is so rusty.

ngxson

Technically this won't support something like https://myserver.com:8443

eapache · 2026-03-07T23:08:21Z

Why not? Because it doesn't recognize 8443 as a tls port?

ngxson · 2026-03-07T23:40:01Z

There is a check port == 443

llama.cpp/tools/server/server-models.cpp

Lines 1080 to 1101 in c5a7788

    
           server_http_proxy::server_http_proxy( 
        
                   const std::string & method, 
        
                   const std::string & host, 
        
                   int port, 
        
                   const std::string & path, 
        
                   const std::map<std::string, std::string> & headers, 
        
                   const std::string & body, 
        
                   const std::function<bool()> should_stop, 
        
                   int32_t timeout_read, 
        
                   int32_t timeout_write 
        
                   ) { 
        
               // shared between reader and writer threads 
        
               auto cli  = std::make_shared<httplib::ClientImpl>(host, port); 
        
               auto pipe = std::make_shared<pipe_t<msg_t>>(); 
        
               if (port == 443) { 
        
           #ifdef CPPHTTPLIB_OPENSSL_SUPPORT 
        
                   cli.reset(new httplib::SSLClient(host, port)); 
        
           #else 
        
                   throw std::runtime_error("HTTPS requested but CPPHTTPLIB_OPENSSL_SUPPORT is not defined"); 
        
           #endif 
        
               }

eapache · 2026-03-08T00:42:20Z

Ah ok I just need to pass the scheme in explicitly there. I will refresh the patch tomorrow morning when I'm back at my desk.

eapache · 2026-03-08T13:21:31Z

    }
    auto proxy = std::make_unique<server_http_proxy>(
            method,
+            "http",


While adding scheme to be passed around we also now need a scheme here (for the model router). I don't see an existing scheme here, but AFAICT the child processes always just use http so I can hard-code it (model routing still works in a trivial local test with it hard-coded like this). Wanted to call this out in case there's a flag I'm not familiar with for sending the communications to the child inference processes via https or something.

eapache · 2026-03-08T13:22:08Z

    }

-    SRV_INF("proxying %s request to %s://%s%s\n", method.c_str(), parsed_url.scheme.c_str(), parsed_url.host.c_str(), parsed_url.path.c_str());
+    SRV_INF("proxying %s request to %s://%s:%i%s\n", method.c_str(), parsed_url.scheme.c_str(), parsed_url.host.c_str(), parsed_url.port, parsed_url.path.c_str());


I don't love that we now always log the port explicitly, but parsed_url no longer indicates whether the port was explicit or implicit.

eapache · 2026-03-08T13:23:07Z

 #endif

-    httplib::Client cli(parts.scheme + "://" + parts.host);
+    httplib::Client cli(parts.scheme + "://" + parts.host + ":" + std::to_string(parts.port));


I ran it through ChatGPT Codex and it pointed out that I'd broken http downloads on non-standard ports because that had been relying on the port being smushed into the host still. Not quite sure how to test this, but it seems right.

eapache · 2026-03-08T13:26:18Z

        parts.path = "/";
    }
+
+    auto colon_pos = parts.host.find(':');


How do we feel about IPv6 addresses? They can have colons in their literals which will break here. But parsing those properly feels way outside the scope of a little internal helper method, and starts to feel like we should be pulling in a library which can do proper URL parsing...

eapache · 2026-03-08T14:16:53Z

@ngxson I went down a bit of a rabbit-hole trying to support IPv6 literals and it turned into a substantial refactor which I didn't manage to get to the bottom of before giving up. In particular, it adds more logic anywhere we convert from parts to full-URL or back (because a host of ::1 needs to be printed and parsed as [::1] in the context of a URL).

I haven't tried but I'm pretty sure those are already broken for MCP in master, because we're not stripping the [ ] when passing the host to server_http_proxy, so I'd rather not consider that blocking this fix. And as I mentioned in my previous comment, this level of complexity is getting to the point where I'd consider just using a library to do the URL manipulation, it's not as bad as email addresses but there are still a lot of edge cases.

Putting all that aside, I've addressed the tls-inferred-from-port issue you pointed out, and I think otherwise this PR is a reasonable improvement over the current state of master.

ngxson

I think it's fine as a quick fix, we can refactor the parser to support ipv6 in a dedicated PR, but I think not many people will actually use it

…rg#20208) * Parse port numbers from MCP server URLs * Pass scheme to http proxy for determining whether to use SSL * Fix download on non-standard port and re-add port to logging * add test --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

kurnevsky · 2026-03-21T18:06:54Z

There is one problem still left - server sets Host header with host only, but it should include port as well. There are some strict mcp servers that reject requests that have invalid Host header. Created MR: #20843

…rg#20208) * Parse port numbers from MCP server URLs * Pass scheme to http proxy for determining whether to use SSL * Fix download on non-standard port and re-add port to logging * add test --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>

eapache requested review from angt, ggerganov and ngxson as code owners March 7, 2026 19:18

github-actions Bot added examples server labels Mar 7, 2026

ngxson reviewed Mar 7, 2026

View reviewed changes

moonlightelite mentioned this pull request Mar 7, 2026

webui: Agentic Loop + MCP Client with support for Tools, Resources and Prompts #18655

Merged

eapache added 2 commits March 8, 2026 08:40

Parse port numbers from MCP server URLs

ed3ed90

Pass scheme to http proxy for determining whether to use SSL

a9daf8a

eapache force-pushed the parse-port-from-mcp-urls branch from a9f3981 to a9daf8a Compare March 8, 2026 13:03

Fix download on non-standard port and re-add port to logging

4584f14

eapache commented Mar 8, 2026

View reviewed changes

eapache mentioned this pull request Mar 8, 2026

Eval bug: autoparser breaks tool calls from LFM2.5-Instruct #20245

Closed

ngxson approved these changes Mar 9, 2026

View reviewed changes

add test

192c1a5

github-actions Bot added the python python script changes label Mar 9, 2026

ngxson mentioned this pull request Mar 9, 2026

[Mirror] Parse port numbers from MCP server URLs in CORS proxy ngxson/llama.cpp#88

Closed

ngxson merged commit 23fbfcb into ggml-org:master Mar 9, 2026
75 of 78 checks passed

eapache deleted the parse-port-from-mcp-urls branch March 11, 2026 02:29

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Parse port numbers from MCP server URLs in CORS proxy#20208

Parse port numbers from MCP server URLs in CORS proxy#20208
ngxson merged 4 commits intoggml-org:masterfrom
eapache:parse-port-from-mcp-urls

eapache commented Mar 7, 2026 •

edited

Loading

Uh oh!

ngxson left a comment

Uh oh!

eapache commented Mar 7, 2026

Uh oh!

ngxson commented Mar 7, 2026 •

edited

Loading

Uh oh!

eapache commented Mar 8, 2026

Uh oh!

eapache Mar 8, 2026

Uh oh!

eapache Mar 8, 2026

Uh oh!

eapache Mar 8, 2026

Uh oh!

eapache Mar 8, 2026

Uh oh!

eapache commented Mar 8, 2026 •

edited

Loading

Uh oh!

ngxson left a comment

Uh oh!

Uh oh!

kurnevsky commented Mar 21, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

eapache commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngxson left a comment

Choose a reason for hiding this comment

Uh oh!

eapache commented Mar 7, 2026

Uh oh!

ngxson commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

eapache commented Mar 8, 2026

Uh oh!

eapache Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

eapache Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

eapache Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

eapache Mar 8, 2026

Choose a reason for hiding this comment

Uh oh!

eapache commented Mar 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ngxson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kurnevsky commented Mar 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

eapache commented Mar 7, 2026 •

edited

Loading

ngxson commented Mar 7, 2026 •

edited

Loading

eapache commented Mar 8, 2026 •

edited

Loading

kurnevsky commented Mar 21, 2026 •

edited

Loading