feat: support dynamic context for lance namespace

[jackye1995]

This feature is similar to lancedb HeaderProvider, but implemented in a more generic way.

In lance-namespace 0.4.5, we introduced per-request context which is a free-form map that can be passed in, and processed by different implementations differently. Based on that, we add a DynamicContextProvivder.

Then specifically for RestNamespace, we define that any context starting with headers. will be translated to a request level header.

Users can define an implementation of the DynamicContextProvivder in python and java, and load it with for example:

Python:

from lance.namespace import DynamicContextProvider

class MyAuthProvider(DynamicContextProvider):
    def __init__(self, token: str, refresh_url: str):
        self.token = token
        self.refresh_url = refresh_url

    def provide_context(self, info: dict) -> dict:
        # info = {"operation": "list_tables", "object_id": "workspace"}
        return {
            "headers.Authorization": f"Bearer {self.token}",
            "headers.X-Request-Id": str(uuid.uuid4()),
        }

from lance_namespace import connect, CreateNamespaceRequest, ListTablesRequest

ns = connect("rest", {
    "uri": "http://localhost:8080",
    "dynamic_context_provider.impl": "my_module.auth.MyAuthProvider",
    "dynamic_context_provider.token": "secret-token",
    "dynamic_context_provider.refresh_url": "https://auth.example.com/refresh",
})

# Use the namespace
ns.create_namespace(CreateNamespaceRequest(id=["workspace"]))
tables = ns.list_tables(ListTablesRequest(id=["workspace"]))

Java:

package com.example.auth;

import org.lance.namespace.DynamicContextProvider;
import java.util.*;

public class MyAuthProvider implements DynamicContextProvider {
    private final String token;
    private final String refreshUrl;

    // Constructor MUST accept Map<String, String>
    public MyAuthProvider(Map<String, String> properties) {
        this.token = properties.get("token");
        this.refreshUrl = properties.get("refresh_url");
    }

    @Override
    public Map<String, String> provideContext(String operation, String objectId) {
        // operation = "list_tables", objectId = "workspace"
        Map<String, String> context = new HashMap<>();
        context.put("headers.Authorization", "Bearer " + token);
        context.put("headers.X-Request-Id", UUID.randomUUID().toString());
        return context;
    }
}

package com.example.app;

import org.lance.namespace.LanceNamespace;
import org.lance.namespace.model.*;
import org.apache.arrow.memory.RootAllocator;
import java.util.*;

Map<String, String> properties = new HashMap<>();
properties.put("uri", "http://localhost:8080");
properties.put("dynamic_context_provider.impl", "com.example.auth.MyAuthProvider");
properties.put("dynamic_context_provider.token", "secret-token");
properties.put("dynamic_context_provider.refresh_url", "https://auth.example.com/refresh");

try (BufferAllocator allocator = new RootAllocator();
     LanceNamespace ns = LanceNamespace.connect("rest", properties, allocator)) {

    // Use the namespace
    ns.createNamespace(new CreateNamespaceRequest().id(Arrays.asList("workspace")));
    ListTablesResponse tables = ns.listTables(new ListTablesRequest().id(Arrays.asList("workspace")));
}

Because of the requirement to dynamically inject per-request headers, we also moved to use raw reqwest client now for all HTTP requests in RestNamespace.

[github-actions]

PR Review: Dynamic Context Provider for Lance Namespace

Summary

This PR adds a DynamicContextProvider trait that enables per-request context injection (e.g., dynamic authentication headers) for namespace operations. The implementation spans Rust core, Python bindings, and Java bindings.

P0/P1 Issues

P1 - Performance: Rebuilding reqwest client on every request (rust/lance-namespace-impls/src/rest.rs)

The config_with_context() method creates a new reqwest::Client on every single request that has context headers:

fn config_with_context(&self, operation: &str, object_id: &str) -> Configuration {
    // ...
    let client = reqwest::Client::builder()
        .default_headers(default_headers)
        .build()
        .unwrap_or_else(|_| reqwest::Client::new());
    // ...
}

Creating a new reqwest::Client is expensive as it initializes connection pools, TLS configurations, etc. This should instead reuse the existing client and add headers at the request level. Consider:

• Using the client's request() builder to add headers per-request instead of creating new clients
• Or caching clients based on header sets if truly needed

P1 - Missing TLS/SSL configuration when rebuilding client (rust/lance-namespace-impls/src/rest.rs)

When creating the new client in config_with_context(), all TLS configuration from the builder (cert_file, key_file, ssl_ca_cert, assert_hostname) is lost:

let client = reqwest::Client::builder()
    .default_headers(default_headers)
    .build()  // No TLS configuration\!

This means any REST namespace configured with client certificates or custom CA will silently lose those settings when a context provider is used.

---

Other Observations (informational, not blocking)

• Test coverage is comprehensive with tests for both Java and Python bindings
• Thread safety requirements are documented (Send + Sync)
• The headers. prefix convention for extracting HTTP headers is clear and well-documented

[codecov]

Codecov Report

❌ Patch coverage is 72.02925% with 153 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
rust/lance-namespace-impls/src/rest.rs	70.43%	103 Missing and 7 partials ⚠️
rust/lance-namespace-impls/src/connect.rs	20.00%	22 Missing and 2 partials ⚠️
rust/lance-namespace-impls/src/dir.rs	13.63%	19 Missing ⚠️

📢 Thoughts on this report? Let us know!