Skip to content

Add POST-backed query transport for long SQL and search requests #5

@cferrys

Description

@cferrys

Summary

The Perl client currently sends SQL and search queries through GET query parameters, which can fail for long or complex requests and exposes query text in URLs, logs, proxies, and browser/history tooling. Add POST-backed query transport with an automatic fallback/threshold policy for SQL, collection search, and SAM search helpers.

Context

Recent changes added SQL and SAM helpers in lib/Hlquery/Client.pm, including Sql, SqlSearch, Hlquery::Client::SQL::Query, and Hlquery::Client::SAM::Search. These methods route user-supplied SQL, search terms, filters, and options into query strings via ExecuteRequest.

For a high-performance search engine backed by RocksDB, realistic queries can grow quickly: nested filters, highlight fields, SQL projections, long WHERE clauses, user search text, pagination, and ranking options. Encoding these into URLs creates several problems:

  • Request failures from web server, proxy, or client URL-length limits.
  • Sensitive query content appearing in access logs and monitoring systems.
  • Poor ergonomics for advanced SQL/search payloads that naturally belong in JSON bodies.
  • Inconsistent behavior, since ExecSql already uses POST while read-oriented SQL uses GET.

Proposed Implementation

  1. Add a request option or client-level setting for query transport, such as query_transport => 'auto' | 'get' | 'post', defaulting to auto.
  2. Update Sql, SqlSearch, Search, and SAM::Search to use JSON POST bodies when payloads exceed a conservative encoded-size threshold or when explicitly configured.
  3. Preserve backwards compatibility by continuing to support GET for small requests where the server expects it.
  4. Normalize request payload construction so arrays, booleans, and scalar values behave consistently across GET query params and POST JSON bodies.
  5. Add unit tests around generated method/path/body/query behavior using a mocked LWP::UserAgent, especially for long SQL strings and long search queries.
  6. Update README examples to show POST-safe advanced SQL/search usage and document the transport setting.

Impact

This makes the Perl client reliable for production-scale query workloads, reduces accidental leakage of query text, and gives users a safer path for complex search and SQL features without hand-writing raw HTTP calls. It also aligns the client with hlquery’s role as a high-performance search/database wrapper where large structured queries should be expected rather than treated as an edge case.

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions