BinaryImport

[SoftStoneDevelop]

Implementation of sending data packets manual.

Usage:

var import = await client.StartInsertAsync(tableName, columns);

using var batch1 = import.StartNewBatch();
// fill batch 1
batch1.WriteData(0, 1);
batch1.WriteData(1, "my product value");
batch1.WriteData(2, "my category value");

batch1.CompleteWrite();

var sendTask = import.SendBatchAsync(batch1);

using var batch2 = import.StartNewBatch();
// fill batch 2
batch2.WriteData(0, 2);
batch2.WriteData(1, "my product value");
batch2.WriteData(2, "my category value");

batch2.CompleteWrite();

var sendTask2 = import.SendBatchAsync(batch2);

await Task.WhenAll(sendTask, sendTask2);

Thus, creating packages is the prerogative of the library user, not the library itself. How they choose to do so is entirely up to them. PR for example, to show what I mean.

No attributes needed, no object[] array needed. Where the developer gets the data to write to Write is his problem and solution.

Based on the benchmark results, 'BinaryImport' is faster and consumes less memory for any collection size.

Benchmark result:

Method	Count	Mean	Error	StdDev	Median	Ratio	RatioSD	Gen0	Gen1	Gen2	Allocated	Alloc Ratio
ObjectArray	100000	113.19 ms	4.358 ms	9.287 ms	114.23 ms	1.01	0.11	1000.0000	-	-	14.53 MB	1.00
Poco	100000	112.13 ms	5.806 ms	12.745 ms	111.68 ms	1.00	0.14	1000.0000	-	-	13.76 MB	0.95
BinaryImport	100000	94.73 ms	4.412 ms	9.776 ms	89.33 ms	0.84	0.11	-	-	-	4.87 MB	0.34
ObjectArray	500000	679.48 ms	13.408 ms	29.988 ms	677.99 ms	1.00	0.06	9000.0000	8000.0000	1000.0000	70.54 MB	1.00
Poco	500000	655.85 ms	13.758 ms	30.198 ms	657.03 ms	0.97	0.06	9000.0000	7000.0000	2000.0000	66.73 MB	0.95
BinaryImport	500000	543.13 ms	12.932 ms	28.386 ms	549.10 ms	0.80	0.05	2000.0000	-	-	21.21 MB	0.30

#328

---

Note

Medium Risk
Adds a new streaming/batched insert path and refactors ClickHouse binary type serialization to use generic Write<T>/CanWrite<T>, which could affect all binary inserts if any type coercion or null-handling behavior changes.

Overview
Adds a new ClickHouseClient.StartInsertAsync API that returns a BinaryImport helper, letting callers manually build and send multiple gzipped RowBinary batches via BatchData.WriteData/SendBatchAsync with per-batch query IDs.

Refactors the binary type system to use generic Write<T> (and CanWrite<T> where applicable) across many ClickHouseType implementations, with minor coercion tweaks (e.g., numeric/UUID/IP parsing) and corresponding test updates for null cases. Adds a benchmark comparing BinaryImport to existing object[] and POCO binary insert paths, plus a new test covering inserts with a subset of table columns.

^{Reviewed by Cursor Bugbot for commit b459199. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

This PR adds a new, packet-oriented binary insert API to ClickHouseClient, allowing callers to prepare an insert plan once, manually build multiple gzip-compressed RowBinary batches, and send them asynchronously with unique query IDs.

Changes:

• Added StartInsertAsync(...) to prepare insert metadata and return a BinaryImport helper.
• Introduced ClickHouseClient.BinaryImport with StartNewBatch() and SendBatchAsync(...) for user-managed batch creation/sending.
• Broadened InsertPlan visibility to internal to support the new helper type.

[Copilot]

Pull request overview

Copilot reviewed 1 out of 1 changed files in this pull request and generated 7 comments.

[Copilot]

Pull request overview

Copilot reviewed 50 out of 50 changed files in this pull request and generated 6 comments.

[SoftStoneDevelop]

Of course, there's still some work to be done (InsertOptions not for BinaryImport completly) to ensure it's ready for merging into 'main,' but I think it's clear what I mean and what benefits the user will receive.

If you're generally on board, I can refine the pull request.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{Reviewed by Cursor Bugbot for commit d458d03. Configure here.}