-
Notifications
You must be signed in to change notification settings - Fork 618
Closed
Labels
Milestone
Description
Background
v0.3.2 was a minor release scheduled to be released months ago, but now it's a complete rewrite mainly for two reasons:
-
decoupling(see Proposal of restructuring code (RFC) #570 for details)
- Java client is async and lightweight
- JDBC driver is built on top of Java client
-
switching data format to RowBinary to fix issues and improve performance
Benchmark results...0.3.2-test1...
clickhouse-grpc-jdbcandclickhouse-http-jdbcare new JDBC driver(0.3.2) usingRowBinarydata formatclickhouse-jdbcis the old JDBC driver(0.3.1-patch) based onTabSeparatedclickhouse-native-jdbcis ClickHouse-Native-JDBC 2.6.0
Benchmark settings: thread=1, sampleSize=100000, fetchSize=10000, mode=throughput(ops/s).

0.3.2-test3...
Unlike previous round of testing, ClickHouse container is re-created a few minutes before benchmarking each driver.
- Single thread
- Comparison

Note: HttpClient is async(uses more than one thread in runtime); gRPC uses gzip(why?) which is slower than lz4. - VM utilization

Note: on client side, the new driver consumes less memory and CPU than others, BUT higher CPU on server side(due to overhead of http protocol?).
- Comparison
- 4 threads
0.3.2-test1, 0.3.2-test2, and 0.3.2-test3 are pre-release for public testing.
Downloads
Maven dependency:
<dependency>
<!-- will stop using group id "ru.yandex.clickhouse" starting from 0.4.0 -->
<groupId>com.clickhouse</groupId>
<!-- or clickhouse-grpc-client to use gRPC client -->
<artifactId>clickhouse-http-client</artifactId>
<version>0.3.2-test3</version>
</dependency>To download JDBC drivers:
| Package | Size | Legacy | New | HTTP | gRPC | Remark |
|---|---|---|---|---|---|---|
| clickhouse-jdbc-0.3.2-all.jar | 18.6MB | Y | Y | Y | Y | Both old and new JDBC drivers(besides netty, okhttp is included as well) |
| clickhouse-jdbc-0.3.2-http.jar | 756KB | N | Y | Y | N | New JDBC driver with only http support |
| clickhouse-jdbc-0.3.2-grpc.jar | 17.3MB | N | Y | N | Y | New JDBC driver with only grpc support(only netty, okhttp is excluded) |
| clickhouse-jdbc-0.3.2-shaded.jar | 2.8MB | Y | Y | Y | N | Both old and new JDBC drivers |
Note: the first two are recommended. grpc is experimental so you'd better use http.
Known Issues
- new driver(com.clickhouse.jdbc.ClickHouseDriver) does not work with version before 21.3
java.io.IOException: HTTP/1.1 header parser received no byteswhen using JDK 11+ andhttp_connection_provideris set toHTTP_CLIENTRESOURCE_EXHAUSTED: Compressed gRPC message exceeds maximum size- increasemax_inbound_message_sizeto resolveselect 1 format JSONworks in http but not grpc, because grpc client is not aware of response formatinsert into table values(?, ?)is slow in batch mode - tryinsert into table select c2,c3 from input('c1 String, c2 UInt8, c3 Nullable(UInt32)')insteaduse_time_zoneanduse_server_time_zone_for_datesproperties do not work- no table/index show up under
jdbc(*)database - roaringbitmap is not included in the shaded jar
Key Changes
- Java client and JDBC driver are now in different modules, along with JPMS support
- Replaced data format from TabSeparated to RowBinary
- Support more data types including Date32, Geo types, and mixed use of nested types
- JDBC connection URL now supports abbrebation, protocol and optional port
jdbc:ch://localhostis same asjdbc:clickhouse:http://localhost:8123jdbc:ch:grpc://localhost/dbis same asjdbc:clickhouse:grpc://localhost:9100/db
- New JDBC driver class is
com.clickhouse.jdbc.ClickHouseDriver(will removeru.yandex.clickhouse.ClickHouseDriverstarting from 0.4.0) - JDBC connection properties are simplified
- use
custom_http_headersandcustom_http_paramsfor customization - won't work for grpc client jdbcCompliant(defaults to true) to support fake transaction and standard synchronous UPDATE and DELETE statementstypeMappingsto customize type mapping(e.g.DateTime=java.lang.String,DateTime32=java.lang.String)
- use
Some more details can be found at #736, #747, #769, and #777.
DavinQi and JackyWoo





