Skip to content

[HELP] 0.3.2 pre-release for public testing #768

@zhicwu

Description

@zhicwu

Background

v0.3.2 was a minor release scheduled to be released months ago, but now it's a complete rewrite mainly for two reasons:

  1. decoupling(see Proposal of restructuring code (RFC) #570 for details)

  2. switching data format to RowBinary to fix issues and improve performance
    Benchmark results...

    0.3.2-test1...
    • clickhouse-grpc-jdbc and clickhouse-http-jdbc are new JDBC driver(0.3.2) using RowBinary data format
    • clickhouse-jdbc is the old JDBC driver(0.3.1-patch) based on TabSeparated
    • clickhouse-native-jdbc is ClickHouse-Native-JDBC 2.6.0
      Benchmark settings: thread=1, sampleSize=100000, fetchSize=10000, mode=throughput(ops/s).
      image
    0.3.2-test3...

    Unlike previous round of testing, ClickHouse container is re-created a few minutes before benchmarking each driver.

    • Single thread
      • Comparison
        image
        Note: HttpClient is async(uses more than one thread in runtime); gRPC uses gzip(why?) which is slower than lz4.
      • VM utilization
        image
        Note: on client side, the new driver consumes less memory and CPU than others, BUT higher CPU on server side(due to overhead of http protocol?).
    • 4 threads
      • Comparison
        image

      • VM utilization
        image

    0.3.2...

    image

    Query performance is similar as shown in 0.3.2-test3 so this time we only focus on insertion.
    image
    Note: gRPC does not support LZ4 compression so we use GZIP in the test.

    • Single thread
      image
    • 4 threads
      image

0.3.2-test1, 0.3.2-test2, and 0.3.2-test3 are pre-release for public testing.

Downloads

Maven dependency:

<dependency>
    <!-- will stop using group id "ru.yandex.clickhouse" starting from 0.4.0  -->
    <groupId>com.clickhouse</groupId>
    <!-- or clickhouse-grpc-client to use gRPC client  -->
    <artifactId>clickhouse-http-client</artifactId>
    <version>0.3.2-test3</version>
</dependency>

To download JDBC drivers:

Package Size Legacy New HTTP gRPC Remark
clickhouse-jdbc-0.3.2-all.jar 18.6MB Y Y Y Y Both old and new JDBC drivers(besides netty, okhttp is included as well)
clickhouse-jdbc-0.3.2-http.jar 756KB N Y Y N New JDBC driver with only http support
clickhouse-jdbc-0.3.2-grpc.jar 17.3MB N Y N Y New JDBC driver with only grpc support(only netty, okhttp is excluded)
clickhouse-jdbc-0.3.2-shaded.jar 2.8MB Y Y Y N Both old and new JDBC drivers

Note: the first two are recommended. grpc is experimental so you'd better use http.

Known Issues

  • new driver(com.clickhouse.jdbc.ClickHouseDriver) does not work with version before 21.3
  • java.io.IOException: HTTP/1.1 header parser received no bytes when using JDK 11+ and http_connection_provider is set to HTTP_CLIENT
  • RESOURCE_EXHAUSTED: Compressed gRPC message exceeds maximum size - increase max_inbound_message_size to resolve
  • select 1 format JSON works in http but not grpc, because grpc client is not aware of response format
  • insert into table values(?, ?) is slow in batch mode - try insert into table select c2,c3 from input('c1 String, c2 UInt8, c3 Nullable(UInt32)') instead
  • use_time_zone and use_server_time_zone_for_dates properties do not work
  • no table/index show up under jdbc(*) database
  • roaringbitmap is not included in the shaded jar

Key Changes

  • Java client and JDBC driver are now in different modules, along with JPMS support
  • Replaced data format from TabSeparated to RowBinary
  • Support more data types including Date32, Geo types, and mixed use of nested types
  • JDBC connection URL now supports abbrebation, protocol and optional port
    • jdbc:ch://localhost is same as jdbc:clickhouse:http://localhost:8123
    • jdbc:ch:grpc://localhost/db is same as jdbc:clickhouse:grpc://localhost:9100/db
  • New JDBC driver class is com.clickhouse.jdbc.ClickHouseDriver(will remove ru.yandex.clickhouse.ClickHouseDriver starting from 0.4.0)
  • JDBC connection properties are simplified
    • use custom_http_headers and custom_http_params for customization - won't work for grpc client
    • jdbcCompliant(defaults to true) to support fake transaction and standard synchronous UPDATE and DELETE statements
    • typeMappings to customize type mapping(e.g. DateTime=java.lang.String,DateTime32=java.lang.String)

Some more details can be found at #736, #747, #769, and #777.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions