HTTP GET Request is too big

[k3rnL]

Hello,

I got a problem with batched insert on a large table with long column names.
It seems that the generated HTTP header is too long and is rejected by clickhouse server, leading in a 1002 exception on the JDBC client. This is because the column names of the insert query are embedded in the request URL (which is embedded in a GET request, and so is limited in size, apparently 16000 bytes) and not in the subsequent POST request (that can be the case using TabSeparatedWithNames for example).

It results in a limitation of the number of columns supported by the driver.

There is a minimal reproducible example here : Main.java.gz

I used the version 0.2.6 of the JDBC server and I'm running ClickHouse server 21.3.3.14
There is the exception I got :

Exception in thread "main" ru.yandex.clickhouse.except.ClickHouseUnknownException: ClickHouse exception, code: 1002, host: 172.29.100.83, port: 8123; 
	at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.getException(ClickHouseExceptionSpecifier.java:91)
	at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:55)
	at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:28)
	at ru.yandex.clickhouse.ClickHouseStatementImpl.checkForErrorAndThrow(ClickHouseStatementImpl.java:1046)
	at ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:1016)
	at ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:985)
	at ru.yandex.clickhouse.ClickHouseStatementImpl.sendStream(ClickHouseStatementImpl.java:978)
	at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:372)
	at ru.yandex.clickhouse.ClickHousePreparedStatementImpl.executeBatch(ClickHousePreparedStatementImpl.java:349)
	at com.zaxxer.hikari.proxy.StatementProxy.executeBatch(StatementProxy.java:116)
	at com.zaxxer.hikari.proxy.PreparedStatementJavassistProxy.executeBatch(PreparedStatementJavassistProxy.java)
	at com.k3rnl.test.Main.main(Main.java:83)
Caused by: java.lang.Throwable: 
	at ru.yandex.clickhouse.except.ClickHouseExceptionSpecifier.specify(ClickHouseExceptionSpecifier.java:53)
	... 10 more

I think the method ClickHousePreparedStatementImpl.executeBatch(Map<ClickHouseQueryParam, String> additionalDBParams) must extract the column names and put it as first data line. And of course transform the original request by adding FORMAT TabSeparatedWithNames

[zhicwu]

Thanks for reporting the issue with detailed inputs. I was able to reproduce at my end using your code. According to ClickHouse/ClickHouse#15571, we can move query from URL to request body. I just tried and it seems working.

Apart from this issue, the error message is not very helpful. When error code/message is unavailable, we'd better return server response status instead(400 bad request in this case).

Will submit a PR to fix this later this week.

[zhicwu]

@k3rnL, PR #611 is a quick workaround to address the issue you ran into.