java.lang.IllegalStateException: Batch statement cannot contain more than 65535 statements.

[baibingzhi]

复现步骤

1. 创建属性，顶点，创建边，都有索引。
2. 导入数据，自己做的数据，约200万顶点，100万边
3. 删除schema，删除的顺序是 边索引->边->顶点索引->顶点->属性

现象：
数据量小时步骤都正常。后来导了一批数据，大约200万顶点100万边，在删除第一个边的索引时报错，请问如何解决，谢谢！

具体报错

2019-03-08 10:32:29 666937320 [task-worker-3] [WARN ] com.baidu.hugegraph.task.HugeTask [] - An exception occurred when running task: 331
java.lang.IllegalStateException: Batch statement cannot contain more than 65535 statements.
at com.datastax.driver.core.BatchStatement.add(BatchStatement.java:154) ~[cassandra-driver-core-3.2.0.jar:?]
at com.baidu.hugegraph.backend.store.cassandra.CassandraSessionPool$Session.add(CassandraSessionPool.java:156) ~[hugegraph-cassandra-0.8.0.jar:?]
at com.baidu.hugegraph.backend.store.cassandra.CassandraTables$SecondaryIndex.delete(CassandraTables.java:566) ~[hugegraph-cassandra-0.8.0.jar:?]
at com.baidu.hugegraph.backend.store.cassandra.CassandraStore.mutate(CassandraStore.java:201) ~[hugegraph-cassandra-0.8.0.jar:?]
at com.baidu.hugegraph.backend.store.cassandra.CassandraStore.mutate(CassandraStore.java:174) ~[hugegraph-cassandra-0.8.0.jar:?]
at com.baidu.hugegraph.backend.tx.AbstractTransaction.commitMutation2Backend(AbstractTransaction.java:243) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.backend.cache.CachedGraphTransaction.commitMutation2Backend(CachedGraphTransaction.java:135) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.backend.tx.IndexableTransaction.commit2Backend(IndexableTransaction.java:57) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.backend.tx.AbstractTransaction.commit(AbstractTransaction.java:166) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.backend.tx.GraphTransaction.commit(GraphTransaction.java:289) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.HugeGraph$TinkerpopTransaction.doCommit(HugeGraph.java:650) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:105) ~[gremlin-core-3.2.5.jar:3.2.5]
at com.baidu.hugegraph.HugeGraph$TinkerpopTransaction.commit(HugeGraph.java:612) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.job.schema.IndexLabelRemoveCallable.removeIndexLabel(IndexLabelRemoveCallable.java:64) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.job.schema.IndexLabelRemoveCallable.execute(IndexLabelRemoveCallable.java:39) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at com.baidu.hugegraph.job.Job.call(Job.java:36) ~[hugegraph-core-0.8.0.jar:0.8.0.0]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at com.baidu.hugegraph.task.HugeTask.run(HugeTask.java:203) [hugegraph-core-0.8.0.jar:0.8.0.0]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [?:1.8.0_181]
at java.util.concurrent.FutureTask.run(FutureTask.java:266) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_181]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_181]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_181]

[zhoney]

@baibingzhi 感谢反馈！

这是由于删除IndexLabel时会同时删除对应的所有索引信息，Cassandra后端删除时超过了65535的限制（Cassandra client本身的限制）。目前已经改进为分批处理，参见 #386 #387

0.8版本的删除schema导致的删除顶点或者边，实现效率相对比较低，在大数据量的情况下，所需时间较长。该问题在0.9版本会优化。

对于Cassandra这种标记删除的后端，不建议进行大数据量的反复删除，墓碑会降低后续查询的效率，请慎重使用大规模删除。实在需要的话，也要合理配置Cassandra的compaction，清理墓碑。