[Bug]Load Data From HDFS Report Checksum Error

[lanerbean]

Bug Type (问题类型)

exception / error (异常报错)

Before submit

• I had searched in the issues and found no similar issues.

Environment (环境信息)

• Server Version: v1.0.0
• Toolchain Version: v1.0.0
• Data Size: 7 vertices, 18 edges

Expected & Actual behavior (期望与实际表现)

When loading data from HDFS using HugeGraph-Loader, get an error

Vertex/Edge example (问题点 / 边数据举例)

No response

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

No response

[imbajin]

Seems same with #132, do u try to enable the checksum function in HDFS?

And could u provide more info about how to reproduce it? (in a clean/pure HDFS cluster, the error will not appear

[lanerbean]

Hello, the HDFS checksum function is enabled, but the problem still appears.

I build a 3-node Hadoop cluster. The following graph shows the cluster information.

I installed HugeGraph in one node of Hadoop cluster.

[simon824]

hi @lanerbean ,what is your hadoop version?

[simon824]

@lanerbean you can try manually downloading and replacing dependencies that match your cluster version, including hadoop-common, hadoop-hdfs-client.

[lanerbean]

hi @lanerbean ,what is your hadoop version?

My hadoop version is Hadoop 3.3.1.
I'll try to download these dependencies.
Thank you!

[simon824]

Ok, I can try to download these dependencies. Could you tell me where to download these jars and which folder to put these new jars?

@lanerbean You can search for the package you need in mvnrepository , download them and put them in the ./lib/

And remember to remove the old version dependencies.

[lanerbean]

@simon824 Yes, as you said, I replaced the "hadoop-common" and "hadoop-hdfs-client" dependencies in the "lib" folder of HugeGraph-Loader, it worked ok! Thank you very much!