-
Notifications
You must be signed in to change notification settings - Fork 3.7k
Description
Describe the bug
While using hdfs broker to load hdfs files with username configuration, hdfs broker always uses system environment variable HADOOP_USER_NAME as hadoop user to read hdfs files. Otherwise if HADOOP_USER_NAME is not set, it will use the linux user launchs the hdfs broker service.
And the load script is like this.
LOAD LABEL db.xx (
DATA INFILE(
"xx"
) INTO TABLE xx
COLUMNS TERMINATED BY "\x01"
) WITH BROKER hdfs_broker (
"hadoop.security.authentication" = "simple",
"username" = "hadoop_user_name",
"password" = "hadoop_password",
"dfs.nameservices" = "nameservices",
"dfs.ha.namenodes.nameservices" = "nn1,nn2",
"dfs.namenode.rpc-address.nameservices.nn1" = "host1:port2",
"dfs.namenode.rpc-address.nameservices.nn2" = "host2:port2"
) PROPERTIES (
"exec_mem_limit" = "4294967296"
);