Skip to content

HDFS BROKER LOAD username configuration is not effective #3234

@licheeCode

Description

@licheeCode

Describe the bug
While using hdfs broker to load hdfs files with username configuration, hdfs broker always uses system environment variable HADOOP_USER_NAME as hadoop user to read hdfs files. Otherwise if HADOOP_USER_NAME is not set, it will use the linux user launchs the hdfs broker service.

And the load script is like this.

LOAD LABEL db.xx (
DATA INFILE(
"xx"
) INTO TABLE xx
COLUMNS TERMINATED BY "\x01"
) WITH BROKER hdfs_broker (
"hadoop.security.authentication" = "simple",
"username" = "hadoop_user_name",
"password" = "hadoop_password",
"dfs.nameservices" = "nameservices",
"dfs.ha.namenodes.nameservices" = "nn1,nn2",
"dfs.namenode.rpc-address.nameservices.nn1" = "host1:port2",
"dfs.namenode.rpc-address.nameservices.nn2" = "host2:port2"
) PROPERTIES (
"exec_mem_limit" = "4294967296"
);

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/brokerIssues or PRs related to brokerkind/fixCategorizes issue or PR as related to a bug.

    Type

    No type

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions