[cherry-pick](branch-2.1) pick hive text write from master #40537
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
run buildall |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
|
run buildall |
1. Support write hive text table 2. Add SessionVariable `hive_text_compression` to write compressed hive text table 3. Supported compression type: gzip, bzip2, snappy, lz4, zstd
## abstract
introduces the pluginization of the Authenticator component, enabling
greater flexibility and modularity within the authentication system. By
refactoring the Authenticator into a pluggable SPI (Service Provider
Interface), we allow for custom authentication mechanisms to be
seamlessly integrated and managed.
## Key Changes
AuthenticatorFactory Interface:
This interface defines the create method, which is used to create an
Authenticator instance based on initialization properties, enabling
support for various authentication mechanisms.
The factoryIdentifier method provides an identifier for the factory,
allowing differentiation between various AuthenticatorFactory
implementations (e.g., LDAP, default).
Implemented Specific Authentication Factories:
We have implemented factories for different authentication mechanisms
(such as LDAP), providing the necessary configuration support for each.
## eg
In your Maven pom.xml, add the fe-core dependency:
```
<dependency>
<groupId>org.apache.doris</groupId>
<artifactId>fe-core</artifactId>
<version>1.2-SNAPSHOT</version>
<scope>provided</scope>
</dependency>
```
```
import org.apache.doris.common.ConfigBase;
public class LocalAuthenticatorFactory implements AuthenticatorFactory {
@OverRide
public LocalAuthenticator create(Properties initProps) {
return new LocalAuthenticator();
}
@OverRide
public String factoryIdentifier() {
return "local";
}
}
import java.util.HashMap;
import java.util.Map;
public class LocalAuthenticator implements Authenticator {
private Map<String, String> userStore = new HashMap<>();
public LocalAuthenticator() {
// Hardcoded usernames and passwords
userStore.put("user1", "password1");
userStore.put("user2", "password2");
userStore.put("admin", "adminpass");
}
@OverRide
public boolean authenticate(String username, String password) {
// Authenticate by checking the hardcoded user store
String storedPassword = userStore.get(username);
return storedPassword != null && storedPassword.equals(password);
}
}
```
To integrate with the SPI (Service Provider Interface) mechanism, we
created a file at
**src/main/resources/META-INF/services/org.apache.doris.mysql.authenticate.AuthenticatorFactory**
containing **org.example.LocalAuthenticatorFactory,** which allows the
system to dynamically load and utilize the LocalAuthenticatorFactory
implementation.
Your plugin needs to include all necessary dependencies, and it is
generally recommended to package them into an Uber JAR (also known as a
Fat JAR) to ensure that the plugin can run independently without
additional dependency management. Once packaged, this JAR can be placed
directly in the **fe/custom_lib** directory.
Impl ZstdDecompressor and support read hive text table which is compressed by zstd.
suxiaogang223
force-pushed
the
hive_text_write21
branch
from
fe4ef6e to
b95fdce
Compare
|
run buildall |
|
TeamCity be ut coverage result: |
|
run p0 |
|
run buildall |
|
TeamCity be ut coverage result: |
|
run buildall |
suxiaogang223
changed the title
github-actions
bot
added
the
approved
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
|
TeamCity be ut coverage result: |
|
run p0 |
5 participants
Proposed changes
pick prs:
#38549
#40183
#40315