Skip to content

TEZ-4555: Fail fast in LocalClient if the dirs (log, local) haven't been created #348

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ayushtkn merged 1 commit into from
Apr 25, 2024
Merged

TEZ-4555: Fail fast in LocalClient if the dirs (log, local) haven't been created #348

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 11 additions & 2 deletions tez-dag/src/main/java/org/apache/tez/client/LocalClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -331,8 +331,17 @@ public void run() {
// Prepare Environment
Path logDir = new Path(userDir, "localmode-log-dir");
Path localDir = new Path(userDir, "localmode-local-dir");
localFs.mkdirs(logDir);
localFs.mkdirs(localDir);

// fail fast if the local directories (on the paths that were used on HDFS) cannot be created
// in this case, user might want to choose a different staging path, which works on the local FS too
if (!localFs.mkdirs(logDir)) {
throw new IOException(
"Unable to create log directory, try to create it manually for further insights: " + logDir);
}
if (!localFs.mkdirs(localDir)) {
throw new IOException(
"Unable to create local directory, try to create it manually for further insights: " + localDir);
}

UserGroupInformation.setConfiguration(conf);
// Add session specific credentials to the AM credentials.
Expand Down