Skip to content

Conversation

@kw2542
Copy link
Contributor

@kw2542 kw2542 commented Jan 10, 2020

Design:
https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner

Changes:

  1. Depending the on the existence of job.config.loader.factory, YarnJob will alternatively wraps job submission configs to Yarn.
  2. In a separate PR, ClusterBasedJobCoordinator will be updated to read full job config from loader or coordinator stream based on the existence of submission config environment variable.

API Changes:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Upgrade Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Usage Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Tests:
Unit Tests

Ke Wu added 3 commits January 9, 2020 17:57
Design:
https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner

Changes:
1. Depending the on the existence of job.config.loader.factory, YarnJob will alternatively wraps job submission configs to Yarn.

API Changes:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Upgrade Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Usage Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Tests:
Unit Tests
Design:
https://cwiki.apache.org/confluence/display/SAMZA/SEP-23%3A+Simplify+Job+Runner

Changes:
1. Depending the on the existence of job.config.loader.factory, YarnJob will alternatively wraps job submission configs to Yarn.

API Changes:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Upgrade Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Usage Instructions:
N/A. This is part of a series PRs, detailed information will be provided in the last/main PR.

Tests:
Unit Tests
Copy link
Contributor

@bkonold bkonold left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cameronlee314 cameronlee314 merged commit dd7b9b8 into apache:master Jan 17, 2020
@kw2542 kw2542 deleted the SAMZA-2409 branch January 17, 2020 23:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants