Skip to content

Conversation

@yaswanthrajyadiki
Copy link
Contributor

Make sure you have checked all steps below.

Jira

Description

Added below attributes as template_fields, now those params will accept Jinja template and values will changes in run time

S3DeleteObjectsOperator

  1. keys
  2. bucket

S3CopyObjectOperator

  1. source_bucket_key
  2. dest_bucket_key
  3. source_bucket_name
  4. dest_bucket_name

Commits

  • My commits all reference Jira issues in their subject lines, and I have squashed multiple commits if they address the same issue. In addition, my commits follow the guidelines from "How to write a good git commit message":
    1. Subject is separated from body by a blank line
    2. Subject is limited to 50 characters (not including Jira issue reference)
    3. Subject does not end with a period
    4. Subject uses the imperative mood ("add", not "adding")
    5. Body wraps at 72 characters
    6. Body explains "what" and "why", not "how"

Documentation

  • In case of new functionality, my PR adds documentation that describes how to use it.
    • When adding new operators/hooks/sensors, the autoclass documentation generation needs to be added.
    • All the public functions and the classes in the PR contain docstrings that explain what it does
    • If you implement backwards incompatible changes, please leave a note in the Updating.md so we can assign it to a appropriate release

Code Quality

  • Passes flake8

Added below attributes as template_fields in S3CopyObjectOperator,
now those params will accept Jinja template and values will changes in run time
* source_bucket_key
* dest_bucket_key
* source_bucket_name
* dest_bucket_name
@XD-DENG
Copy link
Member

XD-DENG commented Mar 15, 2019

Restarted a few failing CI tests, which may be due to transient error.

@codecov-io
Copy link

codecov-io commented Mar 16, 2019

Codecov Report

Merging #4920 into master will increase coverage by 1.39%.
The diff coverage is 100%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master    #4920      +/-   ##
==========================================
+ Coverage   75.57%   76.96%   +1.39%     
==========================================
  Files         454      454              
  Lines       29159    31768    +2609     
==========================================
+ Hits        22036    24450    +2414     
- Misses       7123     7318     +195
Impacted Files Coverage Δ
...rflow/contrib/operators/s3_copy_object_operator.py 100% <100%> (ø) ⬆️
...ow/contrib/operators/s3_delete_objects_operator.py 90.47% <100%> (+0.47%) ⬆️
airflow/operators/dagrun_operator.py 91.11% <0%> (-3.63%) ⬇️
airflow/contrib/utils/gcp_field_sanitizer.py 90% <0%> (-0.91%) ⬇️
airflow/operators/dummy_operator.py 100% <0%> (ø) ⬆️
airflow/contrib/operators/gcs_list_operator.py 100% <0%> (ø) ⬆️
airflow/contrib/operators/adls_list_operator.py 100% <0%> (ø) ⬆️
airflow/utils/weight_rule.py 100% <0%> (ø) ⬆️
airflow/utils/trigger_rule.py 100% <0%> (ø) ⬆️
airflow/contrib/operators/s3_list_operator.py 100% <0%> (ø) ⬆️
... and 31 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 419e301...443b648. Read the comment docs.

@XD-DENG XD-DENG merged commit 3fdd3d9 into apache:master Mar 16, 2019
@XD-DENG
Copy link
Member

XD-DENG commented Mar 16, 2019

Thanks @yaswanthrajyadiki

@XD-DENG XD-DENG changed the title [AIRFLOW-4095] Adding some params as template fields in S3 Operators [AIRFLOW-4095] Add template_fields for S3CopyObjectOperator & S3DeleteObjectsOperator Mar 16, 2019
@XD-DENG
Copy link
Member

XD-DENG commented Mar 16, 2019

Hi @ashb , this PR is an enhancement on #3823.

#3823 (https://issues.apache.org/jira/browse/AIRFLOW-2985) is already labelled with fix version 1.10.3. So I have labelled this PR with fix version 1.10.3 as well. FYI.

ashb pushed a commit that referenced this pull request Mar 18, 2019
andriisoldatenko pushed a commit to andriisoldatenko/airflow that referenced this pull request Jul 26, 2019
wmorris75 pushed a commit to modmed-external/incubator-airflow that referenced this pull request Jul 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants