Export DynamoDB table to S3 with PITR

[hyangminj]

Description

Airflow provides the Amazon DynamoDB to Amazon S3 below.
https://airflow.apache.org/docs/apache-airflow-providers-amazon/stable/operators/transfer/dynamodb_to_s3.html

Most of Data Engineer build their "export DDB data to s3" pipeline using "within the point in time recovery window".
https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/dynamodb.html#DynamoDB.Client.export_table_to_point_in_time

I appreciate if airflow has this function as a native function.

Use case/motivation

My daily batch job exports its data with pitr option. All of tasks is written by apache-airflow-providers-amazon except "export_table_to_point_in_time" task.
"export_table_to_point_in_time" task only used the python operator. I expect I can unify the task as apache-airflow-providers-amazon library.

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg]

Thanks for opening your first issue here! Be sure to follow the issue template!

[Taragolis]

Feel free to make a PR.

[Taragolis]

BTW, Hooks which provided by amazon-provider is basically just a wrapper around of boto3 clients / resources.
So you always could access to all boto3 clients methods within the hook

from airflow.providers.amazon.aws.hooks.dynamodb import DynamoDBHook

hook = DynamoDBHook(aws_conn_id="awesome-connection-id", region_name="us-east-1")
# DynamoDBHook create `resource` as high level client
# but you have an access to regular client by call `meta.client`.
# This potentially could be uniform in the future, see: https://github.com/apache/airflow/discussions/28560
client = hook.conn.meta.client
client.export_table_to_point_in_time(
    TableArn='string',
    ExportTime=datetime(2015, 1, 1),
    ClientToken='string',
    S3Bucket='string',
    S3BucketOwner='string',
    S3Prefix='string',
    S3SseAlgorithm='AES256'|'KMS',
    S3SseKmsKeyId='string',
    ExportFormat='DYNAMODB_JSON'|'ION'
)

[hyangminj]

Thank you for your information.

However, the exporting job is not final task in some data pipelines. It might be first steps of a dag to analysis data.
So the staring stage (Sensors) might be needed.
For example, If you create EMR job using Create the Job Flow, you need "Wait on an Amazon EMR job flow state" or "Wait on an Amazon EMR step state" methods .

Anyway, I will prepare the PR for this function and ask any questions if it is needed.

[hyangminj]

Your code and explanation would be best practice.

[Taragolis]

You could create multiple PRs one by one which could implemented separate parts: one for operator one for sensors, it handy especially if it your first contribution: less time for review, less time for change requests.

[hyangminj]

Sure, I will create separate PRs for operator and sensors.