Skip to content

[BEAM-12164]: display the metadata table name for Spanner Change Streams Connector #16845

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
pabloem merged 1 commit into from
Mar 10, 2022
Merged

[BEAM-12164]: display the metadata table name for Spanner Change Streams Connector #16845

pabloem merged 1 commit into from
Mar 10, 2022

Conversation

@hengfengli
Copy link
Contributor

@hengfengli hengfengli commented Feb 14, 2022 •
edited
Loading

This adds the name of the metadata table to display in the Pipeline Options of the Dataflow UI.

@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 14, 2022

CC @thiagotnunes

@hengfengli hengfengli force-pushed the allow-custom-metadata-table branch 5 times, most recently from b18372b to 9ae1b87 Compare February 15, 2022 06:05
@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 15, 2022

@pabloem Can you please take a quick review on this PR? Thanks!

@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 16, 2022

Please do not merge at the moment. I am investigating if we can pass the state via pipeline options between two jobs during the hot deploy.

@aaltay
Copy link
Member

aaltay commented Feb 25, 2022

@hengfengli - What is the next step on this PR?

@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 28, 2022

@aaltay Sorry for the delay. I have tested that the pipeline option would not pass between jobs during a hot deploy (set the pipeline option in the old job and see it in the updated job). So this PR can be merged now.

@hengfengli hengfengli force-pushed the allow-custom-metadata-table branch from 401bba5 to 4990612 Compare February 28, 2022 06:11
@hengfengli hengfengli changed the title [BEAM-12164]: allow to specify a custom metadata table for Spanner Change Streams Connector [BEAM-12164]: display the metadata table name for Spanner Change Streams Connector Feb 28, 2022
@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 28, 2022

As the change to allow to specify a custom metadata table is implemented in #16846, this PR is only for displaying the name of the metadata table on the Dataflow UI.

@hengfengli hengfengli force-pushed the allow-custom-metadata-table branch from 4990612 to 7dba96b Compare February 28, 2022 06:21
@hengfengli
Copy link
Contributor Author

hengfengli commented Feb 28, 2022

The failed test is unrelated to this change.

@hengfengli hengfengli force-pushed the allow-custom-metadata-table branch from 7dba96b to cb4efc7 Compare March 6, 2022 22:53
@hengfengli
Copy link
Contributor Author

hengfengli commented Mar 7, 2022

@pabloem Can you please take a look and help us to merge this PR? Thanks.

@aaltay aaltay requested a review from pabloem March 7, 2022 16:26
@pabloem
Copy link
Member

pabloem commented Mar 10, 2022

LGTM

@pabloem pabloem merged commit e1a84ff into apache:master Mar 10, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@pabloem pabloem Awaiting requested review from pabloem

2 more reviewers

@thiagotnunes thiagotnunes thiagotnunes approved these changes

@zoercai zoercai zoercai approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

gcp io java

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@hengfengli @aaltay @pabloem @thiagotnunes @zoercai