Deserialize proto based (mutable key range) change stream return records #35408

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Merged

Abacn merged 1 commit into apache:master from changliiu:v2-deserialization

Jun 27, 2025

Contributor

changliiu commented Jun 24, 2025 •

edited

Loading

Parse proto type change stream return records into connector models.

Including:

DataChangeRecord
HeartbeatRecord
PartitionStartRecord
PartitionEndRecord
PartitionEventRecord

Also:

increase grpc java client lib version so that change stream new records e.g. PartitionStartRecords can be used.
add unit tests for each type or records.

github-actions bot added java build io gcp spanner labels

changliiu force-pushed the v2-deserialization branch from 10e07a1 to 254762f Compare

June 24, 2025 00:14

changliiu marked this pull request as ready for review

June 24, 2025 00:19

changliiu force-pushed the v2-deserialization branch from 254762f to 71ddfe4 Compare

June 24, 2025 00:36

Contributor

github-actions bot commented Jun 24, 2025

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

changliiu force-pushed the v2-deserialization branch 2 times, most recently from 53081b9 to 458398f Compare

June 24, 2025 20:45

dedocibula reviewed

View reviewed changes

...n/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/mapper/ChangeStreamRecordMapper.java Outdated Show resolved Hide resolved

dedocibula reviewed

View reviewed changes

...rc/main/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/dao/ChangeStreamResultSet.java Outdated Show resolved Hide resolved

dedocibula reviewed

View reviewed changes

...n/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/mapper/ChangeStreamRecordMapper.java Outdated Show resolved Hide resolved

dedocibula reviewed

View reviewed changes

...n/java/org/apache/beam/sdk/io/gcp/spanner/changestreams/mapper/ChangeStreamRecordMapper.java Show resolved Hide resolved

dedocibula reviewed

View reviewed changes

Contributor

dedocibula left a comment

First batch of comments

changliiu force-pushed the v2-deserialization branch 2 times, most recently from 684c368 to 52ebaac Compare

June 25, 2025 18:16


          Parse v2 record.

eff8905

pick 4d2a5a7 # Fix API surface test (apache#35028)

Deserialize change stream proto based (mutable key range) return types.

changliiu force-pushed the v2-deserialization branch from 52ebaac to eff8905 Compare

June 25, 2025 18:28

Contributor Author

changliiu commented Jun 25, 2025

Friendly ping :)

thiagotnunes approved these changes

View reviewed changes

Abacn merged commit 3f3f214 into apache:master

22 of 23 checks passed

changliiu deleted the v2-deserialization branch

June 27, 2025 17:20

Abacn added a commit that referenced this pull request


          Revert "Parse v2 record. (#35408)"

6dc3c27

This reverts commit 3f3f214.

Abacn mentioned this pull request

Revert "Deserialize proto based (mutable key range) change stream return records" #35469

Merged

liferoad pushed a commit that referenced this pull request


          Revert "Parse v2 record. (#35408)" (#35469)

f9b7c74

This reverts commit 3f3f214.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

7523e30

This reverts commit f9b7c74.

changliiu mentioned this pull request

Revert "Revert "Deserialize proto based (mutable key range) change stream return records"" #35481

Closed

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

dbf98da

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

196c0dd

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

f7761e9

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

cb7e67a

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

75ba9cb

This reverts commit f9b7c74.

changliiu added a commit to changliiu/beam that referenced this pull request


          Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"

517f589

This reverts commit f9b7c74.

changliiu mentioned this pull request

Revert "Revert "Parse v2 record. (#35408)" (#35469)" #35497

Merged

jrmccluskey pushed a commit to jrmccluskey/beam that referenced this pull request


          Revert "Parse v2 record. (apache#35408)" (apache#35469)

101a4e5

This reverts commit 3f3f214.

Abacn pushed a commit that referenced this pull request


          Revert "Revert "Parse v2 record. (#35408)" (#35469)" (#35497)

This reverts commit f9b7c74.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

build gcp io java spanner