Skip to content

reference/tools: add ticdc open protocol docs#2284

Merged
sre-bot merged 5 commits into
pingcap:masterfrom
TomShawn:cdc-open-protocol
Apr 26, 2020
Merged

reference/tools: add ticdc open protocol docs#2284
sre-bot merged 5 commits into
pingcap:masterfrom
TomShawn:cdc-open-protocol

Conversation

@TomShawn
Copy link
Copy Markdown
Contributor

@TomShawn TomShawn commented Apr 15, 2020

What is changed, added or deleted? (Required)

Add TiCDC Open Protocol documents.

Which TiDB version(s) do your changes apply to? (Required)

  • master (the latest development version)
  • v4.0 (TiDB 4.0 versions)
  • v3.1 (TiDB 3.1 versions)
  • v3.0 (TiDB 3.0 versions)
  • v2.1 (TiDB 2.1 versions)

If you select two or more versions from above, to trigger the bot to cherry-pick this PR to your desired release version branch(es), you must add corresponding labels such as needs-cherry-pick-4.0, needs-cherry-pick-3.1, needs-cherry-pick-3.0, and needs-cherry-pick-2.1.

What is the related PR or file link(s)?

@TomShawn TomShawn added translation/from-docs-cn This PR is translated from a PR in pingcap/docs-cn. size/large Changes of a large size. status/PTAL This PR is ready for reviewing. needs-cherry-pick-4.0 labels Apr 15, 2020
@TomShawn TomShawn requested review from ran-huang and zier-one April 15, 2020 08:03
@zier-one
Copy link
Copy Markdown
Contributor

note that there are some change in pingcap/docs-cn#2763

Comment thread reference/tools/ticdc/column-ddl-type.md Outdated
Comment thread reference/tools/ticdc/column-ddl-type.md Outdated
Comment thread reference/tools/ticdc/column-ddl-type.md Outdated
Comment thread reference/tools/ticdc/column-ddl-type.md Outdated
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 17, 2020

@leoppro, @ran-huang, PTAL.

1 similar comment
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 19, 2020

@leoppro, @ran-huang, PTAL.

Comment thread reference/tools/ticdc/open-protocol.md Outdated
Comment thread reference/tools/ticdc/open-protocol.md Outdated
Comment thread reference/tools/ticdc/open-protocol.md Outdated

* In most cases, the Row Changed Event of a version is sent only once, but in special situations such as node failure and network partition, the Row Changed Event of the same version might be sent multiple times.
* Row Changed Events of the same version first sent on the same table are incremented in the order of timestamps (TS) in the Event stream.
* Resolved Events are periodically broadcasted to each MQ Partition. The Resolved Event means that any Event with TS less than Resolved Event TS has been sent to the downstream.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel less than is weird. Is smaller than or earlier than better?

Suggested change
* Resolved Events are periodically broadcasted to each MQ Partition. The Resolved Event means that any Event with TS less than Resolved Event TS has been sent to the downstream.
* Resolved Events are periodically broadcasted to each MQ Partition. The Resolved Event means that any Event with a TS less than Resolved Event TS has been sent to the downstream.

Comment thread reference/tools/ticdc/open-protocol.md Outdated

## Examples of the Event stream output

This section shows and displays the output logs of Events.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
This section shows and displays the output logs of Events.
This section shows and displays the output logs of the Event stream.

Comment thread reference/tools/ticdc/open-protocol.md Outdated
CREATE TABLE test.t1(id int primary key, val varchar(16));
```

From the following Log 1 and Log 3, you can see that the DDL Event is broadcasted to all MQ Partitions.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
From the following Log 1 and Log 3, you can see that the DDL Event is broadcasted to all MQ Partitions.
From the following Log 1 and Log 3, you can see that the DDL Event is broadcasted to all MQ Partitions, and that the Resolved Event is periodically broadcasted to each MQ Partition.

Comment thread reference/tools/ticdc/open-protocol.md Outdated
```

+ From the following Log 5 and Log 6, you can see that Row Changed Events on the same table might be sent to different partitions based on the primary key, but changes to the same row are sent to the same partition so that the downstream can easily process the Event concurrently.
+ From Log 6, multiple changes to the same row in a transaction only send one Row Changed Event.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
+ From Log 6, multiple changes to the same row in a transaction only send one Row Changed Event.
+ From Log 6, multiple changes to the same row in a transaction are only sent in one Row Changed Event.

Comment thread reference/tools/ticdc/open-protocol.md Outdated

+ From the following Log 5 and Log 6, you can see that Row Changed Events on the same table might be sent to different partitions based on the primary key, but changes to the same row are sent to the same partition so that the downstream can easily process the Event concurrently.
+ From Log 6, multiple changes to the same row in a transaction only send one Row Changed Event.
+ Log 8 is a repeated event of Log 7. Row Changed Event might be repeated, but each version of a first Event is sent orderly.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@leoppro Please confirm whether this is what you mean.

Suggested change
+ Log 8 is a repeated event of Log 7. Row Changed Event might be repeated, but each version of a first Event is sent orderly.
+ Log 8 is a repeated event of Log 7. Row Changed Event might be repeated, but the first Event of each version is sent orderly.

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 22, 2020

@leoppro, @ran-huang, PTAL.

@ran-huang ran-huang requested a review from zier-one April 22, 2020 10:46
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 24, 2020

@leoppro, @ran-huang, PTAL.

@TomShawn
Copy link
Copy Markdown
Contributor Author

@ran-huang @leoppro PTAL again, thanks!

Copy link
Copy Markdown
Contributor

@ran-huang ran-huang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zier-one
Copy link
Copy Markdown
Contributor

LGTM

@TomShawn TomShawn added status/can-merge Indicates a PR has been approved by a committer. and removed status/PTAL This PR is ready for reviewing. labels Apr 26, 2020
@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 26, 2020

/run-all-tests

@sre-bot sre-bot merged commit 48649da into pingcap:master Apr 26, 2020
@TomShawn TomShawn deleted the cdc-open-protocol branch April 26, 2020 07:22
@TomShawn
Copy link
Copy Markdown
Contributor Author

/run-cherry-picker

@sre-bot
Copy link
Copy Markdown
Contributor

sre-bot commented Apr 26, 2020

cherry pick to release-4.0 in PR #2399

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/large Changes of a large size. status/can-merge Indicates a PR has been approved by a committer. translation/from-docs-cn This PR is translated from a PR in pingcap/docs-cn.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants