Handle clock skew after replication slot creation #22446
Assignees
Labels
area/cdc
Change Data Capture
area/cdcsdk
CDC SDK
jira-originated
kind/new-feature
This is a request for a completely new feature
priority/high
High Priority
Comments
yugabyte-ci
added
area/cdc
dr0pdb
added a commit
to dr0pdb/yugabyte-db
that referenced
this issue
May 22, 2024
…andle clock skew
Summary:
This revision introduces a sleep operation for max_clock_skew us after the replication slot creation.
The creation of a replication slot establishes a boundry between the
snapshot and change records. This is represented as a hybrid time.
This hybrid time can be chosen up to max_clock_skew us in the future.
We do not want to return the control back to the client before this
time passes otherwise the below scenario can cause confusion.
T1: Slot creation (snapshot time chosen as Tsnap)
... command returns to the client before Tsnap is in the past.
T2: Insert operation where T2 < Tsnap
The user would expect the insert operation to be part of the change
operations as it was done after the slot creation but it'll be
treated as snapshot operation.
Sleeping after the slot creation prevents that. It is fine to sleep like this because slot creation is not expected to be a frequent operation.
Test Plan: Existing CDC tests.
Differential Revision: https://phorge.dev.yugabyte.com/D35189
dr0pdb
added a commit
that referenced
this issue
May 22, 2024
…ock skew Summary: This revision introduces a sleep operation for max_clock_skew us after the replication slot creation. The creation of a replication slot establishes a boundry between the snapshot and change records. This is represented as a hybrid time. This hybrid time can be chosen up to max_clock_skew us in the future. We do not want to return the control back to the client before this time passes otherwise the below scenario can cause confusion. ``` T1: Slot creation (snapshot time chosen as Tsnap) ... command returns to the client before Tsnap is in the past. T2: Insert operation where T2 < Tsnap ``` The user would expect the insert operation to be part of the change operations as it was done after the slot creation but it'll be treated as snapshot operation. Sleeping after the slot creation prevents that. It is fine to sleep like this because slot creation is not expected to be a frequent operation. Jira: DB-11363 Test Plan: Existing CDC tests. Reviewers: asrinivasan Reviewed By: asrinivasan Subscribers: ycdcxcluster, yql Differential Revision: https://phorge.dev.yugabyte.com/D35189
svarnau
pushed a commit
that referenced
this issue
May 25, 2024
…ock skew Summary: This revision introduces a sleep operation for max_clock_skew us after the replication slot creation. The creation of a replication slot establishes a boundry between the snapshot and change records. This is represented as a hybrid time. This hybrid time can be chosen up to max_clock_skew us in the future. We do not want to return the control back to the client before this time passes otherwise the below scenario can cause confusion. ``` T1: Slot creation (snapshot time chosen as Tsnap) ... command returns to the client before Tsnap is in the past. T2: Insert operation where T2 < Tsnap ``` The user would expect the insert operation to be part of the change operations as it was done after the slot creation but it'll be treated as snapshot operation. Sleeping after the slot creation prevents that. It is fine to sleep like this because slot creation is not expected to be a frequent operation. Jira: DB-11363 Test Plan: Existing CDC tests. Reviewers: asrinivasan Reviewed By: asrinivasan Subscribers: ycdcxcluster, yql Differential Revision: https://phorge.dev.yugabyte.com/D35189
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Jira Link: DB-11363
The text was updated successfully, but these errors were encountered: