Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sync-service): Query Postgres server version at the start of connection setup #1784

Merged
merged 3 commits into from
Oct 3, 2024

Conversation

alco
Copy link
Member

@alco alco commented Oct 2, 2024

We were querying it after starting the replication client before, which was too late, because by that time the shape recovery process might have already reached Electric.Postgres.Configuration.configure_tables_for_replication!() where the version is used to branch the control flow. This was causing flake in integration tests due to the version not always being there and the unsupported query being issue against Postgres version 14.

In short, this PR resolves the flakiness in integration tests, they should be consistently green from now on.

The choice of augmenting the LockConnection module with version querying and a crude state machine might seem dubious but the end result is simpler this way, compared to adding version querying to the replication client, for example.

We were querying it after starting the replication client before, which
was too late, because by that time the shape recovery process might have
already reached
Electric.Postgres.Configuration.configure_tables_for_replication!()
where the version is used to branch the control flow.

The choice of augmenting the LockConnection module with version querying
and a crude state machine might seem dubious but the end result is
simpler this way, compared to adding version querying to the replication
client, for example.
Copy link

netlify bot commented Oct 3, 2024

Deploy Preview for electric-next ready!

Name Link
🔨 Latest commit c91bd35
🔍 Latest deploy log https://app.netlify.com/sites/electric-next/deploys/66fe5e0109c7a200080263ad
😎 Deploy Preview https://deploy-preview-1784--electric-next.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@alco alco force-pushed the alco/pg-version-query-race branch from 442c827 to f99ad93 Compare October 3, 2024 09:27
@alco alco requested a review from robacourt October 3, 2024 10:17
Copy link
Contributor

@robacourt robacourt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice one!

Copy link
Contributor

@msfstef msfstef left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! We might want to rename LockConnection to something like PrepareDatabaseConnection or something more generic like that, as it seems to be evolving to all steps required before any other interaction with the db

@alco
Copy link
Member Author

alco commented Oct 3, 2024

@msfstef good point! I'll consider that when working on #1792.

@alco alco merged commit c886f86 into main Oct 3, 2024
24 checks passed
@alco alco deleted the alco/pg-version-query-race branch October 3, 2024 14:37
KyleAMathews pushed a commit that referenced this pull request Nov 1, 2024
…ection setup (#1784)

We were querying it after starting the replication client before, which
was too late, because by that time the shape recovery process might have
already reached
`Electric.Postgres.Configuration.configure_tables_for_replication!()`
where the version is used to branch the control flow. This was causing
flake in integration tests due to the version not always being there and
the unsupported query being issue against Postgres version 14.

In short, this PR resolves the flakiness in integration tests, they
should be consistently green from now on.

The choice of augmenting the `LockConnection` module with version
querying and a crude state machine might seem dubious but the end result
is simpler this way, compared to adding version querying to the
replication client, for example.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants