Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(sync-service): Move WHERE clause evaluation to single process #1761

Merged
merged 2 commits into from
Oct 10, 2024

Conversation

robacourt
Copy link
Contributor

@robacourt robacourt commented Sep 26, 2024

This is the simplest change to meet the criteria of #1744 .

It's faster and uses less memory:
Screenshot 2024-10-10 at 13 48 21

Than our current version:
Screenshot 2024-10-10 at 13 48 38

Because of the reduction in memory use it can handle bigger transactions at 100k shapes:
Screenshot 2024-10-10 at 13 36 12

vs our la
Screenshot 2024-10-10 at 13 35 59
test version:

@robacourt robacourt force-pushed the rob/single-process-shape-filter branch 2 times, most recently from 93c6505 to 85ba982 Compare September 26, 2024 12:24
Copy link
Contributor

@KyleAMathews KyleAMathews left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like a good first step 👍

@balegas
Copy link
Contributor

balegas commented Sep 26, 2024

Yep. Sounds good.

@robacourt robacourt force-pushed the rob/single-process-shape-filter branch from c9841a8 to 85ba982 Compare September 26, 2024 15:36
@robacourt robacourt force-pushed the rob/single-process-shape-filter branch from 85ba982 to 4fd17d8 Compare October 10, 2024 10:52
@robacourt robacourt marked this pull request as ready for review October 10, 2024 13:00
@robacourt robacourt merged commit c2076c8 into main Oct 10, 2024
13 of 15 checks passed
@robacourt robacourt deleted the rob/single-process-shape-filter branch October 10, 2024 13:02
msfstef pushed a commit that referenced this pull request Oct 10, 2024
Follow-up to #1761 - no changeset as it fixes a bug introduced there
KyleAMathews pushed a commit that referenced this pull request Nov 1, 2024
…1761)

This is the simplest change to meet the criteria of #1744 .

It's faster and uses less memory:
<img width="795" alt="Screenshot 2024-10-10 at 13 48 21"
src="https://github.com/user-attachments/assets/a508d078-9cbd-4610-babb-909d66de682e">

Than our current version:
<img width="790" alt="Screenshot 2024-10-10 at 13 48 38"
src="https://github.com/user-attachments/assets/a1a0d0da-39fd-4201-b468-5aa64a649120">

Because of the reduction in memory use it can handle bigger transactions
at 100k shapes:
<img width="824" alt="Screenshot 2024-10-10 at 13 36 12"
src="https://github.com/user-attachments/assets/da0a62f8-97c6-4dee-b63b-a8222ed54856">

vs our la
<img width="822" alt="Screenshot 2024-10-10 at 13 35 59"
src="https://github.com/user-attachments/assets/2f7aec8b-e840-49ba-b4b7-2d4733b264f9">
test version:
KyleAMathews pushed a commit that referenced this pull request Nov 1, 2024
Follow-up to #1761 - no changeset as it fixes a bug introduced there
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants