Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Scylla] - Investigate performance improvements #184

Open
Tracked by #256
jr1221 opened this issue Sep 11, 2024 · 1 comment · May be fixed by #262
Open
Tracked by #256

[Scylla] - Investigate performance improvements #184

jr1221 opened this issue Sep 11, 2024 · 1 comment · May be fixed by #262
Labels
backend Development Focused in the Backend

Comments

@jr1221
Copy link
Contributor

jr1221 commented Sep 11, 2024

Description

Now that the rust implementation is finished, there could be many performance improvements added to it. These boil down to three categories. CPU time, memory allocation, and polling/async frequencies.

Acceptance Criteria

  • Async: Investigate polling/async frequencies and ensure select! is not providing an overhead, probably using tokio_console
  • Memory alloc: Investigate any data cloning in hot routes, like the query or especially the batch insert, and eliminate them.
  • CPU time: Investigate the use of rayon to parallelize db batching or other large functions.
  • CPU Time: Search for any computational overheads and see if SIMD compliant functions exist.

Proposed Solution

Possibly rearrange selects into dedicated tasks, implement rayon par_iter, implement simd common, etc.

Mocks

No response

@jr1221 jr1221 added the backend Development Focused in the Backend label Sep 11, 2024
@jr1221
Copy link
Contributor Author

jr1221 commented Sep 11, 2024

Fix this clone!

self.data_queue.clear();

@jr1221 jr1221 mentioned this issue Dec 6, 2024
18 tasks
@jr1221 jr1221 linked a pull request Dec 21, 2024 that will close this issue
12 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend Development Focused in the Backend
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant