Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add state to the iterator to avoid re-scans for children #44

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

michaelplaing
Copy link

@michaelplaing michaelplaing commented May 1, 2022

This PR:

  • Adds a byte stack to raxIterator that contains child offsets.
  • Adds a separate, more complicated and slightly slower raxLowWalkSeek() for use by raxSeek() - this preserves the speed of the simpler original raxLowWalk() which is heavily used by raxFind() etc.
  • Modifies the raxSeek(), raxNext() and raxPrior() functions to use the above.
  • Also incorporates a substantive simplification made to raxSeek() taken from Redis 7.0.

The default rax-tests pass, including the fuzz tests.

@jcorporation
Copy link

jcorporation commented May 5, 2022

I merged this in a branch of my fork for testing: https://github.com/jcorporation/rax/tree/michaelplaing-iter_scan_upgrade

I have run the benchmark on master (without this patch) and your patch merged (branch michaelplaing-iter_scan_upgrade) and could see only minor timing changes. Do I miss anything?

@michaelplaing
Copy link
Author

There are only minor improvements on the original default tests as they have a pretty sparse keyspaces. Since the spans in each node therefore are short, there is not much room to improve. Tests using binary data can show much more improvement, depending on density. timestamps might be a good example, if they are issued close together. Redis uses millions of them and they are iterated in the event loop. Sequential IDs might be another usage.

@jcorporation
Copy link

jcorporation commented May 15, 2022

How large must be the keyspace to see improvements? A benchmark for illustration would be good.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants