Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: use correct shard index when creating encoder #1885

Merged
merged 1 commit into from
Apr 25, 2024

Conversation

adklempner
Copy link
Member

@adklempner adklempner commented Mar 5, 2024

Problem

When library consumer specifies a cluster ID and shard index when creating an encoder, it means they are using static sharding and explicitly specifying which shard index (and thus pubsub topic) should be used.

Currently, when an encoder is created this way, it uses the autosharding algorithm to determine the shard index based on content topic, which can lead to unexpected results if it doesn't match the shard index specified by the consumer.

Solution

  • Make shard property optional in SingleShardInfo.
  • Fix conditional that evaluates to falsy when shard is set to 0 (this was preventing static sharding from being used)

Notes

@adklempner adklempner requested a review from a team as a code owner March 5, 2024 02:41
Copy link

github-actions bot commented Mar 5, 2024

size-limit report 📦

Path Size Loading time (3g) Running time (snapdragon) Total time
Waku node 180.73 KB (-0.07% 🔽) 3.7 s (-0.07% 🔽) 2.7 s (+8.76% 🔺) 6.4 s
Waku Simple Light Node 180.86 KB (-0.07% 🔽) 3.7 s (-0.07% 🔽) 2.1 s (+2.04% 🔺) 5.7 s
ECIES encryption 23.11 KB (-0.02% 🔽) 463 ms (-0.02% 🔽) 538 ms (-16.89% 🔽) 1000 ms
Symmetric encryption 22.59 KB (+0.05% 🔺) 452 ms (+0.05% 🔺) 789 ms (+56.95% 🔺) 1.3 s
DNS discovery 72.42 KB (0%) 1.5 s (0%) 1.8 s (+19.54% 🔺) 3.2 s
Peer Exchange discovery 73.96 KB (0%) 1.5 s (0%) 929 ms (+22.21% 🔺) 2.5 s
Local Peer Cache Discovery 67.64 KB (0%) 1.4 s (0%) 1.5 s (+82.95% 🔺) 2.9 s
Privacy preserving protocols 38.87 KB (0%) 778 ms (0%) 1.1 s (+0.17% 🔺) 1.8 s
Waku Filter 111.25 KB (-0.17% 🔽) 2.3 s (-0.17% 🔽) 1.5 s (-11.53% 🔽) 3.7 s
Waku LightPush 110.23 KB (0%) 2.3 s (0%) 1.3 s (-11.78% 🔽) 3.5 s
History retrieval protocols 110.71 KB (0%) 2.3 s (0%) 2 s (+53.11% 🔺) 4.2 s
Deterministic Message Hashing 4.83 KB (0%) 97 ms (0%) 40 ms (-61.62% 🔽) 137 ms

@adklempner adklempner force-pushed the fix/encoder-cluster-id branch 2 times, most recently from 31275b0 to ac16ae8 Compare March 6, 2024 03:24
Copy link
Collaborator

@danisharora099 danisharora099 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand this right, the issue arises as 0 is considered to be false by JavaScript.

If that's the case, we should add a check to handle 0 gracefully, instead of making the type optional (which leads to cases where we have to enforce it's definition by adding !)

Is that a fair interpretation? @adklempner

@adklempner
Copy link
Member Author

adklempner commented Mar 6, 2024

@danisharora099 The reason I made the shard field optional is because right now there's no other way for createEncoder to differentiate between static sharding and autosharding. e.g. in the test { clusterId: 0 } makes the function determine shard index using the content topic, but {clusterId: 0, shard: 1} will always use shard index 1, no matter the content topic.

@adklempner adklempner force-pushed the fix/encoder-cluster-id branch from ac16ae8 to 52c53a5 Compare March 6, 2024 15:40
Copy link
Collaborator

@weboko weboko left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

seems fine to me

@adklempner adklempner force-pushed the fix/encoder-cluster-id branch from 59f8146 to b58a9e9 Compare April 25, 2024 01:04
@adklempner adklempner force-pushed the fix/encoder-cluster-id branch from b58a9e9 to 9514653 Compare April 25, 2024 01:40
@adklempner adklempner merged commit 66081d6 into master Apr 25, 2024
9 of 10 checks passed
@adklempner adklempner deleted the fix/encoder-cluster-id branch April 25, 2024 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

bug: passing shard 0 to createEncoder will generate a pubsubtopic with shard 1
3 participants