Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ratelimiter] Add IP allowlist #19

Closed
wants to merge 1 commit into from
Closed

[ratelimiter] Add IP allowlist #19

wants to merge 1 commit into from

Conversation

ian-shim
Copy link
Contributor

@ian-shim ian-shim commented Nov 14, 2023

Why are these changes needed?

We can use this flag to manually add IPs that can be rate limited at 10x the setting.

Checks

  • I've made sure the lint is passing in this PR.
  • I've made sure the tests are passing. Note that there might be a few flaky tests, in that case, please comment that they are not relevant.
  • Testing Strategy
    • Unit tests
    • Integration tests
    • This PR is not tested :(

@ian-shim ian-shim marked this pull request as ready for review November 14, 2023 05:01
logger: logger,
}
}

// Checks whether a request from the given requesterID is allowed
func (d *rateLimiter) AllowRequest(ctx context.Context, requesterID common.RequesterID, blobSize uint, rate common.RateParam) (bool, error) {
for _, id := range d.allowlist {
if id == requesterID {
// Allow 10x the rate for allowlisted IDs
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we just drop the limit for these IPs, i.e. allowing them to push the traffic to the global limit?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I initially implemented that way, but decided to go this way to ensure no single IP can hog the global throughput, which will make all requests to fail globally. wdyt? maybe we can make it 100x?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about 1MiB/s / len(d.allowlist)?

logger: logger,
}
}

// Checks whether a request from the given requesterID is allowed
func (d *rateLimiter) AllowRequest(ctx context.Context, requesterID common.RequesterID, blobSize uint, rate common.RateParam) (bool, error) {
for _, id := range d.allowlist {
if id == requesterID {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will fail because the disperser constructs the requesterID as ip:quorumID.

Copy link
Collaborator

@mooselumph mooselumph Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could change the AllowRequest to accept both requesterID and a tag variable, and construct the key internally to the library.

@ian-shim ian-shim closed this by deleting the head repository Nov 15, 2023
teddyknox pushed a commit that referenced this pull request Nov 16, 2023
Signed-off-by: Wellington Barbosa <[email protected]>
Signed-off-by: Wellington Barbosa <[email protected]>
Co-authored-by: Wellington Barbosa <[email protected]>
Co-authored-by: Wellington Barbosa <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants