Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use multiple threads for grpc requests #14

Open
davidor opened this issue Feb 1, 2021 · 4 comments
Open

Use multiple threads for grpc requests #14

davidor opened this issue Feb 1, 2021 · 4 comments

Comments

@davidor
Copy link
Contributor

davidor commented Feb 1, 2021

The server uses several threads when attending HTTP requests, but only uses 1 when attending grpc requests. I have a WIP branch that fixes this, but it has another problem: both actix and tonic create N threads where N = number of cpus. Tonic seems to use only the threads it creates, but Actix uses also the ones created by Tonic. We'd need to either create just N threads in total and make both frameworks use them or make Actix use only the threads it creates. This only happens when using the async redis storage implementation.

WIP branch: https://github.com/3scale-labs/limitador/tree/run-grpc-reqs-in-multiple-threads

@alexsnaps
Copy link
Member

We'd benefit most from this by having something along #69
While some parallelism while resolving the counters would be great, at least in an in-memory world, these threads would contend on the write locks protecting counters (they also do in the redis powered world, just further down the pipes and off our control to even be able to introduce parallelism)

@alexsnaps alexsnaps added this to the Limitador v0.4 milestone Jun 17, 2022
@alexsnaps
Copy link
Member

This crate here, by quickly glancing through the code, starts up a single tokio::runtime, used by both actix-web & tonic, then controlling how many worker actix-web should use and explicitly telling how many threads the tokio::runtime should create… Not arguing to use that crate, "just" sharing their approach…

@alexsnaps
Copy link
Member

See #37

@alexsnaps
Copy link
Member

This will require looking at where things are today first... These APIs changed a bit since, but we need to get a better handle on this and certainly prioritize the gRPC workload over the REST APIs one

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Todo
Status: No status
Development

No branches or pull requests

2 participants