Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support batch processing with the GetRegion stream call #8690

Open
4 tasks
JmPotato opened this issue Oct 11, 2024 · 1 comment
Open
4 tasks

Support batch processing with the GetRegion stream call #8690

JmPotato opened this issue Oct 11, 2024 · 1 comment
Labels
type/development The issue belongs to a development tasks

Comments

@JmPotato
Copy link
Member

We have encountered several issues related to the GetRegion call and region cache updates, such as #2843 and #8426. At present, a highly feasible solution to alleviate this type of problem is to change GetRegion from a unary gRPC call to a stream call that supports batch processing, which should reduce the CPU pressure on PD in scenarios such as cache expiration and error retries strom.

  • Implement the gRPC stream version of GetRegion which supports handling various queries, such as GetPrevRegion and GetRegionByID.
  • Implement the batch processing logic within the PD client.
  • Integrate the new GetRegion method into the client-go/TiDB.
  • Conduct the benchmark tests with the previous version to evaluate the QPS/CPU usage performance.
@JmPotato JmPotato added the type/development The issue belongs to a development tasks label Oct 11, 2024
@JmPotato
Copy link
Member Author

/cc @JackL9u

ti-chi-bot bot pushed a commit that referenced this issue Nov 11, 2024
ref #8690

Genericize the batch controller for the later reuse.

Signed-off-by: JmPotato <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue Nov 12, 2024
ref #8690

Aggregate the utils package within the client.

Signed-off-by: JmPotato <[email protected]>
@JmPotato JmPotato pinned this issue Nov 12, 2024
ti-chi-bot bot pushed a commit that referenced this issue Nov 14, 2024
ref #8690

Independent the client option to reduce the number of files under the client root directory.

Signed-off-by: JmPotato <[email protected]>
ti-chi-bot bot pushed a commit that referenced this issue Nov 19, 2024
ref #8690

Split the meta storage client from the client definitions.

Signed-off-by: JmPotato <[email protected]>
ti-chi-bot bot added a commit that referenced this issue Nov 22, 2024
ref #8690

Separate the client metrics package.

Signed-off-by: JmPotato <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Nov 25, 2024
ref #8690

Make the service discovery module of the client a separate package.

Signed-off-by: JmPotato <[email protected]>
ti-chi-bot bot added a commit that referenced this issue Nov 26, 2024
ref #8690

Separate the TSO client implementation.

Signed-off-by: JmPotato <[email protected]>

Co-authored-by: ti-chi-bot[bot] <108142056+ti-chi-bot[bot]@users.noreply.github.com>
ti-chi-bot bot pushed a commit that referenced this issue Nov 26, 2024
ref #8690

Move some common libraries to the `client/pkg` directory.

Signed-off-by: JmPotato <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/development The issue belongs to a development tasks
Projects
None yet
Development

No branches or pull requests

1 participant