ZooKeeper connection pooling #53
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…eating too many connections This is necessary because Terraform will invoke the Provider's `ConfigureContextFunc` once for each operation a resource requires. Invoking `NewClient` in there was leading to spawning a connection each time. The underlying client DOES SUPPORT concurrent use, so we create a _pool_ where each `Client` is identified by a key made of all it's construction parameters.
detro
changed the title
…t spawn a ton anymore
detro
force-pushed
the
detro/debug_connections_issue
branch
from
cc071de to
376f219
Compare
|
@abarabash-sift any chance you want to take a look and see what you think? I did a bit of test and I'm fairly sure this is good. But it's late at night and I'd appreciate a second pair of eyes on this one. If you can't no worries. |
detro
changed the title
detro
force-pushed
the
detro/debug_connections_issue
branch
from
b9f36ea to
0309b9c
Compare
detro
force-pushed
the
detro/debug_connections_issue
branch
6 times, most recently
from
399bf59 to
cc3035a
Compare
detro
force-pushed
the
detro/debug_connections_issue
branch
2 times, most recently
from
9866451 to
bb43814
Compare
detro
force-pushed
the
detro/debug_connections_issue
branch
from
bb43814 to
5821625
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This makes the operator hold the connection to ZooKeeper in a pool of
client.Client, indexed by their construction parameters.The underlying ZooKeeper client is perfectly able to handle concurrent use, as well as automatic reconnection. So this approach ensures we keep around the minimum amount of connections necessary, and delegate to the underlying client the concurrency.
Additionally, this also stops the
resource.ParallelTestprovided by Terraform SDKv2, spawn 1 provider for each test execution. Instead, it shares the one instance.This also solves the issue identified by @abarabash-sift in #49.