Add support for fast fail when token bucket empty

[jamesls]

Describe the feature

In adaptive mode, acquiring a send token from the token bucket always blocks:

botocore/botocore/retries/adaptive.py

Line 59 in 70514ef

self._token_bucket.acquire()

botocore/botocore/retries/bucket.py

Line 66 in 70514ef

def acquire(self, amount=1, block=True):

We should also support the fast fail mode (block=False) where we propagate a client exception when the token bucket is empty.

Use Case

This is required behavior for the SDKs to allow the option to fast fail when the token bucket is empty.

The use case here is that propagating the token bucket acquisition failure lets the caller decide what to do. They could then continue to propagate that up the stack until something has enough context to decide the best course of action. With the current behavior of blocking, the caller has to always accept a block.

A concrete scenario of this would be when botocore is being used in the context of a web service. In some cases, the desired behavior when the token bucket is empty is to send back a 429/4xx response back to the caller asking them to slow down instead of the request handler blocking until capacity becomes available.

Proposed Solution

• Add some config option in the retries dict of the config option. Not sure what to call it, maybe fast_fail, token_bucket_block, etc. Open to ideas.
• Plumb this through into the retry handler.
• Add a new exception when this condition is hit, maybe class NoCapacityError(BotoCoreError), class TokenBucketThrottleError(BotoCoreError)?

Other Information

No response

Acknowledgements

• I may be able to implement this feature request
• This feature might incur a breaking change

SDK version used

All

Environment details (OS name and version, etc.)

All