Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Added extensions and Rate limiting options #314

Merged
merged 28 commits into from
Apr 19, 2024
Merged

Added extensions and Rate limiting options #314

merged 28 commits into from
Apr 19, 2024

Conversation

patduin
Copy link
Contributor

@patduin patduin commented Apr 12, 2024

  • Code is up-to-date with the main branch.
  • You've successfully built and run the tests locally.
  • There are new or updated unit tests validating the change.

📝 Description

Adding extensions module to allow options features to be added to WD.
First one of those features is a Rate Limiting option. This has come out of a lack of rate limiting options we have for the Hive Metastore (HMS). Since Waggle Dance is the proxy in front it is a good point to limit certain.
The underlying problem we're trying to address is performance issues in heavily loaded HMS + RDS causing degradation of service under spiky load.
Please have look at the respective README.md changes in this PR for more information.

🔗 Related Issues

@patduin patduin requested a review from a team as a code owner April 12, 2024 08:57
JayGreeeen
JayGreeeen reviewed Apr 12, 2024
View reviewed changes
CHANGELOG.md Outdated Show resolved Hide resolved
@patduin
Copy link
Contributor Author

patduin commented Apr 12, 2024

Looks like WaggleDanceRateLimitIntegrationTest is failing in the github actions, I will look into that (working locally).

Patrick Duin and others added 8 commits April 12, 2024 15:00
@JayGreeeen
Update waggle-dance-extensions/src/main/java/com/hotels/bdp/waggledan…
afb47be 
…ce/extensions/ExtensionBeans.java

Co-authored-by: Jay Green-Stevens <[email protected]>
@JayGreeeen
Update waggle-dance-extensions/src/main/java/com/hotels/bdp/waggledan…
2bf08a7 
…ce/extensions/ExtensionBeans.java

Co-authored-by: Jay Green-Stevens <[email protected]>
@JayGreeeen
Update waggle-dance-extensions/src/main/java/com/hotels/bdp/waggledan…
e9d0886 
…ce/extensions/client/ratelimit/BucketBandwithProvider.java

Co-authored-by: Jay Green-Stevens <[email protected]>
@JayGreeeen
Update waggle-dance-extensions/src/main/java/com/hotels/bdp/waggledan…
45470a1 
…ce/extensions/client/ratelimit/BucketBandwithProvider.java

Co-authored-by: Jay Green-Stevens <[email protected]>
@JayGreeeen
Update waggle-dance-integration-tests/src/test/java/com/hotels/bdp/wa…
91fa874 
…ggledance/extensions/WaggleDanceRateLimitIntegrationTest.java

Co-authored-by: Jay Green-Stevens <[email protected]>
@patduin
renamed and aded test
070ce28
@patduin
Adding optional prefix
fec5715
@patduin
Adding format fix
2f3e301
Update CHANGELOG.md
0121c58 
added release date.
@patduin patduin merged commit 2524256 into main Apr 19, 2024
1 of 3 checks passed
@patduin patduin deleted the feature/extensions-rate-limit branch April 19, 2024 14:19
flaming-archer pushed a commit to flaming-archer/waggle-dance that referenced this pull request Jun 26, 2024
@flaming-archer
backport Added extensions and Rate limiting options (ExpediaGroup#314)
e95c21e
@flaming-archer flaming-archer mentioned this pull request Jul 4, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JayGreeeen JayGreeeen JayGreeeen approved these changes

@DhrubajyotiSadhu DhrubajyotiSadhu DhrubajyotiSadhu approved these changes

@jmnunezizu jmnunezizu jmnunezizu approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@patduin @jmnunezizu @DhrubajyotiSadhu @JayGreeeen