Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Release] v0.3.0 #61

Merged
merged 7 commits into from
Oct 15, 2024
Merged

[Release] v0.3.0 #61

merged 7 commits into from
Oct 15, 2024

Conversation

8naama
Copy link
Contributor

@8naama 8naama commented Oct 9, 2024

Description

this pr closes #49 closes #50, closes #52, closes #58, closes #59

  • Move all subscription filter actions to the 'background'
    • Rename eventbridge-lambda >> log-group-events-lambda
    • Move subscription filter functions out of common and cfn-lambda into log-group-events-lambda
    • Create new custom SubscriptionFilterEvent event type, which is structured based on the EventBridge events format
    • Generate individual roles for per Lambda instead of a shared role, to ensure each has only necessary permissions
  • Use goroutines to parallelize the subscription filters actions
    • Add retry mechanism upon ThrottlingException: Rate exceeded error
  • Improve the unit tests
    • Add mocks to AWS services
  • Add prefixes support for CustomLogGroups via wildcard (e.g, prefix*)
  • Update readme

What type of PR is this?

(check all applicable)

  • 🍕 Feature
  • 🐛 Bug Fix
  • 🧑‍💻 Code Refactor
  • 🔥 Performance Improvements
  • ✅ Test
  • 🤖 Build / CI
  • ⏩ Revert

Added tests?

  • 👍 yes
  • 🙅 no, because they aren't needed
  • 🙋 no, because I need help from somebody

@8naama 8naama requested review from bardabun and yotamloe October 9, 2024 13:34
}

type SubscriptionFilterEvent struct {
Detail Detail `json:"detail"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is the advantage of having this struct instead of using Detail Detail

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's making it more ordered and clear, because without it the NewSubscriptionFilterEvent function looks a bit messy:

func NewSubscriptionFilterEvent(params RequestParameters) struct {
	Detail Detail `json:"detail"`
} {
	return struct {
		Detail Detail `json:"detail"`
	}{
		Detail: Detail{
			EventName:         "SubscriptionFilterEvent",
			RequestParameters: params,
		},
	}
}

what do you think?

@8naama 8naama requested a review from yotamloe October 13, 2024 12:05
added := make([]string, 0, len(logGroups))
var result *multierror.Error

var wg sync.WaitGroup
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not a part of the struct like the mutex?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

updateSubscriptionFilters method performs both add and delete operations, and we run each of them separately to manage the concurrent calls to AWS (because for large amount of log groups it may be high in each operation). So running them independently helps reduce the overall concurrency, which means each needs it's own wait group

@8naama 8naama merged commit 421620a into master Oct 15, 2024
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants