Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(topic): performance for read and creation #1393

Closed
wants to merge 1 commit into from

Conversation

ivan-savciuc
Copy link
Contributor

About this change—what it does

Fix Kafka Topic create and read performance

@ivan-savciuc ivan-savciuc added the no changelog No changelog entries are required for this PR label Oct 13, 2023
@ivan-savciuc ivan-savciuc requested a review from a team October 13, 2023 09:39
@ivan-savciuc ivan-savciuc force-pushed the ivans-fix-kafka-creation branch from 2aa4078 to 456dac3 Compare October 13, 2023 09:44
@byashimov
Copy link
Contributor

byashimov commented Oct 13, 2023

@ivan-savciuc hey is this for v4 or v3, or both?
Currently it aimed to v5 (main).

byashimov
byashimov previously approved these changes Oct 13, 2023
Copy link
Contributor

@byashimov byashimov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@ivan-savciuc ivan-savciuc changed the base branch from main to v4 October 16, 2023 07:04
@ivan-savciuc ivan-savciuc changed the base branch from v4 to main October 16, 2023 07:04
@ivan-savciuc ivan-savciuc dismissed byashimov’s stale review October 16, 2023 07:04

The base branch was changed.

// No error means topic exists
if err == nil {
client := m.(*aiven.Client)
if isTopicExists(ctx, client, project, serviceName, topicName) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider adding a comment above this block to briefly explain its purpose for clarity

@@ -201,3 +201,11 @@ func DeleteTopicFromCache(projectName, serviceName, topicName string) {
}
t.Unlock()
}

// GetFullQueue retrieves a topics queue
func (t *kafkaTopicCache) GetFullQueue(projectName, serviceName string) []string {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ensure the method name accurately represents its purpose; If the function retrieves a specific type of queue or has a unique behavior, be more explicit in the name

t.RLock()
defer t.RUnlock()

return t.inQueue[projectName+serviceName]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider handling a scenario where the key might not exist in the inQueue map; directly accessing a non-existent key will return a nil slice, which might be unexpected for the function caller

"golang.org/x/exp/slices"
)

var onceForService sync.Map
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

consider renaming the variable to be more descriptive, like onceCheckTopicForService or something similar


var err error
once, _ := onceForService.LoadOrStore(project+"/"+serviceName, new(sync.Once))
once.(*sync.Once).Do(func() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add comments for better clarity, especially on the significance of using sync.Once

c.SetV1List(project, serviceName, list)
})

if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this approach can be confusing: if possible, consider handling the error within the once.Do block, or refactoring the approach to make the flow clearer

return false
}

if slices.Contains(c.GetV1List(project, serviceName), topic) || slices.Contains(c.GetFullQueue(project, serviceName), topic) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

for better performance, if the topic is found in the first c.GetV1List check, there's no need to check in the second c.GetFullQueue call: you can return true immediately

@Serpentiel Serpentiel changed the title fix kafka topic creation and read performance fix(topic): performance for read and creation Oct 17, 2023
@ivan-savciuc
Copy link
Contributor Author

Converting this patch for v4 instead of v5 (current master), will open another RP

@Serpentiel Serpentiel added the invalid This doesn't seem right label Oct 23, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
invalid This doesn't seem right no changelog No changelog entries are required for this PR
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants