Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DEVPROD-6266 Fix duplication for --repeat-patch #7892

Merged
merged 11 commits into from
May 24, 2024

Conversation

malikchaya2
Copy link
Member

DEVPROD-6266

Description
--repeat-patch was relying on BuildProjectTVPairs to create the variant tasks for patches. However, that resulted in a previous patch of: {bv1: t1, bv2: t2} ending up with {bv1: t1, bv1: t2, bv2: t1, bv2: t2}. This fixes that by reusing the variant tasks from the previous patch and applying the filtering on top of that instead of just on the tasks.

Testing
Existing unit tests and testing on staging.

@evergreen-ci evergreen-ci deleted a comment from evergreen-ci-prod bot May 17, 2024
@evergreen-ci evergreen-ci deleted a comment from evergreen-ci-prod bot May 17, 2024
@malikchaya2 malikchaya2 requested a review from a team May 17, 2024 15:53
units/patch_intent.go Outdated Show resolved Hide resolved
units/patch_intent.go Outdated Show resolved Hide resolved
// we can rely on them being there from the previous patch.
if failedOnly {
failedPlusNeeded, err := AddTasksNeededByFailedForReuse(failedTasks, failedTaskDisplayNames, project, vt)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

could we change "needed" language to dependency language? This is more consistent

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It needs both dependencies and tasks from single host task groups

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Perhaps "addDependenciesAndTaskGroups" then? I think right now the function includes details that really aren't relevant to the usage (that only failed tasks are passed in, and that its called by the reuse function).

Also, should we change the error message to say "dependencies and task groups"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks like the first half of this message got missed -- I'd still suggest changing the function title and variables to be more specific about what's being done rather than why its being called.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so strange, I did the change but it was somehow lost. My apologies, pushed it now.

units/patch_intent.go Outdated Show resolved Hide resolved
}

// AddTasksNeededByFailedForReuse add tasks that failed tasks need to run including dependencies and tasks from single host task groups.
func AddTasksNeededByFailedForReuse(failedTasks []task.Task, failedTaskDisplayNames []string, project *model.Project, vt patch.VariantTasks) ([]string, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can includeDependencies be used here instead, if we just convert the failedTasks to tv pairs?

(Also, i think this function can be private)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I made it private. It also adds all tasks in a single host task group, so includeDependencies would only cover part of it.

func setToFilteredTasks(patchDoc, reusePatch *patch.Patch, project *model.Project, failedOnly bool) error {
activatedTasks, err := filterToActiveForReuse(reusePatch, project)
if err != nil {
return errors.Wrap(err, "getting dependencies for activated tasks")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this error message doesn't seem right

units/patch_intent.go Show resolved Hide resolved
units/patch_intent.go Show resolved Hide resolved
units/patch_intent.go Outdated Show resolved Hide resolved
}

patchDoc.Tasks = failedTasks
if failedOnly {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this if statement necessary since filteredTasks = allFailedPlusNeededTasks at 691?

@malikchaya2 malikchaya2 requested a review from ablack12 May 22, 2024 21:19
// we can rely on them being there from the previous patch.
if failedOnly {
failedPlusNeeded, err := AddTasksNeededByFailedForReuse(failedTasks, failedTaskDisplayNames, project, vt)
if err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah. Perhaps "addDependenciesAndTaskGroups" then? I think right now the function includes details that really aren't relevant to the usage (that only failed tasks are passed in, and that its called by the reuse function).

Also, should we change the error message to say "dependencies and task groups"

units/patch_intent.go Show resolved Hide resolved
units/patch_intent.go Outdated Show resolved Hide resolved
units/patch_intent.go Outdated Show resolved Hide resolved
units/patch_intent_test.go Outdated Show resolved Hide resolved
units/patch_intent_test.go Outdated Show resolved Hide resolved
return nil
}

func filterToActiveForReuse(reusePatch *patch.Patch) ([]task.Task, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be moved to task_db.go (doesn't need to reference reuse specifically, it's really just byActivatedTasksWithoutDisplay or something)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moving it creates an import cycle. It's specific enough to this use case that I think it's okay here

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this comment got missed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I forgot to hit enter to submit all of my comments, sorry

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that's only because you're passing in the whole patch -- you can change this to just pass in the ID and then it should be fine

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(may want to do that anyway)

@malikchaya2 malikchaya2 requested a review from ablack12 May 24, 2024 14:04
Copy link
Contributor

@ablack12 ablack12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM with two missed comments (forgot to click save before publishing review, this is the other: #7892 (comment))

return nil
}

func filterToActiveForReuse(reusePatch *patch.Patch) ([]task.Task, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this comment got missed

@malikchaya2 malikchaya2 merged commit a8d4c90 into evergreen-ci:main May 24, 2024
8 checks passed
@malikchaya2 malikchaya2 deleted the DEVPROD-6266 branch May 24, 2024 20:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants