Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

catch go mod download overwriting changes to vendor #3008

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Conversation

jaxesn
Copy link
Member

@jaxesn jaxesn commented May 10, 2024

Issue #, if available:

Description of changes:

From pretty much the beginning of eks-d and then eks-a after it, we have run go mod download before building. Some upstream projects, such as k/k, actually check in their vendor folder so in those cases we do not technically need to run go mod vendor (and in the case of k/k we actually do not, never have) but we do anyway to be consistent and avoid having to have made a case by case decision. This was also important for reproducible builds, there is some oddities with go dependencies that have been download vs checked in and pulled. I have previous PRs which talk about this. Always running go mod vendor ensured all projects would build regardless of if the vendor dir was checked in or not.

About a year ago, the harbor build in eks-a was updated with a patch that actually did patch something in the vendor directory. At that time we added GO_MODS_VENDORED in aws/eks-anywhere-build-tooling#2078 to allow projects to skip running go mod vendor.

Recently a patch was added to CCM for versions 1.28+ which 1) adds the vendor directory and 2) patches something in that directory. As is, this patch is not actually taking affect because it gets overwritten when the go mod vendor call is ran. The fix, which @markapruett is taking, is to port over the skipping mechanism from eks-a to eks-d and setting GO_MODS_VENDORED to true for the ccm build.

While discussing with Mark this nuance, I was thinking of ways we could catch this and avoid the silent "failure". This is my attempt, if the vendor dir exists before running go mod vendor then use git to determine if anything changed in that folder due to go mod vendor, if so, then error.

With this change it will require that any time we patch a go.mod/sum file for a project with vendored deps, we need to make sure the patch includes the vendor folder updates, otherwise git will see a difference and trigger this error. The fix should be pretty easy/obvious in this case.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

@jaxesn jaxesn changed the title TESTING: catch go mod download overwriting changes to vendor catch go mod download overwriting changes to vendor May 13, 2024
@jaxesn jaxesn force-pushed the jgw/catch-overwrite-vendor branch from 3a8de5d to a6a15bc Compare May 13, 2024 16:33
jaxesn added a commit to aws/eks-anywhere-build-tooling that referenced this pull request Jun 3, 2024
harbor vendors its go mods and there used to be a patch that changed files in the vendor directory. setting this flag was necessary to avoid losing the change since go mod download happened after patching. see aws/eks-distro#3008 for more info. I will send another PR adding a similar check to this repo to catch this kind of in the future automatically.
eks-distro-bot pushed a commit to aws/eks-anywhere-build-tooling that referenced this pull request Jun 3, 2024
* harbor no longer needs to skip go mod download

harbor vendors its go mods and there used to be a patch that changed files in the vendor directory. setting this flag was necessary to avoid losing the change since go mod download happened after patching. see aws/eks-distro#3008 for more info. I will send another PR adding a similar check to this repo to catch this kind of in the future automatically.

* Update CHECKSUMS
@eks-distro-bot
Copy link
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign torredil for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@rcrozean
Copy link
Member

/retest

@rcrozean
Copy link
Member

/retest aws-cloud-controller-manager-1-26-presubmit

@eks-distro-bot
Copy link
Collaborator

@rcrozean: The /retest command does not accept any targets.
The following commands are available to trigger required jobs:

  • /test aws-cloud-controller-manager-1-25-presubmit
  • /test aws-cloud-controller-manager-1-26-presubmit
  • /test aws-cloud-controller-manager-1-27-presubmit
  • /test aws-cloud-controller-manager-1-28-presubmit
  • /test aws-cloud-controller-manager-1-29-presubmit
  • /test aws-cloud-controller-manager-1-30-presubmit
  • /test aws-iam-authenticator-1-25-presubmit
  • /test aws-iam-authenticator-1-26-presubmit
  • /test aws-iam-authenticator-1-27-presubmit
  • /test aws-iam-authenticator-1-28-presubmit
  • /test aws-iam-authenticator-1-29-presubmit
  • /test aws-iam-authenticator-1-30-presubmit
  • /test cni-plugins-1-25-presubmit
  • /test cni-plugins-1-26-presubmit
  • /test cni-plugins-1-27-presubmit
  • /test cni-plugins-1-28-presubmit
  • /test cni-plugins-1-29-presubmit
  • /test cni-plugins-1-30-presubmit
  • /test coredns-1-25-presubmit
  • /test coredns-1-26-presubmit
  • /test coredns-1-27-presubmit
  • /test coredns-1-28-presubmit
  • /test coredns-1-29-presubmit
  • /test coredns-1-30-presubmit
  • /test eks-distro-docs-presubmit
  • /test etcd-1-25-presubmit
  • /test etcd-1-26-presubmit
  • /test etcd-1-27-presubmit
  • /test etcd-1-28-presubmit
  • /test etcd-1-29-presubmit
  • /test etcd-1-30-presubmit
  • /test external-attacher-1-25-presubmit
  • /test external-attacher-1-26-presubmit
  • /test external-attacher-1-27-presubmit
  • /test external-attacher-1-28-presubmit
  • /test external-attacher-1-29-presubmit
  • /test external-attacher-1-30-presubmit
  • /test external-provisioner-1-25-presubmit
  • /test external-provisioner-1-26-presubmit
  • /test external-provisioner-1-27-presubmit
  • /test external-provisioner-1-28-presubmit
  • /test external-provisioner-1-29-presubmit
  • /test external-provisioner-1-30-presubmit
  • /test external-resizer-1-25-presubmit
  • /test external-resizer-1-26-presubmit
  • /test external-resizer-1-27-presubmit
  • /test external-resizer-1-28-presubmit
  • /test external-resizer-1-29-presubmit
  • /test external-resizer-1-30-presubmit
  • /test external-snapshotter-1-25-presubmit
  • /test external-snapshotter-1-26-presubmit
  • /test external-snapshotter-1-27-presubmit
  • /test external-snapshotter-1-28-presubmit
  • /test external-snapshotter-1-29-presubmit
  • /test external-snapshotter-1-30-presubmit
  • /test kops-build-1-25-presubmits
  • /test kops-build-1-26-presubmits
  • /test kops-build-1-27-presubmits
  • /test kops-build-1-28-presubmits
  • /test kops-build-1-29-presubmits
  • /test kops-build-1-30-presubmits
  • /test kubernetes-1-23-presubmit
  • /test kubernetes-1-23-test-presubmit
  • /test kubernetes-1-24-presubmit
  • /test kubernetes-1-24-test-presubmit
  • /test kubernetes-1-25-presubmit
  • /test kubernetes-1-25-test-presubmit
  • /test kubernetes-1-26-presubmit
  • /test kubernetes-1-26-test-presubmit
  • /test kubernetes-1-27-presubmit
  • /test kubernetes-1-27-test-presubmit
  • /test kubernetes-1-28-presubmit
  • /test kubernetes-1-28-test-presubmit
  • /test kubernetes-1-29-presubmit
  • /test kubernetes-1-29-test-presubmit
  • /test kubernetes-1-30-presubmit
  • /test kubernetes-1-30-test-presubmit
  • /test kubernetes-release-1-25-presubmit
  • /test kubernetes-release-1-26-presubmit
  • /test kubernetes-release-1-27-presubmit
  • /test kubernetes-release-1-28-presubmit
  • /test kubernetes-release-1-29-presubmit
  • /test kubernetes-release-1-30-presubmit
  • /test livenessprobe-1-25-presubmit
  • /test livenessprobe-1-26-presubmit
  • /test livenessprobe-1-27-presubmit
  • /test livenessprobe-1-28-presubmit
  • /test livenessprobe-1-29-presubmit
  • /test livenessprobe-1-30-presubmit
  • /test main-presubmit
  • /test metrics-server-1-25-presubmit
  • /test metrics-server-1-26-presubmit
  • /test metrics-server-1-27-presubmit
  • /test metrics-server-1-28-presubmit
  • /test metrics-server-1-29-presubmit
  • /test metrics-server-1-30-presubmit
  • /test node-driver-registrar-1-25-presubmit
  • /test node-driver-registrar-1-26-presubmit
  • /test node-driver-registrar-1-27-presubmit
  • /test node-driver-registrar-1-28-presubmit
  • /test node-driver-registrar-1-29-presubmit
  • /test node-driver-registrar-1-30-presubmit

Use /test all to run the following jobs that were automatically triggered:

  • aws-cloud-controller-manager-1-25-presubmit
  • aws-cloud-controller-manager-1-26-presubmit
  • aws-cloud-controller-manager-1-27-presubmit
  • aws-cloud-controller-manager-1-28-presubmit
  • aws-cloud-controller-manager-1-29-presubmit
  • aws-cloud-controller-manager-1-30-presubmit
  • aws-iam-authenticator-1-25-presubmit
  • aws-iam-authenticator-1-26-presubmit
  • aws-iam-authenticator-1-27-presubmit
  • aws-iam-authenticator-1-28-presubmit
  • aws-iam-authenticator-1-29-presubmit
  • aws-iam-authenticator-1-30-presubmit
  • cni-plugins-1-25-presubmit
  • cni-plugins-1-26-presubmit
  • cni-plugins-1-27-presubmit
  • cni-plugins-1-28-presubmit
  • cni-plugins-1-29-presubmit
  • cni-plugins-1-30-presubmit
  • coredns-1-25-presubmit
  • coredns-1-26-presubmit
  • coredns-1-27-presubmit
  • coredns-1-28-presubmit
  • coredns-1-29-presubmit
  • coredns-1-30-presubmit
  • etcd-1-25-presubmit
  • etcd-1-26-presubmit
  • etcd-1-27-presubmit
  • etcd-1-28-presubmit
  • etcd-1-29-presubmit
  • etcd-1-30-presubmit
  • external-attacher-1-25-presubmit
  • external-attacher-1-26-presubmit
  • external-attacher-1-27-presubmit
  • external-attacher-1-28-presubmit
  • external-attacher-1-29-presubmit
  • external-attacher-1-30-presubmit
  • external-provisioner-1-25-presubmit
  • external-provisioner-1-26-presubmit
  • external-provisioner-1-27-presubmit
  • external-provisioner-1-28-presubmit
  • external-provisioner-1-29-presubmit
  • external-provisioner-1-30-presubmit
  • external-resizer-1-25-presubmit
  • external-resizer-1-26-presubmit
  • external-resizer-1-27-presubmit
  • external-resizer-1-28-presubmit
  • external-resizer-1-29-presubmit
  • external-resizer-1-30-presubmit
  • external-snapshotter-1-25-presubmit
  • external-snapshotter-1-26-presubmit
  • external-snapshotter-1-27-presubmit
  • external-snapshotter-1-28-presubmit
  • external-snapshotter-1-29-presubmit
  • external-snapshotter-1-30-presubmit
  • kubernetes-1-23-presubmit
  • kubernetes-1-24-presubmit
  • kubernetes-1-25-presubmit
  • kubernetes-1-26-presubmit
  • kubernetes-1-27-presubmit
  • kubernetes-1-28-presubmit
  • kubernetes-1-29-presubmit
  • kubernetes-1-30-presubmit
  • kubernetes-release-1-25-presubmit
  • kubernetes-release-1-26-presubmit
  • kubernetes-release-1-27-presubmit
  • kubernetes-release-1-28-presubmit
  • kubernetes-release-1-29-presubmit
  • kubernetes-release-1-30-presubmit
  • livenessprobe-1-25-presubmit
  • livenessprobe-1-26-presubmit
  • livenessprobe-1-27-presubmit
  • livenessprobe-1-28-presubmit
  • livenessprobe-1-29-presubmit
  • livenessprobe-1-30-presubmit
  • metrics-server-1-25-presubmit
  • metrics-server-1-26-presubmit
  • metrics-server-1-27-presubmit
  • metrics-server-1-28-presubmit
  • metrics-server-1-29-presubmit
  • metrics-server-1-30-presubmit
  • node-driver-registrar-1-25-presubmit
  • node-driver-registrar-1-26-presubmit
  • node-driver-registrar-1-27-presubmit
  • node-driver-registrar-1-28-presubmit
  • node-driver-registrar-1-29-presubmit
  • node-driver-registrar-1-30-presubmit

In response to this:

/retest aws-cloud-controller-manager-1-26-presubmit

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@rcrozean
Copy link
Member

/test aws-cloud-controller-manager-1-26-presubmit

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants