Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use blobs in integration specs instead of bucket artifacts #2582

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

ystros
Copy link
Contributor

@ystros ystros commented Nov 8, 2024

What is this change about?

We are making an effort to consolidate or eliminate buckets from various pipelines across the ecosystem. Currently the integration specs download several artifacts from S3 buckets. Some of the artifacts being downloaded did not represent the current version used in the Director, such as the NATS server.

Instead of downloading these artifacts from out-of-date buckets, the binaries are extracted from the Director blobs directly. The config server binary must now be provided via environment variables because it is not a blob, and is only used in testing.

One notable change is that since only Linux versions of binaries are included in the current blobs, it is not possible to run the integration specs on a Mac. The previous code appears to have allows running on a Mac, though I personally could not get it to work.

What tests have you run against this PR?

  • ✅ Integration specs via bundle exec rake fly:integration

@ystros ystros force-pushed the use-blobs-in-integration branch 3 times, most recently from 2c2369e to 419a48c Compare November 12, 2024 20:06
@aramprice aramprice force-pushed the use-blobs-in-integration branch 2 times, most recently from 48a038e to 5cee757 Compare November 16, 2024 01:31
@ystros ystros force-pushed the use-blobs-in-integration branch 6 times, most recently from 4269c3f to 0e628e4 Compare November 21, 2024 00:58
@ystros ystros marked this pull request as ready for review November 21, 2024 01:08
@jpalermo jpalermo requested review from a team, aramprice and a-hassanin and removed request for a team November 21, 2024 16:01
Copy link
Member

@aramprice aramprice left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎉

The version of verify-multidigest being downloaded from the bucket is
hardcoded and many versions behind the one currently used by the real
BOSH Director.

This removes the ability to run integration specs on a Mac because the
blob only includes a compiled linux binary.
Also make spacing consistent
The nats-server binary replaced the legacy forked gnatsd binary in 2022:
#2391

However, the integration specs were still using an old version of gnatsd
from an S3 bucket.

Now, the integration specs use the current nats-server blob directly.
This required allowing all users connecting to the NATS server to
subscribe and publish to any channel, because the integration specs
require an admin-level user but the nats-sync process does not allow
custom users (only the Director, Health Monitor, and individual agents).
* config-server and bosh-agent repos are now required to be cloned at
  the same level as the bosh repo
* Services and binaries are only compiled once

Signed-off-by: Aram Price <[email protected]>
@ystros
Copy link
Contributor Author

ystros commented Nov 21, 2024

Latest force push rebases against main branch again, and rewords the nats-server related commit (previously tried to run nats-sync locally but had to abandon that approach, but the commit still referenced running it)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Pending Merge | Prioritized
Development

Successfully merging this pull request may close these issues.

2 participants