Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Certstore system tests fail if certificates are not installed. #3984

Closed
FAlbertDev opened this issue Apr 8, 2024 · 2 comments
Closed

Certstore system tests fail if certificates are not installed. #3984

FAlbertDev opened this issue Apr 8, 2024 · 2 comments
Milestone

Comments

@FAlbertDev
Copy link
Collaborator

After building Botan on Windows, the Botan certstore system tests (src/tests/test_certstor_system.cpp) failed on my system. The root certificate (D-TRUST Root Class 3 CA 2 EV 2009) was somehow not installed on my system, which caused this test to fail. The test implicitly asserts that some well-known CA certificates are installed. There are various options for how to handle this issue:

  1. We can ignore it and hope my system was the only one where the respective certificate was not installed. Maybe we could at least add a comment to the test that it fails if the certificate is not found on the system.
  2. We skip the test if the certificate is not found instead of failing it.
  3. Same as 2, but we add a flag to botan-test to avoid that we skip this test in CI.

In my opinion, it's not very urgent since no one has complained so far. It could be frustrating for a user when installing Botan, though.

@reneme
Copy link
Collaborator

reneme commented Apr 8, 2024

The dependence on "certain" well-known root certificates in the test has always been somewhat an Achilles heel, unfortunately.

I'm in favor of (3), in a sense that we "somehow" flag the tests that assume the trust in certain root certs as "may-fail" in the test setup. AFAIK Botan's test system doesn't have this functionality, yet.

I'm envisioning something like:

auto test_that_assumes_existence_of_some_root() {
   Test::Result result("Test OS certstor, MAY_FAIL);
   /* ... test all the things ... */
   return result
}

... if the test fails, it should print a warning by default but not set the return code of ./botan-test.exe. Except the tests were invoked with ./botan-test.exe --strict (or similar).

@reneme reneme added this to the Botan 3.5.0 milestone Apr 8, 2024
@randombit randombit modified the milestones: Botan 3.5.0, Botan 3.6.0 Jul 9, 2024
@reneme
Copy link
Collaborator

reneme commented Oct 7, 2024

This has been mitigated by #4280 which adapted the test to rely on a handful of certificates and fail only if none of them was found. Obviously, that doesn't fully fix this issue, but hopefully mitigates it for enough people. Closing...

@reneme reneme closed this as completed Oct 7, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants