-
Notifications
You must be signed in to change notification settings - Fork 516
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Change integration testing #3194
Conversation
2072bfc
to
40deadb
Compare
The remaining warnings aren't an issue. Using http inside the test container network. |
fad5fd7
to
298d418
Compare
Signed-off-by: jamshale <[email protected]>
298d418
to
9fbfa7a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Left a couple minor comments, but overall looks good to me.
Still wondering about executing ALL interop tests every time a PR gets opened/updated/reopened as it is potentially a big overhead and use of executor time. We could maybe create a matrix mapping code paths with test tags and dynamically specify which sets to execute (e.g.: only issue-credential and not connections) if nothing has changed, but this would definitely be for a future improvement 🤔
I do think that's a good idea and I could do it in an upcoming PR. I think we should only run a subset that only includes most of the happy path flows in AATH and the new scenarios. They will catch a vast majority of mistakes. Then for nightly and release runs we could do a more thorough testing suite on off hours. The ultimate way would be to only run tests which matched the code changes. I don't think there's really a path forward for this with how complicated the codebase can be. |
Signed-off-by: jamshale <[email protected]>
Signed-off-by: jamshale <[email protected]>
@esune I fixed the spelling mistakes and change the acapy-controller dependency in scenarios to target a release. I'm going to create 3 tickets for additional work.
|
Quality Gate failedFailed conditions |
* Change integration testing plan Signed-off-by: jamshale <[email protected]> * Fix grammer and spelling Signed-off-by: jamshale <[email protected]> * Use tag release for acapy-controller dependency Signed-off-by: jamshale <[email protected]> --------- Signed-off-by: jamshale <[email protected]>
* Change integration testing plan Signed-off-by: jamshale <[email protected]> * Fix grammer and spelling Signed-off-by: jamshale <[email protected]> * Use tag release for acapy-controller dependency Signed-off-by: jamshale <[email protected]> --------- Signed-off-by: jamshale <[email protected]>
Changes the automated integration testing so that there is 3 different types of tests.
The scenario testing could probably be extended to have for e2e tests like what happens in the aries-acapy-tools repo. I imagine being able to test things like shut downs and upgrades and changing configurations from this setup.
TODO:
dev
so we can wipe the env more often?