Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] Replication tests are flaky (index [books-names] blocked by: [FORBIDDEN/1000/index read-only(cross-cluster-replication)) #695

Closed
dblock opened this issue Nov 25, 2024 · 2 comments · Fixed by #696
Labels
bug Something isn't working flaky test

Comments

@dblock
Copy link
Member

dblock commented Nov 25, 2024

What is the bug?

OpenSearch 2.18.0


ERROR   autofollow.yaml (tests/plugins/replication/autofollow.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/_autofollow
  /_plugins/_replication/autofollow_stats

    ERROR   EPILOGUES
        PASSED  POST /_plugins/_replication/{index}/_stop
        ERROR   DELETE /books-names (index [books-names] blocked by: [FORBIDDEN/1000/index read-only(cross-cluster-replication)];)
        PASSED  DELETE /books


ERROR   index.yaml (tests/plugins/replication/index.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/{index}/_start
  /_plugins/_replication/{index}/_update

    ERROR   CHAPTERS
        ERROR   Start replication.
            PASSED  PARAMETERS
                PASSED  index
            PASSED  REQUEST BODY
            ERROR   RESPONSE STATUS (Expected status 200, but received 400: application/json. Cant use same index again for replication. Delete the index:books-names)
            SKIPPED RESPONSE PAYLOAD BODY
            SKIPPED RESPONSE PAYLOAD SCHEMA
            SKIPPED RESPONSE OUTPUT VALUES
        SKIPPED Update settings.
    ERROR   EPILOGUES
        ERROR   POST /_plugins/_replication/{index}/_stop (No replication in progress for index:books-names)
        PASSED  DELETE /books-names
        PASSED  DELETE /books

PASSED  stats.yaml (tests/plugins/replication/stats.yaml)
WARNING Multiple paths detected, please group similar tests together and move paths not being tested to prologues or epilogues.
  /_plugins/_replication/{index}/_status
  /_plugins/_replication/leader_stats
  /_plugins/_replication/follower_stats
  /_plugins/_replication/autofollow_stats


Tested 7/5[28](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:29) paths.
Writing coverage/test-spec-coverage-2.18.0-218a289a83d99f103c6e063e594[29](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:30)0a9879b80c1b2c08[30](https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694#step:7:31)15fa9752b7a36c9db.json ...
Error: Process completed with exit code 1.

How can one reproduce the bug?

https://github.com/opensearch-project/opensearch-api-specification/actions/runs/12014616796/job/33491014455?pr=694

What is the expected behavior?

Tests to always pass.

What is your host/environment?

GHA.

Do you have any screenshots?

Screenshot 2024-11-25 at 11 46 40 AM

Do you have any additional context?

Maybe a simple retry can work around this or use books in one set of tests and movies in another?

@dblock dblock added bug Something isn't working untriaged labels Nov 25, 2024
@dblock dblock changed the title [BUG] Replication tests are flaky [BUG] Replication tests are flaky (index [books-names] blocked by: [FORBIDDEN/1000/index read-only(cross-cluster-replication)) Nov 25, 2024
@dblock
Copy link
Member Author

dblock commented Nov 25, 2024

@Tokesh maybe you want to take a look when you get a chance

@Tokesh
Copy link
Collaborator

Tokesh commented Nov 26, 2024

@dblock
Oh! I'm sorry about that. I've already prepared a PR for your review. You can check it now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working flaky test
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants