Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fix for kafka source issue #3264 (aws glue excetion handling) #3265

Merged
merged 1 commit into from
Aug 27, 2023
Merged

Fix for kafka source issue #3264 (aws glue excetion handling) #3265

merged 1 commit into from
Aug 27, 2023

Conversation

hshardeesi
Copy link
Contributor

Description

  • Fixed Aws glue exception handling to retry only in case of AccessDenied exception.
  • Reduced Thread.sleep() to 100ms from 10s in Unit Tests requiring acknowledgements.

Issues Resolved

Resolves #3264

Testing

  • Added UT case
  • Manually tested by removing glue permissions and verified poll() is retried as expected until permissions are added back.
  • Manually injected plaintext records into Kafka topic with avro schema and verified they are skipped and stream processing is not blocked.

@asifsmohammed asifsmohammed merged commit d908c40 into opensearch-project:main Aug 27, 2023
24 checks passed
@opensearch-trigger-bot opensearch-trigger-bot bot mentioned this pull request Aug 27, 2023
Merged
opensearch-trigger-bot bot pushed a commit that referenced this pull request Aug 27, 2023
@hshardeesi @github-actions
Fix for kafka source issue #3264 (aws glue excetion handling) (#3265)
0cc489a 
(cherry picked from commit d908c40)
asifsmohammed pushed a commit that referenced this pull request Aug 27, 2023
@opensearch-trigger-bot @hshardeesi
Fix for kafka source issue #3264 (aws glue excetion handling) (#3265) (…
1cbb58c 
…#3266)

(cherry picked from commit d908c40)

Co-authored-by: Hardeep Singh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dinujoh dinujoh dinujoh approved these changes

@asifsmohammed asifsmohammed asifsmohammed approved these changes

@chenqi0805 chenqi0805 Awaiting requested review from chenqi0805 chenqi0805 is a code owner

@engechas engechas Awaiting requested review from engechas engechas is a code owner

@graytaylor0 graytaylor0 Awaiting requested review from graytaylor0 graytaylor0 is a code owner

@kkondaka kkondaka Awaiting requested review from kkondaka kkondaka is a code owner

@dlvenable dlvenable Awaiting requested review from dlvenable dlvenable is a code owner

@oeyh oeyh Awaiting requested review from oeyh oeyh is a code owner

Assignees
No one assigned
Labels
backport 2.4
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[BUG] Exception handling for aws glue schema registry
3 participants
@hshardeesi @asifsmohammed @dinujoh