Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Translator Job Record #58

Merged
merged 7 commits into from
Apr 18, 2024
Merged

Conversation

samleeflang
Copy link
Contributor

@samleeflang samleeflang commented Apr 8, 2024

  • Added the Translator Job Record
    • At the start it will insert a new Job Record
    • At the finish it will fill the record with an end status (COMPLETED/FAILED), an end time and a processed records. When it was FAILED it will also insert an ERROR_CODE which in this case will be DISSCO_EXCEPTION
  • Changed a bit of login in the translators
    • They now produce a result which can be COMPLETED or FAILED and they count how many records they have successfully processed
    • Changed a bit of exception handling. It stays tricky, but there are some exception which are critical and will fail the translation. Others, mainly at record level, are not critical. This is because some parsing may fail for that particular record, but we probably still want all other records ingested. Might need further refinement in the future.
      https://naturalis.atlassian.net/browse/DD-1012
      https://naturalis.atlassian.net/browse/DD-954

@samleeflang samleeflang requested a review from southeo April 8, 2024 15:24
Copy link
Contributor

@southeo southeo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very solid improvements. This will be useful. Have a few comments on next steps in design. 🌵

Copy link

@samleeflang samleeflang merged commit c3810a2 into main Apr 18, 2024
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants