Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix unexpected reader_type kwarg error #443

Merged
merged 2 commits into from
Jan 10, 2025
Merged

Fix unexpected reader_type kwarg error #443

merged 2 commits into from
Jan 10, 2025

Conversation

yankevn
Copy link
Collaborator

@yankevn yankevn commented Jan 10, 2025

Summary

This change always pops the reader_type keyword argument, regardless of whether the encoding is identity or not.

Rationale

We have had jobs fail due to using Parquet content type and Gzip file encoding. This change fixes that bug.

Changes

  • Always pop the reader_type kwarg

Impact

All calls to s3_file_to_table will have the reader_type kwarg automatically popped, unless the reader type is daft or partial_file_download_params are defined.

Testing

Regression Risk

Low

Checklist

  • Unit tests covering the changes have been added

    • If this is a bugfix, regression tests have been added
  • E2E testing has been performed

Additional Notes

Any additional information or context relevant to this PR.

@yankevn yankevn marked this pull request as ready for review January 10, 2025 20:32
@yankevn yankevn requested a review from raghumdani January 10, 2025 21:43
Copy link
Collaborator

@raghumdani raghumdani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good.

@yankevn yankevn merged commit 3d9ae37 into main Jan 10, 2025
3 checks passed
@yankevn yankevn deleted the read_table_kwarg branch January 10, 2025 23:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants