Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[CSV Reader] Set better buffer/chunk size defaults for local reader. #1512

Closed
clarkzinzow opened this issue Oct 20, 2023 · 1 comment
Closed

[CSV Reader] Set better buffer/chunk size defaults for local reader. #1512

clarkzinzow opened this issue Oct 20, 2023 · 1 comment
Labels
data-catalogs Related to support for Data Catalogs

Comments

@clarkzinzow
Copy link
Contributor

We currently use the same default buffer/chunk size configurations for both the cloud reader and the local reader, but they are currently tailored to the cloud reader. We should do some benchmarking for the local reader and set some more tailored local reader defaults.
@clarkzinzow clarkzinzow added the data-catalogs Related to support for Data Catalogs label Oct 20, 2023
@github-project-automation github-project-automation bot moved this to On Deck in Daft-OSS Oct 20, 2023
@jaychia jaychia moved this from On Deck to Traige in Daft-OSS Jan 10, 2024
@colin-ho
Copy link
Contributor

Closed with new local csv reader: #3055

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
data-catalogs Related to support for Data Catalogs
Projects
No open projects
Daft-OSS
Status: Needs Triage
Milestone
No milestone
Development

No branches or pull requests
2 participants
@clarkzinzow @colin-ho