We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Read and write directly to S3 to avoid need for large local disks
osmpbf supports arbitrary inputs that support Read + Send https://github.com/b-r-u/osmpbf/blob/main/src/reader.rs#L17
Read + Send
The parquet rust library has an object reader - https://arrow.apache.org/rust/parquet/arrow/async_reader/struct.ParquetObjectReader.html but doesn't appear to have an object writer. Can probably implement an S3AsyncWriter conforming to https://arrow.apache.org/rust/parquet/arrow/async_writer/trait.AsyncFileWriter.html
https://docs.rs/object_store/latest/object_store/aws/struct.AmazonS3.html
The text was updated successfully, but these errors were encountered:
This has been implemented for writing out to S3
Sorry, something went wrong.
No branches or pull requests
Read and write directly to S3 to avoid need for large local disks
osmpbf supports arbitrary inputs that support
Read + Send
https://github.com/b-r-u/osmpbf/blob/main/src/reader.rs#L17
The parquet rust library has an object reader - https://arrow.apache.org/rust/parquet/arrow/async_reader/struct.ParquetObjectReader.html but doesn't appear to have an object writer. Can probably implement an S3AsyncWriter conforming to https://arrow.apache.org/rust/parquet/arrow/async_writer/trait.AsyncFileWriter.html
https://docs.rs/object_store/latest/object_store/aws/struct.AmazonS3.html
The text was updated successfully, but these errors were encountered: