Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AVRO-4086][C++] Fix missing data file reader close handle on windows #3230

Merged
merged 7 commits into from
Dec 24, 2024

Conversation

glywk
Copy link
Contributor

@glywk glywk commented Oct 29, 2024

What is the purpose of the change

On windows datafile reader does not close resource handle. It blocks file deletion, fixing AVRO-4086

Verifying this change

This change is already covered by existing tests, such as DataFileTests and CommonSchemaTests.
I have also add tests on manipulation after closed reader.

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not applicable)

@github-actions github-actions bot added the C++ Pull Requests for C++ binding label Oct 29, 2024
Copy link
Contributor

@thiru-mg thiru-mg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this change would break the symmetry between DataFileReaderBase and DataFileWriterBase. To remain symmetric we can simply reset() stream_ in close() instead of replacing it with ClosedInputStream.
Any attempt to read from the file after closing it should ideally throw an exception. But std::unique_ptr specification make it undefined behavior. But then, the behavior is the same if one tries to write into DataFileWriterBase after closing it.

glywk and others added 5 commits October 29, 2024 11:45
Python 3.12 needs `--break-system-packages`

```
Run python3 -m pip install --break-system-packages --upgrade pip setuptools tox
  python3 -m pip install --break-system-packages --upgrade pip setuptools tox
  python3 -m pip install --break-system-packages python-snappy zstandard
  shell: /usr/bin/bash -e {0}

Usage:
  /usr/bin/python3 -m pip install [options] <requirement specifier> [package-index-options] ...
  /usr/bin/python3 -m pip install [options] -r <requirements file> [package-index-options] ...
  /usr/bin/python3 -m pip install [options] [-e] <vcs project url> ...
  /usr/bin/python3 -m pip install [options] [-e] <local project path> ...
  /usr/bin/python3 -m pip install [options] <archive url/path> ...
```

Signed-off-by: Martin Tzvetanov Grigorov <[email protected]>
@github-actions github-actions bot added the build label Oct 29, 2024
@glywk
Copy link
Contributor Author

glywk commented Oct 30, 2024

I think this change would break the symmetry between DataFileReaderBase and DataFileWriterBase. To remain symmetric we can simply reset() stream_ in close() instead of replacing it with ClosedInputStream. Any attempt to read from the file after closing it should ideally throw an exception. But std::unique_ptr specification make it undefined behavior. But then, the behavior is the same if one tries to write into DataFileWriterBase after closing it.

Thank you for your analysis. Generally, I prefer to keep object instance in valid state in any lifetime step. But I can remove ClosedInputStream and simply reset() stream_ without make it undefined behaviour. And symmetry is respected between DataFileReaderBase and DataFileWriterBase.

@glywk
Copy link
Contributor Author

glywk commented Nov 13, 2024

Hi, does it miss something else to merge?

Copy link
Contributor

@thiru-mg thiru-mg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great.

@thiru-mg thiru-mg merged commit 3621ef2 into apache:main Dec 24, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build C++ Pull Requests for C++ binding
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants