Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AVRO-2511: add case for sync #2392

Merged
merged 3 commits into from
Sep 20, 2023
Merged

AVRO-2511: add case for sync #2392

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
3d114b4
AVRO-2511: add case for sync
clesaec Jul 25, 2023
e336214
AVRO-2511: add javadoc
clesaec Sep 20, 2023
b547524
AVRO-2511: with spotless apply
clesaec Sep 20, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 8 additions & 5 deletions lang/java/avro/src/main/java/org/apache/avro/file/DataFileWriter.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import java.io.BufferedOutputStream;
import java.io.Closeable;
import java.io.File;
import java.io.FileOutputStream;
import java.io.FilterOutputStream;
import java.io.Flushable;
import java.io.IOException;
Expand Down Expand Up @@ -51,7 +52,7 @@
* <i>blocks</i>. A synchronization marker is written between blocks, so that
* files may be split. Blocks may be compressed. Extensible metadata is stored
* at the end of the file. Files may be appended to.
*
*
* @see DataFileReader
*/
public class DataFileWriter<D> implements Closeable, Flushable {
Expand Down Expand Up @@ -181,7 +182,7 @@ public DataFileWriter<D> create(Schema schema, OutputStream outs, byte[] sync) t
* sync marker is written. By default, the writer will flush the buffer each
* time a sync marker is written (if the block size limit is reached or the
* {@linkplain #sync()} is called.
*
*
* @param flushOnEveryBlock - If set to false, this writer will not flush the
* block to the stream until {@linkplain #flush()} is
* explicitly called.
Expand Down Expand Up @@ -211,7 +212,7 @@ public DataFileWriter<D> appendTo(File file) throws IOException {
/**
* Open a writer appending to an existing file. <strong>Since 1.9.0 this method
* does not close in.</strong>
*
*
* @param in reading the existing file.
* @param out positioned at the end of the existing file.
*/
Expand Down Expand Up @@ -304,7 +305,7 @@ public AppendWriteException(Exception e) {

/**
* Append a datum to the file.
*
*
* @see AppendWriteException
*/
public void append(D datum) throws IOException {
Expand Down Expand Up @@ -365,7 +366,7 @@ private void writeIfBlockFull() throws IOException {
* at compression level 7. If <i>recompress</i> is false, blocks will be copied
* without changing the compression level. If true, they will be converted to
* the new compression level.
*
*
* @param otherFile
* @param recompress
* @throws IOException
Expand Down Expand Up @@ -450,6 +451,8 @@ public void fSync() throws IOException {
flush();
if (underlyingStream instanceof Syncable) {
((Syncable) underlyingStream).sync();
} else if (underlyingStream instanceof FileOutputStream) {
((FileOutputStream) underlyingStream).getFD().sync();
}
}

Expand Down