From 756bbbd521c084fef9731a9f9ae51d90bdd4d7c6 Mon Sep 17 00:00:00 2001 From: Rodrigo Meneses Date: Wed, 1 May 2024 08:10:22 -0700 Subject: [PATCH] A new implementation of an Iceberg Sink [WIP] thta will be used with upcoming Flink Compaction jobs --- .../main/java/org/apache/iceberg/flink/sink/IcebergSink.java | 4 ++-- .../apache/iceberg/flink/sink/committer/SinkCommittable.java | 5 +++-- 2 files changed, 5 insertions(+), 4 deletions(-) diff --git a/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java b/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java index f7d6f6b27847..3b93f1e48979 100644 --- a/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java +++ b/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/IcebergSink.java @@ -105,8 +105,8 @@ * org.apache.iceberg.io.WriteResult}s to a single {@link * org.apache.iceberg.flink.sink.DeltaManifests} *
  • {@link Committer} which stores the incoming {@link - * org.apache.iceberg.flink.sink.DeltaManifests}s in state for recovery, and commits - * them to the Iceberg table + * org.apache.iceberg.flink.sink.DeltaManifests}s in state for recovery, and commits them to + * the Iceberg table *
  • {@link WithPostCommitTopology} we could use for incremental compaction later. This is not * implemented yet. * diff --git a/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommittable.java b/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommittable.java index 69c41b3224f9..10b658fe3f15 100644 --- a/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommittable.java +++ b/flink/v1.19/flink/src/main/java/org/apache/iceberg/flink/sink/committer/SinkCommittable.java @@ -30,8 +30,9 @@ *
  • WriterResult - The output files generated by the individual writers. Containing only a * {@link WriteResult} *
  • Committable - The aggregated results of a single checkpoint which should be committed. - * Containing the serialized {@link org.apache.iceberg.flink.sink.DeltaManifests} file - which contains the commit data, and - * the jobId, operatorId, checkpointId triplet which helps identifying the specific commit + * Containing the serialized {@link org.apache.iceberg.flink.sink.DeltaManifests} file - which + * contains the commit data, and the jobId, operatorId, checkpointId triplet which helps + * identifying the specific commit * * * {@link SinkCommittableSerializer} is used for serializing the objects between the Writer and the