From a47fbc51bd51374cbe90b01256d48c132c7401fd Mon Sep 17 00:00:00 2001 From: Marko Topolnik Date: Mon, 19 Oct 2020 12:44:35 +0200 Subject: [PATCH] 4.3 Release Notes and Changelog (#2577) Also remove stale attribution from NOTICE --- NOTICE | 5 +- .../src/root/changelog.txt | 19 +++++- .../src/root/release_notes.txt | 65 +++++++++++++++++++ 3 files changed, 84 insertions(+), 5 deletions(-) diff --git a/NOTICE b/NOTICE index 590a4f5b7d6..832c9a51d13 100644 --- a/NOTICE +++ b/NOTICE @@ -1,5 +1,2 @@ -The class com.hazelcast.jet.impl.TopologicalSorter contains code -derived from the Apache Tez project. - The class com.hazelcast.jet.kafka.impl.ResumeTransactionUtil contains -code derived from the Apache Flink project. \ No newline at end of file +code derived from the Apache Flink project. diff --git a/hazelcast-jet-distribution/src/root/changelog.txt b/hazelcast-jet-distribution/src/root/changelog.txt index efd2374e90a..434ef79239d 100644 --- a/hazelcast-jet-distribution/src/root/changelog.txt +++ b/hazelcast-jet-distribution/src/root/changelog.txt @@ -1,3 +1,21 @@ +** Hazelcast Jet 4.3 ** + +* @MohamedMandouh implemented distributed sorting: `BatchStage.sort()`. +* @guenter-hesse added a high-throughput source with precisely controlled + timing, to be used to benchmark Jet's throughput and latency. +* We added the ability for Jet to suspend an unbounded, fault-tolerant job + when it fails due to an exception in the pipeline code. This allows you + to diagnose the problem, remove its cause, and resume the pipeline without + data loss. +* We have improved the consistency of reconnect behaviour across CDC sources and + introduced a uniform API to configure the reconnect strategy. +* We improved the behavior of streaming pipelines based on ingestion time: a + low event rate in any partition of the source used to stall the progress, + now the watermark advances even without events. +* We moved the `Pipeline` to `DAG` transformation from the client to the server, + in preparation for the upcoming feature of preventing event reordering in the + pipeline. + ** Hazelcast Jet 4.2 ** * We added Change Data Capture (CDC) support for PostgreSQL and MySql. New @@ -39,7 +57,6 @@ directly instead of requiring custom code in the `mapToOutputFn`. Credit for this one goes to @TomaszGaweda. - ** Hazelcast Jet 4.0 ** * Updated Hazelcast IMDG version to 4.0, which brings many new features diff --git a/hazelcast-jet-distribution/src/root/release_notes.txt b/hazelcast-jet-distribution/src/root/release_notes.txt index d2d642da140..a04bd759b93 100644 --- a/hazelcast-jet-distribution/src/root/release_notes.txt +++ b/hazelcast-jet-distribution/src/root/release_notes.txt @@ -23,6 +23,70 @@ To expand an issue number into a full GitHub link, use this template: https://github.com/hazelcast/hazelcast-jet/issues/ + +** Hazelcast Jet 4.3 ** + +Members of the open source community that appear in these release notes: + +@caioguedes +@guenter-hesse +@MohamedMandouh + +Thank you for your valuable contributions! + +1. New Features + +[pipeline-api] [014] @MohamedMandouh implemented distributed sorting: `BatchStage.sort()` + (#2469, #2544) +[core] [012] Added `JobConfig.suspendOnFailure`: suspend a job on exception instead of + cancelling it (#2411) +[cdc] Improved the consistency of reconnect behaviour across CDC sources, new uniform API + to configure the reconnect strategy (#2419) + +2. Enhancements + +[core] @guenter-hesse contributed a test source to benchmark Jet's throughput and latency (#2382) +[core] [013] Improved watermark semantics that prevent low event rate from stalling an + ingestion time-based pipeline (#2485, #2514) +[cdc] Exposed the sequence number in the CDC `ChangeRecord` that orders the events (#2390) +[core] Two new DAG edge types: `distributeToOne` (sending all data to one member) and `ordered` + (maintaining the sort order) (#2394, #2469, #2544) +[core] Disabled access to external XML entities when parsing XML config, this was a potential + XXE attack vector (#2528) + +3. Fixes + +[core] Fixed error handling during job startup that could result in inconsistent job state (#2383) +[core] Fixed an internal exception that leaked out of Observable (#2313, #2389) +[core] Prevented Observable from processing in-flight items after cancellation (#2415, #2418) +[cli] @caioguedes fixed an issue with `--targets` option in CLI where it would overwrite other + settings (#2373, #2421) +[metrics] Fixed a problem where an internally added DAG vertex would show up as a source instead + of the actual source vertex (#2475, #2476) +[core] Fixed a race that could cause `getJobStatus()` to throw an exception if called right after + `newJob()` (#2481, #2484) +[core] Fix a race between snapshotting and restarting (#2487, #2503) +[core] Fixed a race where `getJobStatus()` would report `RUNNING` even though it was actually + `COMPLETING`. (#2507) +[core] Fixed an issue where a `DONE_ITEM` could get lost due to connection failure, preventing + the job from completing (#2158, #2532) +[core] Fixed a job failure related to the coordinator member failing (#2461, #2546) +[core] Fixed a job failure related to a member reconnecting (#2542, #2547) +[core] Improved robustness related to Jet's internal `IMap` operations (#2533, #2550) +[cdc] Upgraded Jackson jr dep, solving a null handling issue (#2459) +[cdc] Upgraded the Debezium dep, solving a Postgress issue resulting in data loss when + snapshotting (#2406) +[core] Fixed a bug where a non-keyed aggregating stage would produce no output when no input + (#2560, #2567) + +4. Breaking Changes + +[pipeline-api] Breaking signature change to `Sources.streamFromProcessorWithWatermarks()` +[pipeline-api] Deprecated `Pipeline.toDag()`, made `Pipeline` and all its components `Serializable`. +[core-api] Breaking signature change to `StreamEventJournalP`, methods `streamRemoteMapSupplier()` + and `streamRemoteCacheSupplier` + + ** Hazelcast Jet 4.2 ** Members of the open source community that appear in these release notes: @@ -70,6 +134,7 @@ Thank you for your valuable contributions! [jet-cli] Deprecated --address and --cluster-name options from command-line as --targets should now be used instead. (#2276) + ** Hazelcast Jet 4.1.1 ** 1. Enhancements