4.2.2

Addax 4.2.2 has released! 🎉

Changes

b19cba6 [enhancement][core][rdbms] Introduce console.enabled property to control console logging
9b7fd3e [bugfix][lib][rdbms] Append jdbc suffix before invoking validJbdcUrl
1474331 [update][common] move OverFlowUtil from element package to util package
4ac6e37 [update][lib][rdbms] Use '+' for simple string concatenation instead of String.format
0cd43a0 [update][core] Remove ExceptionTracker.java
4120791 [bugfix][plugin][kudu] Fix the timezone issue when writing and reading timestamp type data. (#1177)
e99ce1c [improve][core] try-catch is only used for code that needs to catch exceptions, not for the entire code block.
aaba2b1 [update][core] Remove unused codes
42bbf2b [improve][plugin][starrockswriter] Replace Thread.sleep by TimeUnit.sleep to avoid busy-waiting
4be0d47 [update][plugin][hdfsreader] use try-with-resources statement
fbe2f40 [update][plugin][hbase20xsqlreader] Remove redundant initializer
fe7e2b2 [update][plugin][hbase20xreader] Remove redundant initializer
b33492b [update][plugin][hbase11xreader] Remove redundant initializer
dec3e51 [update][plugin][tdenginereader] Add isPresent() check for Optional.get() method
00eded8 [update][plugin][esreader] Simplify condition judge
373febe [update][plugin][hbase11xreader] Remove the condition of always true
0251c44 [update][common] Remove the condition of always false
22687d4 [update][plugin][tdenginereader] Take the default switch to the last case
a6050c7 [improve][modules] code cleanup
f1e637b [update][core] Optimize code for better performance and readability
3e1e210 [update][doc] Add description for item ignoreError
9370284 [update][plugin] Refactor: take the shell execute into a single common util class
69c1c46 [feature][plugin][hdfswriter] Add new item preShell and postShell (#1175)
4442889 [update][script] migrate docker image from docker.io to quay.io

4.2.1

Addax 4.2.1 has released! 🎉

Changes

3e54c09 [update][script] Add module build option
b2a75a4 [update][plugin][hdfsreader] Refactor: extract orc file reader logic into separator class
2d73b23 [update][plugin][hdfsreader] Refactor: extract file type judge logical into separate class
738fc23 [update][plugin][hdfsreader] Refactor: extract Parquet handling logic into separate class
87bddc6 [bugfix][plugin][hdfsreader] Fix exception when reading an optional fields with null values in Parquet files
2ad2a5b [update][plugin][txtfilewriter] Improve handling of file suffixes
f825a49 [update][plugin][kudureader] Remove duplicate key definition
160d86e [update][core] Enhance statistical output for better readability
37852c2 [update][common] Refactor code
8f0eb49 [bugfix][lib][rdbms] Fix missing where keyword in generated split SQL for spliting table
ca62f8d [update][lib][rdbms] Refactor primary key types to use enum class
2c6fbf3 [update][plugin][doriswriter] code improvements
f31dea1 [update][action] Modify changelog output
cb5e75f [update][core] Minor code change
7d45b9e [update][plugin][streamwriter] Refactor code to enhance readability and efficiency
2b674cb [update][lib][storage] Simplify a few functions logic
b65b4b3 [update][plugin][datareader] Replace deprecated methods
83366e5 [maven-release-plugin] prepare for next development iteration

4.2.0

Addax 4.2.0 has released! 🎉

Compatibility Break Release Note for Version 4.2.0

This release note outlines the changes introduced in Version 4.2.0 that may affect compatibility with previous versions. Please review these changes carefully to ensure a smooth transition.

The plugins before 4.2.0 can not used in the 4.2.0 because of the major changes in the plugin framework

Changes

40dad25 [maven-release-plugin] prepare release 4.2.0
e5f045c [update][action] Fix variable definition
cb47451 [update][docs] Update json layout for new version requirements
0ae2421 [feature][lib][rdbms] Add experimental support for DDL statement
04ee37c [update][lib][rdbms] Remove unused function parameters and improve code readability
3334246 [bugfix][lib][rdbms] Fix NullPointerException when password is not set
bcf76c9 [update][plugin][redisreader] Refactor connection.uri to use a list instead of a string
ea35dae [bugfix][plugin][redisreader] Fix missing configuration for standalone mode
e681280 [update][lib][rdbms] Refactor writer.parameter.connection to use a map instead of a list
e26003e [update][plugin][excelwriter] Simplify dependencies, add missing log4j-core deps
4ea03b3 [update][plugin][redisreader] Refactor connection item from list to string
e9cdef5 [update][reader][rdbmsreader] Refactor connection item from list to string
8ef27e9 [update][reader][mongodbreader] Refactor connection item from list to string
283f1b9 [update][plugin][influxdb2reader] Remove unused HTTPComponents dependencies
b9faed6 [update][plugin][influxdbreader] Migrate Apache HttpComponents from 4.x to 5.x
77a1fd7 [improve][core] Fallback to Addax engine for downloads when curl is unavailable
28baad9 [bugfix][script] fix incorrect module path
2705b0e [update][dep] Bump commons-codec:commons-codec from 1.15 to 1.17.1
09ab6d6 [update][reader][redisreader] Migrate Apache HttpComponents from 4.x to 5.x
febf6bd [update][workflow] Adjust maven compile options
47f7f76 [update][plugin][starrockswriter] Switch to HttpComponents 4.x due to incompatibility with Starrocks backend
3205a1d [update][plugin][clickhousewriter] Remove unused HTTPComponents dependencies
fada373 [update][plugin][doriswriter] Switch to HttpComponents 4.x due to incompatibility with Doris backend
1e84fbb [update][core] Migrate Apache HttpComponents from 4.x to 5.x
4abc386 [update][plugin][databendwriter] Remove unused HTTPComponents dependencies
ef2a867 [update][plugin][httpreader] Migrate Apache HttpComponents from 4.x to 5.x
c088d94 [update][action] Fix maven options
bd269f8 [update][dep] Bump some dependencies to latest version
87dc6f9 [update][action] Bump maven from 3.5.4 to 3.8.8
d3dd5e5 [update][reader][hbase20xsql] Use common split function
b8455b8 Merge branch 'master' of github.com:wgzhao/Addax
1c7e162 [update][lib][rdbms] Delete RdbmsRangeSplitWrap class, it's not used no longer
0db4dda [update][dep] Bump actions/setup-java from 4.3.0 to 4.4.0 (#1143)
da9150e [update][dep] Bump some dependencies to latest version
92a47be [update][script] Add support for remote deployment using
7942831 [improve][lib][rdbms] Refactor table split strategy (#1145)
291fd69 [update][lib][rdbms] It should split the table even when the is set to 1.
e7a0774 [improve][plugin][clickhousewriter] Optimize Codebase: Update JDBC Driver and Enhance HTTP Connections
0750c8b [bugfix][plugin][kafkareader] Resolve the issue of consumed data not being output by invoking the flush method
d057a05 [feature][plugin][jsonreader] Add support for multiline json file (#1140)
d6e586c Refactor ftpreader (#1139)
0822901 [update][scripts] Skip Java source and Java doc compile for building module
498a1f7 [update][plugin][excelreader] Minor code change
78c23e2 [update][dep] Bump org.apache.commons-io from 2.15.1 to 2.16.1
6479634 [update][core] Correct the dbf plugin name for the demo files.
9a937ea [update][lib][storage] Removed the unused method .
62b0627 [update][plugin][txtfilewriter] Made minor code changes and removed unused code.
7aa16da [improve][plugin][txtfilereader] Cancel the subsequent tasks if the list of files to read is empty.
18de511 Refactor wildcard file handling using Paths and Files classes (#1133)
4eef124 Refactor errorcode (#1132)
a891423 [update][plugin][hivereader] Simplify the exception handling to reduce complexity.
c7bc7ef [improve][plugin][s3writer] Improve performance by refactoring code
1a90bf4 [improve][plugin][s3reader] Improve performance by refactor codes
ecd236d [update][readme] Add missing databend logo
93b63e5 [improve][plugin][databendwriter] Handle binary / blob types as strings to avoid error
a07018c [update][dep] Bump com.databend:databend-jdbc from 0.1.2 to 0.3.2
4f300ef [update][dep] Bump org.apache.commons:commons-configuration2 from 2.9.0 to 2.11.0 (#1129)
43a50ef [update][dep] Bump fastjson2.version from 2.0.50 to 2.0.52 (#1115)
1d65baf [update][github][action] Generate dynamic changelog using git log
0d3ceb0 [update][dep] Bump com.github.luben:zstd-jni from 1.5.5-11 to 1.5.6-5 (#1118)
4746b55 [update][dep] Bump commons-cli:commons-cli from 1.6.0 to 1.9.0 (#1120)
6147e16 [update][dep] Bump org.apache.commons:commons-rng-simple from 1.5 to 1.6 (#1121)
bfd2903 [update][docker] Use label to replace deprecated maintainer
aa080ba [update][github][action] Bump maven version from 3.6.3 to 3.8.8
282dbea [update][github][action] migrate deprecating set-output commands
fa1e8ea [maven-release-plugin] prepare for next development iteration

4.1.8

Addax 4.1.8 has released! ㊗️

changelog

8d38bda [improve][lib][rdbms] Handle unknown or unsupported types as strings to improve support for various data types
ceefb94 [improve][plugin][hdfsreader] Handle data type mismatch by converting to string when expected type is not met
216b08d [improve][plugin][hdfsreader] Add support for float data type
33dae27 [improve][plugin][hdfswriter] Enhance handling of time-relative data types
b8a9725 [improve][plugin][hdfswriter] Refine timestamp data type handling in Parquet and ORC file formats
950701f [bugfix][lib][rdbms] fix incorrect jdbc path
53767a4 [bugfix][core] fix NPE when doest not exists
127f998 [bugfix][core] use conf instead content to valid path
696f0e4 [improve][core] Improved JSON structure for job content and connection parameters (#1125)
e44d1cf [bugfix][plugin][hdfswriter] fix incorrect parquet decimal value display when value scale differs from column definition
56b0414 [improve][plugin][hdfswriter] add check for writable permissions on HDFS path before writing

4.1.7

Addax 4.1.7 has released! ㊗️

changelog

558b709 [bugfix][plugin][hdfswriter] Fix NullPointerException when writing Parquet file with null decimal field value
629d6dc [update][core] create basic addax layout while first building module
bc1f326 Merge branch 'master' of github.com:wgzhao/addax
e093389 [improve][plugin][hdfswriter] enhance the output for geater clarity
dacd3b6 [update][dep] Bump com.alibaba:druid from 1.2.20 to 1.2.23 (#1108)
787a9c2 [update][dep] Bump org.apache.maven.plugins:maven-source-plugin from 3.3.0 to 3.3.1 (#1101)
07aa414 [improve][plugin][hdfswriter] Refactor HdfsHelper: Extract file type writer methods into separate classes
4923ad9 [bugfix][plugin][hdfswriter] Fix NullPointerException when writing Parquet file with null string field value
9be92e9 [update][core] improve the usage help
19f7e1a [update][plugin][ftpwriter] Replace jsch with com.github.mwiede to fix invalid private key issue
3fb3245 [feature][plugin][ftpreader] Add public key authentication support for sftp
be6cef6 [plugin][ftpreader] Replace jsch with com.github.mwiede to fix invalid private key issue
c138140 [update][plugins] update json templates
02c714a [update][dep] Remove error-prone compiler plugin due to lack of JDK 8 support
0723c07 [update][dep] Bump com.fasterxml.woodstox:woodstox-core from 6.5.1 to 7.0.0 (#1088)
34f3829 [update][dep] Bump org.apache.maven.plugins:maven-dependency-plugin from 3.6.1 to 3.8.0 (#1098)
eddf5e0 [update][docs] Correct inaccurate description of the column field for kudu writer
efb1724 [update][docker] Add y param to impilicate archive the target
012a571 [update][docker] remove custom maven settings
ff25da8 [update][dep] Bump org.apache.maven.plugins:maven-enforcer-plugin from 3.4.1 to 3.5.0 (#1091)
a736fa8 [update][dep] Bump commons-net:commons-net from 3.10.0 to 3.11.1 (#1093)
9176bcb [update][docs] improve the document
69bb958 [update][docs] improve the document
f7bc24c [bugfix][plugin][oraclewriter] Fix closed connection exception caused by update a blob field, fix #1092
89a1834 [bugfix][common] Add missing param for split function
18d6562 [update][docs] change lite docker image to latest
5c7692d [update][action] update the release version for issue template

4.1.6

Addax 4.1.6 has released!

changelog

99fe304 [update][dep] Bump com.sap.cloud.db.jdbc:ngdbc from 2.20.17 to 2.21.11 (#1084)
c01e317 [update][dep] Bump org.sonatype.plugins:nexus-staging-maven-plugin (#1083)
fb7b1f0 [update][core] Add a simplier batch process script to invoke addax in windows
6b65414 [bugfix][plugin][rdbmswriter] Fix incorrect package name
3966b36 Create CITATION.cff
50cc09c [update][dep] Bump org.tukaani:xz from 1.9 to 1.10 (#1077)
025eac4 [update][dep] Bump com.github.albfernandez:javadbf from 1.14.0 to 1.14.1 (#1074)
3a58b47 [update][dep] Bump poi.version from 5.2.5 to 5.3.0 (#1075)
6b67552 [docs][improve] convert starrocks logo to svg
f10eba1 Merge branch 'master' of github.com:wgzhao/addax
4ebe019 [improve][lib][rdbms] improve the split strategy, fix #1072
27782de [improve][docs] adjust logos color for dark theme
14434fe [improve][docs] Enhancing readability through emoji use
fa6bc43 [bugfix][plugin][influxdbreader] Fix incorrect package path, fix #1071
3b1b43a [improve][docs] remove logos background
405b049 [update][github][action]Bump actions/setup-java from 4.2.1 to 4.2.2 (#1067)
d3c694a [bugfix][plugin][oraclewriter] Fix ora-03106 while updating clob column, fix #1030
821bc0e Bump com.google.code.gson:gson from 2.10.1 to 2.11.0 (#1046)
145d131 [update][doc] Add documentation build/publish guide
9cb12ca [maven-release-plugin] prepare for next development iteration

4.1.5

Addax 4.1.5 has released!

changelog

e42ab57 (origin/master, origin/HEAD) Bump fastjson2.version from 2.0.43 to 2.0.50 (#1045)
e36708f [improve][plugin][hdfsreader] Refactor parquet reading method to support timestamp data type
27c5761 [improve][core] improve exception print
81801c0 [update][core] remove commons-logging
4d20cbd Merge remote-tracking branch 'origin/master'
6c7ea46 [update][core] remove commons-logging
0c53159 Bump org.apache.maven.plugins:maven-gpg-plugin from 3.1.0 to 3.2.4 (#1043)
07791e7 (origin/remove_commons_logging) [update][core] Add build module script for develop
1305d14 Bump org.codehaus.plexus:plexus-compiler-javac-errorprone (#1019)
14f917d Bump org.junit.jupiter:junit-jupiter-api from 5.10.1 to 5.10.2 (#1014)
9f0dced Bump com.sap.cloud.db.jdbc:ngdbc from 2.19.11 to 2.20.17 (#1028)
5e258a7 Bump org.influxdb:influxdb-java from 2.23 to 2.24 (#989)
ecee535 Bump com.moilioncircle:redis-replicator from 3.8.0 to 3.8.1 (#981)
1f2703c Bump softprops/action-gh-release from 1 to 2 (#1018)
33cc436 Bump actions/setup-java from 4.0.0 to 4.2.1 (#1024)
11e1ccf Improve exception print (#1038)
8e452e4 [improve][core] disable mongodb drive monitor mass log
a13cfa5 [bugfix][reader][mongodb] read timeout with replica cluster (#1037)
4897020 [update][writer][oracle] Remove unused method
143e107 [bugfix][writer][oracle] Fix ORA-01461 With MERGE INTO Statement And Clob Column #1030
4650d3d [update][plugin] Bump databend jdbc from 0.1.4 to 0.1.9
b292042 [bugfix][lib][storage] Fix skip header logic in StorageReaderUtil.java
9c1503a Add ExitOnOutOfMemoryError to return 3 instead of 0 when oom occurred
b45a7ea Bump org.codehaus.plexus:plexus-compiler-javac from 2.8.8 to 2.14.2 (#991)
c00c877 Bump stCarolas/setup-maven from 4 to 5 (#1010)
3d7a453 Bump com.jayway.jsonpath:json-path in /plugin/reader/jsonfilereader (#1002)
bbb6b0e [improve][writer][doris] improve code, reduce standard log output
c41c765 [improve][writer][doris] Refactor Doris writer plugin (#1007)
02944a8 [update][reader][jsonfile] Bump json-path from 2.8.0 to 2.9.0
8d5dfb8 [improve][reader][http] remove unused variable.
1ca5afb [improve][reader][hdfs] remove unused variable.
2cd981a [improve][writer][tdengine] remove unused variable.
a9f5b5a [improve][writer][elasticsearch] remove unused variable.
114fb7f [improve][reader][elasticsearch] remove unused variable.
f7119b3 [improve][writer][clickhouse] remove unused variable.
40c9ed9 [bugfix][core] remove redundant initializer for maxLogNum variable
fa2b95d [bugfix][writer][starRocks] remove the condition which is always true
3b7f919 [improve][writer][elasticsearch] use StringBuilder to replace string concat
fa63fb8 [improve][reader][cassandra] add element type for container class
a981417 [improve][writer][starRocks] use lambda to replace Anonymous function
442e97a [bugfix][writer][starRocks] fix some bugs
d6fca01 [update][common] Add build script for docker images

4.1.4

Addax 4.1.4 has released!

Changelog

be765d1 [improve][reader][tdengine] Use primitive type instead of wrapper type
361aa4c (origin/master, origin/HEAD) [improve][common] improve EncryptUtil class
a852982 [bugfix][common] Fixed Tag mismatch exception
70995c3 Bump github/codeql-action from 2 to 3 (#985)
7844518 [bugfix][lib][storage] Fix dateFormat is invalid when data type is timestamp
d165f76 [improve][plugin][datareader] Add support for generating email address (#983)
97e4c0b [update][3rd] Bump multiple 3rd-party deps to updated version
00c5355 [update][core] Remove ununsed import
c0889a9 [update][docs] Add SAP HANA logo
bdd51e8 [update][docs] Add hana guide
603322d [add][plugin] Add support for SAP HANA database (#977)
7e5dcc4 [update][plugin][mongowriter] Replace deprecated MongoClient method
5cfae66 [update][plugin][esreader] Remove deprecated JsonParser#JsonParser method
4f1ca9c [update][plugin][influxdbreader] Remove unused InfluxDBReaderTask#post method
d72fa75 [update][plugins] Remove unused import or variables
23fd22d [update][plugin] Remove unused logger instance
4de0743 [update][rdbms] Remove unused PerfTrace
d0e3e23 [improve][plugin][streamreader] Remove deprecated RandomUtils class
e5161fd [improve][plugin][datareader] Remove deprecated RandomUtils class
a72b0b3 [improve][common] Remove unused import to TimestampColumn class
0dda008 [bugfix][plugin][sqlitereader] Fix incorrect package name
5372e9c [update][3rd] Bump multiple 3rd-party deps to updated version
1ccd78f [update][3rd] Bump multiple 3rd-party deps to updated version
e499017 [update][3rd] Bump ch.qos.logback from 1.3.11 to 1.3.12
75e3438 Bump actions/setup-java from 3.13.0 to 4.0.0 (#968)
b5d4ad8 Bump org.fusesource.jansi:jansi from 2.4.0 to 2.4.1 (#967)
04a4619 [update][3rd] Bump multiple 3rd-party deps to updated version
3434020 [update][doc][difference] Add access reader and writer plugin
797bcbd [update][doc][readme] improve
ad3c50d [update][3rd] Bump multiple 3rd-party deps to updated version
b6c95a0 [update][doc][readme] Add star history chart
8af30ad [update][3rd] Bump multiple 3rd-party deps to updated version
9963d36 [update][github] update action template

4.1.3

Addax 4.1.3 has released!

Changelog

General

• [Add][plugin][reader] Add support for Access Database (#931)
• [Add][plugin][writer] Add support for Access Database (#932)
• [improve][plugin][txtfilewriter] convert the log to english (#933)
• [add][plugin][txtfilewriter] Add support for writing to a file using SQL statements (#934)
• [update][plugin][databendwriter]Refactor databendwriter with jdbc, support replace mode (#864)
• [improve][docs] improve docs (#941)
• [improve][plugin][doriswriter] improve column delimiter parser (#947)

4.1.2

Addax 4.1.2 has released!

General

• [bug][doris-writer]Fix "Endpoint must be set" exception (#908)
• [improve][doris-writer] Code refactor (#909)
• [bugfix][hivereader] Add missing 3rd deps (#919)
• [bugfix][hive-reader] HiveBaseResultSet#getTimestamp method does not support pass Calendar (#920)
• [improve][hive-reader] Add support for CDH 6.2.1 (#922)
• [improve][docs] correct the description for speed.channel (#927)