Releases: exasol/script-languages-release
9.0.0: New script options parser
Summary
This release introduces the new Script Options parser used in the Java UDF's. The new parser provides additional escape syntax which allows a more flexible definition of option values.
Note: In this release, the old options parser is still the default parser, the new parser needs to be activated manually.
Additionally, the release contains several bugfixes for the internal toolchain of building script languages containers, and updates several dependencies.
The flavor template-Exasol-all-python-3.10-cuda-conda was renamed to template-Exasol-8-python-3.10-cuda-conda.
Script-Language-Container-Tool (Exaslct)
This release uses version 1.0.0 of the container tool.
Features
- #962: Add a GH workflow which uploads trivy databases and updated Ubuntu packages
Security Issues
- #956: Updated dependencies
- #1006: Updated Dependencies on top of 8.3.0
- #1019: Update Dependencies on top of 8.3.0
Refactorings
- #950: Refactored existing script-options parser
- #944: Removed exaudfclient_bin_py3
- #967: Added ctpg script options parser
- #972: Refactor exception handling for Script Options parser
- #973: Support new lines and white spaces at the beginning in script options values
- #969: Use new CTPG Parser in Java VM
- #980: Moved flavor template-Exasol-all-java-17 to script-languages repo
- #971: Removed dead code in Java VM
- #982: Added a performance test for ScriptOptions parser for a single line UDF
- #983: Refactor CTPG script options Java parser code
- #986: Added environment variable for new ScriptOptionsLine parser in UDF client
- #990: Forward JAR options without changing order to classpath
- #989: Trim script class and import script options
- #988: Adapted Java integration tests for new ScriptOptions parser
- #878: Fixed jvmoption and parameters containing a space
- #1009: Removed NVIDIA GPU Driver Libraries from template-CUDA SLC
- Bumped version to 9.0.0
Bugs
- #977: Fixed Trivy update cache workflow
- #993: Added escape sequence for backslash in new Script Options parser
- #1002: Use ECR fallback repository for Trivy caching
- #958: Fix conda version conflict
- #1014: Fixed Push Docker Test Container workflow
- #1017: Implemented a GH workflow which validates correct target branch of PR
- #772: Added check of installation of R packages
Doc
- #954: added release checklist
8.3.0: Updated exaslct to 1.0.0 and Exasol manifest
Summary
This release uses version 1.0.0 of exasol-script-languages-container-tool, which had removed the starter-scripts and added support for the empty Exasol manifest file in the Script Languages Container.
Besides, it fixes several security issues by updating dependencies of the Script Languages Container.
Internally, a new version of Bazel is used to build the UDF client.
Script-Language-Container-Tool (Exaslct)
This release uses version 1.0.0 of the container tool.
Security Issues
Refactorings
8.2.0: Package updates.
Summary
This release updates some Ubuntu and Python packages. Also, it fixes a bug where a Pip package could be downgraded during the Script-Languages-Container build process.
Package Version Comparison between Release 8.1.0 and 8.2.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.19.0 of the container tool.
Bug Fixes
- #794: Fixed install_via_pip.pl might downgrade already installed packages
Features / Enhancements
n/a
Documentation
n/a
Refactoring
n/a
Security
- #930: Updated Packages
Attention: Kernel CVE's can't be fixed by the container itself, rather need to be addressed by the host system executing them.
8.1.0: Fixed memory leak for Python Dataframe Extension
Summary
This release fixes 3 memory related bugs in the Python Dataframe Extension, which were the root cause of a memory leak. Besides,
this version uses the latest version of exaslct (0.19.0) and Python3.10 as runtime for the CI builds. The security scanner was changed in order to ignore all Linux Kernel related CVE's during the CI and release builds.
Package Version Comparison between Release 8.0.0 and 8.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.19.0 of the container tool.
Bug Fixes
- #895: Fixed GH Action 'Publish Docker Test Container'
- #902: fixed memory related bugs with emit dataframe
Features / Enhancements
- #889: Upgraded all python template flavors
- #892: Filtered out Linux Kernel related CVE's
- #856: Changed mirror for installing R packages
- #921: Use exasol-python-test-framework 0.5.0
Documentation
n/a
Refactoring
Security
- #898: Ignored new Linux Kernel CVE's
- #904: Ignored Kernel CVE
- #906: Updated APT package
- #910: Updated APT package
- #917: Updated Ubuntu JDK package
Attention: Kernel CVE's can't be fixed by the container itself, rather need to be addressed by the host system executing them.
8.0.0: Single standard container
Summary
Standard flavors (7.1 and 8.0) are simplified into a single flavor. The standard flavor now is based on Ubuntu 22.04 and supports Python3.10 and R4.4.
This release fixed the conda-based SLC builds and updated packages. Additionally, new integration tests using a module-based Java UDF have been added.
Package Version Comparison between Release 7.1.0 and 8.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.18.2 of the container tool.
Bug Fixes
- #870: Added tests for pyexasol export/import and that sys.executable returns a comparable python interpreter to the UDF
Features / Enhancements
- #883: Merged standard flavors
Documentation
n/a
Refactoring
- #877: Add tests using a module-based Java UDF
Security
- #867: Fixed conda-based SLC builds, updated packages and ignore Kernel CVEs
- #879: Updated package
- #891: Ignored Kernel CVE
Attention: The CVE's below can't be fixed by the container itself, rather need to be addressed by the host system executing them.
7.1.0: Template flavor with Python 3.10 support
Summary
This release added a new template flavor with Python 3.10 and updated packages.
Package Version Comparison between Release 7.0.0 and 7.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
n/a
Features / Enhancements
- #861: Added flavor template-Exasol-all-python-3.10 with Python 3.10 support
Documentation
n/a
Refactoring
n/a
Security
- #861: Updated Packages and ignore CVE-2023-6176, because it only effects the Kernel
Attention: The CVE's below can't be fixed by the container itself, rather need to be addressed by the host system executing them.
(Updated script-languages ref)
- Ignored CVE-2023-6176 because they are kernel issues and not container issues
7.0.0: Flavor were Upgraded to at least Ubuntu 20.04
Summary
This release upgrades all flavors at least to Ubuntu 20.04 or newer.
- The standard-EXASOL-7.1.0 flavor was upgraded to Python 3.8, R 4.3 and Ubuntu 20.04
- The r-4-minimal flavor was upgraded to R 4.3 and Ubuntu 22.04
- The flavors python-3.7-minimal-EXASOL-6.2.0 and flavor standard-EXASOL-7.0.0 were removed, because EOL
Furthermore, many bugs and security issues were fixed and packages updated.
Package Version Comparison between Release 6.1.0 and 7.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
- #836: ModuleNotFoundError: No module named 'numpy.testing.nosetester'
- #850: Fixed package management tests by updating Ubuntu 20.04
- #851: Fixed AvailableRPackages tests fail for standard flavor 8.0
Features / Enhancements
- #857: Updated R to 4.3 and updated R packages in standard 7.1 flavor
Documentation
n/a
Refactoring
- #376: Removed unused and outdated cpp and python clients (update script-languages ref)
- #843: Switch udf-script-signature-generator to Python 3 (#351)
Security
- #805: Upgraded r-4-minimal flavor to R 4.3 and Ubuntu 22.04
- #806: Removed flavor python-3.7-minimal-EXASOL-6.2.0
- #807: Removed flavor standard-EXASOL-7.0.0
- #827: Updated packages and ignore kernel CVEs
- #831: Updated conda flavors to cryptography 41.0.3 and openssl 3.1.2
- #839: Updated dependencies and ignore kernel CVEs
- #847: Updated packages and ignore Kernel CVEs
- #808: Updated standard-EXASOL-7.1.0 flavor to Ubuntu 20.04
- #824: Updated scipy for standard 7.* flavor to fix CVE-2023-29824
- #854: Updated pyarrow to fix CVE-2023-47248
Attention: The CVE's bellow can't be fixed by the container itself, rather need to be addressed by the host system executing them.
(Updated script-languages ref)
- Ignored CVE-2023-4622, CVE-2023-4623, CVE-2023-4921, CVE-2023-4244, because they are kernel issues and not container issues
- Ignored CVE-2023-20569, because it is an issue of the host system (AMD CPUs)
6.1.0: Pandas 2 support
Summary
This releases adds support for Pandas 2 pyarrow dtype columns for emitting dataframes from Python UDFs. Furthermore, it fixes a silent data corruption when emitting dateframes with float16 dtype columns from Python UDFs.
Note: We could not update scipy in the standard 7.* flavors such that these still contain CVE-2023-29824, because the fixed scipy version is not available for Python 3.7 which is still used in these flavors. We are going to replace Python 3.7 with Python 3.8 in the next release and than also update scipy. See also following GitHub issue.
Note: We also needed to remove numba-scipy from the standard 8.0.0 flavor, because it blocked the security update for scipy.
Package Version Comparison between Release 6.0.0 and 6.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.17.0 of the container tool.
Bug Fixes
- #792: Fixed Github workflow publish-test-container by updating script-languages-container-tool to 0.17.0 and script-languages-container-ci to 1.1.0
- #796: Fixed silent data corruption when emitting dataframes with float16 dtype columns from Python UDFs
- #819: Fixed CI, adapted project to Exasol 8 and updated apt packages
Features / Enhancements
- #793: Added support for Pandas 2 pyarrow dtype columns for emitting data from Python UDFs
Documentation
n/a
Refactoring
n/a
Security
- #785: Updated Ubuntu packages
- #789: Updated Ubuntu packages
- #777: Updated redis-py to fix CVE-2023-28859
- #800: Updated Ubuntu packages
- #803: Updated Ubuntu packages
- #815: Updated Ubuntu packages and poetry.lock
- #822: Updated scipy to 1.10.1 in standard 8.0.0 flavor to fix CVE-2023-29824. For this we had to remove scipy-numba. Updated apt packages.
6.0.0: Python 2 and Python 3.6 Removal
Summary
In this release, we removed Python 3.6 and Python 2 from the flavors and replaced it with Python 3.7, because the former ones are long time end of life and don't receive security fixes anymore.
Note: We had to remove samba support from the standard flavor 7.*, because it depended on Python 2. If you need samba support, please use the standard flavor 8.0.
Furthermore, this release fixed a bug with R SCALAR RETURNS UDFs where either a empty run function or returning null from the run function lead to a hanging UDF, which will fail after a while with a out of memory error.
Package Version Comparison between Release 5.0.0 and 6.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.15.0 of the container tool.
Bug Fixes
- #703: Fixed package diff workflow
- #714: Pinned Python3 cryptography package on legacy standard containers
- #754: Fixed R SCALAR RETURNS UDFs with empty run function hangs
Features / Enhancements
n/a
Documentation
n/a
Refactoring
- #697: Disabled version check GH action for master branch
- #702: Updated toolchain dependencies
- #706: Updated script-languages-container-ci-setup dependency
- #774: Updated CI to the version released on PyPi
- #768: Removed or replaced Python 2 tests
Security
- #710: Updated curl
- #713: Updated Python3.7
- #720: Updated ubuntu dependencies
- #723: Updated curl
- #725: Updated pip protobuf package on standard flavors
- #727: Ignored CVE-2022-23960
- #732: Updated openjdk for standard flavors
- #734: Ignored CVE-2022-43945 and update Ubuntu packages
- #737: Updated ubuntu packages
- #740: Updated Ubuntu packages
- #745: Updated Ubuntu python packages
- #748: Updated Ubuntu packages and removed Python 3.6
- #752: Updated ubuntu packages
- #756: Updated Ubuntu packages
- #759: Updated Ubuntu packages
- #762: Updated Conda/Ubuntu/Pypi packages
- #764: Removed Python 2 from the flavors
- #767: Updated libprotobuf-dev
- #775: Updated dependencies
5.0.0: Conda-Container and UDF client refactoring
Summary
This release adds two new python-flavors, which use the conda-package manager: A template flavor for Python3.8 with conda (template-Exasol-all-python-3.8-conda) and a template flavor for Python3.8 with conda and cuda (template-Exasol-all-python-3.8-cuda-conda), which replaces the old cuda Python3.6 container.
Also, the C++ part of the UDF client was refactored. Besides, there are several minor refactorings and features and many security updates for Ubuntu and Python packages.
Package Version Comparison between Release 4.3.0 and 5.0.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.14.0 of the container tool.
Bug Fixes
- #686: Fixed bazel crashes on conda flavor
Features / Enhancements
- #666: Make package diffs more robust for CI
- #590: Added flavor template-Exasol-all-python-3.8-conda
- #673: Added dev-tools-script to test container
Documentation
n/a
Refactoring
- #588: Removed virtualschema-jdbc-adapter.jar
- #329: Splitted exaudflib cc into several files
- #646: Updated ci dependencies and removed GCloud CI
- #650: Changed nightly cron and avoid running nightly builds during the weekend
- #648: Moved swig-2.0.4.tar.gz to AWS
- #572: Refactored package import tests
- #677: Improved poetry project structure
- #679: Added a duplicated pyproject.toml
- #451: Removed legacy install scripts
- #503: Added flavor template-Exasol-all-python-3.8-cuda-conda