Releases: exasol/script-languages-release
Updates and small Bug Fixes
This release updates the script-language-container-tool and libssl. It also resolves an issue with long dependency lists for R packages and removes python-distutils-extra from python-3.6-minimal-EXASOL-6.2.0, because it pulled python2 into the flavor.
Package Version Comparison between Release 2.1.0 and 2.2.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.2.1 of the container tool.
Bug Fixes
- #293: Remove python-distutils-extra package from python-3.6-minimal-EXASOL-6.2.0
- #317: Fix install_via_r_remotes.pl for long depenendency list
Features / Enhancements
- #298: Fix Gcloud Trigger for develop branch
- #294: Created develop branch (#297)
- #308: Update script-language-container-tool to version 0.2.1
- #312: Updated libssl
- #233: Add script to generate package diffs between releases
Refactoring
- #138: Removed duplicated line in ExaWrapper.java
New flavor without Python 2
This release introduces a new flavor: standard-EXASOL-7.1.0-without-python2.7.
Background: Python 2 went officially out of support on January 1st 2020. From this point on, no updates have been provided anymore. This includes security updates.
Exasol plans to remove Python2 support for UDF's in future releases, and thus this new flavor will become then the standard flavor in the database.
Package Version Comparison to Release 2.0.0
-
r-4.0-minimal-EXASOL-6.2.0 in 2.0.0
was renamed tor-4-minimal-EXASOL-6.2.0 in 2.1.0
. r-base-core and r-base-dev were increased to version 4.1.0-1.1804.0. -
Comparison of standard-EXASOL-7.1.0 with standard-EXASOL-7.1.0_without_python2.7
- Please be aware that
python3-smbc
was removed in this flavor, too, as it has an implicit dependency to the python2 libs.
- Please be aware that
-
standard-EXASOL flavors (6.2/7.0/7.1): pinned EXASOL/websocket-api to latest version compatible with Python2.
(Packages in other flavors have not been changed)
Script-Language-Container-Tool (Exaslct)
This release uses version 0.1.0 of the container tool.
Bug Fixes
- #235: Update script-languages to fix echo timestamp unit test. (#277)
- #278: Exasol WebSocket change broke CI builds (#279)
- #281:Update exaslct (#283)
Features / Enhancements
Refactoring
Use initial release of script-languages-container-tool
This release uses the first release of the script-languages-container-tool. It also updates selected Python libraries, including the Google Cloud API, Microsoft Azure SDK and AWS SDK. Besides, this release provides bug fixes, small improvements.
Please note that due to the restructuring in version 2.0.0 the file flavors//flavor_base/build_steps.py has been changed. If you have your own flavors, you need to change the import of the exaslct package.
Package Version Comparison to Release 1.1.0
- Comparison of standard-EXASOL-7.1.0 in 2.0.0 with standard-EXASOL-7.1.0 in 1.1.0
- Comparison of standard-EXASOL-7.0.0 in 2.0.0 with standard-EXASOL-7.0.0 in 1.1.0
- Comparison of standard-EXASOL-6.2.0 in 2.0.0 with standard-EXASOL-6.2.0 in 1.1.0
Script-Language-Container-Tool (Exaslct)
This release uses version 0.1.0 of the container tool, all relevant changes can be found here.
Bug Fixes
- #65: ForkingPickler PicklingError under MacOSX with Python 3.8
- #229: Fix substitution for empty versions in install script for R remotes and apt
- #228: Fix error caused by consecutive COPY in the Dockefiles
- #247: Update apt-packages for standard flavors
- #256: Fix link in flavor description of standard 7.1 flavor for flavor_base_deps/packages/*
- #264: Fix broken link in flavor description of Python-3.6-minimal-EXASOL-6.2.0
Features / Enhancements
- #66: Investigate to run exalsct in docker container to better support MacOSX and Windows
- #226: Create python 3.6 minimal flavor for easier customization
- #261: Check links in markdown files automatically
- #254: Updated script-languages-container-tools to v0.1.0
- #255: Updated selected Python libraries, including the Google Cloud API, Microsoft Azure SDK and AWS SDK
- #270: Prepared release notes and changelog for v2.0.0
Refactoring
1.1.0: Add new standard flavor for Exasol 7.1 and a minimal flavor for R 4.0 and other bug fixes and improvements
Summary
This release introduces a new standard flavor for Exasol 7.1 with selected Python libraries for the Google Cloud API and the Microsoft Azure SDK. Additionally, this release introduces a minimal flavor for R 4.0 with only the minimal set of packages to simplify customization. Furthermore, flavors for Exasol 6.0 or 6.1 were removed or updated to at least Exasol 6.2. The flavors python3-ds-* and fancyr-* were renamed to python-3.6-data-science* and r-3.5-data-science, respectively. Besides, these larger changes, this release provides bug fixes, small improvements and minor package updates.
Package Version Comparison to Release 1.0.0
- Comparison of standard-EXASOL-7.1.0 in 1.1.0 with standard-EXASOL-7.0.0 in 1.0.0
- Comparison of standard-EXASOL-7.0.0 in 1.1.0 with standard-EXASOL-7.0.0 in 1.0.0
- Comparison of standard-EXASOL-6.2.0 in 1.1.0 with standard-EXASOL-6.2.0 in 1.0.0
Bug Fixes
-
Script-Language Containers:
- #204: Update jinja2 in Pipfile from 2.10.3 to 2.11.3
- #201: Fix url for get-pip.py in install_python2.7_pip.sh
- #182: Pin rsa package to 4.5 for Python 2, because newer versions don't work with Python 2
- #169: Pin the pip version in flavors to lower than version 21.0 and use special install script for Python 2.7
- #166: Flavor builds fails with Syntax Error while installing pip with get-pip.py
- #151: Enable legacy resolver in install_via_pip.pl to avoid pip resolution issues
-
Script-Language-Container-Tool (Exaslct)
- #177: Fix comparison in find_images_by_tag in docker_utils.py
- #176: Make the --path-in-bucket command line argument optional for the upload command
- #170: Fix the exit code of the exaslct start script
- #167: Nightly builds sometimes don't fail when errors during the build happen
- #160: CLI option --path-in-bucket still required for container upload
- #127: Fix incorrect file extension for checksum files of the flavors
- #63: Fix alter session generation for standard-EXASOL-7.0.0
- #64: SyntaxWarning: "is not" with a literal in docker_utils.py
Features / Enhancements
- Script-Language Containers:
- #213: Add option --allow-no-version-for-urls in install_via_pip.pl
- #211: Add selected GCP SDK python packages to standard flavor 7.1
- #210: Add selected Azure SDK python packages to the standard container 7.1
- #206: Add script to list newer candidate versions for pip packages from a package list
- #183: Add R 4.0 Minimal flavor
- #174: Add standard flavor for Exasol 7.1
- #146: Add test case for hadoop-etl-udfs with dockerized hive
Refactoring
-
Script-Language Containers:
- #216: Remove standard-EXASOL-6.1.0 flavor
- #215: Update flavors for Exasol 6.1.0 to Exasol 6.2.0, because Exasol 6.1 reached EOL
- #209: Rename python3-ds flavor to python-3.6-data-sience
- #208: Rename fancyr flavors to r-4.0-data-science
- #184: Pin rsa python package for standard flavors
- #117: Remove 6.0 flavors, because Exasol 6.0 reached EOL
-
Script-Language-Container-Tool (Exaslct)
- #113: Update test environment and import it as a python package instead as a git submodule
Add dedicated standard flavor for Exasol 6.2, other improvements and bugfixes
Summary
This release introduces a dedicated standard flavor for Exasol 6.2 and adds the python package simplejson to all standard-flavors. Additionally, it updates some packages in the standard flavors and fixes some bugs. Furthermore, we introduce with this release semantic versioning. We start the semantic versioning with version 1.0.0, because this project underwent its initial development under the old versioning scheme and should have now a somewhat stable interface.
Bug Fixes
- Script-Language Containers:
- #119: Update apt packages
- #116: Add NGA default uid to the passwd of the flavors
- #114: Fix requests package import for flavor standard-EXASOL-6.1.0
- #112: Update apt-packages for standard-EXASOL-700 flavor
- #108: Update apt-packages for standard-EXASOL-700 flavor
- #104: Fix R install_script using the versions package doesn't work anymore
- #98: Update python-samba in standard-EXASOL-7.0.0
Features / Enhancements
Standard-flavor for Exasol 7.0+, Improved Flavor Customization and Error Reporting
Summary
This release introduces the new standard-flavor for Exasol 7.0+. Additionally, it adds new packages to the standard-flavors, such as boto3, pysftp and paramiko and fixes several bugs for the UDFs, the script-language containers and exaslct. Furthermore, we added more details to the error reporting of the UDFs. Error messages now contain error codes and for Java and Python they include untruncated stacktraces and chained exceptions.
However, most importantly, this release greatly simplifies the customization of the script-language containers. Each flavor now provides package lists under the directory flavors/<flavor-name>/flavor_customization/packages
. Additional packages entered into these lists get automatically installed when building the flavor. Furthermore, we provide detailed instructions on how to do other kinds of modifications to the container by changing the Dockerfile at flavors/<flavor-name>/flavor_customization/Dockerfile
. Please find more information about this, in our Readme.
NOTE: We cannot upload the pre-packaged python3-ds-cuda-preview container to GitHub, because it is larger than GitHub's file size limit for release artifacts. For this reason, we provide it under the following link. You can find the checksum of this container in the release artifacts, if you want to check its consistency.
Bug Fixes
-
UDFs:
- script-languages-release@#67: Fix error in Python3 UDFs when emitting only a dataframe with one Timestamp column
- script-languages-release@#48: Fix Python3 UDF bug during reset after get_dataframe reaches the end of the iterator
- script-languages-release@#31: Java EMITS UDF Scripts cannot emit null
-
Script-Language Containers:
- script-languages@#133: Pin tensorflow_hub version for python3-ds flavors
- script-languages-release@#23: Pin azure-storage to version 0.36.0 until it is clear how we handle the deprecation with version 0.37.0
-
Exaslct:
- script-languages@#109: Fix bug in exaslct upload command
- script-languages@#77: Handle tar return code during export command to detect potentially failed exports
- script-languages-release@#60: Handle multiple lines at once in LogHandlers in exaslct to fix some bugs on MacOSx (but MacOSx is still highly experimental)
Features / Enhancements
-
UDFs:
- script-languages-release@#49: Introduce error codes into UDFClient
- script-languages-release@#50: Introduce error codes for Java, Python and R UDFs. Return untruncated stacktraces for Java and Python. Include causes into Java stacktraces.
- script-languages-release@#54: Support for chained exception for Python 3 UDFs
- script-languages-release@#40: Improve shutdown handling of the UDFClient to reduce the risk of orphaned exaudfclient processes in error cases
-
Script-Language Containers:
- script-languages-release@#47: Add seperate standard-flavor for Exasol 7.0 with pinned package versions
- script-languages-release@#38: Cleanup standard-flavor for Exasol 6.1+ to reduce the container size
- script-languages-release@#26: Add boto3 to standard flavor
- script-languages-release@#25: Add pysftp and paramiko to standard flavor
- script-languages-release@#32: Add exasol websocket-api for Python3 to standard flavor
- script-languages-release@#57: Simplify the customization of flavors for users
- script-languages-release@#23: Add script to install specific version of R packages
- script-languages@#127: Add more robust package install scripts which support package versions
- script-languages-release@#94: Exclude unnecessary directories from /var during export
- script-languages-release@#93: Add lists for actual installed packages to the release build step of the flavors
- script-languages-release@#84: Copy apt/dpkg package database into release image
-
Exslsct:
- script-languages-release@#42: Add a command to generate a language activation statement
- script-languages-release@#95: Generate and store checksum of the exported container besides the container
Refactorings
-
Exaslct:
- script-languages-release@#46: Reorganize exaslct and extract the test environment to exasol/integration-test-docker-environment
- script-languages@#108: Add a cleanup method to exaslct tasks and scripts to run exaslct tests
- script-languages@#98: Pass the docker socket to the test container
- script-languages@#142: Extract and refactor functionalities for performance testing into exatest
- script-languages@#105: Fix a reference to luigis FrozenOrderedDict and pin dependency versions in pipfile
-
Continuous Integration:
- script-languages-release@#99: Add a build breaker to the CI which fails if the commit was not rebuild without cache
- script-languages-release@#87: Add pull-request blocker which checks if the HEAD of submodules is at a ancestor of origin/master
Fix memory leak in the UDFClient
This is a bugfix release which fixes a memory leak for Scalar-UDFs which get called for very large number of tuples.
NOTE: Github Releases doesn't allow the upload of files larger than 2 GB. So, for the time being, we provide the pre-packaged python3-ds-cuda-preview container here.
Fix bug in Java UDFs with non-Docker-DB Exasol Instances
The release 20190726 of the script-languages introduced a bug for Java UDF. This release provides a fix for the bug. As a safety precaution, we added the ability to run the tests against external Exasol Instances which allows the tests to run against other non-Docker-DB Exasol Instances.
Minor changes:
- Used Bazel version for compiling the UDFClient is pinned to a fixed version
- Using the same executable name for the UDFClient in release and test container
- Support for multiple Docker-DB Version for testing
- Improve the Exaslct BaseTask to simplify the extension of Exaslct