User visible changes
- The extensions to CWL that
cwltool
knows about will no longer results in an error
Operations changes:
- After we run
cwltool
, we now try harder to cleanup the process.
Dependecies upgrade:
- Bump jackson-core from 2.14.1 to 2.14.2
- Bump mockito-inline from 5.0.0 to 5.1.1
User visible changes
- Better identify repository licenses through the Licensee Ruby Gem
- When processing a directory, the presence of errors will cause a afile to be skipped instead of erroring out the entire process
- Many user visible error message have been made more helpful; especially in the case of a missing or private repository
Operations changes:
- Unused RO Bundles and Git repositories are now deleted in case of errors, instead of filling up the disk with unusable files
- Temporary directories older than 1 day are now cleared via a daily cron job
See
src/main/resources/application.properties
for configuration - Error handling and retries are improved for git checkouts
Minor changes:
- mockito-all was replaced by mockito-inline as it supports mocking static methods
Dependecies upgrade:
- Bump jackson from 2.13.4 to 2.14.1
- Bump spring-data-commons from 2.7.3 to 2.7.7
- Bump hibernate-types-55 from 2.19.2 to 2.21.1
- Bump liquibase-core from 4.16.1 to 4.19.0
- Bump snakeyaml-engine from 2.4 to 2.6
- Bump common-compress from 1.21 to 1.22
- Bump junit-jupiter from 1.17.4 to 1.17.6
- Bump postgresql from 1.17.4 to 1.17.6
- Bump jena-core from 4.6.1 to 4.7.0
Changes related to the migration of view.commonwl.org from Curii, Inc to HPC4AI, University of Turin:
- Removed Data controller section in the About page
- Added HPC4AI logo in the page footer
- Updated docker-compose.yml file to explicitly set
ulimit -n
to65536
User visible changes
- Switch to snakeyaml-engine (YAML 1.2) from snakeyaml (YAML 1.1) to improve YAML parsing (less errors)
- Made explicit that only gitlab.com and github.com are supported as shortcuts, and that specifying workflows on other hosts needs an explicit git repo URL + branch + path.
- Fewer errors due to the use of complex types
- Support workflows using
MultipleInputFeatureRequirement
Minor changes:
- Always pass --disable-color to cwltool to improve the logs
- Removed explicit html5lib Python dependency
- Improvements to container building.
- Added diagram of operation to the docs
Dependecies upgrade:
- Bump maven-enforcer-plugin from 3.0.0 to 3.1.0
- Bump liquibase-core from 4.11.0 to 4.16.1
- Bump spring-data-commons from 2.7.0 to 2.7.3
- Bump hibernate-types-55 from 2.16.2 to 2.19.2
- Bump postgresql from 1.17.2 to 1.17.4
- Bump junit-jupiter from 1.17.2 to 1.17.3
- Bump jena-core from 4.5.0 to 4.6.1
- Bump jackson-core from 2.13.3 to 2.13.4
Rebuild container to pull in newer version of Python (3.6.9) Use Java 17 LTS Update Docker command java location
Bump postgresql from 1.17.1 to 1.17.2 Bump spring-data-commons from 2.6.3 to 2.7.0 Bump hibernate-types-55 from 2.16.1 to 2.16.2 Bump jena-core from 4.4.0 to 4.5.0 Bump jackson-core from 2.13.2 to 2.13.3 Update spring boot (2.6.1 → 2.6.7), liquibase (unpinned → 4.10.), jgit (5.13 → 6.1) Bump liquibase-core from 4.10.0 to 4.11.0 Bump junit-jupiter from 1.17.1 to 1.17.2
A few workflows were not rendering correctly after the upgrade from MongoDB to PostgreSQL. The reason was a security update in Thymeleaf that stopped Java static methods from being invoked in templates. Only a few workflows triggered the part of the code with static calls, and the production instance of CWL Viewer, https://view.commonwl.org, was not migrated yet - this issue should only affect users using v1.4.2 locally.
Smaller changes:
- Small fixes for issues found during dev/prod deployment (includes Thymeleaf fix) #391 @kinow
Misc fixes:
- Add mongo_to_pg Python script and notebook, replace
dump.sh
bydump.py
(supports pagination) #396 @kinow
Dependencies upgrade:
- Bump junit-jupiter from 1.16.2 to 1.16.3 #387 @mr-c
- Bump postgresql from 1.16.2 to 1.16.3 #386 @mr-c
- Bump jackson-core from 2.13.1 to 2.13.2 #390 @kinow
- Bump spring-data-commons from 2.6.2 to 2.6.3 #392 @kinow
- Bump hibernate-types-55 from 2.14.0 to 2.14.1 #393 @kinow
- Bump hibernate-types-55 from 2.14.0 to 2.14.1 #393 @kinow
- Bump hibernate-types-55 from 2.14.1 to 2.15.2 #400 @kinow
- Bump postgresql from 1.16.3 to 1.17.0 #401 @kinow
- Bump hibernate-types-55 from 2.15.2 to 2.16.0 #403 @kinow
- Bump junit-jupiter from 1.16.3 to 1.17.0 #402 @kinow
- Bump postgresql from 1.17.0 to 1.17.1 #404 @kinow
- Bump junit-jupiter from 1.17.0 to 1.17.1 #405 @kinow
- Bump hibernate-types-55 from 2.16.0 to 2.16.1 #406 @kinow
The database model has been migrated from MongoDB to PostgreSQL. README instructions and Docker images updated. New version will be deployed to https://view.commonwl.org soon.
Smaller changes:
- Added Maven default build goal #377 @kinow
Misc fixes:
- Document Git, GitHub, Docker Hub, and Quay.io release process #360 @mr-c
- Migrate from MongoDB to PostgreSQL #378 @kinow
Dependencies upgrade:
- Bump spring-data-commons from 2.6.0 to 2.6.1 #383 @mr-c
- Bump jena-core from 4.3.2 to 4.4.0 #382 @mr-c
- Bump hibernate-validator from 7.0.1.Final to 7.0.2.Final #381 @kinow
- Bump spring-data-commons from 2.6.1 to 2.6.2 #380 @mr-c
This version started using SpringBoot 2.6.1, and had other small code changes, and many dependencies upgraded.
Smaller changes:
- Convert label-schema to OCI annotations #361 @mr-c
Misc fixes:
- Added plug-in configuration for avoiding any usages of outdated log4j2 versions, some of which are subject to the RCE CVE-2021-44228 ("Log4Shell") and CVE-2021-45046 20c58b9d782802de04cdce3e975198940900a504 @mr-c
Dependencies upgrade:
- Upgrade to SpringBoot 2.6.1 #284 @mr-c @etzanis @kinow
- update pip & setuptools #364 @mr-c
- Dependency updates, courtesy of @dependabot-bot
- Bump jackson-core from 2.12.5 to 2.13.0 #358
- Bump jsonld-java from 0.13.3 to 0.13.4 #365
- Bump snakeyaml from 1.29 to 1.30 #366
- Bump hibernate-validator from 6.0.13.Final to 6.0.20.Final #370
- Bump jena-core from 4.3.1 to 4.3.2 #372
- Bump hibernate-validator from 6.0.20.Final to 7.0.1.Final #373
- Bump jackson-core from 2.13.0 to 2.13.1 #374
Many updates since 2018, but the most important is the fix (#355) for CVE-2021-41110 courtesy of @kinow
New features:
- Streamable CWL graph images #240 @stain
- Schedule recurrent CWL Viewer maintained cron-job for purging of old queued workflows from database #326 @obasekiosa
Smaller changes:
- Separate workflow URL from repository URL in "retrieved from" column of workflows page #316 @obasekiosa
- Fix replace non working deleteByRetrievedFrom function with working delete function #321 @obasekiosa
Documentation updates:
- Update README.md to explain better how to get started #308 @Anushka-shukla
- README.d: correct example URL to use port 8080 #311 @yichiehc
- Add links to the 2017 Video overview & Mark's report 612f5b4 b83b4fd 3380d44 @mr-c @stain
- Notes on running Mongo/Jena in Docker, and spring boot on host #334 @tetron
- Typos in README.md #349 @kinow
Misc fixes:
- Update jena & switch to Turtle syntax for SPARQL connection #213 @stain
- fix reversed class & id attributes #235 @mr-c & fixed by @kinow in #352 #353
- Use HTTPS instead of HTTP to resolve dependencies #250 [security update!] @JLLeitschuh
- Indexing retrievedOn in mongo to fix "Clicking 'Last' on the Explore page gives ISE" (#270) 922b434 @stain
- Update copyright year (#286) @stain
- add skip-schemas to the
cwltool
invocation so we are more lenient
Changes related to the migration of view.commonwl.org from University of Manchester to Curii, Inc (Many thanks to @stain and UNIMAN for their years of service to the public!)
- Add restart:always in docker-compose.yml #294 @cure
- Update the data controller for the https://view.commonwl.org instance. #297 @cure
- Tweaks for the docker-compose.yml file & set a larger internalQueryExecMaxBlockingSortBytes value for mongod #298 @cure
- Remove mention of dev instance 51a7d38 @tetron
Dependencies upgrade:
- Dependency upgrades to patch security vulnerabilities in transitive dependencies (Fixes CVE-2017-5929 CVE-2018-7489 CVE-2017-7525 CVE-2017-15095 CVE-2017-17485 CVE-2018-5968 CVE-2017-5651 CVE-2016-3093 CVE-2017-5648 CVE-2017-5650 CVE-2017-5647) #209 @MarkRobbo
- More dependency upgrades via @snyk-bot #223 #225 #289 #342
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHETHRIFT-173706
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHETOMCATEMBED-451342
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHETOMCATEMBED-451343
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHETOMCATEMBED-451458
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHETOMCATEMBED-451459
- https://snyk.io/vuln/SNYK-JAVA-ORGAPACHECOMMONS-460507
- https://snyk.io/vuln/SNYK-RUBY-FFI-22037
- https://snyk.io/vuln/SNYK-RUBY-JEKYLL-451462
- https://snyk.io/vuln/SNYK-RUBY-KRAMDOWN-585939
- https://snyk.io/vuln/SNYK-RUBY-ADDRESSABLE-1316242
- https://snyk.io/vuln/SNYK-RUBY-KRAMDOWN-1087436
- https://snyk.io/vuln/SNYK-RUBY-REXML-1244518
- Even more dependency updates, courtesy of @dependabot-preview
- Bump snakeyaml from 1.23 to 1.29 #233 #253 #276 #300 #340
- Bump jsonld-java from 0.12.1 to 0.13.3 #232 #242 #277 #315
- Bump jackson-core from 2.9.6 to 2.12.5 #229 #238 #246 #256 #267 #272 #278 #317 #241 #346
- Bump org.eclipse.jgit from 4.9.7.201810191756-r to 5.11.0.202103091610-r #230 #245 #251 #258 #259 #266 #271 #275#302
- Bump jena-osgi from 3.11.0 to 4.1.0 #237 #248 #264 #269 #309 #338
- Bump commons-compress from 1.19 to 1.21 #330 #343
- spring-boot 1.5.22 @mr-c ea1e273
CI updates:
- stop double testing PRs with Travis (#234) @mr-c
- speed up Travis by caching maven ea47b4a76fb9072f8aa4d2847edfc48a4f11d825 @mr-c
- Install codeql-analysis.yml #268 @mr-c (this helped @kinow and I find CVE-2021-41110; thanks https://github.com/github/codeql-action !)
- Update primary branch name to
main
3e3865c3f5b779458f8395f1cfc6c0589e89cb59 @mr-c - Mergify: configuration update a867bd3 1178ed2 @mr-c
- Upgrade to GitHub-native Dependabot 0a5b427 @mr-c
- Update codeql-analysis.yml to run less often 50401a5 @mr-c
- codeql: git checkout HEAD^2 is no longer necessary 2502986 @mr-c
- Permalinks with content negotiation
- Show workflow license
- Added Privacy Policy
- Support packed workflows in permalinks
- Use Python 3 and nodejs
- Support git tags
- UI: Shrink description text
- Fix for invalid branch names being accepted in some circumstances
- Fix regression in supporting slashes in branch names
- References
- Adds directory listing functionality
- Better support for packed workflows
- Use %26 (#) in the URL to view individual workflows as per cwltool
- Links to subworkflows within a packed file now function correctly
- Improved error reporting
- Fix for array and optional types not being parsed correctly
- Fix for twitter meta tags not containing the correct image URL
- Fix for a concurrency issue
- Brings API documentation up to date with the current state of the JSON API
- This release contains no changes - please use v1.2.1 instead
- References
- This patch release of CWL Viewer fixes documentation and a potential git checkout issue.
- References
- Support for any workflow stored in a Git repository
- Specific additional features eg linking directly to files for Github, Gitlab, Bitbucket
- Adds cwltool use for parsing - uses RDF representation with triple store
- Intermediate loading screen
- Research Object enhancements
- Add visualisation images as bundled aggregates
- RDF representation as annotation
- Packed version of workflow files as annotation
- Git2prov link for the repository as history
- Schema.org support for author attribution in CWL descriptions
- API and documentation backing current features
- Visualisation improvements
- Intermediate value names
- Ontology information linked and name given for format fields on inputs and outputs
- "About" page with best practices for writing CWL for parsing by the viewer
- Label and description searching on the explore page
- Various bug fixes
- References
- Bug fixes and attribution.
- References