PyPI v1.11.0 - ClearML

New Features and Improvements

• Add offline mode for ClearML Datasets (#854)
• Improve performance of Dataset.add_external_files() (#962, thanks @john-zielke-snkeos!)
• Add Task.launch_multi_node() for distributed experiment execution
• Add Task.get_all_reported_scalars() to fetch all available scalar data
• Add Task.remove_input_models() to disassociate input models from a task
• Improve Dataset.list_datasets() to include dataset version in the result
• Add support for PyTorch Lightning CLI >=2.0
• Add support for automatic Dataset content preview for JSON files
• Improve docstrings

Bug Fixes

• Fix multiple local parallel experiments conflicting on storage access validation (#897)
• Fix code running with joblib hangs (#1009)
• Fix comment for storage scheme gs:// (#1018, thanks @pzarfos!)
• Fix typo "Categories per column" (#1021, thanks @antonaleks!)
• Fix downloading artifacts from GCP when the download URI includes unsafe characters
• Fix pipeline step cache is not cleared when the only change to the step is that the docker image changed
• Fix Task.report_text() sometimes reporting to an incorrect task when multiple tasks run simultaneously
• Fix Task.set_offline(offline_mode=False) raising an UnboundLocalError
• Fix scikit-image package (skimage) is sometimes not detected as a dependency
• Fix PipelineDecorator sometimes causing a race condition when starting a remote execution

PyPI v1.10.4 - ClearML

New Features and Improvements

• Update references to GetAllRequest in Task docstrings (#982, thanks @Make42!)

Bug Fixes

• Fix issue with JSON conversion when using jsonargparse during remote execution (clearml-agent #153 and #1010)
• Fix ProxyDictPostWrite.update() not triggering a write-back to the backend (#985)
• Fix standard streams being patched in subtasks, which was causing the streams to be reported to the wrong task
• Fix TypeError when using Task.query_tasks() without specifying the task_filter parameter
• Fix Tensorboard negative iterations, they are now zeroed
• Fix Task.init(continue_last_task=0) when running remotely, now no longer ignored
• Fix urllib3 import error when using urllib3>=2.0.0

PyPI v1.10.3 - ClearML

Bug Fixes

• Fix mixed-up logs when using pipelines. Now pipeline controller logs won't be visible in pipeline steps' logs.

PyPI v1.10.2 - ClearML

New Features and Improvements

• Allow task filtering with Task.query_tasks() using TaskTypes (#938 )
• Allow adding tags to individual pipeline steps and to the PipelineController
• Restrict Dataset.create() usage for old server API versions
• Improve docstrings

Bug Fixes

• Fix Task.get_tasks docstring (#937)
• Fix Task.get_reported_scalars() docstring for x_axis parameter when set to "timestamp" (#964, thanks @jday1!)
• Fix missing plots on models in Windows
• Fix Pipeline version auto-bumping
• Fix APIClient.events.debug_images() for latest server API version

PyPI v1.10.1 - ClearML

New Features and Improvements

• Add cast parameter to Task.get_parameter method (#958, thanks @harry-optimised!)

Bug Fixes

• Fix backward compatibility issue when using datetime filtering with APIClient().models.get_all and APIClient().tasks.get_all requests
• Fix docstrings explaining tags filtering on Model and Task

PyPI v1.10.0 - ClearML

Breaking Changes

• When using MinIO or Azure storage, downloading using StorageManager.download_file() and StorageManager.download_folder() will not create a subfolder with the bucket name in the generated download path (#709)

New Features and Improvements

• Improve the Task.mark_completed, Task.close, and the hyperparameter example (#927, thanks @Make42!)
• Add Task.get_debug_samples() to retrieve task's debug samples (#761)
• Add custom artifact serialization for pipelines using the new artifact_{serialization,deserialization}_function parameters in PipelineController and PipelineDecorator (#689)
• Allow registering offline models (#653)
• Add table layout configuration to Logger.report_table() through extra_data parameter, solving (#796)
• Add support for attaching plots and scalars to models using the new Model.report_* methods
• Add force_download argument to Model.get_local_copy() to force the downloading of a new copy of a model even if already in the cache
• Add PipelineController.connect_configuration() to add configuration dictionaries to a pipeline
• Add Dataset.delete configurable hosted contents deletion support, allowing deletion of both ClearML file server hosted files, and external storage hosted files
• Add PipelineController.is_successful() criteria customization
• Adjust CLEARML_VCS_DIFF environment variable behavior, setting it to an empty string now forces the task to not log uncommitted changes
• Add support for API v2.23
• Add Getting Started Notebooks
• Add the output_uri=True argument in the hook config for the OpenMMLab example
• Add local HPO command line execution with the --local argument
• Add support for python package namespace detection

Bug Fixes

• Fix logging dependencies that use the subdirectory argument when pip installing from a git repo (#947, thanks @jday1!)
• Fix missing comma leading to a SyntaxError (#959, thanks @jday1!)
• Fix auto-logging for multiple models trained by the same framework in the same task (#767)
• Fix wrong overriding of custom server URLs when running clearml-init (clearml-server #181 and #910)
• Fix parallel coordinates plot display of categorical variables (#907)
• Fix task.connect() docstring (ClearML Docs issue #473)
• Fix auto-publishing in Dataset.finalize() raises exception due to backward compatibility issue (#908)
• Fix auto-detection of Jupyter Notebooks in Sagemaker
• Fix Gradio binding being triggered even when gradio is not imported
• Fix reconnection following an offline session. Only allow closed tasks to be reconnected
• Fix python-fire support
• Fix mishandling nested dictionaries in Task.connect() inside PipelineDecorator
• Fix get_or_create_project crashes when run in parallel
• Fix task.get_user_properties(value_only=True), previously raising an exception
• Fix silence_errors parameter not working as expected in StorageManager.get_file_size_bytes()
• Fix setting Dataset metadata name to a string prefixed with data_ breaking Dataset.get
• Fix Pipeline fails when creating tasks from multi-argument functions, due to a parsing error when working with Python comma slices

PyPI v1.9.3 - ClearML

Bug Fixes

• Fix broken Task._get_status() which was breaking clearml-session in the latest version
• Fix path substitution, making it possible now to store unsubstituted URL for models (#935 thanks @john-zielke-snkeos!)

PyPI v1.9.2 - ClearML

New Features and Improvements

• Support parsing queue name when providing execution queue in pipelines code (#857, thanks @Anton-Cherepkov!)
• Ignore None values for keys in the click argument parser (#903, thanks @chengzegang!)
• Improve docstrings for Task.mark_completed() and Task.close() (#921, #920, thanks @Make42!)
• Add pre/post execution callbacks to pipeline steps using @PipelineDecorator.component
• Add status-change callback to pipeline steps using PipelineController.add_step(), PipelineController.add_function_step(), and @PipelineDecorator.component

Bug Fixes

• Fix missing debug samples when reporting using TensorBoard (#924, thanks @jday1!)
• Fix wrong Jupyter API token during repository detection (#904, thanks @RoseGoldIsntGay!)
• Fix typo in the warning for very large git diffs (#932, thanks @yiftachbeer!)
• Fix pipelines from tasks don't propagate parameter_override values in PipelineController.add_step()
• Fix folders and files uploaded to S3 and Azure with StorageManager.upload_file() have wrong MIME types
• Fix CSV file preview in Datasets
• Fix Task.connect_configuration() doesn't work with non-string dictionary keys
• Fix LightGBM example deprecation warning
• Fix potential race condition in get_or_create_project()

PyPI v1.9.1 - ClearML

New Features and Improvements

• Add signature version to boto3 configuration (#884, thanks @cgaudreau-ubisoft!)
• Allow requesting custom token expiration using the api.auth.req_token_expiration_sec configuration setting
• Add Python 3.11 support

Bug Fixes

• Fix UniformParameterRange.tolist() throws error when step size is not defined (#859, thanks @davyx8!)
• Fix StorageManager.list() does not return size metadata (#865)
• Fix storage with path substitutions (#877, thanks @john-zielke-snkeos!)
• Fix extras in ClearML installation prevents clearml from being included in requirements (#885, thanks @cajewsa!)
• Fix metadata set on an uploaded model object is not accessible (#890, thanks @supritmkulkarni!)
• Fix TriggerScheduler docstrings (#881)
• Fix Azure storage upload not working (#868)
• Fix connect list of dicts parsed incorrectly in remote
• Fix casting None to int fails uploads and permission checks
• Fix numpy 1.24 support
• Fix clearml-data previews are saved on file server even when output_uri is specified
• Fix connecting a dictionary to task sometimes raises an exception
• Fix authentication headers are not set on substituted fileserver URLs
• Fix Task.get_project_id() cannot find hidden projects

PyPI v1.9.0 - ClearML

New Features and Improvements

• Add r prefix to re.match() strings (#837, thanks @daugihao!)
• Add path_substitution to clearml.conf example file (#842)
• Clarify deferred_init usage in Task.init() (#855)
• Add pipeline decorator argument to control docker image (#856)
• Add StorageManager.set_report_upload_chunk_size() and StorageManager.set_report_download_chunk_size() to set chunk size for upload and download
• Add allow_archived argument in Task.get_tasks()
• Support querying model metadata in Model.query_models()
• Add Dataset.set_metadata() and Dataset.get_metadata()
• Add delete_from_storage (default True) to Task.delete_artifacts()

Bug Fixes

• Fix jsonargparse and pytorch lightning integration broken for remote execution (#403)
• Fix error when using TaskScheduler with 'limit_execution_time' (#648)
• Fix dataset not synced if the changes are only modified files (#835, thanks @fjean!)
• Fix StorageHelper.delete() does not respect path substitutions (#838)
• Fix can't write more than 2 GB to a file
• Fix StorageManager.get_file_size_bytes() returns ClientError instead of None for invalid S3 links
• Fix Dataset lineage view is broken with multiple dataset dependencies
• Fix tensorflow_macos support
• Fix crash when calling task.flush(wait_for_uploads=True) while executing remotely
• Fix None values get casted to empty strings when connecting a dictionary