[DAR-4788][External] Allow download of frames extracted from videos & multi-slotted items #972
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
darwin-py
will download 2 files for every frame that was extracted from a video upon upload: The frame itself, and the video file. This results in massively duplicated data, much slower downloads (due to re-downloading videos), and an incorrect local folder structuredarwin-py
will crash when trying to pull a release where any file is a multi-slotted item or a DICOM seriesSolution
Problem 1 occurs because on upload, each extracted frame is associated with two source files:
When darwin-py encounters a slot associated with >1 source file, it tries to download every source file. This PR makes the following change: There are only 2 supported types of data where >1 source file can be associated with a slot, so we check:
If we detect an extracted frame, we select only the image for download. If we find that the slot is neither supported type of data, we raise an error
Problem 2 occurs because we are trying to access each source file with
source_file.file_name
.source_file
is a dictionary, so instead we need to dosource_file["file_name"]
Changelog
Resolved issues with: