MAAP-Project
-
|
By default the output of a DPS job is nested pretty deep, something like I assume that /year/month/day/hour/minute... is approximately the timestamp of the end of the job execution. As this is unknown to the user and job specific, it makes the task of finding output pretty cumbersome for a many-jobs workflow. Is this path configurable ? @sujen1412 ? If not I think we should have it as a to-do. |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
This path currently is not configurable other than the identifier. Some context of the path being the way it is: |
Beta Was this translation helpful? Give feedback.
This path currently is not configurable other than the identifier.
We (cc @rtapella) are exploring multiple options and will reach out to the users with those options before implementing them for any suggestions.
Some context of the path being the way it is:
The DPS we are using is the same one being used by Earth Science missions for their data processing which have some strict requirements on provenance tracking. Adapting this to something like MAAP, where users have much more autonomy on what they run/write out, we needed a way to always have unique paths and not have jobs overwrite their outputs (even if the same job was run twice).