You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In special_cases::harvester::ignore_processed_files::[site] (introduced in #341), we use a YAML reference to an array of files. At the time, I thought this would be both DRY and easily extensible - YAML can use references to merge dicts, so why not arrays?
It turns out that YAML can't merge arrays, at least not in a way that results in a predictably flat results. (- *DUPLICATED_PASAT_FILES translates to a list of lists.) Under the current structure, you can only extend the referenced array, or you're out of luck.
This should only require a straightforward change to config_utils.py::flatten_path_dict, which checks if val is None, and if so, output.append(new_prefix).
The text was updated successfully, but these errors were encountered:
In
special_cases::harvester::ignore_processed_files::[site]
(introduced in #341), we use a YAML reference to an array of files. At the time, I thought this would be both DRY and easily extensible - YAML can use references to merge dicts, so why not arrays?It turns out that YAML can't merge arrays, at least not in a way that results in a predictably flat results. (
- *DUPLICATED_PASAT_FILES
translates to a list of lists.) Under the current structure, you can only extend the referenced array, or you're out of luck.A workaround is a value-less dict that looks like this:
After parsing, this comes out as
This should only require a straightforward change to config_utils.py::flatten_path_dict, which checks if
val
isNone
, and if so,output.append(new_prefix)
.The text was updated successfully, but these errors were encountered: