Cleanup IntervalList orphans in weekly job only (#1195)

* cleanup interval orphans in nightly job only

* update changelog

* Update docs/src/ForDevelopers/Management.md

Co-authored-by: Chris Broz <[email protected]>

* suggest less frequent running of IntervalList cleanup

---------

Co-authored-by: Chris Broz <[email protected]>

CHANGELOG.md

@@ -44,8 +44,10 @@ dj.FreeTable(dj.conn(), "common_session.session_group").drop()
 - Merge table delete removes orphaned master entries #1164
 - Edit `merge_fetch` to expect positional before keyword arguments #1181
 - Allow part restriction `SpyglassMixinPart.delete` #1192
+- Move cleanup of `IntervalList` orphan entries to cron job cleanup process #1195
 - Add mixin method `get_fully_defined_key` #1198
 
+
 ### Pipelines
 
 - Common

docs/src/ForDevelopers/Management.md

@@ -228,10 +228,16 @@ disk. There are several tables that retain lists of files that have been
 generated during analyses. If someone deletes analysis entries, files will still
 be on disk.
 
-To remove orphaned files, we run the following commands in our cron jobs:
+Additionally, there are periphery tables such as `IntervalList` which are used
+to store entries created by downstream tables. These entries are not
+automatically deleted when the downstream entry is removed. To minimize interference
+with ongoing user entry creation, we recommend running these cleanups on a less frequent
+basis (e.g. weekly).
+
+To remove orphaned files and entries, we run the following commands in our cron jobs:
 
 ```python
-from spyglass.common import AnalysisNwbfile
+from spyglass.common import AnalysisNwbfile, IntervalList
 from spyglass.spikesorting import SpikeSorting
 from spyglass.common.common_nwbfile import schema as nwbfile_schema
 from spyglass.decoding.v1.sorted_spikes import schema as spikes_schema
@@ -241,6 +247,7 @@ from spyglass.decoding.v1.clusterless import schema as clusterless_schema
 def main():
     AnalysisNwbfile().nightly_cleanup()
     SpikeSorting().nightly_cleanup()
+    IntervalList().cleanup()
     nwbfile_schema.external['analysis'].delete(delete_external_files=True))
     nwbfile_schema.external['raw'].delete(delete_external_files=True))
     spikes_schema.external['analysis'].delete(delete_external_files=True))

src/spyglass/common/common_interval.py

@@ -158,7 +158,7 @@ def plot_epoch_pos_raw_intervals(self, figsize=(20, 5), return_fig=False):
         if return_fig:
             return fig
 
-    def nightly_cleanup(self, dry_run=True):
+    def cleanup(self, dry_run=True):
         """Clean up orphaned IntervalList entries."""
         orphans = self - get_child_tables(self)
         if dry_run:

src/spyglass/utils/dj_mixin.py

@@ -507,9 +507,6 @@ def cautious_delete(
                 delete_external_files=True, display_progress=False
             )
 
-        if not self._test_mode:
-            _ = IntervalList().nightly_cleanup(dry_run=False)
-
     def delete(self, *args, **kwargs):
         """Alias for cautious_delete, overwrites datajoint.table.Table.delete"""
         self.cautious_delete(*args, **kwargs)