Cleanup task executions older than n days - composed tasks 2.11.0

[klopfdreh]

Description:
I just tested the new feature of clean up task executions older than n days and found out that composed tasks are not cleaned up correctly. The entries remain in the list but with Duration / Start date / End date set to N/A.

Release versions:
2.11.0

Custom apps:
N/A

Steps to reproduce:

• Launch some compose tasks
• Adjust their dates to be in the past in the task executions table
• Cleanup older than n days in between some executions

Screenshots:
N/A

Additional context:
N/A

[klopfdreh]

I just performed two calls with a CTR and edit the TASK_EXECUTION table.

The first task and sub tasks I set to 09-20-2023 and the seconds to 09-19-2023

1. I performed a cleanup with older than 1 days which deleted exactly the task-execution of 09-19-2023
2. I performed another cleanup with 09-20-2023 and cleanup older than 0 days which deleted the other one.

The new functionality is working like expected.

Maybe it would be nice to make the option Cleanup older execution(s) not depend on Only complete execution(s) which was @oodamien's first implementation and add a way to also cleanup tasks with no timestamp which are not completed.

Then you could cleanup all executions older than n days or with no timestamp that are not completed, too.

This is important as the jobs might fail because of some infrastructure stuff in kubernetes or an OOM Exception occurred and they are still in RUNNING as the state couldn't be written.

WDYT?

[onobc]

way to also cleanup tasks with no timestamp which are not completed.

My only concern is that this would also kill any currently executing tasks. Maybe we could do a sanity check so that the start timestamp is not within the last N minutes.

[klopfdreh]

I understand your concern, but a task / job can fail at any point even if there is no start / end time. In this case you can’t clear them but only with a full clean of all tasks

[onobc]

understand your concern, but a task / job can fail at any point even if there is no start / end time.

I think this feature will be fine w/ these added constraints:

• If a task has no start time then I think it could be considered ready for removal
• If a task has a start time but no end time and that start time is passed some time period (15mins??) then it is ready for removal

[klopfdreh]

We have tasks that run 2 hours - I think it would be good to store this value when the task is ready for removal.

But this would be good yes!

[onobc]

We have tasks that run 2 hours

Yeh we would likely need to expose a property for the min cull time (or something like this).