Job preservation for cron scheduled jobs

[bforma]

We have config.good_job.preserve_job_records = false configured, however in the good job UI I still see succeeded jobs being preserved. It seems this is only the case for jobs that have been enqueued via a cron schedule. Is this a feature, or a bug? And how can I configure good job to also not preserve jobs that have been enqueued via a cron schedule?

[Earlopain]

Good question! I recently had a similar though about how preserving jobs interacts with cron.

In short, these jobs must be preserved in order to guarantee that they don't double-enqueue, for example with multiple processes or during restarts with the newly added cron_graceful_restart_period.

Some small tidbits in #1488 / #767 /#731. For that reason, it's not possible to not preserve them. What is your use-case for wanting them deleted like other jobs?

[bforma]

What is your use-case for wanting them deleted like other jobs?

Keeping the good job queue tables small in row count. I'm assuming performance will degrade when they grow too much (indefinitely in this case). The UI becomes sluggish as well (although I think some improvements have been made, such as showing total counts with async requests).

[Earlopain]

I'm assuming performance will degrade when they grow too much

Yes, table size does play a role in query performance. Personally I can't say much about when it will actually become a problem (depends on your setup as well) but there are some good data points in the issues/discussion here if you want to take a look.

indefinitely in this case

The default for preservation is 14 days. I'm pretty sure that is supposed to clean up these cron jobs as well (and if it doesn't, probably a bug). Is that what you are observing?

If you're enqueueing so many cron jobs that those alone are a problem over that period, you can probably lower cleanup_preserved_jobs_before_seconds_ago to purge them earlier. https://github.com/bensheldon/good_job?tab=readme-ov-file#monitor-and-preserve-worked-jobs
I haven't tested this but am somewhat confident that those values will do their job regardless of preserve_job_records.

[bforma]

Thanks @Earlopain for the help and tips! I'll have a look at what we are actually experiencing and will report back here.

[bforma]

I think I can report that there is no bug here regarding to cleaning up cron scheduled jobs. I've configured the following:

Rails.application.configure do
  config.good_job.preserve_job_records = false

  # Cron scheduled jobs are preserved, even when preserve_job_records is false,
  # so configure cleanup for those jobs in a short timespan (the default is 14 days).
  config.good_job.cleanup_preserved_jobs_before_seconds_ago = 2.days
end

And I noticed that cron scheduled jobs were indeed being cleaned up.

For "future us" I would however like this to be documented a bit more clear.