Saving data when SIGTERM is sent to the solver.

[bigfooted]

I would like SU2 to save the current solution when SIGTERM is sent to the job.
So here is my current understanding of how this is done by others:

Some (CFD) software packages will save the solution at the current iteration when you kill the job. I think what all these packages do is that they specifically capture SIGTERM. As far as I know, packages like slurm and LSF send 3 signals, SIGCONT or SIGINT, SIGTERM and SIGKILL, in this order. By capturing SIGTERM, you do not save files if you just hit on the command line when you start SU2 there (sending SIGINT) and with cluster jobs you can still force the job to not write solutions files by sending SIGKILL immediately.

So I think it's a matter of catching the signal and then exit the main loop as if it finished all the iterations. Maybe somebody with more experience in this matter can comment on this approach...

[pcarruscag]

We'd have to try, I don't know if all MPI ranks get the signal for example.
And the signal handler is kind of detached (async) from the main parts of the code, so signaling back to the solver might be tricky.
Many codes also look for a specific file and stop based on that.