I think in this context we should add some manual overwritting for the batching as well.
For Example we have one person that wants to run the some MDSuite Job in the background, does not care about the speed and wants to do something else in the meantime.
On the other side there is someone who wants the results asap and MDSuites batching is to agressiv and forcefully slows down the computation.

I think this could be handled by an argument in mdsuite.config which by default is config.batch_scaling = 1.

This is more in the context of testing the scaling functions than the actual scaling though.

I would go for 8 GB though.

The initial idea was that you set the memory consumption which is hardcoded to be super conservative rn because the scale functions are not accurate enough. I think having a dev option for manual batching is a good idea but I would leave it to setting memory consumption for users e.g., use 50% of memory vs 98% if you want that super performance.

The reason I think 4GB is because it makes the scaling very obvious. The smaller the better in fact, if it is safe enough to run on 1GB then it is safe to run on 100. By setting it to 4GB we pretty much assure that runs on all standard GPUs and that the scaling is good, hence I see no reason to move to 8.

Having it set through config would be great. I was having problems today with getting TF to stop using the GPU so we should probably make the config file something that is easier to work with on a user end where we set stuff like turn of GPU, use this much memory, set logging and so on.