Docker: Add GPU Support for Tensorflow

[DiogenesAnalytics]

Problem

Currently the Docker image being built does not have the correct drivers to work with the version of tensorflow pinned in the pyproject.toml file:

autoencoder/pyproject.toml

Line 20 in ca2c35b

tensorflow = ">=2.15.0"

Solution

What seems to work is installing tensorflow[and-cuda] via pip according to the Tensorflow install documentation. Now the question becomes: how to handle this optional install with the autoencoder library (which uses poetry as its build backend)?

References

• https://blog.tensorflow.org/2023/11/whats-new-in-tensorflow-2-15.html
• https://discuss.tensorflow.org/t/tensorflow-install-error/20491
• TF 2.14 minimum nvidia driver version tensorflow/tensorflow#62032
• https://python-poetry.org/docs/pyproject/

[DiogenesAnalytics]

Implemented (finally) with: 1c7b1f6

[DiogenesAnalytics]

Problem

Installing the and-conda extras (i.e. as tensorflow[and-conda]) makes the resulting Docker image MASSIVE:

Solution

Need to work with Poetry docs to figure out the correct way to make these CUDA libs optional ...

[DiogenesAnalytics]

References

Some references specific to this issue.

Poetry

• https://python-poetry.org/docs/pyproject/
• https://python-poetry.org/docs/pyproject/#extras

TensorFlow

• https://blog.tensorflow.org/2023/11/whats-new-in-tensorflow-2-15.html
• https://www.tensorflow.org/install/pip#linux

NVIDIA

• https://developer.nvidia.com/cuda-gpus
• https://www.nvidia.com/download/index.aspx?lang=en-us
• https://developer.nvidia.com/cuda-toolkit-archive
• https://docs.nvidia.com/cuda/archive/11.8.0/
• https://developer.nvidia.com/cudnn
• https://docs.nvidia.com/deeplearning/tensorrt/archives/index.html#trt_7
• https://stackoverflow.com/questions/61533291/is-it-still-necessary-to-install-cuda-before-using-the-conda-tensorflow-gpu-pack/61538568#61538568

PIP

• https://stackoverflow.com/questions/30239152/specify-extras-require-with-pip-install-e