Look into lossless compression for raw video storage

[bgottula]

To get the most out of our camera we need to store the data in RAW format. At high resolutions and frame rates RAW format consumes a huge amount of bandwidth and disk space. It should be possible to reduce these resource demands (at the expense of CPU time) substantially using lossless compression. Investigate this and see how we might apply it. Some ideas:

• Apply lossless compression to video data during capture before writing to disk. This may require too much CPU power but would reduce disk write bandwidth.
• Apply lossless compression as a post-processing step after the capture is completed to conserve disk space. This could be done either immediately or later for archiving.

Some lossless video codecs exist (OpenCV's VideoWriter documentation mentions FFMPEG FFV1, Huffman HFYU, and Lagarith LAGS). Most of these codecs appear to assume grayscale, RBG, or YCbCr representations; however, the RAW data from our camera is in the form of a Bayer mosaic pattern. Thus use of these codecs would likely require application of a demosaic operation (interpolation to fill in the missing color information for each pixel), which is itself (potentially) lossy and which typically triples the size of the source. These issues are discussed in at least the following papers which propose alternative compression algorithms for mosaic images:

• http://www4.comp.polyu.edu.hk/~cslzhang/paper/RTI_05_Aug.pdf
• https://arxiv.org/pdf/0903.2272.pdf

I do not know if any widely available codec is similarly well-suited for mosaic images.

See also seeing-things/track#119 which has additional ideas for imagery post-processing, which arguably does not belong in the track project.