Optimize image acquisition and processing by reducing image write and read operations

[georgeslabreche]

Background

The execution time between the moment the camera captures an image and when that image gets classified can be reduced by eliminating the number of times an image is being written to file and the read back into memory.

Current Process

1. Image acquisition is trigger by executing the ims100_testapp binary which outputs a raw image file (ims_rgb) and a png file.
2. The png file is read, downsampled, and converted to a thumbnails jpeg file with the following piped Netpbm programs outputs: pngtopam {F} | pamscale {S} | pnmtojpeg -quality {Q} > {O}.
3. The png file is read, downsampled, and converted to an image classification input file with the following piped Netpbm programs outputs: pngtopam {F} | pamscale -xsize {X} -ysize {Y} | pnmtojpeg -quality {Q} > {O}.
4. The image classification input file is read by the image classifier binary executable and then classified.
5. Delete the image classification input file created in step 3.

Optimization

Option A – Full Optimization:

• Integrate image acquisition and classification into a single binary that processes the image file in a memory buffer and only writes the ims_rgb, png, and thumbnail jpeg file at the very end when classification is complete.
• Implement a threadpool / queue to classify the image as soon as the camera is done writing the image data in memory so that the camera is not blocked waiting for the image processing and classification process to be completed before it can start capturing the next picture.

Option B – Partial Optimization (In-Memory):
Get rid of steps 2 and 3 in the current process and replace them with in-memory image processing within the image classifier.

Option C – Partial Optimization (File writing):
Get rid of steps 2 and 3 in the current process and replace them with a custom dedicated executable binary.

Resources

Use the stb headers to process images in-memory:

• stb_image.h
• stb_image_resize.h
• stb_image_write.h

[georgeslabreche]

There's ongoing work for Option C. Here's the custom dedicated executable binary to replace the SmartCam's dependency on piped Netpbm program outputs:
https://github.com/georgeslabreche/image-resizer

[georgeslabreche]

Option C was implemented as part of issue #13 and merged with PR #18.