Quick and dirty example of using Unity Barracuda v1.0 for inference using TensorFlow/Keras-created models using a custom made and trained CNN in Keras (TensorFlow v2.1 backend) on Dogs vs. Cats dataset.
barracuda_inference.cs explains how to perform inference step-by-step
If you're looking on how to convert Keras' .h5 models to .ONNX, check out example at this repo.
- Unity 2019.x (didn't test on older versions, but confirmed working on all 2019 versions)
- Unity Barracuda v1.0 (installation instructions: https://docs.unity3d.com/Packages/[email protected]/manual/index.html)
There are two provided images, cat2.png and dog2.png, both 150x150 RGB images (required by model) on which inference is run. The script is the most barebones needed for inference on images; can be a good starting point for further development.
Don't forget to check out official docs for Barracuda here!!!
- Create a new 2D project in Unity/Unity Hub
- Install Unity Barracuda v1.0 package (see link under Requirements)
- Import all files in Assets folder into the project's assets (drag and drop)
- Select cat2 and dog2 images in Project window inside Unity, and inside Inspector window, check Read/Write Enabled under Advanced (where to find it).
- Select Main Camera in the Hierarchy window, go to the Inspector window, click Add Component, and select barracuda_inference scritpt.
- In the same Inspector window (with Main Camera still selected under Hierarchy), drag and drop cats_dog_small_2.onnx, dog2.png into Sprite1, and cat2.png from Assets into Sprite2 attribute (how it's supposed to look).
- Press Run from within Unity, and observe output in the Console. Output is approx. 0 for cat photos, and approx. 1 for Dog photos.
