LK and HS Optical Flow simulated with a Particle Engine using CUDA/OpenGL InterOp.

Theory - Determining Optical Flow

Horn Schunck Method Lucas Kanade Method

Performance Stats

1. Final render: 3840x2160x12 @ 37 fps on an Nvidia 1060 GTX GPU assembled on a Surface Book Pro 2
2. Final render: 3840x2160x12 @ 93 fps on an Nvidia 2070 RTX (OFF) assembled on an MSI GS66 Stealth

Demo:

You can find the coolest state of the project here on Vimeo.

EDIT: The one with CFD physics is being located from my drive.

Dependencies:

1. CMake v3.8+ [for CUDA support within CMake]
2. CUDA v9.0+
3. OpenCV 4.3.0 [for reading images and webcam]
4. GLFW
5. GLEW (Only Windows)
6. GLM

Installation:

1. Let $PROJECT_ROOT be the root directory of the project.
2. Download an example video from here and place it in $PROJECT_ROOT/src/resources.
3. After installing the dependencies please follow the cmake build steps in the code block below.
4. After the binary has been compiled, place it in $PROJECT_ROOT/src directory.
5. Run .\particle_atractor

cd $PROJECT_ROOT
mkdir build
cd build
cmake ..
make -j4

Roadmap:

• CUDA based particle engine.

  • Event system for Mouse, Keyboard and Window
  • Object file reader
  • Shader code parser
  • Rendering Engine
  • Instanced Rendering for "pixel" objects
  • Materials and Lighting (Specular, Ambient and Diffuse)

• Simulate gravity using CUDA OpenGL Interop.

• Integrating OpenCV with the particle engine to render frames captured from the video file.

• CUDA Kernel for LK and HS based velocity vector computation.

  • Central difference approximation
  • HS Kernel
  • LK Kernel

• Update particle engine physics to simulate Hooke's Law Gradient descent.

• Navier Stokes based CFD Physics.

  • 2D advection
  • 2D Diffusion
  • Cavity Flow

  Lessons learnt

  1. Heap allocations are expensive.
  2. Locality of reference is critical.
  3. Random access is costlier than sliding things in a cache.