basic-non-linear-optimisation

Examples of solving non-linear optimisation problems using ceres and lietorch.

ceres

The program could give a good idea on how to model a simple optimisation program based on re-projection error. In addition, since we're optimising for rotation coordinates, it shows how to explicitly set the rotation on a quaternion manifold to ensure the updates yield a valid rotation.

installation

Make sure to install ceres solver.

Important

The program has been tested with ceres-2.2 - it contains some functions that were introduced with later versions of ceres, and may not work with previous versions.

The only dependencies required to run make_noisy.py are scipy and numpy.

problem statement

We are given 5 images in the data folder, and their correspondences in the Correspondences sub-directory. Our goal is to optimise for the extrinsics of each image using reprojection loss, given camera intrinsics of $f_x=f_y=721.5$ and $(c_x, c_y)=(256, 176)$.

The correspondences are provided in the format p_x p_y X Y Z i:

• p_x, p_y are the 2D image coordinates
• X, Y, Z are the coordinates of the 3D correspondences for the 2D coordinate.
• i is the 3D point's index in the point cloud and can be ignored for this problem.

The extrinsics directory contains ground-truth extrinsics for us to compare against.

running

The rest of the instructions assume you're in the ceres directory:

cd ceres

First, create some initialisations for our non-linear optimisation, done here by noising the ground-truth extrinsics and storing them in init.

python make_noisy.py

Note

Ceres successfully solves for the extrinsics even with very large noise of, say, 50 along the translation coordinates and 2 (radians) along the rotation coordinates. Alter the noise in the make_noisy file to experiment with this.

Create the executable

mkdir -p src/build
cd src/build
cmake ..
make 
cd -

For each pose "index" (denoting the image index in the data), solve for the extrinsics:

./src/build/optimize 1

To run all of the above steps, and solve for all of the extrinsics, storing the results in the results directory, simply run:

bash run.sh

lietorch

The program should give a simple example on how to create lie group parameters in lietorch to enable tangent-space backpropogation.

installation

The general dependencies are torch, open3d and lietorch. The optimisation itself is not dependent on open3d but has been added to enable the creation of some visualisations later.

An env file with the dependencies in a system with cuda 12.4 can be found at deps.yml.

lietorch is not easy to install. The only method that I could get to work consistently is:

• install all non-lietorch dependencies first (torch, open3d)
• install the dependencies needed for lietorch (scipy, pyyaml)
• install lietorch from source. The binaries on pip and conda for lietorch have never worked for me.

problem statement

ICP Slam: We are given N points over M frames. The reading of the points in the various frames are noisy. We optimise for the point coordinates in some 'base' frame, and the transformation from the 'base' frame to the other frames in consideration.

The residual can be framed as:

$$ \sum_{i=1}^m\sum_{j=1}^n ||\vec{X_{ij}} - \hat{T_{i0}}\hat{X_{0j}}|| $$

credits: Sai Shubodh

We optimise this residual.

Note

The points in the pointcloud toothless.ply are used as input to create a viable ground truth point coordinate set.

running

Simply run optimize.

cd lietorch
python optimize.py

Misc.

The above programs formed a subset of a few assignments of the Mobile Robotics course, IIIT-Hyderabad, Monsoon '24.