Update the README for the pre-release (#14)

README.md

@@ -1,19 +1,38 @@
 # Oxford Spires Dataset
-This repository contains scripts that are used to evaluate Lidar/Visual SLAM on the Oxford Spires Dataset.
+This repository contains scripts that are used to evaluate localisation, 3D reconstruction and radiance field methods using the [Oxford Spires Dataset](https://dynamic.robots.ox.ac.uk/datasets/oxford-spires/). 
 
-## Installation
-### oxford_spires_utils (Python)
+This is a pre-release of the software. The codebase will be refactored in the near future.
+
+## Localisation Benchmark
+The localisation benchmark runs LiDAR SLAM methods ([Fast-LIO-SLAM](https://github.com/gisbi-kim/FAST_LIO_SLAM), [SC-LIO-SAM](https://github.com/gisbi-kim/SC-LIO-SAM), [ImMesh](https://github.com/ori-drs/ImMesh_hesai)) and LiDAR Bundle Adjustment method ([HBA](https://github.com/hku-mars/HBA)). The resultant trajectory are evaluated against the ground truth trajectory using [evo](https://github.com/MichaelGrupp/evo).
+
+Build the docker container and run the methods:
 ```bash
-pip install -e .
+cd oxford_spires_dataset
+docker compose -f .docker_loc/docker-compose.yml run --build spires
+
+# in the docker
+python scripts/localisation_benchmark/colmap.py
+python scripts/localisation_benchmark/fast_lio_slam.py
+python scripts/localisation_benchmark/immesh.py
+python scripts/localisation_benchmark/vilens_hba.py
 ```
 
-### spires_cpp (C++ Pybind)
-Install [octomap](https://github.com/OctoMap/octomap) and [PCL](https://github.com/PointCloudLibrary/pcl) to your system, then
+
+## Reconstruction Benchmark
+The reconstruction benchmark runs Structure-from-Motion ([COLMAP](https://colmap.github.io/)), Multi-view Stereo ([OpenMVS](https://github.com/cdcseacave/openMVS)), radiance field methods ([Nerfstudio](https://github.com/nerfstudio-project/nerfstudio/tree/main/nerfstudio)'s Nerfacto and Splatfacto), and generates 3D point cloud reconstruction, which is evaluated against the TLS-captured ground truth 3D point cloud.
+
+Build the docker container and run the methods:
 ```bash
-cd spires_cpp
-pip install -e .
+cd oxford_spires_dataset
+docker compose -f .docker/docker-compose.yml run --build spires
+
+# inside the docker
+python scripts/reconstruction_benchmark/main.py --config-file config/recon_benchmark.yaml
 ```
 
+## Novel-view Synthesis Benchmark
+Currently, the NVS benchmakr is included in the reconstruction benchmark script, since it builds upon output from COLMAP. 
 
 ## Contributing
 Please refer to Angular's guide for contributing(https://github.com/angular/angular/blob/22b96b96902e1a42ee8c5e807720424abad3082a/CONTRIBUTING.md).

oxford_spires_utils/eval.py

@@ -46,7 +46,7 @@ def get_recon_metrics(
 
 
 def get_recon_metrics_multi_thresholds(
-    input_cloud: np.ndarray, gt_cloud: np.ndarray, thresholds: list = [0.02, 0.05, 0.1], max_distance=2.0
+    input_cloud: np.ndarray, gt_cloud: np.ndarray, thresholds: list = [0.02, 0.05, 0.1], max_distance=9999.0
 ):
     assert isinstance(input_cloud, np.ndarray) and isinstance(gt_cloud, np.ndarray)
     assert input_cloud.shape[1] == 3 and gt_cloud.shape[1] == 3

scripts/reconstruction_benchmark/main.py

@@ -217,9 +217,9 @@ def run_nerfstudio(
             self.evaluate_reconstruction(metric_cloud_gt_frame)
 
     def evaluate_reconstruction(self, input_cloud_path, results_dir=None):
-        assert input_cloud_path.exists(), f"Input cloud not found at {input_cloud_path}"
+        assert Path(input_cloud_path).exists(), f"Input cloud not found at {input_cloud_path}"
         assert Path(input_cloud_path).suffix == ".pcd", "Input cloud must be a pcd file"
-        assert self.gt_octree_path.exists(), f"Ground truth octree not found at {self.gt_octree_path}"
+        assert Path(self.gt_octree_path).exists(), f"Ground truth octree not found at {self.gt_octree_path}"
         recon_thresholds = [0.03, 0.05, 0.1, 0.2]
         results_dir = self.recon_benchmark_dir if results_dir is None else Path(results_dir)
         input_cloud = o3d.io.read_point_cloud(str(input_cloud_path))