Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training with a live streaming #105

Open
jinn9403 opened this issue Apr 11, 2024 · 1 comment
Open

Training with a live streaming #105

jinn9403 opened this issue Apr 11, 2024 · 1 comment
Labels
enhancement New feature or request question Further information is requested

Comments

@jinn9403
Copy link

Really good research and thanks for sharing the source. Especially the reconstruction using Gaussian splatting in fast time seems to be a step closer to real-time reconstruction.

When doing online training, I noticed that instantNGP has a function to show the training process while receiving streaming via nerfcapture app.
You used the dataset_capture_loop function to stream the dataset to a folder and train based on it. Is it possible to apply the def live_streaming_loop function to train in real time?

And the output is supposed to be viewable in open3d. If I want to view it in nerfstudio viewer or any other viewer, please let me know where to modify it.

@Nik-V9
Copy link
Contributor

Nik-V9 commented Jun 8, 2024

Thanks!

Yes, updating the code to show live training should be possible. However, we currently don't have this implemented. Furthermore, SplaTAM needs further optimization to enable faster reconstruction.

Regarding other viewers for Gaussian Splats: We support exporting our output to the standard .ply format used by various viewers like SuperSplat & PolyCam (not sure what NeRFStudio uses):

python scripts/export_ply.py configs/...

@Nik-V9 Nik-V9 added enhancement New feature or request question Further information is requested labels Jun 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants