Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.KSplat Color issue with spherical harmonic #375

Open
hyqshr opened this issue Nov 19, 2024 · 11 comments
Open

.KSplat Color issue with spherical harmonic #375

hyqshr opened this issue Nov 19, 2024 · 11 comments

Comments

@hyqshr
Copy link

hyqshr commented Nov 19, 2024

While rendering my scene with .ksplat with 0 sphericalHarmonicsDegree, it worked well:

        const viewer = new GaussianSplats3D.DropInViewer({
            sharedMemoryForWorkers: false,
            showLoadingUI: true,
            format: GaussianSplats3D.SceneFormat.KSplat,
            sphericalHarmonicsDegree: 0
        });
image

But when I change the sphericalHarmonicsDegree to 1 or 2, the color seems to be broken:

image
@mkkellogg
Copy link
Owner

Out of curiosity, how many splats are in your scene?

@hyqshr
Copy link
Author

hyqshr commented Nov 21, 2024

Only 1 splat.

The broken color issue happens on .ply as well, but only when sh_degree is 1 (0 or 2 is fine). My model is trained from splatfacto-w (https://docs.nerf.studio/nerfology/methods/splatw.html) not sure if this the reason?

@mkkellogg
Copy link
Owner

Sorry let me rephrase, how many individual splat "blobs" are in your scene? If you press the 'I" key, you'll bring up an info panel that will give you that info:

image

@hyqshr
Copy link
Author

hyqshr commented Nov 21, 2024

I press "I" and I can not bring up the info panel. Is there other way I could print this?

@hyqshr
Copy link
Author

hyqshr commented Nov 21, 2024

Also - is there a python script to convert ply to ksplat? I think it can help a lot of folk's workflow. I can create a PR if needed

@mkkellogg
Copy link
Owner

If you want to send me your .ksplat or .ply file, I can tell you how many splats it has. My suspicion is that it has a large number, which can cause issues because there's not enough memory to store spherical harmonics data.

You could also load it in the online viewer here: https://projects.markkellogg.org/threejs/demo_gaussian_splats_3d.php and if you press 'I' in that viewer, it should bring up the info panel.

@hyqshr
Copy link
Author

hyqshr commented Nov 22, 2024

Below is the info panel. The weird thing is the broken color seems only happens at sh_degree=1 for this KSplat. 0 and 2 are fine

image

@hyqshr
Copy link
Author

hyqshr commented Nov 22, 2024

I have another model have the same issue when sh_degree is 1 in viewer

image

@mkkellogg
Copy link
Owner

That is pretty weird, would you mind sharing your .ply or .ksplat file and I can try to debug?

@hyqshr
Copy link
Author

hyqshr commented Nov 22, 2024

@mkkellogg
Copy link
Owner

mkkellogg commented Nov 23, 2024

Would you also be able to share your original .ply as well? And do you know how many SH degrees should be present in the file?

Also, I get an error when I try to access the above link:
image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants