Need help with state and timing of gl.render #3283

dayunbao · 2024-06-10T13:56:50Z

dayunbao
Jun 10, 2024

Hi,

I'm using Next.js, Zustand, and React Three Fiber in my app. What I want to do is take 2 screenshots of the canvas: 1 of just the character, and 1 of just the background. There is a button that when clicked is supposed to take the screenshots. However, it isn't working. Here are the most relevant components:

export default function GenerateBackgroundModel({url}: {url: string}) {
  const { backgroundVisible } = useImageGenerationEditorStore((state) => state);
  const { scene } = useGLTF(url);
  
  return (
        <primitive visible={ backgroundVisible } position={[0,0,0]} object={scene} />
  )
}

export default function GenerateCharacterModel({url}: {url: string}) {
  const characterRef = useRef<any>();
  const { selectedPose, time, showControls, characterVisible } = useImageGenerationEditorStore((state) => state);
  const { scene } = useGLTF(url);
  const { animations } = useGLTF(selectedPose.animation.glb);
  const { actions, names, clips } = useAnimations([animations[0]], scene);

  if(clips[0].tracks[0].name === "Hips.position") {
    clips[0].tracks.shift();
  }

  useEffect(() => {
    const action = actions[names[0]];
    action!.play();
    action!.paused = true;
    const updateTime = time * action!.getClip().duration;
    action!.time = updateTime;
  }, [actions, time]);
  
  return (
    <group>
      <PivotControls
        anchor={[0, 0.25, 0]} 
        scale={50} 
        depthTest={false} 
        fixed 
        lineWidth={2}
        visible={showControls}
    >
        <mesh
          ref={characterRef}
          >
        <primitive 
          castShadow 
          receiveShadow 
          object={scene} 
          visible={ characterVisible }
        />
      </mesh>
    </PivotControls>
  </group>
  )
}

export default function CharacterStage() {
           ...
           <CameraControls
                ref={cameraControlRef}
                makeDefault
                mouseButtons={{left: ROTATE, right: TRUCK, middle: NONE, wheel: DOLLY }}
            />
            <Stage
                adjustCamera={false}
                intensity={0.5}
                preset="soft"
            >
                    <GizmoHelper alignment="top-right" >
                        <GizmoViewport axisColors={['red', 'green', 'blue']} />
                    </GizmoHelper>
                    <Center position={[0, 0, 0]}>
                        {selectedBackground &&
                            <Suspense fallback={<Html center> {progress} % loaded </Html>}>
                                <GenerateBackgroundModel url={selectedBackground} />
                            </Suspense>
                        }
                        <Center position={[0, 0, 3]}>
                            {selectedCharacter.url &&
                                <Suspense fallback={<Html center> {progress} % loaded </Html>}>
                                    <GenerateCharacterModel url={selectedCharacter.url} />
                                </Suspense>
                            }
                        </Center>
                </Center>
            </Stage>
           ...

So what I've been trying to do is something like this (in Character Stage):

useEffect(() => {
        if(!backgroundVisible) {
            gl.render(scene, camera);
            const characterImg = gl.domElement.toDataURL();
            setCharacterImage(characterImg);
            setBackgroundVisible(true);
            setCharacterVisible(false);
        }

        if(!characterVisible) {
            gl.render(scene, camera);
            const backgroundImg = gl.domElement.toDataURL();
            setBackgroundImage(backgroundImg);
            setCharacterVisible(true);
        }

    }, [backgroundVisible, characterVisible])

The problem is getting the timing of the toDataURL calls to sync with the re-render so that the correct model has been hidden. With my current code, I get screenshots when both the character and background are hidden.

What I want is to hide the background, take a screenshot; show the background and hide the character, then take a screenshot.

Can anyone help me figure out the best way to do this?

dayunbao · 2024-06-12T12:44:28Z

dayunbao
Jun 12, 2024
Author

I'm open to other solutions, as well, like layering two canvases, putting the background model in one, and the character model in the other. However, this issue is blocking me in my work, so I'd love to hear some suggestions! What do you think @drcmda?

Thanks!

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Need help with state and timing of gl.render #3283

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 1 comment

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Need help with state and timing of gl.render #3283

dayunbao Jun 10, 2024

Replies: 1 comment

dayunbao Jun 12, 2024 Author

dayunbao
Jun 10, 2024

dayunbao
Jun 12, 2024
Author