How can I smooth transition perspectiveCamera.lookAt(vector) on Click to next marker?

[JustFly1984]

Hello, Have a great time of the day!
Thank you for this amazing breakthrough with react and WebGL.
I had approached 3d in react multiple times on the range of 10 years, and building 3d in browser never been so approachable till now.

I'm making a game to prove the concept.

Currently I have a plane with a grid of polygons. Each polygon in the scene has unique center coordinates.

I've positioned the camera, and I wrote onClick handler for each polygon, so it updates redux with new coordinates for Camera.lookAt()

The movement of the camera is instant, and do not feels natural, so I want to add a smooth transition on redux values change.

I guess that I could use React.useEffect hook in my custom <CameraController /> to track lookAt.x, lookAt.y and lookAt.z props change, but I do not know how to smoothly transition camera targeting from initial position to the next one.

I can't find docs for @react-spring/three, and can't install react-spring/three - it is removed from npm

How do I transition camera.lookAt() smoothly from [0, 0, 0] to [5, -5, 0]?

I have my CameraController component:

import * as React from 'react'
import { Vector3, PerspectiveCamera } from 'three'
import { useSelector } from 'react-redux'
import { useFrame, useThree } from 'react-three-fiber'

import type { ReduxState } from 'src/types/common'
import type { Cube } from 'src/types/game'

interface SelectorProps {
  lookAt: Cube
  position: Cube
}

function selector(state: ReduxState): SelectorProps {
  return {
    lookAt: state.camera.lookAt,
    position: state.camera.position,
  }
}

function CameraController(): JSX.Element {
  const ref = React.useRef<PerspectiveCamera | null>(null)

  const { lookAt, position } = useSelector(selector)

  const vectorLookAt = React.useMemo(
    function memo() {
      return new Vector3(lookAt.x, lookAt.y, lookAt.z)
    },
    [lookAt.x, lookAt.y, lookAt.z]
  )

  const vectorPosition = React.useMemo(
    function memo() {
      return new Vector3(position.x, position.y, position.z)
    },
    [position.x, position.y, position.z]
  )

  const { setDefaultCamera } = useThree()

  // Make the camera known to the system
  React.useEffect(
    function effect() {
      if (ref.current !== null) {
        setDefaultCamera(ref.current)

        ref.current.lookAt(vectorLookAt)
        ref.current.updateMatrixWorld()
      }
    },
    [vectorLookAt, setDefaultCamera]
  )

  // Update it every frame
  useFrame(function renderCallback() {
    if (ref.current !== null) {
      ref.current.updateMatrixWorld()
    }
  })

  return <perspectiveCamera ref={ref} fov={75} position={vectorPosition} />
}

export default React.memo(CameraController)

and the Tile component which dispatches an action to redux to update the camera lookAt position to it's own coordinates onClick

import * as React from 'react'
import { Vector3, Object3D, Shape, DoubleSide } from 'three'
import { useSelector, useDispatch } from 'react-redux'

import { cameraLookAt } from 'src/redux/actions/camera'

import type { ReduxState } from 'src/types/common'

import type {
  Vertex,
  PieceId,
} from 'src/types/game'

interface Props {
  // id: VertexId
  cube: Vertex
  position: Vector3
  color: string
  vertices: Vertex[]
}

function Tile({ position, cube, color, vertices }: Props): JSX.Element {
  const ref = React.useRef<Object3D>()

  const dispatch = useDispatch()

  const onClick = React.useCallback(
    function callback() {
        dispatch(cameraLookAt({ cube }))
    },
    [dispatch, cube]
  )

  const args: [
    shapes: Shape | Shape[],
    curveSegments?: number | undefined
  ] = React.useMemo(
    function memo() {
      const shape = new Shape()

      shape.moveTo(vertices[0].x, vertices[0].y)
      shape.lineTo(vertices[1].x, vertices[1].y)
      shape.lineTo(vertices[2].x, vertices[2].y)
      shape.lineTo(vertices[3].x, vertices[3].y)
      shape.lineTo(vertices[4].x, vertices[4].y)
      shape.lineTo(vertices[5].x, vertices[5].y)
      shape.lineTo(vertices[0].x, vertices[0].y)

      return [shape]
    },
    [vertices]
  )

  return (
    <mesh
      castShadow
      receiveShadow
      position={position}
      ref={ref}
      onClick={onClick}
      onPointerOver={onPointerOver}
      onPointerOut={onPointerOut}
    >
      <shapeBufferGeometry args={args} />
      <meshBasicMaterial
        color={color}
        side={DoubleSide}
      />
    </mesh>
  )
}

export default React.memo(Tile)

PS:I have noticed that <perspectiveCamera/> does not have typescript typings, so I need to cast const ref = React.useRef<PerspectiveCamera | null>(null) Should I open separate issue for that?

I also want to use the same redux.js logic to smoothly transition camera position around current lookAt by the curve then players switch turns.

[drcmda]

function CameraController() {
  const ref = useRef()
  const { lookAt, position } = useSelector(selector)
  const vectorLookAt = useState(() => new THREE.Vector3())
    
  useFrame(() => {    
    ref.current.lookAt(vectorLookAt.lerp(lookAt, 0.1))
    ref.current.position.lerp(position, 0.1)
  })

  return <perspectiveCamera ref={ref} ... />

perspectiveCam is typed in the jsx IntrinsicElements. i dont know how types work, you can sure take a look.

[JustFly1984]

@drcmda I've tested your approach, and it works, but only while I move mouse cursor. If I do not move mouse, it pauses.

I had put console.info('useframe') inside the useFrame() callback, and I see that useFrame is called only then mouse cursor is hovering into any 3dObject. Other time it is not firing at all, so camera animation is paused. I can try to use simple rAF to overcome this issue, but would love to get understanding why useFrame can act like it is acting in my project.

Could it be cos I use [email protected] ?

I see a lot of perf warnings in chrome console:

One more notice in typescript empty vector do not have lerp() method, then correctly providing x, y, z to new Vector() does provide the method.

lookAt from redux is just an object { x, y, z }, so it can't be passed to lerp() without creating a new vector in between.

concerning the types - I guess mergeRefs function has incorrect typings or no types at all, cos it does not infer correct type for ref.

currently my camera component looks next:

import * as React from 'react'
import { Vector3, PerspectiveCamera } from 'three'
import { useSelector } from 'react-redux'
import { useFrame, useThree } from 'react-three-fiber'

import type { ReduxState } from 'src/types/common'
import type { Cube } from 'src/types/game'

interface SelectorProps {
  prevLookAt: Cube
  lookAt: Cube
  position: Cube
}

function selector(state: ReduxState): SelectorProps {
  return {
    prevLookAt: state.camera.prevLookAt,
    lookAt: state.camera.lookAt,
    position: state.camera.position,
  }
}

function CameraController(): JSX.Element {
  const ref = React.useRef<PerspectiveCamera | null>(null)

  const { prevLookAt, lookAt, position } = useSelector(selector)

  const prevVectorLookAt = React.useMemo(
    function memo() {
      return new Vector3(prevLookAt.x, prevLookAt.y, prevLookAt.z)
    },
    [prevLookAt.x, prevLookAt.y, prevLookAt.z]
  )

  const vectorLookAt = React.useMemo(
    function memo() {
      return new Vector3(lookAt.x, lookAt.y, lookAt.z)
    },
    [lookAt.x, lookAt.y, lookAt.z]
  )

  const vectorPosition = React.useMemo(
    function memo() {
      return new Vector3(position.x, position.y, position.z)
    },
    [position.x, position.y, position.z]
  )

  const { setDefaultCamera } = useThree()

  // Make the camera known to the system
  React.useEffect(
    function effect() {
      if (ref.current !== null) {
        setDefaultCamera(ref.current)
      }
    },
    [setDefaultCamera]
  )

  // Update it every frame
  useFrame(function renderCallback() {
    if (ref.current !== null) {
      ref.current.lookAt(prevVectorLookAt.lerp(vectorLookAt, 0.1))
      ref.current.updateMatrixWorld()
    }
  })

  return <perspectiveCamera ref={ref} fov={75} position={vectorPosition} />
}

export default React.memo(CameraController)

I can provide access to the private repository if you want to look at behavior.

[JustFly1984]

@drcmda Even with rAF I need to move mouse, so animation continues without interruption.

On each mouse click I got one onClick React.useCallback call, but in browser console I see 2 Violation warnings:

I run the profiler and I got next screenshot:

and I saved json file, which I can send by email if you need.

Basically each click is creating long running task, which executes for ~191ms, and involves 2 click events: one is natural, and second is synthetic react event.

This is the reason for animation staggering.

Is there any advice to how get rid of long running process?

[JustFly1984]

I made a zoom on one of clicks, and I see that main bottle neck is 3 redux action dispatches. I think I can manage to make a single dispatch action, which will update 3 reducers at once. I use immer under the hood, which in theory should provide immutability and performance gains, but something somewhere went wrong.

[drcmda]

useframe is called 60 times per second all the time unless your canvas is set to invalidateFrameloop, in which case the user is responsible for invalidating.

[JustFly1984]

@drcmda Thank you, indeed I had invalidateFrameloop prop enabled, and removing it fixed the behavior for lerp()!

[JustFly1984]

final working code for smooth camera transition:

import * as React from 'react'
import { Vector3, PerspectiveCamera } from 'three'
import { useSelector } from 'react-redux'
import { useThree, useFrame } from 'react-three-fiber'

import type { ReduxState } from 'src/types/common'
import type { Cube } from 'src/types/game'

interface SelectorProps {
  prevLookAt: Cube
  lookAt: Cube
  position: Cube
}

function selector(state: ReduxState): SelectorProps {
  return {
    prevLookAt: state.camera.prevLookAt,
    lookAt: state.camera.lookAt,
    position: state.camera.position,
  }
}

function CameraController(): JSX.Element {
  const ref = React.useRef<PerspectiveCamera | null>(null)

  const { prevLookAt, lookAt, position } = useSelector(selector)

  const prevVectorLookAt = React.useMemo(
    function memo() {
      return new Vector3(prevLookAt.x, prevLookAt.y, prevLookAt.z)
    },
    [prevLookAt.x, prevLookAt.y, prevLookAt.z]
  )

  const vectorLookAt = React.useMemo(
    function memo() {
      return new Vector3(lookAt.x, lookAt.y, lookAt.z)
    },
    [lookAt.x, lookAt.y, lookAt.z]
  )

  const vectorPosition = React.useMemo(
    function memo() {
      return new Vector3(position.x, position.y, position.z)
    },
    [position.x, position.y, position.z]
  )

  const { setDefaultCamera } = useThree()

  // Make the camera known to the system
  React.useEffect(
    function effect() {
      if (ref.current !== null) {
        setDefaultCamera(ref.current)
      }
    },
    [setDefaultCamera]
  )

  // const requestRef = React.useRef<number>(0)

  // const animate = React.useCallback(
  //   function callback(/* time */): void {
  //     // console.info('animate')

  //     if (ref.current !== null) {
  //       ref.current.lookAt(prevVectorLookAt.lerp(vectorLookAt, 0.1))
  //       ref.current.updateMatrixWorld()
  //       // console.info('updateMatrixWorld')
  //     }

  //     requestRef.current = requestAnimationFrame(animate)
  //   },
  //   [prevVectorLookAt, vectorLookAt]
  // )

  // React.useEffect(() => {
  //   animate()
  //   requestRef.current = window.requestAnimationFrame(animate)

  //   return () => window.cancelAnimationFrame(requestRef.current)
  // }, [animate]) // Make sure the effect runs only once

  // Update it every frame
  useFrame(function renderCallback() {
    if (ref.current !== null) {
      ref.current.lookAt(prevVectorLookAt.lerp(vectorLookAt, 0.1))
      ref.current.updateMatrixWorld()
    }
  })

  return <perspectiveCamera ref={ref} fov={75} position={vectorPosition} />
}

export default React.memo(CameraController)