In React Three Fiber, I have a React component that generates a sprite that I want to persist across camera zooms. The algorithm seems to work (the size doesn't seem to change over time), but I can visibly see it scaling when zooming in and out.

I've seen something similar done to sprites elsewhere, and it seems to work fine without the problematic appearance. I think there is some kind of lag, but I'm not sure where it is or how to fix it.

I have also attached a video of the issue: https://drive.google.com/file/d/121XPY1pB5bNJr5EYNESnyPfLDdtEeIhn/view?usp=sharing

I have attached the code for the React component. Like regular React Three Fiber code, this is just input into the canvas component:

function TestFunction() {
  const [scale, setScale] = useState(new Vector3(1, 1, 1));
  const map = new THREE.TextureLoader().load("src/assets/Joshy.png");

  let state = useThree();
  let zoom = state.camera.zoom / 100;
  let scaler: Vector3 = new Vector3(1 / zoom, 1 / zoom, 1 / zoom);
  const handleMouseScroll = (event: WheelEvent) => {
    // Perform actions when the mouse is scrolled
    scaler.set(1 / zoom, 1 / zoom, 1 / zoom);
    setScale(scaler);
  };
  useFrame(() => {
    window.document.addEventListener("wheel", handleMouseScroll, {
      capture: true,
      passive: true,
    });
  });

  return (
    <sprite scale={scale}>
      <spriteMaterial map={map} />
    </sprite>
  );
}

I hope it doesn't have that laggy/glitchy look to it since I'm updating every frame so I'm not sure what the problem is.

What I'm looking for is this: https://drive.google.com/file/d/1KtEcrikFJFYboC6LX0mFVttx2eSTia0c/view?usp=sharing No noticeable lag. The code that generates this lag-free version is pure JavaScript Three.js.

const cont = document.documentElement;
const [w, h] = [cont.clientWidth - 20, cont.clientHeight - 20];
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(50, w / h, 0.1, 100);
camera.position.set(10, 10, 10);

const renderer = new THREE.WebGLRenderer();
renderer.setSize(w, h);
document.body.appendChild(renderer.domElement);

const hlp = new THREE.AxesHelper(3);
scene.add(hlp);

const map = new THREE.TextureLoader().load(
  'https://source.unsplash.com/random/200x200'
);
const material = new THREE.SpriteMaterial({
  map: map
});

const sprite_scale = 4;
const sprite = new THREE.Sprite(material);
/* sprite.scale.set(sprite_scale, sprite_scale); */
const virtual_d = sprite.position.distanceTo(camera.position) *1.4/ sprite_scale;

scene.add(sprite);

const controls = new THREE.OrbitControls(camera, renderer.domElement);

const render = () => {
  renderer.render(scene, camera);
};

const animate = () => {

  requestAnimationFrame(animate);

  render();

};

document.addEventListener('wheel', scaler);

function scaler() {
  const scale = sprite.position.distanceTo(camera.position)/2;
  console.log(sprite.position.distanceTo(camera.position) )
  console.log("CAMERA POSITION: ", camera.position)
  sprite.scale.set(scale, scale);
}

animate();

The logic of ordinary Three.js and React Three Fiber feels the same, but the results are completely different.