Render the rest of the A-Frame Scene outside the MindAR Target

[marco6ocram]

TLDR: Is there a way to render the rest of the scene outside of the MindAR target?

I know that is stated in the documentation that:

Remember this. The AR engine only does ONE thing! - It updates the visibility and positions of a-entity

But I think that being able to explore the rest of the scene using the Magic Window while interacting with the marker's positions is something that enhances a lot of augmented reality experiences.

In AR.js, this is currently possible, but the positions of the objects anchored to the marker get an offset. There are things that can be worked around, but the thing stopping me from using AR.js is that I don't know how to pause the AR.js engine in order to use the camera for other things, as it is possible in MindAR.

[dariocavada]

I tried placing objects outside the MindAR target, but they are automatically hidden. Here's the behavior I would like to achieve:

1. Start scanning.
2. When the image target is found, capture the anchor.
3. Keep the objects visible (using only sensors like the gyroscope, etc.) even when the target is lost.

I tried using onTargetFound to copy world coordinates and assign them to another entity external to the image entity, but it didn't work.

onTargetFound: function () {
    console.log('Target found');
},
onTargetLost: function () {
    console.log('Target lost');
},

I noticed that in the A-Frame "Hello World" example (https://aframe.io/examples/showcase/helloworld/), the sensors are working quite well.

[notbadjon]

I tried placing objects outside the MindAR target, but they are automatically hidden. Here's the behavior I would like to achieve:

1. Start scanning.
2. When the image target is found, capture the anchor.
3. Keep the objects visible (using only sensors like the gyroscope, etc.) even when the target is lost.

@dariocavada This is exactly what I wanted to do. I basically just want to use the image tracking to initialize the camera location in a virtual scene. The user can then look around, assuming the camera stays in the same place. I kinda, almost got this working with this:

      const target = document.querySelector(".ar-target");
      target.addEventListener("targetFound", event => {
        setTimeout(() => {
          var upsideDown = document.querySelector(".upside-down")

          var targetPosition = new THREE.Vector3();
          var targetOrientation = new THREE.Quaternion();
          target.object3D.getWorldPosition(targetPosition);
          target.object3D.getWorldQuaternion(targetOrientation);

          upsideDown.object3D.setRotationFromQuaternion(targetOrientation);
          upsideDown.object3D.position.x = targetPosition.x / 1000;
          upsideDown.object3D.position.y = targetPosition.y / 1000;
          upsideDown.object3D.position.z = targetPosition.z / 1000;
          camera.components.camera.camera.near = 0.1;
          camera.components.camera.camera.updateProjectionMatrix();
        }, 100);

        console.log("Pausing tracking... ");
        camera.setAttribute("look-controls", "enabled", true);
        mindarSystem.pause(true);
      });

The tricky parts that took me a bit to figure out:

• The position coordinates are divided by 1000. Apparently the anchor is in mm and the aframe scene in m.
• I needed to add a small delay before copying the target pose.
• I needed to change the camera near clipping plane. MindAR sets it to 10 which is way to far for my scene.

This still does not work properly. The virtual scene is not close to staying in sync with the physical scene when using the look-controls.

I would be very grateful if anyone has suggestions to get this working.