Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Virtual Camera #674

Open
crunchwrap89 opened this issue Apr 12, 2023 · 3 comments
Open

Virtual Camera #674

crunchwrap89 opened this issue Apr 12, 2023 · 3 comments
Assignees
Labels
priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@crunchwrap89
Copy link

crunchwrap89 commented Apr 12, 2023

The problem

When i was trying to add the Transformation Controls from the Three.js utils and attach it to a 3D object i stumbled upon some issues.

Attaching the TransformationControls to the 3D Object works fine, i can see the Controls and the position and axis is correct after the latest major update to the package. However.. interacting with the TransformationControls does not work. Nothing happens when clicking/dragging the controls.

When dragging the controls it should translate the objects position on the map.
This currently doesnt work due to how the current camera is positioned.

Feature request

After discussing this with Martin we concluded that the solution to this issue would be to add a virtual camera, that "sees" the same as the actual maps viewport sees. Then that virual camera can be passed to the TransformControls and then the internal raycaster of the Transform Controls should work as intended.

There are more use cases for adding a virtual camera that "sees" the same as the actual maps viewport.
Here is a list of some that i can come up with:

Usecases

  • Ability to frustum cull objects outside of the cameras viewport. (A Very great performance improvement, that will allow building a significantly larger 3D world with higher frame rates.)
  • Ability to perform Animations on objects when they enter or leave the Viewport.
  • Ability to trigger any type of function or world event when the a specific object enter / leaves Viewport.
  • Ability to use object.lookAt(virtualCamera.position) which itself has alot of use cases, such as letting floating player names and texts always focus on the virtual camera no matter which rotation, tilt or zoom is being used.
  • Ability to use the controls that Three.js has such as TransformationControls, there are many of them.

Here is a suggested calculation for setting the position of the Virtual Camera on each frame update.
It Almost works, there is one small factor missing but i could really use your guys help to figure out what piece of the puzzle is missing.


function updateVCam() {
    if (!_camera || !_scene || !_renderer)
      return;

    _vCamMvpMatrix.copy(_camera.projectionMatrix)
    _vCamInverseMvpMatrix.copy(_vCamMvpMatrix).invert()

    _v4.set(0, 0, 1, 0)
    _v4.applyMatrix4(_vCamInverseMvpMatrix)
    _v4.multiplyScalar(1 / _v4.w)
    const cameraPosition = new Vector3()
    cameraPosition.set(_v4.x / _v4.w, _v4.y / _v4.w, _v4.z / _v4.w)

    const c = new Vector3(0, 0, 0).applyMatrix4(_vCamInverseMvpMatrix)
    const x = new Vector3(1, 0, 0).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()
    const y = new Vector3(0, 1, 0).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()
    const z = new Vector3(0, 0, -1).applyMatrix4(_vCamInverseMvpMatrix).sub(c).normalize()

    const near = new Vector3(0, 0, -1).applyMatrix4(_vCamInverseMvpMatrix).sub(cameraPosition).length()
    const far = new Vector3(0, 0, 1).applyMatrix4(_vCamInverseMvpMatrix).sub(cameraPosition).length()

    _m4.makeBasis(x, y, z)

    _vCam.position.copy(cameraPosition)
    _vCam.quaternion.setFromRotationMatrix(_m4)
    _vCam.updateMatrixWorld()

    _vCam.matrixWorldInverse.copy(_vCam.matrixWorld).invert()

    const projectionMatrix = new Matrix4().multiplyMatrices(_vCamMvpMatrix, _vCam.matrixWorldInverse)

    const fov = (2 * Math.atan(1 / projectionMatrix.elements[5]) * 180) / Math.PI
    const aspectRatio = projectionMatrix.elements[5] / projectionMatrix.elements[0]

    _vCam.fov = fov
    _vCam.aspect = aspectRatio
    _vCam.near = near
    _vCam.far = far

    _vCam.updateProjectionMatrix()

    _renderer.render(_scene, _camera)
    _renderer.resetState()
  }

@crunchwrap89 crunchwrap89 added triage me I really want to be triaged. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. labels Apr 12, 2023
@usefulthink usefulthink added priority: p3 Desirable enhancement or fix. May not be included in next release. and removed triage me I really want to be triaged. labels Apr 13, 2023
@crunchwrap89 crunchwrap89 changed the title TransformControls Virtual Camera Jul 28, 2023
@crunchwrap89
Copy link
Author

crunchwrap89 commented Jul 28, 2023

Hi,
I made an update to this issue description, added some useCases for the suggested Virtual Camera and also added an "Almost" working code example of how it might work that was calculated by @usefulthink

The above code example works fine for utilizing object.lookAt(_vCam.position), but there is some small factor missing.
Hopefully you can help and this might be added to the package. :)

Love this package btw! <3

@crunchwrap89
Copy link
Author

Any chance this will be made? :) Im still looking for ways to implement tranform controls on the webgloverlayview.

Br
Marcus

@MstrD
Copy link

MstrD commented Oct 11, 2024

I'm pinging this thread because I am also needing some sort of virtual camera attached to my project. In my specific case, I am trying to work with CSS3DRenderer, but I can't seem to place my instances of CSS3DObject in the correct place; despite I already found how to get the virtual camera position, like @crunchwrap89 reported, there's a small factor missing in the calculations, and I can't seem to find it.
Are you already trying to implement this feature, @usefulthink ?

Any sort of feedback would be appreciated.
Thanks in advance!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
priority: p3 Desirable enhancement or fix. May not be included in next release. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

4 participants