OpenXR: Add documentation page about the new composition layers #9373
Conversation
BastiaanOlij
added
enhancement
topic:xr
BastiaanOlij
force-pushed
the
xr_composition_layers
branch
from
c427fdf
to
8962b22
Compare
BastiaanOlij
force-pushed
the
xr_composition_layers
branch
from
8962b22
to
d894725
Compare
| Hole punching | ||
| ------------- | ||
|
|
||
| This is a feature that is specifically targeted at AR use cases as it requires our |
There was a problem hiding this comment.
I'd add MR in addition to AR in that sentence to avoid giving the impression we're only referring to AR glasses:
This is a feature that is specifically targeted at AR and MR use cases...
There was a problem hiding this comment.
As mentioned during the XR meeting, should have written this down before :) In my humble opinion I want to avoid the MR terminology here. AR and VR reference two distinct approaches, VR referencing fully immersive virtual worlds, while AR references any approach where the real world is augmented with virtual content.
Just for completeness sake for this discussion, XR is a grouping term that was adopted by Khronos and as such by a part of the industry (including by Godot).
MR is a confusing term that means something different depending on who you talk to.
- Originally it referred to mixing real world capture with virtual world capture to create a composite such as solutions as MixCast, Liv and Oculus own Mixed Reality iPad app implement.
- Then Microsoft adopted MR to mean Mixed Reality as the name for their WMR platform, in this case defining it as a platform supported both AR (through Hololens) and VR (through other WMR devices, none of which do passthrough I believe). I also guess they hoped people would see it as an abbreviation of Microsoft Reality, but that's pure speculation on my part :)
- Then in most recent years MR meaning Mixed Reality has become an alternative to XR as an encompassing term for people who don't like the XR moniker.
- And finally, Meta seems to have rebranded it recently to specifically mean AR through the means of passthrough though I am not sure if that has created more clarification or more confusion in the market space. All though Metas marketing here IMHO is inconsistent as its also often referring to the above meaning as a catch all for AR and VR.
So MR in its meaning in the context of this document as I understand it, e.g. AR through passthrough, is a meaning that is fairly contested.
As Godot is a platform agnostic development tool and we try not to care how AR is achieved but want our applications to be portable between various AR devices, it is my personal opinion that AR in this context is sufficient capturing all use cases of this feature, while MR is confusing.
There are features that are only possible on devices that use a camera feed to achieve AR and where passthrough should be named specifically, but this is no such feature as it is as applicable on a passthrough device such as the Quest, as it is on an optical AR device such as the Magic Leap 2.
However, I stress that this is my personal standpoint on the matter, so this is absolutely up for debate.
| was_intersect = NO_INTERSECTION | ||
|
|
||
|
|
||
| Hole punching |
There was a problem hiding this comment.
It may be worth giving a use-case to illustrate how this is being used in actual project (e.g: adding a virtual window to the environment with passthrough in the 'background'.
There was a problem hiding this comment.
I said I was on board with this use case last night at the XR meeting but I need to run that back: hole punched composition layers aren't the correct method for a "window into a virtual world". Since a subviewport can't be rendered in stereo to make it look actually 3D. If you wanted to achieve that effect you'd be better off having the "virtual world" be the main scene, and then have everything that isn't the window be covered up by (hole punched) geometry passthrough.
The main point of hole punching in composition layers is to just let things be displayed in front of the composition layer, like hand models, etc. Adding that use case would make sense.
There was a problem hiding this comment.
@devloglogan actually for those type of use cases you're better off rendering geometry directly with the shadow to opacity option and skip composition layers all together.
Hole punching in relation to composition layers is purely meant to allow the composition layer to be rendered behind the main content so that you can have virtual foreground objects occlusion part of the content of the composition layer.
Right now composition layers are always visible on top even if they clip through geometry.
There was a problem hiding this comment.
@BastiaanOlij I totally agree, and was attempting to say as much in my previous comment. :) Would not use composition layers when going for that use case.
There was a problem hiding this comment.
Ah we're on the same page then :)
|
One more bit of feedback from the XR meeting was the suggestion to add some image captures to the documentation. Especially seeing we're naming this by the technical solution, not by what you can achieve with this solution, a picture tells a thousand words to let people know what this functionality is used for. |
|
Perhaps an image like this would suffice to illustrate hole punching?
|
|
Owh thats pretty clear! Interesting btw that this shows its also viable for VR, so possibly I should rewrite that part of the text to not point out AR at all. |
|
@devloglogan That screenshot looks great!
+1 to this! |
@BastiaanOlij +1! A VR specific example would be to use a quad composition layer for text display to improve the text rendering quality.
@devloglogan During the meeting, I was suggesting the opposite, showing a window into the real world, which should be doable using the
This would make for an interesting sample, especially if the geometry is destructable so the user can 'break out' from the real world into the virtual world (e.g: similar to what First Encounters does). |
|
@m4gr3d asked this on rocketchat as well, but I don't see whats special about FB passthrough geometry, we can already do this with the |
BastiaanOlij
force-pushed
the
xr_composition_layers
branch
from
d894725
to
cbcf3b8
Compare
There was a problem hiding this comment.
This is looking really great! I have just a couple minor notes.
Also, I wonder if it's worth pointing out that an equirect layer can also be used for rendering a sky box? While Godot can of course render the sky, using a composition layer has the same advantages with regard to clarity and performance that using a composition layer for 2D UI has.
This is mentioned specifically in the OpenXR spec:
| There are currently 3 nodes that expose this functionality: | ||
|
|
||
| - :ref:`OpenXRCompositionLayerCylinder <class_OpenXRCompositionLayerCylinder>` shows the contents of the SubViewport on the inside of a cylinder. | ||
| - :ref:`OpenXRCompositionLayerEquirect <class_OpenXRCompositionLayerEquirect>` shows the contents of the SubViewport on a warped rectangle. |
There was a problem hiding this comment.
I think describing an equirect as the interior of sphere makes it easiest to understand:
| - :ref:`OpenXRCompositionLayerEquirect <class_OpenXRCompositionLayerEquirect>` shows the contents of the SubViewport on a warped rectangle. | |
| - :ref:`OpenXRCompositionLayerEquirect <class_OpenXRCompositionLayerEquirect>` shows the contents of the SubViewport on the interior of a sphere (or "slice" of a sphere). |
There was a problem hiding this comment.
I just copied these descriptions from the class definition, so I think if we change it here, we should also change it on the class?
There was a problem hiding this comment.
Oops, probably my fault then. :-) Yeah, we should update it there as well.
|
|
||
| There are currently 3 nodes that expose this functionality: | ||
|
|
||
| - :ref:`OpenXRCompositionLayerCylinder <class_OpenXRCompositionLayerCylinder>` shows the contents of the SubViewport on the inside of a cylinder. |
There was a problem hiding this comment.
| - :ref:`OpenXRCompositionLayerCylinder <class_OpenXRCompositionLayerCylinder>` shows the contents of the SubViewport on the inside of a cylinder. | |
| - :ref:`OpenXRCompositionLayerCylinder <class_OpenXRCompositionLayerCylinder>` shows the contents of the SubViewport on the inside of a cylinder (or "slice" of a cylinder). |
| ... | ||
|
|
||
| Next we check if we're intersecting with our viewport. | ||
| If so we check if our button is pressed and place our pointer at our intersection point. |
There was a problem hiding this comment.
| If so we check if our button is pressed and place our pointer at our intersection point. | |
| If so, we check if our button is pressed and place our pointer at our intersection point. |
| As the composition layer is composited on top of the render result, | ||
| it can be rendered in front of objects that are actually forward of the viewport. |
There was a problem hiding this comment.
Is it worth specifically saying something about "virtual hands appearing behind the composition layer"? I feel like that's the first thing developers are going to notice about composition layers in VR.
| Hole punching | ||
| ------------- | ||
|
|
||
| As the composition layer is composited on top of the render result, |
There was a problem hiding this comment.
This is dependent upon the sort order, so it may be worth clarifying that.
This new help page documents the new composition layer support @dsnopek added to Godot.
I've detailed a simple example project for this. I'll see if I can upload that somewhere so we can reference the source.