# WebXR Augmented Reality Module - Level 1

## W3C First Public Working Draft, 10 October 2019

This version:
  https://www.w3.org/TR/2019/WD-webxr-ar-module-1-20191010/
Latest published version:
  https://www.w3.org/TR/webxr-ar-module-1/
Editor's Draft:
  https://immersive-web.github.io/webxr-ar-module/
Issue Tracking:
  GitHub
  Inline In Spec
Editors:
  Brandon Jones (Google)
  Nell Waliczek (Amazon [Microsoft until 2018])
  Manish Goregaokar (Mozilla)
Participate:
  File an issue (open issues)
  Mailing list archive
  W3C’s #immersive-web IRC

Unstable API

The API represented in this document is under development and may change at any time.

For additional context on the use of this API please reference the WebXR Augmented Reality Module Explainer.

Copyright © 2019 W3C® (MIT, ERCIM, Keio, Beihang). W3C liability, trademark and permissive document license rules apply.

## Abstract

The WebXR Augmented Reality module expands the WebXR Device API with the functionality available on AR hardware.

## Status of this document

This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at https://www.w3.org/TR/.

The Immersive Web Working Group maintains a list of all bug reports that the group has not yet addressed. This draft highlights some of the pending issues that are still to be discussed in the working group. No decision has been taken on the outcome of these issues including whether they are valid. Pull requests with proposed specification text for outstanding issues are strongly encouraged.

This document was published by the Immersive Web Working Group as a Working Draft. This document is intended to become a W3C Recommendation.

This document is a First Public Working Draft.

Publication as a First Public Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.

This document was produced by a group operating under the W3C Patent Policy. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.

This document is governed by the 1 March 2019 W3C Process Document.

This WebXR Argumented Reality Module is designed as a module to be implemented in addition to WebXR Device API, and is originally included in WebXR Device API which was divided into core and modules.

## 1. Introduction

Hardware that enables Virtual Reality (VR) and Augmented Reality (AR) applications are now broadly available to consumers, offering an immersive computing platform with both new opportunities and challenges. The ability to interact directly with immersive hardware is critical to ensuring that the web is well equipped to operate as a first-class citizen in this environment. The WebXR Augmented Reality module expands the functionality available to developers when their code is running on AR hardware.

### 1.1. Terminology

Augmented Reality describes a class of XR experiences in which virtual content is aligned and composed with the real-world environment before being displayed to users.

XR hardware can be divided into categories based on display technology.

- Devices described as additive light, also known as see-through, use transparent optical displays to present virtual content. On these devices, the user may always be able to see through to the real-world environment regardless of developer requests during session creation.

- Devices described as pass-through use an opaque display to combine virtual content with a camera stream of the real-world environment. On these devices, the real-world environment will only be visible when the developer has made an explicit request for it during session creation.

- Devices described as opaque fully obscure the real-world environment and do not provide a way to view the real-world environment.

## 2. WebXR Device API Integration

### 2.1. XRSessionMode

As defined in the WebXR Device API categorizes `XRSession`s based on their `XRSessionMode`. This module expands the allowable values of the `XRSessionMode` enum to include `"immersive-ar"`.

    enum XRSessionMode {
      "inline",
      "immersive-vr",
      "immersive-ar"
    };

A session mode of `"immersive-ar"` indicates that the session’s output will be given exclusive access to the immersive XR device display and that content is intended to be blended with the real-world environment.

The definition of immersive session is also expanded to include sessions with a mode of `"immersive-ar"`. On compatible hardware, user agents MAY support `"immersive-vr"` sessions, `"immersive-ar"` sessions, or both. Supporting for the additional `"immersive-ar"` session mode, does not change the requirement that user agents MUST support `"inline"` sessions.

NOTE: This means that `"immersive-ar"` sessions support all the features and reference spaces that `"immersive-vr"` sessions do, since both are immersive sessions.

The following code checks to see if `"immersive-ar"` sessions are supported.

    navigator.xr.supportsSession('immersive-ar').then(() => {
      // 'immersive-ar' sessions are supported.
      // Page should advertise AR support to the user.
    }

The following code attempts to retrieve an `"immersive-ar"` `XRSession`.

    let xrSession;

    navigator.xr.requestSession("immersive-ar").then((session) => {
      xrSession = session;
    });

### 2.2. XREnvironmentBlendMode

When drawing XR content, it is often useful to understand how the rendered pixels will be blended by the XR Compositor with the real-world environment.

    enum XREnvironmentBlendMode {
      "opaque",
      "alpha-blend",
      "additive"
    };

    partial interface XRSession {
      // Attributes
      readonly attribute XREnvironmentBlendMode environmentBlendMode;
    };

The `environmentBlendMode` attribute MUST report the `XREnvironmentBlendMode` value that matches blend technique currently being performed by the XR Compositor.

- A blend mode of `opaque` MUST be reported if the XR Compositor is using opaque environment blending.

- A blend mode of `alpha-blend` MUST be reported if the XR Compositor is using alpha-blend environment blending.

- A blend mode of `additive` MUST be reported if the XR Compositor is using additive environment blending.

### 2.3. XR Compositor Behaviors

When presenting content to the XR device, the XR Compositor MUST apply the appropriate blend technique to combine virtual pixels with the real-world environment. The appropriate technique is determined based on the XR device's display technology and the mode.

- When performing opaque environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing exclusively 100% opaque black pixels. The composited output is then presented on the XR device. This technique MUST be applied on opaque and pass-through displays when the mode is set to either `"immersive-vr"` or `"inline"`. This technique MUST NOT be applied when the mode is set to `"immersive-ar"`, regardless of the XR Device's display technology.

- When performing alpha-blend environment blending, the rendered buffers obtained by the XR Compositor are composited using source-over blending on top of buffers containing pixel representations of the real-world environment. These pixel representations must be aligned on each `XRFrame` to the `transform` of each view in `views`. The composited output is then presented on the XR device. This technique MUST be applied on pass-through displays when the mode is set `"immersive-ar"`. This technique MUST NOT be applied when the mode is set to `"immersive-vr"` or `"inline"` regardless of the XR Device's display technology.

- When performing additive environment blending, the rendered buffers obtained by the XR Compositor are composited using lighter blending before being presented on the XR device. This technique MUST be applied on additive light displays, regardless of the mode.

The XR Compositor MAY make additional color or pixel adjustments to optimize the experience. The timing of composition MUST NOT depend on the blend technique or source of the real-world environment. but MUST NOT perform occlusion based on pixel depth relative to real-world geometry; only rendered content MUST be composed on top of the real-world background.

NOTE: Future modules may enable automatic or manual pixel occlusion with the real-world environment.

The XR Compositor MUST NOT automatically grant the page access to any additional information such as camera intrinsics, media streams, real-world geometry, etc.

NOTE: Developers may request access to an XR Device's camera, should one be exposed through the existing [Media Capture and Streams](https://www.w3.org/TR/mediacapture-streams/) specification. However, doing so does not provide a mechanism to query the `XRRigidTransform` between the camera’s location and the native origin of the viewer reference space. It also does not provide a guaranteed way to determine the camera intrinsics necessary to match the view of the real-world environment. As such, performing effective computer vision algorithms wil be significantly hampered. Future modules or specifications may enable such functionality.

## 3. Security, Privacy, and Comfort Considerations

### 3.1. Protected functionality

add information about new threats and mitigations introduced by functionality in this module

## 4. Acknowledgements

The following individuals have contributed to the design of the WebXR Device API specification:

- Chris Van Wiemeersch (Mozilla)

- Kearwood Gilbert (Mozilla)

- Rafael Cintron (Microsoft)

- Sebastian Sylvan (Formerly Microsoft)

And a special thanks to Vladimir Vukicevic (Unity) for kick-starting this whole adventure!

## Index

### Terms defined by this specification

- additive, in §2.2
- "additive", in §2.2
- additive environment blending, in §2.3
- additive light, in §1.1
- alpha-blend, in §2.2
- "alpha-blend", in §2.2
- alpha-blend environment blending, in §2.3
- blend technique, in §2.3
- display technology, in §1.1
- environmentBlendMode, in §2.2
- "immersive-ar", in §2.1
- "immersive-vr", in §2.1
- "inline", in §2.1
- opaque
  - definition of, in §1.1
  - enum-value for XREnvironmentBlendMode, in §2.2
- "opaque", in §2.2
- opaque environment blending, in §2.3
- pass-through, in §1.1
- real-world environment, in §1.1
- XREnvironmentBlendMode, in §2.2
- XRSessionMode, in §2.1

https://www.w3.org/TR/compositing-1#porterduffcompositingoperators_plus Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/compositing-1#porterduffcompositingoperators_srcover Referenced in:

- 2.3. XR Compositor Behaviors (2)

https://www.w3.org/TR/webxr/#xrframe-interface Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#xrrigidtransform-interface Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#xrsession-interface Referenced in:

- 2.1. XRSessionMode (2)
- 2.2. XREnvironmentBlendMode

https://www.w3.org/TR/webxr/#exclusive-access Referenced in:

- 2.1. XRSessionMode

https://www.w3.org/TR/webxr/#immersive-session Referenced in:

- 2.1. XRSessionMode (2)

https://www.w3.org/TR/webxr/#xr-immersive-xr-device Referenced in:

- 2.1. XRSessionMode

https://www.w3.org/TR/webxr/#xrsession-mode Referenced in:

- 2.1. XRSessionMode
- 2.3. XR Compositor Behaviors (2) (3) (4) (5) (6)

https://www.w3.org/TR/webxr/#xrspace-native-origin Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#dom-xrview-transform Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#xrsession-viewer-reference-space Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#dom-xrviewerpose-views Referenced in:

- 2.3. XR Compositor Behaviors

https://www.w3.org/TR/webxr/#xr-compositor Referenced in:

- 2.2. XREnvironmentBlendMode (2) (3) (4) (5)
- 2.3. XR Compositor Behaviors (2) (3) (4) (5) (6)

https://www.w3.org/TR/webxr/#xrsession-xr-device Referenced in:

- 2.3. XR Compositor Behaviors (2) (3) (4) (5) (6) (7) (8)

### Terms defined by reference

- [css compositing level 1] defines the following terms:
  - lighter
  - source-over
- [webxr device api - level 1] defines the following terms:
  - XRFrame
  - XRRigidTransform
  - XRSession
  - exclusive access
  - immersive session
  - immersive xr device
  - mode
  - native origin
  - transform
  - viewer reference space
  - views
  - xr compositor
  - xr device

## IDL Index

    enum XRSessionMode {
      "inline",
      "immersive-vr",
      "immersive-ar"
    };

    enum XREnvironmentBlendMode {
      "opaque",
      "alpha-blend",
      "additive"
    };

    partial interface XRSession {
      // Attributes
      readonly attribute XREnvironmentBlendMode environmentBlendMode;
    };

## Issues Index

add information about new threats and mitigations introduced by functionality in this module ↵
#real-world-environment Referenced in:

- 1.1. Terminology (2) (3) (4) (5)
- 2.1. XRSessionMode
- 2.2. XREnvironmentBlendMode
- 2.3. XR Compositor Behaviors (2) (3) (4) (5)

#display-technology Referenced in:

- 2.3. XR Compositor Behaviors (2) (3)

#additive-light Referenced in:

- 2.3. XR Compositor Behaviors

#pass-through Referenced in:

- 2.3. XR Compositor Behaviors (2)

#opaque Referenced in:

- 2.3. XR Compositor Behaviors

#enumdef-xrsessionmode Referenced in:

- 2.1. XRSessionMode (2)

#dom-xrsessionmode-inline Referenced in:

- 2.1. XRSessionMode
- 2.3. XR Compositor Behaviors (2)

#dom-xrsessionmode-immersive-vr Referenced in:

- 2.1. XRSessionMode (2)
- 2.3. XR Compositor Behaviors (2)

#dom-xrsessionmode-immersive-ar Referenced in:

- 2.1. XRSessionMode (2) (3) (4) (5) (6) (7) (8)
- 2.3. XR Compositor Behaviors (2)

#enumdef-xrenvironmentblendmode Referenced in:

- 2.2. XREnvironmentBlendMode (2)

#dom-xrsession-environmentblendmode Referenced in:

- 2.2. XREnvironmentBlendMode

#dom-xrenvironmentblendmode-opaque Referenced in:

- 2.2. XREnvironmentBlendMode

#dom-xrenvironmentblendmode-alpha-blend Referenced in:

- 2.2. XREnvironmentBlendMode

#dom-xrenvironmentblendmode-additive Referenced in:

- 2.2. XREnvironmentBlendMode

#blend-technique Referenced in:

- 2.1. XRSessionMode
- 2.2. XREnvironmentBlendMode
- 2.3. XR Compositor Behaviors

#opaque-environment-blending Referenced in:

- 2.2. XREnvironmentBlendMode

#alpha-blend-environment-blending Referenced in:

- 2.2. XREnvironmentBlendMode

#additive-environment-blending Referenced in:

- 2.2. XREnvironmentBlendMode