What are VR, AR, and MR?

Virtual reality (VR), augmented reality (AR) and mixed reality (MR) are emerging technologies utilizing a variety of digital (artificial) immersion and overlays on the real world that users can interact with.

Virtual Reality (VR) encompasses immersive experiences and content via a VR headset or HMD (head-mounted display). The content is 100% digital and computer-generated. The current reality is replaced with a new 3D digital environment in which the user is isolated from the real world.

Augmented reality (AR) overlays computer-generated content on top of the real world. This superimposed digital overlay can superficially interact with the environment in real-time. AR is primarily experienced via a wearable glass device or through smartphone applications.

Mixed reality (MR) combines several technologies into one wearable device. MR lenses or headsets present an overlay of digital content that interacts with objects in the real world in real-time. The products are, in most cases, in the research and development phase, but MR is viewed through transparent wearable glasses.

Extended reality (XR) is an umbrella term that encompasses all real and virtual environments which include VR, AR, and MR.

Understand VR, AR, MR technologies

The technology industry, and consumers, are inundated with new tech and the terminologies used to describe them. What is different about terms like AR, VR, and MR and where do they come from?

In 1984, Professor Paul Milgram and Fumio Kishino presented the Milgram Mixed Reality Spectrum to explain the range of virtual and augmented reality. This scale was the base most manufacturers had used for 30 years.

VR had been a popular topic before the ease of access and popularity of the internet boomed in the 1990s. The technology then lost momentum, interest, and funding from many technology companies.

Oculus then launched a successful Kickstarter campaign in 2012 which gained a lot of media attention and public interest. This was followed by the company’s subsequent high-profile acquisition by Facebook in 2015. This seemingly started a resurgence in VR research and development across the technology sector.

Microsoft introduced Windows 10 mixed reality in 2015 and released an updated Milgram Scale, as “The Mixed Reality Spectrum”. This included their HoloLens (MR) and VR headsets on opposing sides of the spectrum. It has been criticized by people working in the technology industry. The Microsoft definition has arguably added confusion to the understanding of mixed reality for consumers. Most of their products are considered virtual reality headsets and not mixed reality.

Disclaimer: at Aniwaa we don’t consider Microsoft’s definition of mixed reality to be entirely correct but it could apply to a platform encompassing VR, AR, and MR. We further explain these definitions in the guide below.

The Consumer Technology Association put out a set of AR/MR/VR/XR definitions in December 2016 to try and help standardize the terms. However, most of the industry seems to have ignored these guidelines, possibly as a result of the technologies’ relative infancy.

What are the differences between VR, AR and MR?

Virtual, augmented, and mixed reality have some fundamental differences. The number of products, and the pace of the technology, are moving along at a very fast rate. As is the increased use and application of the technology in a variety of industries and applications.

• Virtual Reality: VR is content that is 100% digital and can be enjoyed in a fully immersive environment.
• Augmented Reality: AR overlays digital content on top of the real world.
• Mixed Reality: MR is a digital overlay that allows interactive virtual elements to integrate and interact with the real-world environment.

What is Virtual Reality (VR)?

VR encompasses all virtually immersive experiences. These could be created using real-world content (360° video), digital content (computer-generated), or a hybrid of both. This medium requires the use of a VR headset or head-mounted display (HMD).

Categories of VR headsets

A number of key manufacturers, startups, and crowdfunded products have emerged to produce a range of VR products. These include technology giants such as Facebook (through their acquisition of Oculus), Google (with a series of Daydream headsets), HTC (VIVE), Samsung (Gear VR), and Windows (who have named their range of devices Windows Mixed Reality, even though a number are VR-only enabled.) Startups have emerged and managed to gain huge investments too.

There are three types of VR devices available and each have their own criteria and pros and cons.

Tethered VR

Tethered virtual reality headsets currently are much more immersive than other types of VR. These tethered VR headsets require a cable connection to a powerful gaming PC for a superior desktop VR experience. Most of the Windows Mixed Reality headsets fall in this VR category and are considered virtual reality within the industry, and not mixed reality (ASUS HC102, Acer Mixed Reality headset, Dell Visor, HP Mixed Reality headset, Samsung HMD Odyssey).

Examples of popular PC VR headsets: Oculus Rift, HTC VIVE Pro.

Standalone VR

These standalone headsets (wireless VR) don’t require a connection to a PC or a smartphone and are generally a more affordable option.

Examples of the most popular standalone VR headsets: Pico Neo, VIVE Focus, Oculus Go.

Smartphone VR

Mobile virtual reality through smartphone headsets has allowed many users to have their first VR experience. The range of devices available, the affordable price range, combined with the number of users who own smartphones, means this VR category is the most diverse in terms of products. The type of screen and its resolution is a key factor in smartphone VR headsets as well as battery life.

Examples of the most popular smartphone VR headsets: Samsung Gear VR, Google Daydream View 2.

Considerations for buying a VR headset

Content

Content is offered by a range of platforms, and each manufacturer may have its own content streaming service. SteamVR is the most widely used VR content platform.

Controllers

Handheld controllers allow for better interaction with the content and depending on the DoF (Degrees of Freedom), the better the immersion in the VR experience.

Tracking

The more precise the tracking is, the better the VR immersion is. Key factors are external or internal positional tracking (this also extends to inside-out and outside-in tracking) as well as eye and hand tracking.

Field of View (FOV)

A human eye sees about 220 degrees but most VR headsets provide around 100 degrees FOV. The smaller the FOV, the less immersive the experience.

Display

Good quality tethered and standalone VR headsets should offer a high resolution and crisp display. Key factors include pixel density, color fidelity, dynamic range, and brightness.

Refresh rate

The higher the refresh rate (also known as frame rate), the better the immersion is. A good starting refresh rate is 60 Hz, although most VR headsets offer rates between 90 Hz and 120 Hz.

Audio

Audio helps improve the immersive experience and most VR headsets come with built-in speakers or basic headphones. “Spatial audio” is the key industry term used to describe this and is key for a believable, immersive experience.

What is Augmented Reality (AR)?

Augmented Reality (AR) overlays digital content on a user’s view of the real world. It means seeing all types of content (2D, 3D, or 4D) and data (ambient temperature, emails, directions, etc.) through an AR device. Augmented content doesn’t recognize or interact with the physical objects within a real-world environment, however, it does enhance the user’s experience.

Augmented reality is believed to have some of the biggest potential for mass consumption compared to virtual reality or mixed reality. It can be delivered on hardware that we already own and use (smartphones).

AR has an almost infinite number of potential applications and it offers practical daily uses that have the ability to change the way people live and interact. Furthermore, AR is considered less restrictive than VR and doesn’t disengage a user from the real world.

Categories of AR products

Wearable AR glasses (smart glasses)

AR glasses are wearable glasses that add information or digital content alongside or to what the wearer sees (source: Wikipedia).

High-end AR smart glasses (microdisplays) are used more widely at an enterprise-level; in commercial, manufacturing, engineering, and sports for example. The price of these devices tends to be quite high, from $1,000 to $5,000. Enterprise and consumer level AR glasses are available, however, but the trend has been for manufacturers to target enterprise businesses.

AR generally requires less processing power compared to VR as it doesn’t need to render a completely new environment. The costs of the AR glass devices remain high compared to VR. This is because it is primarily aimed at enterprise-level customers and the amount of tech placed in a relatively small wearable device.

Apple, Samsung (in partnership with Windows) and Google, are rumored to be working on AR headsets or AR smart glasses.

Examples of popular AR glasses: Vuzix Blade Smart Glasses, Meta 2, ODG R-9

Smartphone AR

Smartphone AR has been accessible for mass consumption through apps utilizing AR. Games (Pokemon Go) as well as retail (IKEA Place) applications are popular examples.

Apps are available for smartphones to experience a

ugmented reality for entertainment, enterprise, and practical purposes. The inclusion of AR on smartphones, for example, offers an immersive and engaging shopping experience. In principle, the user directs the camera on the mobile at an object in the real-world environment and the AR app will overlay content such as an image, animation, or data that the user can see.

Apple, Facebook, Snapchat, Google, and other well-known tech companies already offer AR applications and software development kits (SDK).

Other AR headsets

VR headsets such as the VIVE Pro and Sensics zSight are also able to integrate augmented reality (AR) through ongoing software updates.

The failed crowdfunding product, Ghost augmented reality headset is offered as a smartphone AR device.

Another option is the Optinvent ORA-X as smart headphones featuring a retractable AR display.

The ANTVR MIX is also a crowdfunding campaign that was successful in raising funding and aims to offer a tethered AR device with some VR capabilities.

What is Mixed Reality (MR)?

MR (also known as MX, immersive media, spatial computing, and hybrid reality) intertwines digital content while interacting with a user’s real-world environment. This allows for digital content to integrate, enrich and interact with the user’s real-world environment.

Mixed Reality removes the boundaries between the real and virtual world using occlusion: the computer-generated objects can be visibly obscured by objects in the physical environment from the user’s point of view.

Examples of MR devices

MR devices constantly scan a real-world environment to enable mixed reality experiences. This is needed to seamlessly place digital information within a user’s real world and allow users to interact with these computer-generated objects. There are far fewer mixed reality headsets available compared to augmented reality and virtual reality headsets.

Microsoft’s HoloLens

The Microsoft HoloLens is untethered and doesn’t require any external power to run. It includes 3 distinct types of sensors and 5 cameras to offer mixed reality. The HoloLens has a limited FOV of 50 degrees.

Magic Leap

Magic Leap is developing one of the most talked-about mixed reality headsets, Magic Leap One, and it is still under development. Backed by massive technology companies such as Google and Qualcomm, it aims to project the light straight into the user’s retina through a digital light field. It is backed to be a game-changer when it is released.

Other MR devices

Google is reportedly developing a mixed reality device of their own. Apple has several patents for mixed and virtual reality headsets, and along with Samsung, is rumored to be working on mixed reality devices.

What is Extended Reality (XR)?

Extended Reality (XR) is the umbrella category that covers all the various forms of digital reality, including Augmented Reality (AR), Mixed Reality (MR), and Virtual Reality (VR).  

Where are the VR, MR, and AR technologies heading?

More immersive tracking

As the technology with VR, AR, and MR improve, tracking will become increasingly more significant. As a general rule, the more sensors, the better the tracking which leads to “sensor fusion”. This means multiple sensors will be able to correct themselves and nullify any respective flaws.

Improvements in eye-tracking also mean immersion will lead to better interaction with digital content. For example, virtual/digital characters will know when is a user is looking at them (and where).

In terms of data collection and analysis from tracking, content creators for VR, AR, and MR will have access to an unprecedented level of information in user analytics. This will give them a great level of understanding of users’ behavior and experiences.

DoF (Degrees of Freedom)

Progress in the advancement of DoF in recent years shows how quickly the tech is developing. Increased immersion will be the result of the continued development as headsets and controllers both are aligned in DoF.

FOV (Field Of View)

Increased FOV will lead to better immersion. Developments in bigger FOV in VR headsets like the Pimax 8K offer 200°, compared to the standard of 100°. The Magic Leap One is rumored to have a much larger FOV than the HoloLens.

Untethered future developments for VR and MR

• Will all VR and MR move towards being untethered or will the processing power of PCs or mini PCs still be required?
• Untethered MR and VR devices are likely to decrease in size and price and reach a mass market by doing so.

Expected growth in VR, AR and MR markets

A report from Digi-Capital’s Augmented/Virtual Reality Report Q1 2018 forecasts that AR and VR could reach $150B in revenue by 2020. AR is expected to take the biggest portion of this with $120 billion and VR at $30 billion.

AR is expected to have more mass-market appeal due to the accessibility of smartphones and could have hundreds of millions of users. Also, consider that AR and VR startups have raised over $3.6 billion between Q1 2017 – Q1 2018 in the US alone.

All major tech companies like Google, Apple, Microsoft, Intel, Facebook, and others have been adding resources to mixed reality ventures while emerging tech companies like Magic Leap are expected to disrupt the industry.