How We Tested VR Headsets

By:
David Wise and Austin Palmer

Last Updated:
Thursday
September 21, 2017

Share:
The top headsets on the market  ready for some literal head-to-head testing.
The top headsets on the market, ready for some literal head-to-head testing.

To find out which of VR headset is truly the best, we conducted extensive research, then bought the top model available on the market today. We devised a comprehensive testing plan that pitted these products head-to-head in five weighted rating metrics: Interactiveness, Visual Immersiveness, Comfort, User Friendliness, and Ease of Setup. Each metric had a handful of different evaluations, with each product receiving a subscore in each metric that was then aggregated to determine each product's final score. The sections below explain exactly what we did to test these headsets and our methodology behind scoring. If you are interested in how a specific product performed, then you should consult our comprehensive VR headset review where we compare models side-by-side and highlight the top models of today.

Controller design is very important.
Controller design is very important.

Interactiveness


Comprising the largest portion of the overall score, our Interactiveness metric takes credit for 35% of the total score. This metric boiled down to three tests: how easy it is to interact with each headset, what limitations there are on the space you can use, and how accurate the motion tracking of each headset its.

This first test had two components. First, we compared the different handheld remotes for each headset - if there is one — looking at how they felt to hold, the number of remote configuration available, and the number of inputs.

Mobile headsets have simple controller designs.
Mobile headsets have simple controller designs.

We also looked at if there are any buttons on the headset itself and how useful they were for interacting with your VR environment.

The Samsung Gear is equipped with a built in track pad  home and back button.
The Samsung Gear is equipped with a built in track pad, home and back button.

After we evaluated the different ways you could interact with each headset, we looked at what limitations exist on the amount of space you could use. This is a non-issue for most of the mobile phone headsets, as these are limited to 3 Degrees of Freedom, or 3-DOF. This means that you can look all around from a stationary point, but you aren't meant to walk around. The tethered headsets are all 6 Degrees of Freedom, or 6-DOF, meaning that you can walk around the room, as well as look all around in your VR experience. This is accomplished through external sensors, so this test was basically assessing how much of the room the sensors adequately covered. It was the entire room for some models and only about 7' in front of the sensor for others.

One of two sensors wall mounted for the Vive.
One of two sensors wall mounted for the Vive.

Finally, we looked at how accurate the motion tracking of the headset and the hand controller is for each product. We didn't have a set testing method for this, but after logging hours and hours with each headset, we had a pretty good idea of which controllers drifted and which were highly accurate.

The underwater experience felt very life like; Watch out for sharks!
The underwater experience felt very life like; Watch out for sharks!

Visual Immersiveness


The entire point of VR is to immerse yourself in a situation so deeply that you can believe it is real. The best headsets do a very good job of this and this metric assessed how well each headset did at visually immersing you in the experience. It accounts for a decently large portion of the total score at 20%.

The first test was how well each headset did at blocking ambient light. The best headsets completely blocked any light from leaking in, while the worst let in enough light that it became distracting. We had a handful of different people try on each headset and rate them, aggregating the scores to account for any variation caused by face size.

Next, we compared the resolution per eye of each screen and looked at the field of view. The wider the field of view, the more immersive an experience will feel. We took into account the claimed field of view, as well as how much of an identical test image was shown by each headset when determining scores.

A screen shot used in field of view testing.
A screen shot used in field of view testing.

Finally, we had a group of people give us their holistic view of the visual immersiveness of each headset, as shown through a handful of different VR experiences, ranging from climbing a skyscraper to reading text.

An uncomfortable headset takes away from your overall experience.
An uncomfortable headset takes away from your overall experience.

Comfort


Depending on the experience, you could be wearing a VR headset for a significant amount of time and an uncomfortable headset will definitely become distracting and pull you out of the experience before long. This metric is responsible for 20% of the total score and consists of three different tests.

The first test was how comfortable each model felt on your faces. We again had a handful of people try on each headset, noting how comfortable it was and if there were any irritating pressure points.

The head straps on the Vive can leave you with some interesting hat hair.
The head straps on the Vive can leave you with some interesting hat hair.

The second test assessed if there is sufficient room to wear glasses in the headset. Some model had more than ample room for your glasses, while others had so little room that there is absolutely no way that you could wear glasses with them.

Finally, we evaluated how much ventilation each model had. Insufficient ventilation can cause your face to get sweaty when in use. While this is an unpleasant enough feeling in its own right to make it undesirable, it also can cause the internal optics to fog up.

You are required to plug your Samsung phone into the USB-C connector to use the Gear VR.
You are required to plug your Samsung phone into the USB-C connector to use the Gear VR.

User Friendliness


This metric assessed how much of a hassle it is to use each VR system after the initial setup has been completed. It merited 15% of the overall score for each headset and is based on the results of four tests.

First, we looked at the audio setup for each product. While you can play the sound of your VR experience through the speakers on your phone, computer, or TV, the experience is greatly enhanced by the use of earbuds due to ambisonic sound. This can simulate the location of the sound, causing you to look in the direction of the origin of the sound in your VR experience. This is a similar effect to an ambulance driving by you with the sirens on, where you can track the location of the sound. We based scores for this test on how much of a hassle it is to set up headphones for each model, or better yet, if they were built in.

The integrated headphones on the Rift were by far the most convenient and easy to to use.
The integrated headphones on the Rift were by far the most convenient and easy to to use.

Next, we looked at how much work it was to get the headset ready to use. For the tethered models, this pretty much meant putting on the headset in view of the sensors when the system was on, but the mobile headsets need your smartphone to be inserted properly. This ranged from simply dropping it in the top to elaborate clamping systems that only sort of work.

There isn't much setup  choose the video or game you like and put your phone in.
There isn't much setup, choose the video or game you like and put your phone in.

We also checked to see if you had to take the case off of your phone to get the VR system to work. We also noted if you were likely to press buttons inadvertently on your phone when using the headset — a frequent annoyance with some products.

You need a pretty powerful computer to run the Vive.
You need a pretty powerful computer to run the Vive.

Ease of Setup


The final aspect of our testing process looked at the difficulty in the initial unboxing and setup process for these products, with this group of tests taking credit for the residual 10% of the total score. We looked at what hardware is required to run each system and what the hardware setup and software installation entailed.

For the hardware setup, we looked at how long it would take to completely accomplish, as well as the level of documentation and helpful prompts provided. We performed a similar analysis for the software installation. In both cases, we noted and deducted points when we encountered problems or other difficulties.

To assess hardware required, we awarded more points to models that had a wide range of compatible hardware options and lower scores to products that required particularly expensive, high-end components to run or products that only had a limited range of devices that they were compatible with.

VR headsets ready for testing.
VR headsets ready for testing.

Conclusion


Hopefully, this article has provided more insight into how we tested and scored VR headsets. Check out our comprehensive review here to see how individual products stacked up, or take a glance at our Buying Advice guide for more insight on how to find the perfect VR system to suit your needs.

  • Share this article:
 

Follow Us



Related Review

The Best VR Headsets of 2017

by David Wise and Austin Palmer

Unbiased.