How We Tested Smartwatches

By:
David Wise and Austin Palmer

Last Updated:
Tuesday
September 26, 2017

Share:
How do we test smartwatches? Find out below!
How do we test smartwatches? Find out below!

How do we test smartwatches? We spent over 150 hours doing extensive research, picking through the manufacturer's specification sheets and comparing user experiences with different models to pick the top 8. We took these top model and put them through an intensive side-by-side testing regime, spread across five weighted metrics to determine the overall scores and select the winners.

The rotating bezel on the Gear S3 was a unique -- and much preferable -- input method compared to touchscreen alone.
The rotating bezel on the Gear S3 was a unique -- and much preferable -- input method compared to touchscreen alone.

Ease of Use


This was the most important metric of our testing process, making up 30% of the total score. While the smart functions and fitness tracking features are important, none of these really matter if it is too much of a hassle to even use the watch in the first place. We conducted six different tests for this metric, rating everything from the interface of the watches, to how easy it was to replace the wristbands.

The most important aspect of this metric was our screen responsiveness test — as there is nothing more frustrating than a smartwatch that fails to respond promptly and accurately to your inputs. We compared how easy it was to navigate through the menus on each model using the touchscreen, as well as timed how long it took for the watch to automatically wake when we raised our wrist.

Some watches took much longer to wake than others  such as the Fossil Q.
Some watches took much longer to wake than others, such as the Fossil Q.

These models will automatically dim the display and enter a sleep mode to conserve battery and wake when you raise your wrist to view the screen. Some of the models were much more astute at waking than others, illuminating instantly, whereas others had a noticeable delay.

In addition to the touch screen and side buttons, several of the top models had a crown scroll or bezel scroll. This is where either the largest side button can be rotated to navigate through the menus, or the actual bezel of the watch rotated and served the same purpose. We awarded more points to models that had this feature.

We preferred to use the crown scroll on the LG Watch to navigate  rather than the touchscreen.
We preferred to use the crown scroll on the LG Watch to navigate, rather than the touchscreen.

Next, we looked at the charging setup for each product. While the battery life is always increasing on these products, you most likely are going to be charging these watches every night, or at least every other night. We ranked the charger of each product, looking for a secure, reliable connection to their charging cable or cradle, and deducted points from models that were particularly finicky to connect or prone to disconnect accidentally.

With the huge array of models available  picking the right smartwatch can be a daunting task.
With the huge array of models available, picking the right smartwatch can be a daunting task.

Then, we compared the waterproof ratings given by the manufacturers, awarding higher marks for those that met a more rigorous standard, and low marks for those that couldn't get wet. We also compared how difficult it was to remove the wristband and swap it out, giving more points for models that allowed swappable bands and had a wide range of compatible bands. Last, we evaluated how much work it was to take a screenshot of content on the watch screen, giving low marks to models that made it very difficult.

The display on the Apple Watch was the best of the group and made it very easy to use.
The display on the Apple Watch was the best of the group and made it very easy to use.

Smart Functions


This set of features is what separates these products from a normal watch — features like being able to make and receive calls, text messages, emails, or see your social media notifications. This metric made up 20% of the total score, encompassing 5 different tests.

The first thing we tested was compatibility with popular apps. We attempted to install Uber, Facebook Messenger, Spotify, Strava, IFTTT (If This, Then That), Evernote, Shazam, Instagram, and Twitter on each model.

The rotating bezel on the Gear S3 is a fantastic way to easily navigate through the menu.
The rotating bezel on the Gear S3 is a fantastic way to easily navigate through the menu.

We awarded the most points to the models that were compatible with the most apps, and for those that retained the most functionality for each app. Next, we looked at if these products could receive phone calls and allow you to answer, of if they just showed a notification to alert you to answer your phone

The Nixon will only notify you of an incoming call  requiring you to pull out your phone to answer.
The Nixon will only notify you of an incoming call, requiring you to pull out your phone to answer.

We also checked to see if the watch has the necessary hardware to allow you to pay for things, using either Apple Pay, Android Pay, or Samsung Pay. We also awarded points if there was a built-in GPS unit in the watch, giving you the freedom to leave your phone behind and still retain navigation abilities.

All of the models we tested  including the Gear S3  offered basic music control capabilities.
All of the models we tested, including the Gear S3, offered basic music control capabilities.

Finally, we compared and evaluated the ability of each watch to control your music, whether the controls automatically appeared when you went to play music

You can customize the face of the timepiece to match your personal tastes  whatever they may be.
You can customize the face of the timepiece to match your personal tastes, whatever they may be.

Display


Even the most advanced smartwatch, packed with features and functions is pretty much worthless if the display makes it too difficult to see anything. This metric made up 20% of the overall score and was composed of five tests.

The first — and most important — test was screen quality. To evaluate this, we looked at the size of the screen, its type, and its pixel density. We also did a side-by-side test with the same image shown to a panel of observers that rated which models simply looked the best.

Some of the different styles of watch faces available on these products.
Some of the different styles of watch faces available on these products.

Next, we compared how easy it was to read the screens in bright and dim conditions. We also deducted points from the models that had a truncated screen — something we called a flat tire.

The Gear S3 has a fully-circular display.
The Fossil Q Marshall has a clipped  flat tire display.
 

Finally, we gave points for models that had an auto-brightness feature or an always-on feature.

The Apple Watch does a solid job at fitness tracking.
The Apple Watch does a solid job at fitness tracking.

Fitness Impact


All of these products have the capability to function as a limited fitness tracker, allowing you to use just one piece of tech to count your steps and stay constantly connected to your social media presence. This metric also made up 15% of the total score.

We compared the accuracy of the step counter on each model against a manual mechanical tally counter. We also checked the accuracy of the heart rate monitor against a chest strap mounted monitor to act as a control.

The Gear S3 had a reasonably accurate heart rate monitor when compared to a chest strap.
The Gear S3 had a reasonably accurate heart rate monitor when compared to a chest strap.

Finally, we compared the ability of each watch to track various workouts, such as the number of preset profiles that were available, as well as if the watch could count the flights of stairs climbed.

Battery Life


Finally, we tested the battery life of each model. This metric made up 15% of the total score and was split into three tests: normal use, time to charge to 50%, and time to completely charge.

We timed how long each model lasted during normal use, sending a schedule of calls, texts, and notifications to each watch until it died completely. Then, we timed how long it took to charge to 50% and how long it took to complete charging, awarding points accordingly — faster charging equals a higher score.

Conclusion


Picking the right smartwatch can be difficult but we are here to help. If you are looking at how specific models scored in this testing plan, see the complete results of our side-by-side comprehensive smartwatch review.

  • Share this article:
 

Follow Us



Related Review
The top three models in our test  all winning an Editors' Choice award.

The Best Smartwatches of 2017 for Android & iOS

by David Wise and Austin Palmer

Unbiased.