How do we test smartwatches? We spend hundreds of hours doing extensive research, reading through manufacturer specifications, and comparing user experiences with different models to pick the most promising smartwatches available. From there, we purchase the top models and put them through extensive side-by-side testing, which we split into five metrics, weighted according to importance. We combined individual test results for an overall score within each metric, which helped us identify specific areas where some wearables shine, and others fail. The combined metrics determined the final scores for each device and the winners in our battle for the best smartwatch.
Ease of Use
Our first test metric rated and ranked the user-friendliness of each wearable and was the most heavily weighted metric, comprising 30% of the final score of each product. The metric tested the ease of taking screenshots, device water resistance, charging setup, whether or not devices utilize a crown or bezel scroll, screen responsiveness, and the ease of swapping wristbands.
Our first test evaluated the ability and ease of taking a screenshot on each device. Almost every device we test, where supported, uses a unique method of taking screenshots. Some require complicated navigation through menus or settings and some combination of simultaneous button pressing, while others make taking a screenshot as easy as a swiping motion of the screen.
We then rated the water resistance of each device. A wearable that has to be removed before a shower, doing a sink full of dishes, or a pool workout is neither user-friendly nor is it great for fitness tracking. The best of the devices remained water-resistant to 50-meter depths, and the worst were essentially only splashproof.
All the devices we tested consume power and need to be recharged relatively frequently. We tested the user-friendliness of each wearable's charging setup, scoring each one based on the strength of the connection to its charger, as well as how easy it is to connect to the charger.
Our fourth test rated the presence and performance of a crown or bezel and the ability to use the crown or bezel to navigate or scroll through screens and apps.
Next we looked at screen responsiveness. We observed the time it took for a device to respond to the turn of a wrist, as well as the snappiness of the touch screen to respond to touching and swiping.
The last test in our ease of use metric evaluated the ease of changing wrist bands. Whether you swap a wristband for comfort, fashion, or because the band is simply worn out, there are a lot of reasons to change one. We changed wristbands and rated the process as user-friendly or cumbersome.
The next set of tests we performed made up our smart functions metric, and 20% of each product's final score. For our first test, we assessed the availability and compatibility of popular apps. We attempted to install and use Uber, Messenger, Spotify, Strava, IFTTT, Evernote, Shazam, Twitterrific, Instagram, and Pandora on each device. The popular app test really distinguished smart wearables intended for everyday convenience from those that were less smart and tailored specifically to fitness tracking.
Next, we looked at whether or not these products could receive phone calls. Whether a wearable allows the user to answer a call, or if it just alerts the user to an incoming phone call, varied from device to device. The range of support included everything from just notifying the user of a phone call on their wrist (with the call then having to be picked up on a phone) to some devices allowing users to go through the call log and even dial out from their smartwatch.
We compared and evaluated the ability of each watch to control music. We explored music controls to see if they allowed users to play, stop, pause, skip, adjust volume, and even whether or not they supported thumbs up and thumbs down interactions.
With a growing number of payment processors and locations accepting wireless payment support from mobile devices, commonly referred to by the acronym NFC, we tested support for making payments from each wearable. We awarded points for the ability of each device to allow you to pay for things using wireless payment methods like Apple Pay, Google Pay, or Samsung Pay.
We also awarded points if there was a built-in GPS unit in each watch, giving the user freedom to leave the phone behind but still record location data and maintain navigational abilities. With some exceptions, the bulk of modern devices we tested incorporate a built-in GPS and do not require on the GPS data of a paired phone.
The last evaluation of smart functions we made looked at the cellular capabilities of each device. Standalone cellular capabilities allow users to leave the phone behind whilst keeping the ability for apps on the wearable to use data and take or make phone calls.
Our third metric evaluated the display on each device and contributed 20% of the final score. We evaluated and scored screen quality, readability in different levels of both bright or dark light, auto brightness adjustment and performance, and whether or not each device had an always-on setting.
The first and most important test of the display metric was screen quality. To evaluate this, we looked at the size of the screen, its type, and its pixel density. We also did a side-by-side test with the same image shown to a panel of observers that rated which models simply looked the best. Next, we compared how easy it was to read the screens in bright and dark conditions. Finally, we gave points for models that had an auto-brightness feature or an always-on feature.
Next we moved on to our fitness impact metric, which compared and scored the fitness tracking and monitoring features each watch offered, making up 15% of the final score of each device.
Individual tests within the metric included testing accuracy of the step counter, determined by comparing the count shown on each watch to our manual count collected on a mile-long walk. We also tested the accuracy of device heart rate monitoring against a chest strap monitor. We captured data both during rest and during a range of exercises.
Finally, we compared the ability of each watch to track various workouts, such as the number of preset profiles that were available. We also looked at how accurately every device could track the number of stairs climbed each day.
Our final test metric evaluated battery performance and comprised 15% of each product's overall score. Here we tested the actual battery life of each device under normal use by sending various calls and notifications to each device at timed intervals. We also tested the charging performance by measuring the time it took each device to charge to 25%, 50%, 75%, and full charge.
Picking the right smartwatch can be difficult, so we designed our five-metric scoring system and extensive individual tests and evaluations within each of those metrics to help. If you are interested in seeing how each model we tested performed in our tests, then take a look at the results of our comprehensive smartwatch review.