How We Tested Wifi Thermostats

By:
David Wise and Austin Palmer

Last Updated:
Tuesday
September 26, 2017

Share:
We spent weeks testing every aspect of these thermostats, going so far as to build a custom testing board to compare them side-by-side. This board used a series of components to convert the signal from the thermostat to the control of a standard wall outlet. We did this three times and hooked a space heater, a portable AC unit, and an auxiliary fan to each of the outlets. This meant that when a particular model called for the furnace to turn on, it was actually turning on the space heater, with the same pattern following for AC and the fan.

Some of the top thermostats  ready to go on our testing board.
The reverse side of our thermostat testing board.
 

This allowed us to repeat the test on each thermostat immediately, leading (in our opinion!) to a better, direct comparison between models. This led right into the first scoring category that we assessed: Install & Support.

Having already formed some opinion when we were wiring the base plate of each model, we also installed each model into the building where we work, to more accurately approximate what the typical person would go through. We also evaluated the supplied documentation, and whether or not we needed to consult online resources to obtain the knowledge to feel comfortable installing each thermostat. The final component of install was to assess the difficulty of installing the companion app on a phone and pairing it with the thermostat

Now that we had completed the install, we evaluated the support each manufacturer provided for their customers. After removing the current thermostat in our office, we sent in a picture of the bare wires protruding from the wall to each manufacturer and asked if their thermostat would be compatible. Responses that were expedient and helpfully walked us through where and how to determine which wire was which scored highly, and companies that required follow up emails or were less helpful scored poorly. A similar process was conducted for each company over the phone, though we took the time and number of transfers into account when determining scores.

The most important scoring metric that we looked at, and consequently the metric that made up the majority of our scores, was ease of use. We looked at the initial thermostat setup, setting the date and time, as well as setting a sample schedule — something that would be applicable to a standard office. Initially, we would attempt to accomplish each of these tasks without consulting the manual — as we know there are plenty of people out there that might not necessarily consult it — and only looking at it when we were truly stuck, ranking each product on its ease of use and intuitiveness. Some of the models guide you through a series of prompts, and were by far the easiest, while others definitely required the manual to have any chance of figuring out what to do.

The setup prompts on the Nest.
The setup prompts on the Nest.

After initializing each thermostat, we played with setting up the more advanced features, such as setting a lock on the thermostat, multiple heating stages, and notifications. Finally, the ease of adjusting the current temperature, and the variety of customization each thermostat allowed for holding the adjusted temperature. Some model would simply hold it indefinitely, while others would wait until the next scheduled change.

The Nest gives an estimate of how long it will take to reach the desired temperature  as well as change the background color based on if it is heating or cooling.
The Nest shows the current temperature  desired temperature  and estimated time to hit it on its home screen.
 

While making up the second largest portion of our scores, this next aspect was the first thing we looked at — Install & Support. To assess the level of support provided by each of the manufacturer's, we sent an email stating that upon moving into a home the previous thermostat was gone, leaving only bare wires remaining, We were asking if their particular thermostat was compatible, and to see if they would aid us in determining which wires were which.

The photo we sent into each customer support service. We asked them to assist us in determining where to go from here if we did not know which wire controlled what.
The photo we sent into each customer support service. We asked them to assist us in determining where to go from here if we did not know which wire controlled what.

We also rated the supplied documentation to aid with install for each product, either the hardcopy manual, guided prompts, or online resources. After figuring out how to connect each thermostat, we installed each model on out testing board, as well as in our office.

Finally, we looked at the ease of installing the companion app and connecting the thermostats to our WiFi network. This leads right into the next rating metric we evaluated: Remote Access.

We essentially repeated the majority of our ease of use tests, now attempting to complete the various tasks remotely through the internet connection. We set schedules, modified basic and advanced settings, and adjusted the current temperature through the remote interface, and ranked their performance.

Setting the schedule on the Sensi app.
Setting the schedule on the Honeywell app.
 

The final metric that we assessed — and one that we did not originally think would come into play — was the accuracy of the thermostats. We procured a lab-grade thermometer and compared it to the temperatures read by the thermostats, multiple times and across a large range.

For a more descriptive look at the side-by-side performance of these products, take a glance at our WiFi Smart Thermostat Review, or take a look at our Buying Advice article for a detailed discussion of why you would even want one of these products in the first place.

  • Share this article:
 

Follow Us


Table of Contents


    Related Review
    The Nest took home the top award as the best wifi smart thermostat.

    Best Wifi Smart Thermostats of 2017

    by David Wise and Austin Palmer

    Unbiased.