Reviews You Can Rely On

How We Tested Thermostats

Friday April 7, 2023

We spent weeks testing every aspect of these thermostats, going so far as to build a custom testing board to compare them side-by-side. This board used a series of components to convert the signal from the thermostat to the control of a standard wall outlet. We did this three times and hooked a space heater, a portable AC unit, and an auxiliary fan to each of the outlets. This meant that when a particular model called for the furnace to turn on, it was actually turning on the space heater, with the same pattern following for AC and the fan.

Some of the top thermostats, ready to go on our testing board.
Some of the top thermostats, ready to go on our testing board.
The reverse side of our thermostat testing board.
The reverse side of our thermostat testing board.

This allowed us to repeat the test on each thermostat immediately, leading (in our opinion!) to a better, direct comparison between models.

Many smart thermostats, like the Ecobee, have a touch interface.
Credit: Abriah Wofford

Ease of Use

The most important scoring metric that we looked at, and consequently the metric that made up the majority of our scores, was ease of use. We looked at the initial thermostat setup, setting the date and time, as well as setting a sample schedule — something that would be applicable to a standard office. Initially, we would attempt to accomplish each of these tasks without consulting the manual — as we know there are plenty of people out there that might not necessarily consult it — and only looking at it when we were truly stuck, ranking each product on its ease of use and intuitiveness. Some of the models guide you through a series of prompts, and were by far the easiest, while others definitely required the manual to have any chance of figuring out what to do.

Many smart thermostats allow you to conveniently calibrate the temperature for when you're away.
Credit: Abriah Wofford

After initializing each thermostat, we played with setting up the more advanced features, such as setting a lock on the thermostat, multiple heating stages, and notifications. Finally, the ease of adjusting the current temperature, and the variety of customization each thermostat allowed for holding the adjusted temperature. Some models would simply hold it indefinitely, while others would wait until the next scheduled change.

The Nest gives an estimate of how long it will take to reach the...
The Nest gives an estimate of how long it will take to reach the desired temperature, as well as change the background color based on if it is heating or cooling.
The Nest shows the current temperature, desired temperature, and...
The Nest shows the current temperature, desired temperature, and estimated time to hit it on its home screen.

Installation & Support

Having already formed some opinion when we were wiring the base plate of each model, we also installed each model into the building where we work, to more accurately approximate what the typical person would go through. We also evaluated the supplied documentation, and whether or not we needed to consult online resources to obtain the knowledge to feel comfortable installing each thermostat. The final component of the install was to assess the difficulty of installing the companion app on a phone and pairing it with the thermostat

Now that we had completed the install, we evaluated the support each manufacturer provided for their customers. After removing the current thermostat in our office, we sent in a picture of the bare wires protruding from the wall to each manufacturer and asked if their thermostat would be compatible. Responses that were expedient and helpfully walked us through where and how to determine which wire was which scored highly, and companies that required follow-up emails or were less helpful scored poorly. A similar process was conducted for each company over the phone, though we took the time and number of transfers into account when determining scores.

thermostat - the photo we sent to each customer support service. we asked them to...
The photo we sent to each customer support service. We asked them to assist us in determining where to go from here if we did not know which wire controlled what.
Credit: Austin Palmer

We also rated the supplied documentation to aid with install for each product, either the hardcopy manual, guided prompts, or online resources. After figuring out how to connect each thermostat, we installed each model on our testing board, as well as in our office.

Finally, we looked at the ease of installing the companion app and connecting the thermostats to our WiFi network.

Remote Access

For our remote access metric, we essentially repeated the majority of our ease of use tests, now attempting to complete the various tasks remotely through the internet connection. We set schedules, modified basic and advanced settings, and adjusted the current temperature through the remote interface, and ranked their performance.

Setting the schedule on the Sensi app.
Setting the schedule on the Sensi app.
Setting the schedule on the Honeywell app.
Setting the schedule on the Honeywell app.


The final metric that we assessed — and one that we did not originally think would come into play — was the accuracy of the thermostats. We procured a lab-grade thermometer and compared it to the temperatures read by the thermostats, multiple times and across a large range.

For a more descriptive look at the side-by-side performance of these products, take a glance at our thermostat review, or take a look at our Buying Advice article for a detailed discussion of why you would even want one of these products in the first place.

Ad-free. Influence-free. Powered by Testing.

GearLab is founded on the principle of honest, objective, reviews. Our experts test thousands of products each year using thoughtful test plans that bring out key performance differences between competing products. And, to assure complete independence, we buy all the products we test ourselves. No cherry-picked units sent by manufacturers. No sponsored content. No ads. Just real, honest, side-by-side testing and comparison.

Learn More