Why We Need to Test in the First Place
Have you ever been stoked on an item that you bought until you took it out of the box and used it? There's a big difference between "good on the shelf" and "good out of the box." And when it comes to coffee grinders it's hard to know what you're going to get until you've actually taken it home and given it a good try. Also, with all the variables involved in coffee making it can be difficult to discern which online user reviews truly hold weight, "Does this grinder really 'do the trick?' Or is this reviewer just content with mediocre coffee?" "Is this grinder really a piece? Or is this person just being nitpicky? Maybe their coffee machine is really the problem."
This is why we purchased 11 of the top-rated and most popular grinders on the market and tested them side-by-side to see how they stack up. They all went through the same tests, to the same standards, with the same testers brewing the same high quality coffee. We made coffee with each grinder to see how good it tastes; we ranked how easy they were to use, we cleaned up their little coffee dust doings until it was apparent which ones were mess-makers, and we even considered which grinder was most likely to wake a sleeping baby. We used the same batch of high quality coffee personally roasted by our lead tester, we oversaw that every brewing variable was in keeping with the Specialty Coffee Association of America's (SCAA) guidelines for optimal extraction, and we repeated both French press and drip style taste tests until there was no doubt in our mind where each grinder stood amidst the competition.
Quality of Taste
The Full Flavored Cup — Our Basis for Evaluation
The ultimate goal in grinding fresh coffee is to get a fresh brew that captures and maximizes the beautiful flavors and aromas that are locked up inside the coffee beans. A fresh coffee can be lackluster if it is under-brewed or can show its "not so beautiful" characteristics if it is over-brewed. Oxidized, stale coffee is also a minus when it comes to quality of taste, giving the cup a dingy flavor. A high quality brew is one that delivers the maximum awesomeness of the coffee and is devoid of defective flavor. We called this the "full flavored cup." It was absolutely loaded with natural sweetness, tantalizing aromas, a silky rich mouth feel, layer upon layer of nuance, and a full prismatic range of flavor that exhibits a balance of depth, vibrancy, and everything in between. There are words in the coffee industry for these qualities, such as complexity, resonance, brightness, body and balance. We applied this knowledge to the simple goal of how much the coffee quality would be enjoyed by the unassuming coffee drinker.
Some of our testers have coffee tasting merit badges from their experience in the industry, while others came into our taste tests as laymen with an evaluative vocabulary that mostly consisted of, "Ooh, nice!" "Meh," "Uhh, that's kinda gross," "Yummy," "Wow!" and maybe even an occasional expletive to express strong feelings about the coffee's quality, good or bad. We brewed over 200 cups of coffee, sampling them in all-at-once tastings and in exhaustive side-by-side tastings as well. We tasted each sample while hot and after it cooled. We had both coffee professionals and non-professionals taste these samples with the same definition of "full flavor" in mind. Then we came to a consensus on the ranking of every grinder's cup quality on a scale of 1-10.
Choice of Coffee
Using the right coffee is a big deal—for personal enjoyment but even more so for testing. No matter how incredible your brewing equipment is, using good coffee is an irreplaceable factor in making a full flavored cup. We chose the First Ascent blend from Refuge Coffee in South Lake Tahoe, California because this blend maximized our potential to experience a balanced, full flavored cup from every one of the grinders we tested. The First Ascent blend was comprised of medium roasted coffee from several different origins and was crafted to maximize sweetness, body, layers of nuance, and balance.
Because our lead tester personally roasted this blend, we were given another priceless advantage: our lead tester had long been familiar with each constituent of the blend and their distinct flavor profiles and potential. This provided us even deeper insight into how each grinder performed in presenting different aspects of the coffee's character. "Does the sweetness and body of the Grade One Sumatra really come out with this brew? Or is it light on the tongue?" "Does it have a hint of liveliness and clarity from the juicy Guatemala? Or is it a bit flat, dirty, and muddled?" Our goal was to use a versatile coffee that would help us best understand which grinder would produce a good brew, regardless of the style of coffee put in the hopper.
The many details that contribute to brewed coffee quality include, but are not limited to, the coffee we used, the ratio of water to coffee, temperature of water, temperature of each brewing device, freshness of grind, pour time, extraction time and temperature of the tasting cup and sample. We busted out the gram scales and used manual brewing devices to personally oversee and maintain the integrity of each experiment. We used SCAA guidelines for industry standard coffee tasting and a good handful of common sense.
Dialing it in, For Days — Grind Size
In order to get every grinder onto a level playing field during testing, we made sure that the grind setting was dialed in for every single one. If the grind is more coarse or fine than optimal, the brewing water that flows through or steeps with the ground coffee will not extract the optimal amount of flavor. And because we didn't want a less-than-ideal grind size to influence our scoring of any grinder's peak performance, we gave a LOT attention to "dialing in the grind," finding the unique and optimal grind setting for every machine, for both French press and drip brew.
So we toyed with each grinder, making samples and rubbing them between our fingers until we thought each grinder had a comparable baseline for evaluation. From there we started brewing and tasting, recording and comparing brew times and adjusting each grinder's settings until we felt that each one had produced its best possible cup. If the resulting brew lacked depth we would tighten the grind; if it were a bit bitter or dirty we would grind coarser.
More than a few times, when adjusting the grind for a better brew, we started clicking the grind selector into a range that was indicated on the machine to be for a different kind of brew (i.e., brewing a drip coffee within the grinder's "espresso" range, or brewing French press coffee within the grinder's "drip" range). We felt it was important to discover the grinder's capabilities regardless of whether the recommended settings seemed a bit misleading. In fact, this turned out to be the norm for us. Out of the 11 coffee grinders we tested, only the Breville Smart Grinder Pro, the Capresso Infinity, and the Cuisinart Deluxe Grind brewed their best drip coffee on a drip grind setting; these grinders also brewed their best French press coffee in within their French press grind range. Therefore we recommend that users explore outside of the recommended grind settings to find the best results for their brewing situation. Read more about "dialing in the grind" and how to get the best flavor out of your grinder in our Buying Advice article.
Why We Didn't Evaluate Grind Consistency
Doing our research on the top grinders in the market, we were able to find articles and grinder reviews that compared the consistency of the grind but didn't actually make a cup of coffee and taste it. Why? With coffee brewing there are a lot of variables, and controlling all these variables during experimentation can be difficult and time consuming. Evaluating grain sizes is easier than performing extensive taste tests; but the results are just an inference.
In theory, the consistency of the grind is directly correlated to the quality of brewed coffee flavor. "Boulders" contribute to under-extraction, while "fines" can easily over-extract and yield bitter flavor that overshadows the coffee's virtues. Our test results show that this has some truth to it, but when it comes to evaluating a grinder's ability to produce an epic brew, the most direct way of doing that is to spend the time grinding away, brew the coffee and taste it.
We originally evaluated the grind quality by shaking samples into multiple micron diameters with a sieve set, but once we had a fairly and thoroughly evaluated taste metric in place, we no longer had reason to produce scores for grind consistency. Measuring the consistency of the grind is just a shortcut to really evaluating taste. The quality of a coffee grinder shouldn't be determined by whether it has an impeccable "grind consistency profile" bell curve, but should ultimately be determined by the grinder's end goal: helping you make an excellent cup of coffee.
Ease of Use
When it came to ease of use, our test was simple. We didn't have to collect mounds of data and plug them into a complex spreadsheet. We simply used the machines to perform over 200 taste test samples, and afterwards had a pile of notes indicating which grinders were easy to use and why. How many tasks does it require from the user? How much figuring out? Some grinders, like the Breville Smart Grinder Pro, had a lot of features but were awkward to use and we ultimately wanted to avoid using them. Other grinders, like the OXO On Barista Brain, were so easy to use that even after making 200 test samples, we still wanted to make coffee for everyone in the office as an excuse to keep using it.
The Noise Test
Which grinder is the loudest? Which one is the quietest? The answers seemed apparent to us, but "for science" we busted out the decibel meter and took a reading at an equal distance from each grinder to quantify the volume produced by each machine. We ran each machine with coffee beans in it, jotted down the numbers, confirmed them with a second test and then looked over how they stacked up. But it wasn't too long before we were scratching our heads wondering why the machine we thought was the quietest didn't have the lowest decibel reading. Other grinders seemed tied in their decibel reading but we felt it was obvious that one of them created more ruckus. So we used another means of measuring quietness. We basically gathered in the coffee grinder lab and asked ourselves the question, "If someone was sleeping in the next room and you had to press one of these buttons for a good 10 seconds, which one would it be?" We ranked each grinder accordingly. We found that "noise level" and "volume level" was not always the same thing. How noisy, disagreeable, or abrasive a grinder sounded was a much bigger picture—a dynamic of volume, along with the frequency, tone, or pitch produced by the unit's motor. Ultimately, the Cuisinart Deluxe Grind (our Top Pick for Quietest Grinder) didn't have the absolute lowest decibel reading of the grinders we tested, but it was obviously the most peaceful unit. And similarly the KitchenAid Burr didn't rank lowest because of its volume but because of its screaming, whizzing, "please shut that thing off" motor noise.
Similar to our evaluation of ease of use, mess-free operation rankings revealed themselves over time. You can only brew so many coffees before you get tired of cleaning up after certain units. Some grinders produced mess by spilling coffee retained among their burrs when bumped or moved, others collected "chaff" and coffee dust around the mouth of their grind chamber which eventually found its way on our counters, and several had to be brushed out with every use to avoid the buildup of dingy residue. We noted how big is the mess? How frequently is a cleanup task required from the user? How easy is it to take care of? We evaluated mess-free operation in terms of convenience because at the end of the day, users prefer a mess-free grinder because it just makes life easier. The lowest scoring grinder (the Cuisinart Supreme Grind, which ranked at a 1 for mess-free) was so messy that it felt like our primary task was controlling its chaotic static mess while the process of making coffee became an afterthought. Our top scorers on the other hand left so little mess that they remained unnoticed and cleanup was quick and simple.