Opinions

The world's most read coffee and espresso resource

Our testing and evaluation parameters at CoffeeGeek have evolved and refined themselves over the decades, providing our reviewers with the documentation tools and standards they need to present a fair and balanced review of coffee equipment. Here’s how we ended up with our current evaluation methods for anything espresso related.

Share Article:

18.5 grams in. 35 seconds on the shot timer from the moment the brew button is pressed. 45 grams total liquid and foam weight volume out.

That is the CoffeeGeek Standard today for most espresso related things we test and review. This includes espresso machines, grinders, and espresso blends and single origins. The only change made is the dose volume and time for espresso machines with filter baskets smaller than 57mm.

How did we come to these parameters? Are they just arbitrary, pulled out of the air, “let’s set one standard and just see how everything performs at that standard” numbers?

No, these numbers and methods are the result of over 20 years of testing, evaluating and reviewing coffee equipment and roasted coffee. Here’s how we arrived at them.

Learning to Evaluate Espresso

I make no bones about my feelings towards espresso in the world of specialty coffee: to me, it is the epitome, the pinnacle of what coffee is capable of. It’s by far and away the most demanding, most torturous thing you can do to ground coffee. To hit everything the espresso brewing method is capable of, you have to really pay attention and really be part of the process. Your equipment has to be well maintained. Your water needs to be perfect. Your grinder has to be capable. And the blend you use must be first rate.

During the course of reviewing all things related to espresso, I quickly realised that, to be fair to every espresso machine I test, every grinder reviewed, and every coffee evaluated, I had to have a baseline standard for how we conduct our tests.

It got so serious at one point, we organized a series of round tables with some of the top Baristas in Canada to come to a new espresso testing standard. That journey is detailed in two articles from 2007 on CoffeeGeek, called Establishing an Espresso Evaluation Process, and Battle Espresso: N. America vs. Italy. There have been many other collaborations between CoffeeGeek members and our website contributors before and since.

We've engaged former national and world champions to help test coffees and espresso equipment
As we developed our testing standards, we had lots of help from cafe owners, world barista judges and competitors.

One of the greatest tools I had access to was the CoffeeGeek Lab, which I operated for a time in Vancouver. Well over 1,500 people came through that lab, including people who went on to be world barista champions and some of the best cafe owners in Vancouver, Seattle, and Portland.

All this collaborative training and evaluation served many purposes, not the least of which was helping me to develop a standardized regimen for testing coffee and espresso equipment for the reviews we write. My constant goal when evaluating a product or coffee is to present the fairest playing field I can, so that in turn, you get an unbiased and even review to read.

And it evolved frequently, at least for the first fifteen years of doing this, as I learned more and more about the art of espresso and how people make espresso in their own homes, in their own ways.

One example: For the first 14 years or so, I believed that meant using the best possible equipment to test with. If I was evaluating a grinder, I’d pair it up with a $18,000 Speedster espresso machine. If a $300 espresso machine was under the microscope, a $2,500 single dose Versalab M3 grinder was used. I had an a-ha moment around 2015 that made me move to more approachable, and affordable equipment.

So… 18.5 grams in. 35 second shot pull. 45 grams out. How did we get there? Here’s some of the major stages in our evaluation regimen.

Aiming for the Perfect Double - this is from our Breville Barista Express evaluation, back in 2019.

The Evolution of the CoffeeGeek Espresso Evaluation Program

As mentioned, we’ve been writing professional reviews and evaluating coffee and espresso machines and equipment, and coffees, for nearly 23 years now. How we reviewed these things in 2004 is quite different from how I do so today.

2004 CoffeeGeek Standard

CoffeeGeek launched in 2001, and the first reviews we published, written by myself and several other contributors, were not much more than your typical Amazon contributed review for a toaster is today. We talked about how the product looked, what the workflow was like, and if we felt good about the purchase.

By 2004, I had developed a first standardized test for several major categories: espresso machines, auto drip coffee makers, grinders, and tampers. Yep, espresso tampers.

Our standard espresso we brewed back in 2004 was the ristretto double shot. We used 17.5g of coffee into a double ridged La Marzocco double basket (for 58mm portafilter machines), producing about 1oz (30ml, under 30g) of brewed coffee in 22-28 seconds. Ristretto was all the rage back then and considered, falsely, as the next step in espresso evolution. It was short-lived (though, to this day, Espresso Vivace, a pioneer cafe in espresso, still brews ristrettos).

In 2004, we were using ristretto as the main brewing method and goal in all our espresso based evaluations. Here's a 30ml shot from a 17.5g dose, in late 2004.

2008 CoffeeGeek Standard

We moved away from ristretto around 2005, because I felt at the time it was a bit of a cheat, and thought a real challenge to one’s barista skills was getting a full 2oz double out of a double shot of espresso that tasted good.

We moved to doing full 2oz shots (60ml, 55g) from 18g of coffee, max-packed into that famous La Marzocco Double Ridged filter basket. Still a 25 second shot time (+/- 2 seconds), with no preinfusion. We also moved to using a GS/3 (CoffeeGeek bought one for company use the first month they hit the market), and a Compak K10 Conic WBC espresso grinder.

The prep changed a bit too. The WDT method was born around this time, but we didn’t use it early on. We did a distribution knock and levelling with fingers. A curved bottom tamper was used for the final prep.

When I first got a production GS3, I organized a series of public evaluations for a monster review we published on CoffeeGeek in 2006. Here, the machine is at Elysian Coffee in Vancouver, with over 30 people attending.
The La Marzocco GS3 MP and Compak K10 Conic, in the CoffeeGeek Lab in 2009.

2012 CoffeeGeek Standard

Three big changes came by 2012 for our testing parameters.

We started using the VST baskets, which were a bit revolutionary. Dosing was bumped up to 18.5g in, with 50g (1.75oz) out.

We also started using the Speedster espresso machine with its manually controlled preinfusion mode, and bumped shot times, with preinfusion to 30 seconds. When it’s working, the Speedster was then, and still is the best espresso machine on the planet. (the problem is they break down a lot).

I also switched to using a Versalab M3 grinder for all tests. Its output is luxurious, and I still think it’s the best espresso grinder ever made (albeit with quirks). The Versalab’s belt drive still gives me mild nightmares to this day.

One more change came in 2013 – after a lot of feedback from visitors to the CoffeeGeek Lab, we dropped our shot volumes to 40g out (1.35oz roughly). We were nearing ristretto country again, and it wasn’t a decision I was ever comfortable with.

The amazingly fluffy, well distributed and donut-like dosing straight from a Versalab M3 grinder.
Around 2012, the Versalab Grinder (on left) and Speedster were used when evaluating other products, like the Baratza Forte in the middle of this shot.

2016 CoffeeGeek Standard

In late 2016, we settled into what our current standard is for 58mm filter basket espresso machines: 18.5g of coffee used, 10 second preinfusion, 35 second max shot time, and 45g double shot out. If our testing is on espresso machines with 54mm (or smaller) filter baskets, we adjust both the dose and output down (17.5g used, 40g out).

We started using a WDT distribution method, though the grinder we settled on doesn’t really need it (more on that below). Initially I was sceptical including this in the regimen, but for many folks without an excellent dosing grinder, it’s a must-do for great espresso shots, so it became part of the evaluation parameters.

We also changed a rather big thing: instead of using a $9,500 La Marzocco GS3 or a $18,000 KvdW Speedster to evaluate grinders and coffee roasts, we started using a Breville Dual Boiler as our baseline espresso machine for all tests and comparisons. We still use the 18g VST basket to this day.

For espresso machine reviews and tests, we went from using a $2,500 Versalab M3 grinder (and occasionally, a $2,000 Compak Konic WBC grinder) to our current standard bearer for espresso grinding: the Baratza Sette 270Wi.

A double shot (into one cup) brewed on the Breville Dual Boiler when evaluating a grinder (not pictured), in 2017.

The Reasons

Our goal with our evaluations is to present espresso machines, grinders, and coffees in a way you can achieve at home with your equipment and some TLC to the process. We want our test results to mimic what you are capable of, yourself. Absolute top of the line espresso machines can produce potentially superior and/or more consistent results. That said, they aren’t “real world” results for 95% of our readership on CoffeeGeek.

Here’s how the various elements of our test process were settled on.

The Dose

Across specialty coffee, an 18-19g dose for a 58mm double basket shot of espresso has more or less become standard. We’ve been using 18.5g as our baseline dose volume since the VST filter baskets came out over 10 years ago. For 54/53mm filter baskets, we do 17.5g as our baseline.

Now, not every coffee (single origin or blend) or every espresso machine is ideal at 18.5g/17.5g doses/ This is why our testing and review rules require a standard espresso blend (Social Coffee’s People’s Daily Blend) for all our testing. This blend is well suited for these doses, yet forgiving enough for machines that don’t do well with our standard doses.

When we’re evaluating coffees, we do need a baseline, but in those evaluations, if the coffee just never tastes right at 18.5g in, 45g out, we’ll try other doses, timings and outputs to see if it improves. That said, its scores are based on our baseline standard shot pull.

A well groomed distribution of the coffee grounds during a coffee evaluation.

The Output

Recently I tweeted out about the standard 45g output in our machine tests, and someone replied “wow, that’s a lot of espresso – it’s got to be weak! I do 30g output!”

30g (about 28ml) of espresso from 18.5g of coffee is definitely a double ristretto, and not a normal double shot of espresso. 45g is 1.5oz of espresso, which is still “short” of the standard 2 ounces from a double shot that the espresso industry has run with since 1947, when modern day espresso was born.

I still feel 45g output is a bit of a cheat (not as big a cheat as ristretto shots are); but it is a fair volume and close enough to the usual output volume most enlightened home espresso drinkers aim for. Again, some coffees taste better with less output; some taste just fine with 60g out. Our standard espresso blend works great at 45g out.

A true double, roughly 22.5g of liquid per cup.

The Timing

We’ve settled on 35 seconds, including up to 10 seconds preinfusion time, as our baseline standard for double espresso shot pulls. If the machine doesn’t have preinfusion, this is dropped down to 25-30 seconds. We use this timing as a dial-in point for grinder adjustments, or dose adjustments if the grinder we’re testing has too big of a jump between grind fineness levels.

The only time I will adjust this is if we’re evaluating a coffee designed for espresso brewing. I’ve found some coffees (and bean types) require longer shot pulls (and sometimes shorter ones) to get the best results in the cup. When this happens, I make a judgement call based on past experience to adjust the shot times appropriately.

45 grams in about 35 seconds, when done.

The Machine

If we are evaluating a coffee or a grinder (or an espresso accessory) our standard shot pulling machine for the last eight years has been a Breville Dual Boiler espresso machine. It’s not cheap – at over $1,200 new – but it’s representative of the kinds of shots you’d get on an average (yet capable) home espresso machine. That said, I’m currently mulling a change to the Breville Barista Express as our baseline espresso machine. It is the world’s most popular espresso machine as of this writing.

In the past, we used a La Marzocco GS3 (2006-2009, 2015) and a Kees van der Westen Speedster (2010-2015) for all evaluations of grinders and coffees. Around 2015, I had an espresso class going on, and one of my students said, “what hope do we have, getting shots like that on our $300 home machine” and I realized they were right. That’s the main reason for the switch in our testing setup.

So why not use a $450 Gaggia Classic Evo Pro, $800 Rancilio Silvia or a $450 Breville Bambino Plus? Or even the Breville Infuser, which is a Barista Express without the grinder?

Because the Breville Dual Boiler provides a very stable and consistent brewing platform, something I need when testing a variety of equipment and coffee. The grouphead gets up to ideal temperatures thanks to a PID controller right on it, and the big brew boiler inside has extremely even temperatures and short recovery times. I want to mimic what you’d get at home with a modern and capable espresso machine for most budgets, but I still need consistency across all the review tests I do over the years.

Our workhorse. We actually use two Dual Boilers - the original one we still have from 2011 (pictured above) and a 820 model from 2016.

The Grinder

If we’re evaluating a different espresso machine, or a coffee blend, our standard grinder is the Sette 270Wi. The 270 – the one without the scale but same adjustment ability – would do as well.

I firmly believe the Sette 270Wi is the best espresso grinder you can get under $1000, all things considered (speed, fluffy grind output, no waste, uniformity of grind at espresso settings). Its output is so fluffy and non-compacted, you don’t need to do the WDT distribution method on its output.

What about the Niche Zero? If I had one, I’d consider changing to that as my new standard, but those are hard to get in North America. It is also more than double the price of the Sette 270.

Even though I think the Sette is “the best”, the price is still approachable. The version without a built in scale is just $280, and the Acaia scale enabled version is $420. Keep in mind, a Rancilio Rocky cost you $300 over 20 years ago, and it doesn’t even compare to the Sette’s ability or output.

A Sette 270 prototype, undergoing testing in 2015 next to the Versalab. It's performance here made me change to this grinder for all our espresso testing.

The Coffee

I’ve mentioned it a few times: we have a standard coffee blend we use for testing all espresso machines, grinders and accessories. It is Social Coffee’s People’s Daily blend.

First and foremost: it is absolutely delicious. Primary taste notes are caramel and chocolate / cocoa. Exactly what I love in an espresso shot.

Second, it’s been extremely consistent over the years because Steve Souphanthong, head roaster and owner of Social, has no problems changing the blend’s components or roast levels, to keep the same taste profile in the blend, season by season, year by year.

Third, People’s Daily is one of the more forgiving blends I’ve ever had.

“Forgiving”? That means the coffee blend lends itself to producing good output from a variety of machines, brewing methods, doses, and output volumes. A 30g double ristretto with People’s Daily tastes syrupy awesome. A 45g standard double shot with the blend tastes different than the ristretto shot, but still tastes fantastic. Even lungos – 60g+ shot pulls – taste just fine with this blend.

I have so much experience with this blend over the years that I know it inside and out, and I use that intrinsic knowledge to recognize when the machines I’m testing are the problem.

Social Coffee back in 2014, when we first started using their blends. One of the best decisions I ever made at CoffeeGeek.

Is it Perfect?

Our testing parameters for espresso haven’t really changed in about 7 years now. Prior to that, they did get subtly modified and sometimes drastically overhauled.

That doesn’t mean our testing parameters are perfect. They could evolve again as new technology, new equipment and new expectations evolve in the world of espresso. Back in 2003, the “ristretto” was king, and people thought that was the epitome of the brewing method. Some to this day still do. Back in that same time, the Mazzer Mini was considered the best grinder you could buy for home espresso. It’s been a long time since that was the case.

Back in 2010, I used to “split” espresso shot pours when evaluating coffees, to taste how each segment of the shot tasted. I didn’t find it added much to my evaluations, so I stopped doing it, but now, some 13 years later, I feel I know more about evaluating espresso brews, so I might bring it back, but only if it adds value to any coffee reviews I do in the future for our readers.

This used to be part of our coffee evaluation test for espresso blends and single origins. I might bring it back into the regimen.

We want our testing to reflect real world expectations and results. We want to test grinders using the typical espresso machines you might own and use. We want to test espresso machines using a grinder most folks can budget enough to buy. As we settled into these equipment choices, the parameters for their use evolved and became set.

The home super coffee geek hobbyist – pulling 25g shots after plowing through 150g of coffee dialing everything in – isn’t representative of how the average, enlightened espresso drinker expects and enjoys their morning espresso. Still, God bless them, because these super geeks are some of the primary experimenters / inventors of new and better espresso techniques. Things like the WDT method and RDT static prevention tips came from these kinds of super geeks.

We aren’t trying to mimic the home super coffee nerd. We’re trying to present reviews based on how you would use the equipment, brew the espresso, and grind the coffee. I feel this is the best way we can provide you with real and valuable advice for what is a pretty major appliance purchase.

Did you enjoy this article?
PLEASE CONSIDERSHARING IT:
photo credits:
Mark Prince
Guest Columnist

Mark Prince

Support CoffeeGeek

If you enjoy and learn from this resource, please consider making a one time or recurring donation to help support our work and fund purchases for future reviews.

Donate
donate via Paypal

Other Recent Opinion Articles

The companies that make this website possible

CoffeeGeekSponsors