Our Valued Sponsor
OpinionsConsumer ReviewsGuides and How TosCoffeeGeek ReviewsResourcesForums
Articles: CoffeeGeek Columnist Feedback
Establishing an Espresso Evaluation Process by Mark Prince
Rocket R58 Double Boiler
Rocket Espresso R58 Double Boiler -  Everything you need for the perfect shot!
www.seattlecoffeegear.com
 
Not Logged in: Log In to Postlog in
New Topics updated topics   New Posts new posts   Unanswered Posts new unanswered  
Search Discussion Board search   Discussion Board FAQ faq   Signup sign up  
Discussions > Articles > Columnist... > Establishing an...  
view previous topic | view next topic | view all topics
showing page 9 of 10 first page | last page previous page | next page
Author Messages
Philosopher
Senior Member


Joined: 13 Feb 2007
Posts: 188
Location: Australia
Expertise: I like coffee

Espresso: Silvia
Grinder: Rocky
Posted Tue Oct 9, 2007, 6:22pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JasonBrandtLewis Said:

I view it as the difference between -- say -- a magazine like Car & Driver and another one that focuses only on Italian sports cars with $100,000+ price tags.  

Posted October 9, 2007 link

For the obsessive-compulsive, 'but I need a score' type person:

Which car would you prefer to drive a Ferrari, Porsche or Ford super-car?

They are all fabulous vehicles.  They have different power/weight ratios, handle in different ways and obviously look different yet give the same lap times around a circuit when driven by a professional.    

Would most owners be able to extract the same performance? No
Would many people appreciate how much better they are than a BMW roadster?  No

If one could afford any of these, the final decision is probably not based on straight line acceleration, peak torques, engine capacities or brake horsepower.   It is the overall gestalt of these beasts that make a person choose one over another.  

Once you go beyond driving a Ford Focus or even a Mitsubishi Evo, the debate about 'which is best' becomes moot.  People are not going to be swayed by more redundant tests and numbers.  

Beyond a reasonably robust test to exclude inferior products, I don't see any point in trying to extend the assessment of coffee from a 5 point to 100 point scale.  Beyond a professional making a few recommendation about his top favourites, it is really up to the consumer to determine if he shares the same impression.
back to top
 View Profile Link to this post
Philosopher
Senior Member


Joined: 13 Feb 2007
Posts: 188
Location: Australia
Expertise: I like coffee

Espresso: Silvia
Grinder: Rocky
Posted Tue Oct 9, 2007, 6:42pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JonR10 Said:

it would be a particular bonus to get the barista comments (i.e.  "this blend seemed to work best for me underdosed in a triple basket and pulled as normale shots"). There's no way to make any tasting process completely objective, so I endorse the idea of "transparent subjectivity".  I also prefer the idea of a more subjective rating system like the one that Zin describes and I agree that numerical scores seem out of place for this type of subjective evaluation.  

Posted October 9, 2007 link

I agree - sometimes words are much more effective than numbers.

I suspect this obsession with scores makes people feel more comfortable.  They appear 'rational and objective' and can be assessed quickly, subjected to ranking and statistical analysis.   They are applied to all sorts of situations but they rarely give meaning unless complemented with additional descriptors.  I am not ignoring the expertise of professionals but an over-reliance on cold hard numbers can then become a substitute for others from developing an ability to make their own assessments and conclusions.
back to top
 View Profile Link to this post
JasonBrandtLewis
Senior Member
JasonBrandtLewis
Joined: 9 Dec 2005
Posts: 6,372
Location: Berkeley, CA
Expertise: I live coffee

Espresso: Elektra T1 - La Valentina -...
Grinder: Mahlkönig K30 Vario -...
Vac Pot: Yama 5-cup
Drip: CCD, Chemex
Roaster: No, no, not another...
Posted Tue Oct 9, 2007, 9:50pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JonR10 Said:

Zin - could you please describe what you DO want to see for evaluations in 100 words or less?

Posted October 9, 2007 link

Test every coffee twice:  1) every coffee sample pulled at a "standard" temperature (e.g.: 200F.); 2) every sample tested again, but at whatever specific temperature is recommended by the roaster.  Each time, pull double shots but into two separate cups -- one for tasting as straight espresso; the other for a "milk drink."  

Consider:  a) using a dynometric tamper (e.g.: like this) to eliminate human variations in tamping; b) using volumetric dosing to eliminate another variable; c) a third taste test by brewing using a Technivorm.

Be as descriptive as possible in one's notes, and forego numerical ratings for a five-star type of system.  

(99 words?)

 
A morning without coffee is sleep . . .
back to top
 View Profile Link to this post
MarkPrince
Moderator


Joined: 19 Dec 2001
Posts: 5,590
Location: Vancouver, BC
Expertise: Professional

Espresso: KvdW Speedster
Grinder: Compak K10 WBC
Vac Pot: A bit too many
Drip: Clive Coffee Drip Stand
Roaster: Hario Glass Retro Roaster
Posted Wed Oct 10, 2007, 2:26am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

Oh man did I have to bite my tongue a lot in this long thread! Some of what you guys wrote is exactly what we did in this testing - other things, we did variances on the theme. Some we didn't do at all, which is one of the reasons why discussion is so important. A lot of you brought up some really interesting points and arguments for how to better evaluate espresso - some of which we certainly didn't do in our test, and I wish we had.

But for now, the latest article is up.

Mark

 
CoffeeGeek Senior Editor
www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.instagram.com/coffeegeek (you get the picture)
back to top
 View Profile Visit website Link to this post
Weasel
Senior Member


Joined: 29 May 2007
Posts: 118
Location: San Jose, CA
Expertise: Just starting

Posted Wed Oct 10, 2007, 3:07am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

OK, I'm back for another two cents worth.

I agree with Zin about a simpler rating system. I've always liked the Michelin guide's 3 star - " worth a special trip " rating for restaurants. Perhaps Mark and his entourage should pow wow over how many distinctions should be made for espressos, and what each one means ( I see potential for a lot of fun here ). Personally, I think 5 or 6 would suffice. That, followed by descriptive commentary should be useful to the consumer/geek/reader.

I, however, disagree with standardizing the process via the dynometric tamper or volumetric dosing. This is where the importance and presence of a highly skilled barista comes in. In fact, I thought utilizing the variables was why the first rate Barista was being used to begin with. To me, allowing the Barista to tweak the variables was the most appealing aspect of Mark's throwdown. If the WeRock med-roast blend requires a higher brew temp, heavy dosing, and a light tamp to bring out it's best, then do it. (Just be sure to pass on that valuable info in the review). Don't different blends require different treatments? It seems to me that changing these variables ( pressure too ) is necessary to bring out the best of each blend. Following that, each blend can be evaluated - at it's best.

Granted, the skill of the Barista is the linchpin in this type of evaluation, but Mark seems keenly aware of this. He has made a point of using WBC caliber baristas, that for me, cements the validity of using dynamic variables. Eventually, time - and many espressos will tell. For me, this type of review is overdue, and certainly worth trying.

That said, I do see a place for a second test. I'm sure many of the CG readers/home espresso brewers - don't have/want tweakable machines but would appreciate an evaluation of espresso blends brewed in a fashion they can repeat.  So again, to a small degree, I agree with Zin, this time about testing each coffee twice.

The second test, imo, should cater to the standard of a Gaggia Classic or Rancilio Silvia. Again, allow the Barista to adjust tamp and dose, but use a standard brew temp. and pressure. Think of it as Review for the Common Man. (if done, feel free to play Aaron Copeland's Fanfare as an Anthem before the testing). I suspect that this type of review would speak to many/most? home espresso brewers and to many others who simply want a good/great espresso. Why do that, you say? Glad you asked.

Testing twice will allow the review(s) to speak to a much larger audience, and can demonstrate for that audience ( me included ), the advantages of tweaking a blend. I expect there would be a better overall "score" ( result ) for an Optimized Blend A, than for a Common Man Blend A. How much better? Maybe we'll find out?

And now that I'm done, I see Mark has posted the long awaited article. I off to bed.
back to top
 View Profile Link to this post
bushrod
Senior Member


Joined: 24 Apr 2007
Posts: 304
Location: Alexandria, VA
Expertise: I live coffee

Espresso: Livia 90, Elektra MCaL
Grinder: Vario
Vac Pot: Sunbeam C20B
Posted Wed Oct 10, 2007, 6:38am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

Weasel Said:

That said, I do see a place for a second test. I'm sure many of the CG readers/home espresso brewers - don't have/want tweakable machines but would appreciate an evaluation of espresso blends brewed in a fashion they can repeat.  So again, to a small degree, I agree with Zin, this time about testing each coffee twice.

The second test, imo, should cater to the standard of a Gaggia Classic or Rancilio Silvia. Again, allow the Barista to adjust tamp and dose, but use a standard brew temp. and pressure. Think of it as Review for the Common Man. (if done, feel free to play Aaron Copeland's Fanfare as an Anthem before the testing). I suspect that this type of review would speak to many/most? home espresso brewers and to many others who simply want a good/great espresso. Why do that, you say? Glad you asked.

Testing twice will allow the review(s) to speak to a much larger audience, and can demonstrate for that audience ( me included ), the advantages of tweaking a blend. I expect there would be a better overall "score" ( result ) for an Optimized Blend A, than for a Common Man Blend A. How much better? Maybe we'll find out?

And now that I'm done, I see Mark has posted the long awaited article. I off to bed.

Posted October 10, 2007 link

This is what I was trying to say in the results article thread.  You've said it much better, thanks.

 
Rich A
back to top
 View Profile Link to this post
sharpjd
Senior Member


Joined: 4 Sep 2007
Posts: 7
Location: edinburgh
Expertise: Pro Barista

Posted Wed Oct 10, 2007, 10:07am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

Is the whole thing such a big deal?  The entire process will always be subjective and at the end of the day you can always buy a kilo and decide whether you like a coffee...does it need a score?  Is it really possible to definitively rate 1 coffee better than another?

For example:
- I can't stand Lavazza Qualita Rossa - I just can't get past the robusta smell and all I taste is wood and hessian...but many Italians would rate it as the best coffee you can buy.
- I love Monmouth espresso blend but I know people who dislike such a light roast as an espresso and complain it's not bold enough to be a serious espresso.

For me a simple description of the tastes and a simple score out of 5 (if we need a score at all) would tell me everything I need to know.  Are we just creating a massively complex process where something very simple would do?  This can never become a science so why make such a huge effort in trying to create one.
back to top
 View Profile Link to this post
JasonBrandtLewis
Senior Member
JasonBrandtLewis
Joined: 9 Dec 2005
Posts: 6,372
Location: Berkeley, CA
Expertise: I live coffee

Espresso: Elektra T1 - La Valentina -...
Grinder: Mahlkönig K30 Vario -...
Vac Pot: Yama 5-cup
Drip: CCD, Chemex
Roaster: No, no, not another...
Posted Wed Oct 10, 2007, 10:19am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

Weasel Said:

. . . I, however, disagree with standardizing the process via the dynometric tamper or volumetric dosing.

Posted October 10, 2007 link

Keep in mind, I said "consider."  I'm not sure I like the idea, either, unless . . .

Weasel Said:

That said, I do see a place for a second test. I'm sure many of the CG readers/home espresso brewers - don't have/want tweakable machines but would appreciate an evaluation of espresso blends brewed in a fashion they can repeat.  So again, to a small degree, I agree with Zin, this time about testing each coffee twice.

The second test, imo, should cater to the standard of a Gaggia Classic or Rancilio Silvia. Again, allow the Barista to adjust tamp and dose, but use a standard brew temp. and pressure. Think of it as Review for the Common Man. (if done, feel free to play Aaron Copeland's Fanfare as an Anthem before the testing). I suspect that this type of review would speak to many/most? home espresso brewers and to many others who simply want a good/great espresso. Why do that, you say? Glad you asked.

Testing twice will allow the review(s) to speak to a much larger audience, and can demonstrate for that audience ( me included ), the advantages of tweaking a blend. I expect there would be a better overall "score" ( result ) for an Optimized Blend A, than for a Common Man Blend A. How much better? Maybe we'll find out?

Posted October 10, 2007 link

Perhaps your "standardized" Gaggia Classic/Rancillo Silvia is where -- if it is used at all -- the dynometric tamping and volumetric dosing comes into play, if at all.  (Granted neither machine has volumetric dosing, but bear with me for a moment.)  

The skill of the barista is paramount, and I completely agree the ability to "tweak" is crucial to obtaining the best shot(s) possible.  But since most home users either cannot or do not (want to) tweak to such an extent, the idea of a "standardized" test does have some attraction.

"Scores" could be something along the lines of:

Barista: ***** (with lots of descriptive notes)

Standard: *** (with lots of descriptive notes)

This would also serve to encourage -- on some level -- so-called "standard users" to improve their techniques, while letting them know that "Hairstraightener" may be a better choice in a "standard" environment, while "Calico Cat" night be the better option if you have the ability/desire to "tweak."

 
A morning without coffee is sleep . . .
back to top
 View Profile Link to this post
mitchellb
Senior Member


Joined: 20 Jul 2007
Posts: 635
Location: Houston, TX
Expertise: I love coffee

Espresso: Isomac Zaffiro
Grinder: Mazzer Major
Drip: Hario Pourover
Roaster: Behmor 1600
Posted Wed Oct 10, 2007, 10:40am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

sharpjd Said:

Is the whole thing such a big deal?  The entire process will always be subjective and at the end of the day you can always buy a kilo and decide whether you like a coffee...does it need a score?  Is it really possible to definitively rate 1 coffee better than another?

Posted October 10, 2007 link

snip

sharpjd Said:

Are we just creating a massively complex process where something very simple would do?  This can never become a science so why make such a huge effort in trying to create one.

Posted October 10, 2007 link

very good points. it is a very human thing to try and quantify something that can only be qualified.

that being said.. the reason we find this article interesting is the same reason we are geeks.. though the article is fundamentally flawed in some respects, and can only tell us what a few friends thought about certain espresso blends, that is the fun of it. we get to all argue about it and expand our own views.
back to top
 View Profile Link to this post
MarkPrince
Moderator


Joined: 19 Dec 2001
Posts: 5,590
Location: Vancouver, BC
Expertise: Professional

Espresso: KvdW Speedster
Grinder: Compak K10 WBC
Vac Pot: A bit too many
Drip: Clive Coffee Drip Stand
Roaster: Hario Glass Retro Roaster
Posted Wed Oct 10, 2007, 11:33am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
 

sharpjd Said:

Is the whole thing such a big deal?  The entire process will always be subjective and at the end of the day you can always buy a kilo and decide whether you like a coffee...does it need a score?  Is it really possible to definitively rate 1 coffee better than another?

Posted October 10, 2007 link

On the industry side of things, it is a big deal. Fact is, flawed or not, Ken drives a lot of business for a small roaster. One roaster in particular told me that a 93+ score on Ken's website mean as much as $10-$15K in sales for that month the review is posted.

It's the same with Robert Parker, but I would imagine 10, 20x as much.

For me it becomes a big deal because I'm more and more concerned about the rash of 90-97 point coffees showing up, and while I don't doubt the 97 pointers are fantastic (the Biloya certainly was), it wasn't a 97 point coffee to me - and in a way, that's where the subjectivity comes in.

It's a big deal for me because I know how important a points rating system is for the public (or at least a segment of the public), but only one source seems to currently be driving or running this kind of schema. And no, I do not want to be the counter on a regular basis to Coffee Review. I don't feel I'm skilled enough on coffee evaluation to do so.

I do, however, think that I'm skilled enough to do espresso evaluations, and I have a good core of people in Vancouver to help with that. That said, this test was more complex than it has to be or should be for several reasons. One is, well, it was, as far as I can tell, the first publicly disseminated and explained "process" trying to find new ground in this territory. That alone adds layer upon layer of complexity that comes natural in the first try. Can it be simpler? Sure - we could just run the tests, and post the tasting results - for the reader, simple as pie.

Mark

 
CoffeeGeek Senior Editor
www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.instagram.com/coffeegeek (you get the picture)
back to top
 View Profile Visit website Link to this post
showing page 9 of 10 first page | last page previous page | next page
view previous topic | view next topic | view all topics
Discussions > Articles > Columnist... > Establishing an...  
New Topics updated topics   New Posts new posts   Unanswered Posts new unanswered     Search Discussion Board search   Discussion Board FAQ faq   Signup sign up  
Not Logged in: Log In to Postlog in
Discussions Quick Jump:
Symbols: New Posts= New Posts since your last visit      No New Posts= No New Posts since last visit     Go to most recent post= Newest post
Forum Rules:
No profanity, illegal acts or personal attacks will be tolerated in these discussion boards.
No commercial posting of any nature will be tolerated; only private sales by private individuals, in the "Buy and Sell" forum.
No SEO style postings will be tolerated. SEO related posts will result in immediate ban from CoffeeGeek.
No cross posting allowed - do not post your topic to more than one forum, nor repost a topic to the same forum.
Who Can Read The Forum? Anyone can read posts in these discussion boards.
Who Can Post New Topics? Any registered CoffeeGeek member can post new topics.
Who Can Post Replies? Any registered CoffeeGeek member can post replies.
Can Photos be posted? Anyone can post photos in their new topics or replies.
Who can change or delete posts? Any CoffeeGeek member can edit their own posts. Only moderators can delete posts.
Probationary Period: If you are a new signup for CoffeeGeek, you cannot promote, endorse, criticise or otherwise post an unsolicited endorsement for any company, product or service in your first five postings.
Learn @seattlecoffeegear
Learn all about coffee, watch videos, read how-to articles.
www.seattlecoffeegear.com
Home | Opinions | Consumer Reviews | Guides & How Tos | CoffeeGeek Reviews | Resources | Forums | Contact Us
CoffeeGeek.com, CoffeeGeek, and Coffee Geek, along with all associated content & images are copyright ©2000-2014 by Mark Prince, all rights reserved, unless otherwise indicated. Content, code, and images may not be reused without permission. Usage of this website signifies agreement with our Terms and Conditions. (0.337526082993)
Privacy Policy | Copyright Info | Terms and Conditions | CoffeeGeek Advertisers | RSS | Find us on Google+