Our Valued Sponsor
OpinionsConsumer ReviewsGuides and How TosCoffeeGeek ReviewsResourcesForums
Articles: CoffeeGeek Columnist Feedback
Establishing an Espresso Evaluation Process by Mark Prince
Stefano's Espresso Care
Repair - Parts - Sales
Factory Authorized &
Trained Technician
www.espressocare.com
 
Not Logged in: Log In to Postlog in
New Topics updated topics   New Posts new posts   Unanswered Posts new unanswered  
Search Discussion Board search   Discussion Board FAQ faq   Signup sign up  
Discussions > Articles > Columnist... > Establishing an...  
view previous topic | view next topic | view all topics
showing page 7 of 10 first page | last page previous page | next page
Author Messages
Philosopher
Senior Member


Joined: 13 Feb 2007
Posts: 188
Location: Australia
Expertise: I like coffee

Espresso: Silvia
Grinder: Rocky
Posted Mon Oct 8, 2007, 3:11am
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JasonBrandtLewis Said:

It's not just the essays you point to.  I've often said that you can't have a 100-point scale, that it promotes the concept of "perfection" --and what happens when that perfect wine, or that perfect "god shot" is surpassed by another one?  104 points?

Posted October 7, 2007 link

Moreover, it assumes that you can mark coffee on a linear scale which will be scored the same way no matter who, when and how many times it is marked.  Considering that even green beans will vary batch to batch and season to season, I am not sure if it is possible to achieve this degree of consistency.

In reality every cup has its own particularly unique character which has to be enjoyed on its own merits and interpreted through the preferences of the person drinking it.  

The best advice I have heard about how to use wine scores is to actually find a wine judge whose preferences line up with your own.  More useful are the tasting notes which give more descriptive comments about the vintage.   It is sad when people slavishly follow other's opinions rather trying to develop their own palate and preferences.  It might sound impressive to have a cellar of Parker 95+s but if the big, complex, high alcohol reds do not suit your style then you have really wasted your money (even though you have impressed your wine-geek friends).  Even then your moods will vary about what you prefer drinking on any given day,  at what time and what food you happen to be serving with it.

It is the same with coffee.  If we were honest with ourselves, we tend toward preferences for particular characteristics e.g.  medium or heavy body, chocolate or citrus/floral-like notes, medium or high acidity.  If you are going to use some rating system, it has to be combined with more detailed information to have any meaning.
back to top
 View Profile Link to this post
JasonBrandtLewis
Senior Member
JasonBrandtLewis
Joined: 9 Dec 2005
Posts: 6,368
Location: Berkeley, CA
Expertise: I live coffee

Espresso: Elektra T1 - La Valentina -...
Grinder: Mahlkönig K30 Vario -...
Vac Pot: Yama 5-cup
Drip: CCD, Chemex
Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 7:02am
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

I grant you this is a place to discuss coffee, but . . .

Philosopher Said:

The best advice I have heard about how to use wine scores is to actually find a wine judge whose preferences line up with your own.

Posted October 8, 2007 link

Actually the most important thing is to find a wine critic who is consistent.  This is true whether or not you agree with him/her.  The more consistent the critic is, the more it is possible to "calibrate" your own palate preferences to his/hers.  Parker is perhaps the most consistent critic.  As such, I know that when he uses certain words, regardless of the point score, I know I'm absolutely going to not like that wine at all . . . regardless of the fact it received 95+ points.  Certain other words, and I know I'm going to enjoy the wine, whether it got a "92," an "82," or a "72"!  Someone who is inconsistent in their palate preferences is useless (IMHO, of course).

But, back to coffee:

Philosopher Said:

Moreover, it assumes that you can mark coffee on a linear scale which will be scored the same way no matter who, when and how many times it is marked.  Considering that even green beans will vary batch to batch and season to season, I am not sure if it is possible to achieve this degree of consistency.

In reality every cup has its own particularly unique character which has to be enjoyed on its own merits and interpreted through the preferences of the person drinking it . . . .

It is the same with coffee.  If we were honest with ourselves, we tend toward preferences for particular characteristics e.g.  medium or heavy body, chocolate or citrus/floral-like notes, medium or high acidity.  If you are going to use some rating system, it has to be combined with more detailed information to have any meaning.

Posted October 8, 2007 link

I absolutely agree!  But there is yet another factor to consider:  unlike wine -- even if we all agreed that the extraordinarily rare "Siberian Lentilberry" coffee roasted by "Mooncoins" of Vladivostok was a "100-point" coffee -- there is the variable of the barista.  Whether you go to Mooncoins for an espresso, or if you go to your house . . . or mine! . . . the skill of the individual pulling the shot can instantly turn that alleged 100-point coffee into a 72 in quite a hurry . . . .

Within the wine trade, there are so many variables than can affect a wine -- from temperatures during shipping to storage to food pairings to the allergies or cold that a taster may   be sufferring -- we often say, "There is no such thing as a great wine, only a great bottle of wine."  Perhaps there is no such thing as a great coffee, but only a great cup of coffee (or espresso).

 
A morning without coffee is sleep . . .
back to top
 View Profile Link to this post
mrgnomer
Senior Member
mrgnomer
Joined: 16 Oct 2005
Posts: 2,286
Location: NA
Expertise: I live coffee

Espresso: Vetrano, Silvia, Olympia...
Grinder: Zass., Macap M4 stepless,...
Vac Pot: Yama, Bodum E Santos
Drip: French Press, ibrik, Moka...
Roaster: Hottop programmable, IRoast2
Posted Mon Oct 8, 2007, 8:00am
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

I think issue is more how a roast is evaluated not what kind of rating system is used.  The evaluation process would probably evolve into a rating system.  If you're evaluation is subjective and under what you consider is common extraction conditions like 200F across the board, same grind/dose/distribution for all roasts your rating would be more subjectively based regardless of what system you're using.

If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective.  Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control.  The results of a test like that would be more reflective of actual roast quality.  Yes, tasting is subjective for sure and could be influenced by the sampler's preference.  You could give a biography of the sampler with their personal preferences in a roast.  That would help somewhat with putting a sampler's subjective evaluation into perspective.

Whatever is done roasts need to be evaluated fairly with essential variables of espresso extraction addressed and controlled with a protocol that is not arbitrary or unreasonably inconsistent.  You could really skew your results if you were more focussed on satisfying specific elements of a rating system rather than focussing on the best extraction.  Extraction first then rating system.

With respect to extraction quality, since there is no way to satisfy all possible conditions a roast might be extracted under from equipment capabiltiy to barista skill, evaluations need to be geared towards what a roast is capable of under generally very good to ideal conditions.  There's no other way to know what a roast is capable of, IMHO, and report on it.  Rather than discourage espresso interest I think it would encourage it.  When I started I read about the honey thick pours from dialed in grinds and the deep charactered extractions of extraction temps tuned in to a specific roast it encouraged me to get deeper into espresso rather than give it up.  It makes espresso much more interesting to know how complex and dynamic it is and more rewarding when you approach very good to ideal espresso for yourself.  Espresso's evaluation needs to be uncompromising not only to be competently informative but to encourage appreciation.
back to top
 View Profile Contact via AOL Instant Messenger Link to this post
JasonBrandtLewis
Senior Member
JasonBrandtLewis
Joined: 9 Dec 2005
Posts: 6,368
Location: Berkeley, CA
Expertise: I live coffee

Espresso: Elektra T1 - La Valentina -...
Grinder: Mahlkönig K30 Vario -...
Vac Pot: Yama 5-cup
Drip: CCD, Chemex
Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 12:47pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

mrgnomer Said:

I think issue is more how a roast is evaluated not what kind of rating system is used.  The evaluation process would probably evolve into a rating system.  If you're evaluation is subjective and under what you consider is common extraction conditions like 200F across the board, same grind/dose/distribution for all roasts your rating would be more subjectively based regardless of what system you're using.

If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective.

Posted October 8, 2007 link



Within limits, it's SUBjective, either way.  

mrgnomer Said:

Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control.  The results of a test like that would be more reflective of actual roast quality.  Yes, tasting is subjective for sure and could be influenced by the sampler's preference.  You could give a biography of the sampler with their personal preferences in a roast.  That would help somewhat with putting a sampler's subjective evaluation into perspective.

Posted October 8, 2007 link

In an ideal world, perhaps . . .

But given the realities of the space-time continuum . . . just how much time does one reviewer have to spend on each coffee sample to pull shots? to tweak grind and temperature for each coffee*? to write up the results? to write up his/her CV to accompany the article? to meet his/her deadlines for the next publication date?  (and so on and so on and so on)

Let alone the roast date of the beans, the freshness (or lack thereof), etc.  And IS the "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru the same blend all year 'round?

 
A morning without coffee is sleep . . .
back to top
 View Profile Link to this post
mrgnomer
Senior Member
mrgnomer
Joined: 16 Oct 2005
Posts: 2,286
Location: NA
Expertise: I live coffee

Espresso: Vetrano, Silvia, Olympia...
Grinder: Zass., Macap M4 stepless,...
Vac Pot: Yama, Bodum E Santos
Drip: French Press, ibrik, Moka...
Roaster: Hottop programmable, IRoast2
Posted Mon Oct 8, 2007, 1:31pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JasonBrandtLewis Said:

Within limits, it's SUBjective, either way.  



In an ideal world, perhaps . . .

But given the realities of the space-time continuum . . . just how much time does one reviewer have to spend on each coffee sample to pull shots? to tweak grind and temperature for each coffee*? to write up the results? to write up his/her CV to accompany the article? to meet his/her deadlines for the next publication date?  (and so on and so on and so on)

Let alone the roast date of the beans, the freshness (or lack thereof), etc.  And IS the "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru the same blend all year 'round?

Posted October 8, 2007 link

Philisophically, everything is subjective.  Space-time continuum?  Only if Dr. Who were pulling the shots, I'd imagine or Picard found himself in a wormhole after firing up his GS3.  Maybe you could get Q to do the reviews?

As far as time goes, it's experience that would be more of a factor than time.  We're talking knowledgeable baristas doing the evaluation not casual home users who happen to be writers and general critics.

The evaluation can be broken down, I believe, to objective factors.  How well those factors are controlled as variables depends on the experience of the evaluator.  Saying it can't be done is non sense.  You'd have to be a top ranked world class barista to say that with any credibility.

The point is general critics are putting forward reviews that read like they're expert evaluations.  Au contraire, mon frère, they're turning out not to be experts.  But the average consumer wouldn't know that just like the average consumer who reads the top 10 espresso picks from Consumer Reports wouldn't know that those picks are not even on the radar for really good espresso.  Really good espresso along with the good roasts that make it possible needs to be fairly evaluated and promoted by good, knowledgeable reviews.  Doing otherwise is just marketing.

Yeah, some roasts could be off from ideal for any number of reasons.  Good roasters wouldn't suffer as much, I'd imagine, the inconsistencies of variables like green quality or freshness.  They'd be sourcing out good greens and making sure their packaging and shipping ensures their roasts reach consumers in a way that preserves their quality.  If their blend varies from time to time so what.  It's the quality of the roast/blend that's being assessed, not what's in it.

If big roaster wants to ship their roasts over by barge and it takes months to be accessible to North American consumers, that's their choice.  If a small/mid sized artisan roaster dates their roasts and ships it right after roasting to arrive anywhere in North America within 3-4 days, that's their choice too.  If the decision to source top quality greens and ship right after roasting makes for better espresso than a stale overseas roast an evaluation that's based on predominantly subjective opinion isn't going to be definitive.  The conclusions of an objective evaluation by qualified baristas is the only way to know with any credible certainty.
back to top
 View Profile Contact via AOL Instant Messenger Link to this post
brentling
Senior Member
brentling
Joined: 25 Sep 2002
Posts: 911
Location: Auckland
Expertise: I like coffee

Espresso: wega, fiorenzato
Grinder: Growing collection of...
Drip: yes people call me that...
Roaster: collection building...
Posted Mon Oct 8, 2007, 2:11pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JasonBrandtLewis Said:

Within limits, it's SUBjective, either way.  

Posted October 8, 2007 link

There in lies the intrinsic problem.

It is also why you use a panel...

It is also why you want the best barista you can get to pull the shots...

etc etc etc

but

JasonBrandtLewis Said:

Within limits, it's SUBjective, either way.  

Posted October 8, 2007 link

still sums it all up.

Brent
back to top
 View Profile Visit website Link to this post
JasonBrandtLewis
Senior Member
JasonBrandtLewis
Joined: 9 Dec 2005
Posts: 6,368
Location: Berkeley, CA
Expertise: I live coffee

Espresso: Elektra T1 - La Valentina -...
Grinder: Mahlkönig K30 Vario -...
Vac Pot: Yama 5-cup
Drip: CCD, Chemex
Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 2:18pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

So -- out of curiousity, Mr. Gnomer -- where do you draw the line?

Do you never review such brands as Illycafe?  What about Terroir?  After all, you'll get it a lot quicker if you like in Portland, Maine than you will of you live in Portland, Oregon.  Conversely, one would expect to receive a shipment from Stumptown more quickly if you lived in Portland, Oregon than in Portland, Maine . . . let alone, how fresh is it if you walk in?  

What about reviewing coffees from Starbuck's?  Peet's?  Peerless?  Caribou?  Or do you only review coffees roasted from "micro-roasters" like Blue Bottle or Ecco Caffe?  Is Intellegensia too large?

It's your publication.  You set up the criteria . . .

  • * * * *

Now then, on another tangent, what are the prerequisites that qualify one to serve as a judge in a professional competition?  There are standards put forth by the World Barista Championship.  But rarely are these professional judges the ones who write critiques of various coffees -- such as may be found on coffeereview.com or in the forums of coffeegeek.com.  I don't know -- are only certified judges permitted to write articles for magazines, trade journals, newspapers, etc.?  I rather doubt it, but if the editor of Globe & Mail wants to hire a professional WBC judge to write a column on coffee, rather than a journalist on staff in the "Food" section . . . .

(Note:  if I read the website correctly, Mark Prince is on the 2006-2007 WBC roster as a Certified Judge for Sensory Evaluation only -- not Technical -- having passed the written test with 90%, and the "Sensory Skills Test" with a score of "100/90/70" . . . whatever that means.  I know that doesn't mean that another judge's evaluation will agree exactly with Mark's, nor does it mean my evaluation will, either.  And, no disrespect whatsoever intended -- I know I would probably fail the WBC judges' test -- but whose opinion is more important:  Mark's, another judge's, or my own?  Hey, if I had Mark's taste buds inside my mouth . . . but I don't; I have my own.)

None of this should be taken as being arguementative.  Rather, these are ALL issues I've dealt with for some 25 years as a professional wine judge/writer.

 
A morning without coffee is sleep . . .
back to top
 View Profile Link to this post
MarkPrince
Moderator


Joined: 19 Dec 2001
Posts: 5,582
Location: Vancouver, BC
Expertise: Professional

Espresso: KvdW Speedster
Grinder: Compak K10 WBC
Vac Pot: A bit too many
Drip: Clive Coffee Drip Stand
Roaster: Hario Glass Retro Roaster
Posted Mon Oct 8, 2007, 3:02pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

JasonBrandtLewis Said:

(Note:  if I read the website correctly, Mark Prince is on the 2006-2007 WBC roster as a Certified Judge for Sensory Evaluation only -- not Technical -- having passed the written test with 90%, and the "Sensory Skills Test" with a score of "100/90/70" . . . whatever that means.

Posted October 8, 2007 link

They have me listed as sensory only? Yikes. I didn't see that. Maybe it's because I've told Justin over and over again that I'm not comfortable doing technical... ;)

Re the scores, 100/90/70 (actually, it's 72, or was supposed to be) refers to the FDA-styled Sensory Skills Test that the SCAA does - three rounds, each one harder than the last, the last one pretty freakin' hard. Those are my scores in each round (also, the minimum required to "pass".

I took the sensory skills test informally a second time to see if I fluked the first one, and got 100/100/74 the second time around.

Mark

 
CoffeeGeek Senior Editor
www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.instagram.com/coffeegeek (you get the picture)
back to top
 View Profile Visit website Link to this post
Philosopher
Senior Member


Joined: 13 Feb 2007
Posts: 188
Location: Australia
Expertise: I like coffee

Espresso: Silvia
Grinder: Rocky
Posted Mon Oct 8, 2007, 3:39pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 


Actually the most important thing is to find a wine critic who is consistent.  

Agree

JasonBrandtLewis Said:

Within the wine trade, there are so many variables than can affect a wine -- from temperatures during shipping to storage to food pairings to the allergies or cold that a taster may   be sufferring -- we often say, "There is no such thing as a great wine, only a great bottle of wine."  Perhaps there is no such thing as a great coffee, but only a great cup of coffee (or espresso).

Posted October 8, 2007 link

You could take the analogy further.  The bean is like the un-picked grape from the vine. The roaster and barista are the wine-makers.

Riesling variety can make some very impressive wines.  However, the final result is very dependent on where that grape is grown, which ones selected and what happens before it gets into the bottle.   There are not many wine-makers who can turn grapes into exceptional wines.

This is why this espresso rating system loses its practicability for most people.  We are rating a bean or blend when what we are really rating is a final composite of many factors, some which we have no proper control over.
back to top
 View Profile Link to this post
Philosopher
Senior Member


Joined: 13 Feb 2007
Posts: 188
Location: Australia
Expertise: I like coffee

Espresso: Silvia
Grinder: Rocky
Posted Mon Oct 8, 2007, 4:15pm
Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
 

mrgnomer Said:

If you're controlling the variables like brew temp, grind/dose/distribution, shot style (normale to ristretto), pump pressure, crema amount...your results and rating would be more objective.  Take one test with all variables across the board constant and see how all roasts fair under the control then test with variables tuned in for each different roast and see how the roasts compare to eachother and with themselves vs. the control.  The results of a test like that would be more reflective of actual roast quality.  

Posted October 8, 2007 link

It seems that if we are just talking about espresso, any rating system either becomes too generic (and potentially meaningless) or too overly prescriptive (and too unwieldy and impractical).  I was wondering if you were to prepare a cup by other methods e.g. vac-brew or french press that you could get such a wide scatter of results.

The rating system also needs to be directed to the appropriate target market.  For the majority of people, they want to know if the blend can give good results but is fairly forgiving even when they are preparing for a party of 10 people.  The minority are those caffeine-toxic from tweaking their machine the whole morning trying to find the right zone for their blend.
back to top
 View Profile Link to this post
showing page 7 of 10 first page | last page previous page | next page
view previous topic | view next topic | view all topics
Discussions > Articles > Columnist... > Establishing an...  
New Topics updated topics   New Posts new posts   Unanswered Posts new unanswered     Search Discussion Board search   Discussion Board FAQ faq   Signup sign up  
Not Logged in: Log In to Postlog in
Discussions Quick Jump:
Symbols: New Posts= New Posts since your last visit      No New Posts= No New Posts since last visit     Go to most recent post= Newest post
Forum Rules:
No profanity, illegal acts or personal attacks will be tolerated in these discussion boards.
No commercial posting of any nature will be tolerated; only private sales by private individuals, in the "Buy and Sell" forum.
No SEO style postings will be tolerated. SEO related posts will result in immediate ban from CoffeeGeek.
No cross posting allowed - do not post your topic to more than one forum, nor repost a topic to the same forum.
Who Can Read The Forum? Anyone can read posts in these discussion boards.
Who Can Post New Topics? Any registered CoffeeGeek member can post new topics.
Who Can Post Replies? Any registered CoffeeGeek member can post replies.
Can Photos be posted? Anyone can post photos in their new topics or replies.
Who can change or delete posts? Any CoffeeGeek member can edit their own posts. Only moderators can delete posts.
Probationary Period: If you are a new signup for CoffeeGeek, you cannot promote, endorse, criticise or otherwise post an unsolicited endorsement for any company, product or service in your first five postings.
Commercial Equipment
Nuova Simonelli, La Marzocco, Rancilio. Nationwide installation. Instant financing options.
www.seattlecoffeegear.com
Home | Opinions | Consumer Reviews | Guides & How Tos | CoffeeGeek Reviews | Resources | Forums | Contact Us
CoffeeGeek.com, CoffeeGeek, and Coffee Geek, along with all associated content & images are copyright ©2000-2014 by Mark Prince, all rights reserved, unless otherwise indicated. Content, code, and images may not be reused without permission. Usage of this website signifies agreement with our Terms and Conditions. (0.416280031204)
Privacy Policy | Copyright Info | Terms and Conditions | CoffeeGeek Advertisers | RSS | Find us on Google+