CoffeeGeek - Articles: CoffeeGeek Columnist Feedback, Establishing an Espresso Evaluation Process by Mark Prince

Articles: CoffeeGeek Columnist Feedback
Establishing an Espresso Evaluation Process by Mark Prince

Quality Espresso Machines

Imported Italian Espresso Machines, Coffee Machines & accessories for all your coffee needs!
www.thecoffeebrewers.com

Not Logged in:

updated topics	new posts	new unanswered
search	faq	sign up

Discussions > Articles > Columnist... > Establishing an...

view previous topic | view next topic | view all topics

showing page 2 of 10

first page | last page

previous page | next page

Author

Messages

MarkPrince
Moderator

Joined: 19 Dec 2001
Posts: 5,462
Location: Vancouver, BC
Expertise: Professional

Espresso: KvdW Speedster
Grinder: Compak K10 WBC
Vac Pot: A bit too many
Drip: Clive Coffee Drip Stand
Roaster: Hario Glass Retro Roaster

Posted Mon Oct 1, 2007, 3:17pm Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

RussP Said: I would suggest having a panel of judges and using the average. I say this because many people are going to use this score in the marketing and promotions of their espresso blends. If someone got a bad score (lets say because it was to dark for you) it could be "used against them" on this or any other site talking about espresso scores. Posted October 1, 2007 link We did a panel of three tasters, with a fourth "buffer" in case one of the three tasters was way off the median. That part is very much a work in progress though. I didn't have to use the buffer scores at all, the three panelists were sufficiently close (drop was 20% different from other scores). The final score is indeed an average score across the three. The lowest coffee in the scoring got 34.5 points. The highest, and it was agreed it was a great shot of espresso, was (only) 79.5points. We all thought, around the table, that was not only a fair score, but a high one. Everyone around the table and everyone pulling shots said the best ones we had ever had would probably score in the high eighties - low nineties, almost echoing what Andrew Barnett said when he said he would have given Klaus' 2006 WBC shot a 91 or 92. Mark

CoffeeGeek Senior Editor www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.about.me/coffeegeek

MarkPrince
Moderator

Posted Mon Oct 1, 2007, 3:20pm Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

Mach Said: Ken says at one point in his response that consumers can't change the brewing temperature of their machine. But I just learned, by watching this video, how much water to run through my ECM Giotto portafilter to get the brew temperature right. The difference in my shots has been nothing short of amazing. I'm now consistently getting really beautiful, sweet espresso from several different blends (one of which I'd hated the day before). I'm sure I could dial each of the blends in even further, but my point for the moment is that you definitely can--and must--adjust the brew temperature on a heat-exchanger machine. Posted October 1, 2007 link I also show on this video how to easily surf the Rancilio Silvia to get different temperature ranges on your machine. I made this point in my private correspondence with Ken that I do agree most (maybe 80%) of consumers can't or won't "temperature surf" a machine, or adjust a pstat to get a desired grouphead temperature, or experiment enough to find a blends' sweet spot. But I also told him I'm not talking to that 80%. I'm talking to the 15% who want better espresso (and who don't know about temp surfing, but are willing to try), and the 5% of consumers who do know about these things. Mark

CoffeeGeek Senior Editor www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.about.me/coffeegeek

brentling
Senior Member

Joined: 25 Sep 2002
Posts: 911
Location: By the beach in Auckland, NZ - turn right you can't miss me
Expertise: I like coffee

Espresso: wega, fiorenzato
Grinder: Growing collection of...
Drip: yes people call me that...
Roaster: collection building...

Posted Mon Oct 1, 2007, 7:41pm Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

Ok I will add my 1/2 cent... First off, espresso needs to be evaluated as an espresso, and brewed coffee in the traditional manner, but for either coffee consistency of preparation makes or breaks the evaluation. Second, my personal opinion is that a simple scoring system is best - the barista score is an interesting twist, and worthy of discussion and exploration. next, I think that even if you are doing random order stuff, if you are doing say, hmm 13 or 14 blends, you should cup / taste 20 samples. The early ones should be repeated at some point - once the tasters palletes have settled in. That or there should be a calibration session involving several espressos prior to starting. (that does of course make it hard to find people prepared to put away 20 odd espressos in a short period of time! but I know I can and would do it, so don't mind pushing the barrow:) ) then the issue rolls around to tempreature etc. I will again make the comment that I think that there are two types of great coffee - those that are tweaked etc to be pulled just so, at which point the shot is great ro fantastic or better, otherwise it is perhaps just OK. Then there are the coffees that aren't so dependant on the earth, sun and moon being lined up to make a great shot, the great shots come on a variety of equipment, from a variety of baristas. In my mind these are the stellar blends, and they aren't going to be concerened as much by absolute temperature parameters etc. I don't know if they would score as high as perhaps the super tweaked ones, but in some ways that is the point. Don't misunderstand me, I will line up for that great shot pulled just so so it is fantastic as the moon passes the sun, who wouldn't? Brent

MarkPrince
Moderator

Posted Mon Oct 1, 2007, 8:20pm Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

brentling Said: then the issue rolls around to tempreature etc. I will again make the comment that I think that there are two types of great coffee - those that are tweaked etc to be pulled just so, at which point the shot is great ro fantastic or better, otherwise it is perhaps just OK. Then there are the coffees that aren't so dependant on the earth, sun and moon being lined up to make a great shot, the great shots come on a variety of equipment, from a variety of baristas. In my mind these are the stellar blends, and they aren't going to be concerened as much by absolute temperature parameters etc. I don't know if they would score as high as perhaps the super tweaked ones, but in some ways that is the point. Posted October 1, 2007 link Brent, the ironic thing is, in the discussions on how to score the espressos, I was a strong proponent of giving a higher scoring status to the most forgiving blends. In past CG articles, in these forums, on Coffeed, I sometimes brought up specific praise for "forgiving blends" that seem to not care so much for the machine temperature or other variables - blends that, at a wide range of brewing conditions, deliver a pleasing shot. The shot may taste different at different variables, but is pleasing nonetheless. In the Espresso Awards article I did, I even gave nod to Kid-O, at the time, as the winner based, more than anything else, on the forgiveness of the blend (at that time - it's been much less forgiving thru this summer). BUT.... the idea of doing a "forgiveness" rating was nixed in the discussions (I was vetoed), mainly due to the difficulty in achieving a good score for this kind of thing. How would we score something so subjective? How, if we were doing 7 espresso blends in a session, tasting as much as 21 drinks (2 espresso + 1 capp per blend) could we achieve any idea as tasters about that. One of our parameters in evaluating the espressos was swallowing (read: no spitting) as much as possible. Even with these amounts, we were facing blown palates by the end of the session. If we upped it to 4, 6, 8, 10 shots of each blend, we'd barely be able to creep through 2 blends a day. In the end, I snuck in "forgiveness", but put it on the barista pulling the shots, and explained to them that the forgiveness of the blend was part of their scoring - if it was easy to work with and had a wide range of sweet spots while they were dialing it in, score it up big time. Mark

CoffeeGeek Senior Editor www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.about.me/coffeegeek

fookoonetwork
Senior Member

Joined: 19 Dec 2001
Posts: 224
Location: Los Angeles
Expertise: I like coffee

Espresso: Pasquini Livia 90
Grinder: Pasquini Moka 90, Anfim...
Vac Pot: Royal Balance Brewer
Drip: none
Roaster: Behmor 1600

Posted Mon Oct 1, 2007, 9:28pm Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

Mark, Certainly a thought provoking article but one that seems to be an academic exercise because normal consumers do not have access to a world class barista to pull the shots and tightly control the espresso variables. This may be a fair evaluation of espresso, but not one that normal, non professional consumers will ever come close to experiencing, especially with your declaration that you a truly dedicated amateur, cannot match up to a professional. It is simply too bad that this kind of espresso is not commonly available at the various coffee/espresso shops that indicates just how difficult it is to pull it off on a consistent basis.

brentling
Senior Member

Posted Mon Oct 1, 2007, 9:31pm
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

MarkPrince Said:

Brent, the ironic thing is, in the discussions on how to score the espressos, I was a strong proponent of giving a higher scoring status to the most forgiving blends. In past CG articles, in these forums, on Coffeed, I sometimes brought up specific praise for "forgiving blends" that seem to not care so much for the machine temperature or other variables - blends that, at a wide range of brewing conditions, deliver a pleasing shot. The shot may taste different at different variables, but is pleasing nonetheless.

Posted October 1, 2007 link

And I think I have always agreed :)

MarkPrince Said:

BUT.... the idea of doing a "forgiveness" rating was nixed in the discussions (I was vetoed), mainly due to the difficulty in achieving a good score for this kind of thing.

Posted October 1, 2007 link

had you asked I would have nixed it to - in essence it will be self evident if that makes sense.

MarkPrince Said:

How would we score something so subjective? How, if we were doing 7 espresso blends in a session, tasting as much as 21 drinks (2 espresso + 1 capp per blend) could we achieve any idea as tasters about that. One of our parameters in evaluating the espressos was swallowing (read: no spitting) as much as possible. Even with these amounts, we were facing blown palates by the end of the session. If we upped it to 4, 6, 8, 10 shots of each blend, we'd barely be able to creep through 2 blends a day.

Posted October 1, 2007 link

last time we scored espressos, we were sharing cups, which meant we were all tasting the same shot(s). Interestingly, I always find that I can tell when it's a bad shot versus bad coffee.

I also think there is merit in tasting straight shots seperately to milky ones. My reasoning is kind of self centred - I like some recovery time after lots of milk due to intolernaces to dairy :)

MarkPrince Said:

In the end, I snuck in "forgiveness", but put it on the barista pulling the shots, and explained to them that the forgiveness of the blend was part of their scoring - if it was easy to work with and had a wide range of sweet spots while they were dialing it in, score it up big time.

Posted October 1, 2007 link

which is probably the place for it...

brentling
Senior Member

Posted Mon Oct 1, 2007, 9:36pm
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

fookoonetwork Said:

Mark,

Certainly a thought provoking article but one that seems to be an academic exercise because normal consumers do not have access to a world class barista to pull the shots and tightly control the espresso variables. This may be a fair evaluation of espresso, but not one that normal, non professional consumers will ever come close to experiencing, especially with your declaration that you a truly dedicated amateur, cannot match up to a professional. It is simply too bad that this kind of espresso is not commonly available at the various coffee/espresso shops that indicates just how difficult it is to pull it off on a consistent basis.

Posted October 1, 2007 link

The points I think are - if you are evaluating, use the best tools you can, world class baristas in this case are a tool, and if available shouldbe used.

As for what a consumer can get / pull off - I disagree, great shots can be pulled on a lot of the gear out there, and not being a pro does not negate the ability to pull great or even fantastic shots.

One of the best shots of coffee I have had was made by my 11 year old daughter for me... at home.

Weasel
Senior Member

Joined: 29 May 2007
Posts: 118
Location: San Jose, CA
Expertise: Just starting

Posted Tue Oct 2, 2007, 3:41am Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

For awhile I felt like a kid waiting for Christmas while waiting for the articles to appear. Now I feel like I'm waiting to open the biggest present of all. I am a novice both to coffee and to espresso, and I am fascinated by the cg community and it's enthusiasm with espresso. I enjoyed reading about the methedologies for this tasting and I am inclined to agree with the rationale for them, mostly. That said..... The Score: from my viewpoint, I would like to see a score that represents the "quality" of espresso a consumer can expect in his/her cup. For me, that would be based on Taste and Aftertaste of the espresso, period. Forgive my naivte here, but it seems to me that the forthcoming scores will represent something slightly different ( and perhaps rightly so ). It seems to me that the "with milk" component, and some elements of the barista score ( ease of dialing in, forgiveness, and quality of the shot pour ), will result in higher scores for some blends that don't taste as good as a lower scoring competitor blend. Another concern is the points for flavor elements: acidity, body, crema, aroma, sweetness, and aftertaste. I know these are traditional "taste elements" used to describe coffee blends. I just wonder if there are other "taste elements" that are missing and perhaps ought to be factored in. For example let's compare Blend A to Blend B. Is it possible that blend A, which is a little lacking in one , two, or even three of the aforementioned flavor elements, could taste as good as, if not better than, Blend B? I ask this because I suspect that there is more to espresso flavor and taste than acidity , body, crema, aroma, sweetness, and aftertaste. While I believe these flavor elements are useful in describing how the resulting coffee/espresso will taste, they don't necessarily tell you how good it will ( or can ) taste. That considered, I guess what I would like to see is an overall subjective score by the judges with reference to any and all factors that significantly affected the final number ( perhaps this could be a second score? )..... I guess I could just settle for the judges overall score and quit prattling on. I should explain that I prefer Subjective analysis to Objective analysis. Subjective analysis allows for intangibles, whereas Objective analysis is limited to factors that can be quantified. Too often, there are factors that cannot be measured, or haven't been considered. I am reminded of years ago when I was in college. The important criteria for evaluating turntables ( yes, turntables that play records......I still love turntables - good ones ) were total harmonic distortion and WOW & flutter. Turntables were often evaluated by their THD and wow & flutter numbers, not by the quality of sound they produced. Turns out that the turntables which had low distortion numbers seldom produced good sound. My point ( I think ) is that I am less concerned with quantifying crema and acidity, and am more interested in the overall evalution of the blends. Basically, how good does espresso sound? You know what I mean. Final concern ( for now ), I would rather see the barista as a scoring judge, not as an element of the score. If you need to make up the 20 points, I say increase the overall score element to 30 pts. I also say that based on a condition. The tastings should be blind tastings, and the barista should be included as a judge, only if he/she is also "blind". If it isn't possible for the barista to be blind ( say, due to overfamiliarity with the blends being tasted ), then I think the barista should not be included as a judge. I realize I may be missing the mark ( no pun intended ) in my assumptions and 'thinking', so I'll shut up for now and wait for Christmas morn. Thanks to all for your indulgence.

MarkPrince
Moderator

Posted Tue Oct 2, 2007, 3:46am Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

All great points, and after reading them, I think you'll be happy with the results article, for the most part. The tests were done "double blind", which I will explain in full in the next article. Mark

CoffeeGeek Senior Editor www.twitter.com/coffeegeek www.flickr.com/coffeegeek, www.about.me/coffeegeek

fookoonetwork
Senior Member

Posted Tue Oct 2, 2007, 5:20am
Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince

brentling Said:

Posted October 1, 2007 link

But was she able to repeat it? It is fine to have fantastic shots occasionally, but can it be done on a consistent basis? And if so, has it been referenced against shots pulled by a world class barista who is on his/her game?

showing page 2 of 10

first page | last page

previous page | next page

view previous topic | view next topic | view all topics

Discussions > Articles > Columnist... > Establishing an...

updated topics

new posts

new unanswered

faq

Not Logged in: log in
Discussions Quick Jump:
Symbols: = New Posts since your last visit = No New Posts since last visit = Newest post

Forum Rules:
No profanity, illegal acts or personal attacks will be tolerated in these discussion boards.
No commercial posting of any nature will be tolerated; only private sales by private individuals, in the "Buy and Sell" forum.
No cross posting allowed - do not post your topic to more than one forum, nor repost a topic to the same forum.
Who Can Read The Forum? Anyone can read posts in these discussion boards.
Who Can Post New Topics? Any registered CoffeeGeek member can post new topics.
Who Can Post Replies? Any registered CoffeeGeek member can post replies.
Can Photos be posted? Anyone can post photos in their new topics or replies.
Who can change or delete posts? Any CoffeeGeek member can edit their own posts. Only moderators can delete posts.
Probationary Period: If you are a new signup for CoffeeGeek, you cannot promote, endorse, criticise or otherwise post an unsolicited endorsement for any company, product or service in your first five postings.

Learn @seattlecoffeegear

Learn all about coffee, watch videos, read how-to articles.
www.seattlecoffeegear.com