JasonBrandtLewis Senior Member Joined: 9 Dec 2005 Posts: 6,100 Location: Berkeley, CA Expertise: I live coffee
Espresso: Elektra T1 - La Valentina -... Grinder: Mahlkönig K30 Vario -... Vac Pot: Yama 5-cup Drip: CCD, Chemex Roaster: No, no, not another...
Posted Mon Oct 8, 2007, 6:22pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
MarkPrince Said:
They have me listed as sensory only? Yikes. I didn't see that. Maybe it's because I've told Justin over and over again that I'm not comfortable doing technical... ;)
Glad I had the opportunity to point out that error, Mark . . . ;^)
MarkPrince Said:
Re the scores, 100/90/70 (actually, it's 72, or was supposed to be) refers to the FDA-styled Sensory Skills Test that the SCAA does - three rounds, each one harder than the last, the last one pretty freakin' hard. Those are my scores in each round (also, the minimum required to "pass").
I took the sensory skills test informally a second time to see if I fluked the first one, and got 100/100/74 the second time around.
Posted Mon Oct 8, 2007, 7:42pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
Anyone is free to review roasts relative to espresso quality. The problem for me are reviewers who are not skilled or really qualified to evaluate roasts who put forward their reviews as if they are.
Of course espresso extraction is way more difficult to control than steeping methods. Will a qualified, skilled barista be on every shot? Probably not. But they'll not only be close to ideal more often than someone who's skills and experience are casual they'll have the skills to dial in a roast if it's part of the evaluation process.
With respect to target market, I just had the opportunity to talk with a retired master brewer for a very large, commercial brewer. With his knowledge, skill and experience he had the ability to tailor brews to very specific tastes and characters but never had the chance to change much but basic characteristics of a beer and usually that was done in answer to a change in a competitor's beer. The reason was that the majority of consumers really didn't care about the beer past a few characterists and those were the ones that were targeted by marketing and sales. I think that's true of coffee as well.
Not only is the majority, I imagine, not interested in chasing a God shot by investing in research, equipment, and developing barista skills they probably don't care to. No problem in reviewing for them but just be honest about your evaluation. Most won't likely care if you're giving them Black Cat or *$ and with the machines and skills they have it probably won't make a difference anyway.
If you're looking to evaluate roasts based on good results but is fairly forgiving for the majority you're asking roasters to potentially hobble themselves. Roast for fragile but exceptional character or hardy mediocrity. I hope a roaster that roasts for exceptional character would be supported and not have to choose mediocrity.
In any case, I think to do roasts justice and recognize good roasting the roasts should be evaluated without any market in mind. Why should the assessment of good roasts have anything to do with the mass market anyway? That's rediculous. Bring out the best in a roast for evaluation purposes. Who cares if 95% of present consumers will ever appreciate the same results. 5% will and it's those 5% that'll appreciate a comprehensive evaluation and use it enjoy not only good roasts but to get the best out of the roast. Some of the 95% may be intrigued by good evaluations and come to an appreciation of seriously good espresso. You don't spread the knowledge of what is good by censoring or suppressing it. Spread the knowledge from good evaluations and the 5% minority will most likely grow.
I don't draw any line. I just agree that a fresh roast beats a stale for good espresso extraction and a fresh roast from a concientious roaster is worth investing in more than a stale macro roaster roasting for the 'mass market'. But how should my opinion matter. Review all roasts. Don't bell curve or skew evaluations for the masses. Let extraction draw the line, not opinion.
As far as the competency of an evaluator I believe there's a test for that. Barista experience should be obvious to see even if a resume wasn't enough.
Philosopher Senior Member Joined: 13 Feb 2007 Posts: 188 Location: Australia Expertise: I like coffee
Espresso: Silvia Grinder: Rocky
Posted Mon Oct 8, 2007, 8:24pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
mrgnomer Said:
If you're looking to evaluate roasts based on good results but is fairly forgiving for the majority you're asking roasters to potentially hobble themselves. Roast for fragile but exceptional character or hardy mediocrity. I hope a roaster that roasts for exceptional character would be supported and not have to choose mediocrity.
You don't have to see this as an "either..or" proposition. There are lots of roasters out there whose standard offering is not that great. It would be nice they can score at least an 80+ on any scale. A roaster can both have standard range and a premium one as well. When you hit the strastopheric heights of exceptional coffee then scores and ranking mean less and taste preferences do. Trying to be overly prescriptive about the testing methods becomes less and less relevant.
Who here has said anything about "censoring" or "suppessing"?!?!?
mrgnomer Said:
Anyone is free to review roasts relative to espresso quality. The problem for me are reviewers who are not skilled or really qualified to evaluate roasts who put forward their reviews as if they are.
Welcome to the "real" world of the internet . . . anyone with an opinion can create a blog, or post on a bulletin board -- no credentials required!
mrgnomer Said:
Of course espresso extraction is way more difficult to control than steeping methods. Will a qualified, skilled barista be on every shot? Probably not. But they'll not only be close to ideal more often than someone who's skills and experience are casual they'll have the skills to dial in a roast if it's part of the evaluation process.
I'll grant you any idiot -- OK, most idiots ;^) -- can operate a corkscrew or twist off a screwcap. Thus, the consumer -- the one who buys the bottle of wine -- can be reasonably assured of getting the same wine that was reviewed in the Wine Speculator by that famous wine critic, Jean Deaux. However, this is most certainly NOT the case when the consumer purchases a pound of "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru -- a great deal depends upon the equipment owned by, as well as the individual skill, of that consumer. Just as the skill of a professional barista (and I think you and I agree we're talking about the people the caliber of which enterthe WBC events, and not the person behind the counter at Starbucks) will vary from day-to-day, so will the skills of the consumer vary . . . but, one presumes, over a wider range. That said, that range will/should narrow with experience and -- at least on their own machine -- become increasingly consistent with that experience.
mrgnomer Said:
With respect to target market, I just had the opportunity to talk with a retired master brewer for a very large, commercial brewer. With his knowledge, skill and experience he had the ability to tailor brews to very specific tastes and characters but never had the chance to change much but basic characteristics of a beer and usually that was done in answer to a change in a competitor's beer. The reason was that the majority of consumers really didn't care about the beer past a few characterists and those were the ones that were targeted by marketing and sales. I think that's true of coffee as well.
The same can be said of winemakers at some wineries. The larger you are -- generaly speaking -- the more the accountant is in charge, and the less the winemaker/brewmaster/roaster is.
mrgnomer Said:
Not only is the majority, I imagine, not interested in chasing a God shot by investing in research, equipment, and developing barista skills they probably don't care to. No problem in reviewing for them but just be honest about your evaluation. They won't care if you're giving them Black Cat or *$ and with the machines and skills they have it probably won't make a difference anyway.
If you're looking to evaluate roasts based on good results but is fairly forgiving for the majority you're asking roasters to potentially hobble themselves. Roast for fragile but exceptional character or hardy mediocrity. I hope a roaster that roasts for exceptional character would be supported and not have to choose mediocrity.
I'm not quite sure who here has EVER suggested aspiring for mediocrity. The mere fact that we -- the lurkers and participants ofcoffeegeek.com -- are here means we have an above average interest in coffee/espresso. In this, coffeegeek.com is no different than erobertparker.com -- the participants have an above-average interest AND experience in wine . . . or coffee.
mrgnomer Said:
In any case, I think to do roasts justice and recognize good roasting the roasts should be evaluated without any market in mind.
How do you do that? Does coffee exist in a vacuum? (NO, not a vacuum pot -- a vacuum.)
mrgnomer Said:
Why should the assessment of good roasts have anything to do with the mass market anyway? That's rediculous . Bring out the best in a roast for evaluation purposes. Who cares if 95% of present consumers will ever appreciate the same results. 5% will and it's those 5% that'll appreciate a comprehensive evaluation and use it enjoy not only good roasts but to get the best out of the roast. Some of the 95% may be intrigued by good evaluations and come to an appreciation of seriously good espresso. You don't spread the knowledge of what is good by censoring or suppressing it. Spread the knowledge from good evaluations and the 5% minority will most likely grow.
You are not suggesting to evaluate "without any market in mind"; on the contrary, you are suggesting to evaluate with the very top end -- that top 5 percent -- in mind. That's fine -- but let's be upfront about it. There's nothing wrong with writing/evaluating with the top 5 percent in mind. After all, I'd say that someone like Robert Parker (The Wine Advocate) or Stephen Tanzer (The International Wine Sellar) writes for a far smaller market -- closer to two percent of the market, and those who participate on his (Parker's) website probably represent closer to 0.5% . . .
mrgnomer Said:
I don't draw any line. I just agree that a fresh roast beats a stale for good espresso extraction and a fresh roast from a concientious roaster is worth investing in more than a stale macro roaster roasting for the 'mass market'. But how should my opinion matter. Review all roasts. Don't bell curve or skew evaluations for the masses. Let extraction draw the line, not opinion.
How DO you skew the evaluations for the masses? By doing a standardized test? Hmmmm . . . .
mrgnomer Said:
As far as the competency of an evaluator I believe there's a test for that. Barista experience should be obvious to see even if a resume wasn't enough.
MarkPrince Moderator Joined: 19 Dec 2001 Posts: 5,462 Location: Vancouver, BC Expertise: Professional
Espresso: KvdW Speedster Grinder: Compak K10 WBC Vac Pot: A bit too many Drip: Clive Coffee Drip Stand Roaster: Hario Glass Retro Roaster
Posted Tue Oct 9, 2007, 5:01am Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
Just to keep y'all updated. It's been a crazy weekend for me. CRAZY. I just didn't find the time to finish the photos for part three of this article series. But I'll do all I can today to get it online by this eve.
Posted Tue Oct 9, 2007, 11:33am Subject: Re: Establishing an Espresso Evaluation Process by Mark Prince
MarkPrince Said:
Just to keep y'all updated. It's been a crazy weekend for me. CRAZY. I just didn't find the time to finish the photos for part three of this article series. But I'll do all I can today to get it online by this eve.
JasonBrandtLewis Senior Member Joined: 9 Dec 2005 Posts: 6,100 Location: Berkeley, CA Expertise: I live coffee
Espresso: Elektra T1 - La Valentina -... Grinder: Mahlkönig K30 Vario -... Vac Pot: Yama 5-cup Drip: CCD, Chemex Roaster: No, no, not another...
Posted Tue Oct 9, 2007, 2:50pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
Hmmmm . . . . somehow I missed this the first time around.
JasonBrandtLewis Said:
IFC = In-F'ing-Credible GSM = Good $#!+, Maynard (an homage to Dobie Gilis' best friend, Maynard G. Krebbs) PGS = Pretty Good $#!+ DNS = Does Not Suck DNPIM = Do NOT Put In Mouth
OMG. This is simply brilliant. If you don't mind, I would like to either just use this kind of rating, or something similar (I don't know if many here would know who Dobie Gillis or Maynard are), when I do very informal tasting (not cupping) reviews for future Coffee at the Moment Articles?
Sure . . . for the cost of one GS3 -- installed -- they're all yours!
;^)
Mark, seriously: feel free . . . it's not like I've ever copyrighted my wine evaluation scale! Besides, I actually stole -- with full credit given -- DNPIM from wine writer Bob Thompson, who actually has a rubber stamp that reads "DNPIM," but it stands for "Did Not Put In Mouth," as a reminder to himself -- the wine smelled, shall we say. less-than-thrilling and so why fatigue your palate by tasting a wine you KNOW will not get a medal? I altered the verb tense to "Did Do NOT Put in Mouth!"
But it's really no different -- a bit more colo(u)rful perhaps, but no different from:
Extraordinary ***** Outstanding **** Very Good *** Good ** Standard * Below Standard (no stars = bleecch!)
Posted Tue Oct 9, 2007, 3:56pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
JasonBrandtLewis Said:
Whoa, whoa, whoa, whoa, whoa . . .
Who here has said anything about "censoring" or "suppessing"?!?!?
Welcome to the "real" world of the internet . . . anyone with an opinion can create a blog, or post on a bulletin board -- no credentials required!
I'll grant you any idiot -- OK, most idiots ;^) -- can operate a corkscrew or twist off a screwcap. Thus, the consumer -- the one who buys the bottle of wine -- can be reasonably assured of getting the same wine that was reviewed in the Wine Speculator by that famous wine critic, Jean Deaux. However, this is most certainly NOT the case when the consumer purchases a pound of "Straight Hair Blend" from "Not-So-Brilliant" coffee roasters on the island of Nauru -- a great deal depends upon the equipment owned by, as well as the individual skill, of that consumer. Just as the skill of a professional barista (and I think you and I agree we're talking about the people the caliber of which enterthe WBC events, and not the person behind the counter at Starbucks) will vary from day-to-day, so will the skills of the consumer vary . . . but, one presumes, over a wider range. That said, that range will/should narrow with experience and -- at least on their own machine -- become increasingly consistent with that experience.
The same can be said of winemakers at some wineries. The larger you are -- generaly speaking -- the more the accountant is in charge, and the less the winemaker/brewmaster/roaster is.
I'm not quite sure who here has EVER suggested aspiring for mediocrity. The mere fact that we -- the lurkers and participants ofcoffeegeek.com -- are here means we have an above average interest in coffee/espresso. In this, coffeegeek.com is no different than erobertparker.com -- the participants have an above-average interest AND experience in wine . . . or coffee.
How do you do that? Does coffee exist in a vacuum? (NO, not a vacuum pot -- a vacuum.)
Thanks for breaking down all my comments. Makes me feel special...;)
Anyway, to make things as concise as I can, I'll just try to simply state my position.
(Stepping on to soap box) There are roasters really dedicated to tweaking roasts. Those roasts need to be evaluated with a process and under conditions that are fair to good roasts/roasters. I believe the process is possible, the conditions can be met and roasts can be 'objectively' evaluated. If only a small percentage of home espresso 'geeks' are able to duplicate and appreciate the results of a serious evalutation it shouldn't influence the evalutation in any way nor should 'geeks' be seen as the 'target market' for either the evaluation or the roasts. It's only incidental that they would be able to appreciate both the evaluation and results.
Saying it can't be done because the variables are too difficult to control or because the interest in a serious evaluation is too small is what I believe is not only non sense but does an injustice to good roasts/espresso. Thank you for your time and consideration.
JasonBrandtLewis Senior Member Joined: 9 Dec 2005 Posts: 6,100 Location: Berkeley, CA Expertise: I live coffee
Espresso: Elektra T1 - La Valentina -... Grinder: Mahlkönig K30 Vario -... Vac Pot: Yama 5-cup Drip: CCD, Chemex Roaster: No, no, not another...
Posted Tue Oct 9, 2007, 5:33pm Subject: Re: Some Random Thoughts re: Mark Prince's "Espresso Evaluation Process"
Please keep in mind that a) I am not trying to be argumentative, nor b) am I trying to pick a fight. Far from it. I am, however, trying to understand, and in that attempt, I find it easier to sometimes comment one "section" at a time, rather than to the entire post at once, as I believe some points may/will get lost. So, with that in mind, I hope you take my comments in the spirit in which they are intended.
mrgnomer Said:
There are roasters really dedicated to tweaking roasts. Those roasts need to be evaluated with a process and under conditions that are fair to good roasts/roasters. I believe the process is possible, the conditions can be met and roasts can be 'objectively' evaluated.
That's fine -- indeed, I absolutely agree that there are "roasters really dedicated to tweaking roasts" AND that "roasts need 'should' be evaluated with a process and under conditions which are fair . . . "
But I have to ask: HOW? How can it be done OBjectively? I understand you believe it can be. I don't know the answer to that, and you seem to, so I'm asking you for how you would do an objective evaluation.
If only a small percentage of home espresso 'geeks' are able to duplicate and appreciate the results of a serious evalutation it shouldn't influence the evalutation in any way nor should 'geeks' be seen as the 'target market' for either the evaluation or the roasts. It's only incidental that they would be able to appreciate both the evaluation and results.
Hmmmm . . . I'm not sure it's only incidental, but -- OK.
I view it as the difference between -- say -- a magazine like Car & Driver and another one that focuses only on Italian sports cars with $100,000+ price tags. BOTH publications have a limited audience (i.e.: a lot more people on Planet Earth drive than will ever pick-up a copy of either publication), but Car & Driver, because they review everything from minivans to gigantic SUVs, will have a wider readership than the magazine that only deals with super high-end Italian sports cars. That said, there will always be some people who will never be able to afford/never even consider spending $100,000+ on a car, who will -- not "may," "will" -- pick up and glace at such a magazine. Still, the readership will never be that of Car & Driver.
The question of "knowing one's audience" affects, or should affect, a) how the article is written (e.g.: how much pre-existing knowledge and technical savvy you presume your readership has); b) who you sell your ads to (e.g.: Cartier will be a more likely advertiser if the target audience is the Ferrari buyer; Coors more likely if it's all about pick-up trucks!); and so forth.
But having a "target audience" in mind does NOT limit your readership -- as I said above, a lot of people who may not ever consider a high-end Italian sports car may still pick up the magazine. It's not an "either/or" situation. The Wine Spectator has a HUGE readership consisting of a lot of wine geeks and non-wine geeks alike, everyone (or so it often seems) who has even a casual interest in wine reads it, at least occasionally. Robert Parker's The Wine Advocate has great influence, but a very small readership -- it's by paid subscription only (on- or off-line) -- and one that consists only of the top 1/2 of 1 percent of wine geeks.
mrgnomer Said:
Saying it can't be done because the variables are too difficult to control or because the interest in a serious evaluation is too small is what I believe is not only non sense but does an injustice to good roasts/espresso.
It kinda seems like anyone who posts any opinion differing from yours gets a dissertation from you.... I'll admit I don't have the patience to read through the mounds of text on this thread so I'll just ask this:
Zin - could you please describe what you DO want to see for evaluations in 100 words or less?
I did follow this thread with interest before it got too thick for my limited attention span, and I don't have very much to add. I'll chime in that I like what Mark has proposed as a start, it's transparent and flexible and caters to what I personally would be interested in seeing.
I'm no pro, but like many here I feel like I generally pull pretty decent shots at home and it would be a particular bonus to get the barista comments (i.e. "this blend seemed to work best for me underdosed in a triple basket and pulled as normale shots"). There's no way to make any tasting process completely objective, so I endorse the idea of "transparent subjectivity". I also prefer the idea of a more subjective rating system like the one that Zin describes and I agree that numerical scores seem out of place for this type of subjective evaluation.
Symbols: = New Posts since your last visit = No New Posts since last visit = Newest post
Forum Rules: No profanity, illegal acts or personal attacks will be tolerated in these discussion boards. No commercial posting of any nature will be tolerated; only private sales by private individuals, in the "Buy and Sell" forum. No cross posting allowed - do not post your topic to more than one forum, nor repost a topic to the same forum. Who Can Read The Forum? Anyone can read posts in these discussion boards. Who Can Post New Topics? Any registered CoffeeGeek member can post new topics. Who Can Post Replies? Any registered CoffeeGeek member can post replies. Can Photos be posted? Anyone can post photos in their new topics or replies. Who can change or delete posts? Any CoffeeGeek member can edit their own posts. Only moderators can delete posts. Probationary Period: If you are a new signup for CoffeeGeek, you cannot promote, endorse, criticise or otherwise post an unsolicited endorsement for any company, product or service in your first five postings.
= No New Posts since last visit 