I recently read an article in Scientific American Mind (June/July 2007 issue), which had a great article on getting good advice. Although the article was talking primarily about medical advice, the following paragraph struck me as relevant to anyone who is interested in learning about the tea they drink:
We cannot avoid relying on expert opinion. We simply do not have the factual knowledge required to answer all of our questions. Certain fields are so technical, moreover, that only a true expert’s opinion will do…. But our very need for such advice is also why claims of expertise so easily lend themselves to abuse, much to the detriment of the person looking for help. Professionals in the advertising industry are well aware of the persuasive powers of such appeals to authority. Consequently, they spend billions of dollars on advertising and employ ostensibly trustworthy—or not so trustworthy—experts who try to lure us into buying products or services. It’s one thing to be on our guard when watching commercials, however, and quite another to evaluate the credibility of Web sites, self-help books and the like.
The tea industry is not immune to this. Everyone assumes that their local tea shop worker can teach them something about tea. And often this is the case…but not always. When my career path officially switched from “education” to “tea industry” (though I’d already been teaching about tea under the former category), I had pretty much tapped out the tea information of all the places I bought tea from. When it came time for training at my first tea-shop job, they pretty much skipped over everything having to do with the product itself because I’d been feeding them information about tea for some time.
How do you evaluate whether someone’s really an expert, then? Here, modified from its original form to apply better to the tea industry rather than medical field, is a quick and dirty list:
Impartiality: The tea authority should not be biased. If what they’re telling you about would benefit them, it should lead you to question more deeply what they say. For example, a tea shop that tells you how to evaluate a tea that happens to be one of their best-sellers may be giving you biased information leading you to think theirs is the best, rather than a true judge of the quality of that kind of tea.
Degree/Certification: There are two things to look at here. First, is the credential that someone claims relevant? For example, I have a Level-3 certification in black tea from the Specialty Tea Institute. You might well question whether my advice about green tea is good, since I don’t currently have a Level-3 certification in that type. The other thing to look at is the reputation and requirements of the certifying body is. There are lots of fake degrees, diploma mills, and certifications that set their bar very low. Just because someone says they’re certified, doesn’t mean the certificate represents mastery, or that it’s relevant to what they’re telling you about tea.
Experience: A real expert should have extensive experience in whatever area they’re talking about. If you’ve been reading my blog or taking my classes, you’ll have an idea of how many areas it’s possible to have experience in, relative to tea. Just because someone has been selling tea for a long time doesn’t necessarily mean they have experience in how it’s processed, or agriculture, or biochemistry, or history. I know of a tea company that changed ownership—the old owner had extensive experience in the sensory evaluation of teas, whereas the new owner has more extensive experience in business and marketing. I’ve heard from several of their customers that this change in experience can be detected in both the products they carry and the health of the business.
Affiliations: What affiliations does this particular expert have? What reputation does he or she have with those organizations? Ask what memberships they have, and inquire of those organizations if the expert is a member in good standing. If an expert is not a member of an organization you expect they would, ask why. As a non-tea example, at least one of the Seattle neighborhood chambers of commerce is not itself a member of the national chamber of commerce. At first that seems strange, but upon questioning, they will be happy to tell you that they don’t agree with the profits-above-environment sentiment and policies of the national chamber and have chosen not to be national members for that reason. This bit of information helps suggest what biases both the local and national organization might have.
Publications: While there is very little scholarly publication about tea in the U.S. outside of the medical community, publication is still an area where you can evaluate an expert. Although even books these days can be published without the watchful eye of an informed topic-specific editor, important information can be gleaned from anything published—from books, to newspaper or magazine articles, to blog posts. Does the piece include references? Are opinions backed up with evidence, or labeled as opinions, or stated as plain facts? Have you seen conflicting information elsewhere, and if so, which source meets more of these expert-reliability tests? In the case of publications where comments can be appended (many online articles, blogs, etc.), have industry leaders questioned or supported the information? The more “peer reviewed” the expert’s publications have, the better (assuming the peer-reviews aren’t pointing out errors or questionable sources and so forth).
And, as also provided in the Scientific American Mind article, a great place to learn more about the fallacy of argumentum ad verecundiam, or the “appeal to misleading authority”: http://www.fallacyfiles.org/authorit.html
Now, go forth and question those who tell you things about tea! (And yes, that includes me.)
Great post. I think you captured some really important information here.
When I went to library school, I was taught to use multiple reliable sources to back up an answer given to a patron. Some libraries, for example, require their librarians to offer patrons three mutually-confirming sources when answering a reference question.
The trouble as I see it is that, as you note above, finding accurate tea information in the US can be difficult. Yes, I probably can find three sources that “confirm” a random tea “fact”, but there is a good chance (again, in the US) that those three sources may not only be wrong, but they may have used each other as the source of that fact.
This state of affairs irritates me to no end.