It’s tool awards season, with mentions in my inbox, press releases, tool review offers, story pitches, on social media, and even on product pages.
Some of the emails about new and unfamiliar products and brands are often summed up as: “Check out our new award-winning product!” While it makes for flashy marketing, many awards seem to be full of holes.
Advertisement
Always Ask These Questions
Did a product or brand you’re looking at win an award of some kind? Ask the following questions.
Did they pay for consideration?
What was the criteria for winning?
How many competing products were in the running for that award?
Was the product even tested?
I’ve posted about my dislike with “pay-to-play” awards processes before, and the lack of transparency of it all. I haven’t done so in quite some time because I like the people who run one particular tool-related program, and it’s difficult to convey that my feelings are really about “pay to enter” awards processes in general.
Advertisement
My frustration comes from how some marketers are taking the awards they “won” and implying that they bested competing brands. It can confuse readers into thinking that a product is the best of its kind, when it very well might not be. Not all brands pay to participate in product awards processes, leading them to lose by default, and awards processes typically look at products released in a calendar year.
With most awards programs, there’s not much light shed on any of the process between “entry” and “awards notifications.”
Many awards are won without products even being tested. In those cases, awards are “earned” based on what the manufacturer or brand writes down on a piece of paper.
We have even seen awards given to products that never made it to production or store shelves. So how are awards given to products that never existed?
In many industries, marketers and marketing boast about awards, but I can’t find help but find them to be hollow, even meaningless or misleading, because of the lack of context.
One particular “pay to enter” awards program has many awards categories each with have 3 winner tiers, but some categories only show 2 winners. Does that mean that there were only 2 entries and they both automatically earned awards? If there are too many entries in one category, is another category introduced to ensure that more entries receive prizes?
Consumer and commercial product awards aren’t anything new, but I haven’t thought much about them at all until a few years ago. Back then, I took a lot at face value. Now, it frustrates me when shopping for products – let’s say a specific piece of test equipment, for example – and it dawns on me that many enthusiastically-promoted awards are empty – maybe even dubious – because I know that no other brand released anything to that product category the same year.
Most awards programs at best vaguely discuss their judgement processes. One program, due to the nature of the commercial products they give awards to, definitely aren’t testing any of the products. So are they just judging products based on a photo and marketing language write-up?
There is one thing that would make me feel a little better about all this, and that’s if all product awards programs, paid-entry or not, published a list of entrants and entries.
I only know of one paid-entry awards program that displays a list of entries, but when I last checked, they have a list of last year’s winners, and a list of this next year’s entrants, and without any categorical separation.
Maybe an NCAA-style bracket chart would be a good way to visualize things.
Otherwise, how do I, as a consumer, know that an award holds any meaning, and it isn’t awarded just because there was only one entry in the category? Any discussion of how or why a product is award-worthy would be helpful, too.
See Also:
- Rant: “Tool Awards”
- That “Award” Your Favorite Brand is Boasting About? They Might Have Paid for it.
JeffD
Send me $500 cash and I will award you the 2018 Best-est Tool Blog Web in the Whole InterWebs!
DK
Metabo is one of the worst for this recently, they’ve been touting quite a few tools on their facebook with awards that won the “pro tool innovation awards” which I learned is just a paid award. https://protoolinnovationawards.com
Im disappointed in any company that does this, and it makes me less interested to buy their tools or have confidence in the quality of tool they create if they pay for fake marketing awards
fred
This is akin to trade magazines (there used to be a gazillion of them in hard copy before the Internet) that had placed pieces – with little or no editorial content. They were usually written by the manufacturers or their agents/ vendors. Quite often they were made to look like news stories – but to say that they were “news” in the old journalistic sense – would be disingenuous if not an outright lie. Thinly or even unsupported claims abounded and while “fake news” is a now popular term – IMO it could well have been applied back then. The fact that many of these trade magazines came into your mailroom – at no cost to you – since they were supported by advertisers – was a clue about unbiased content. To be fair – sometimes you learned something interesting by thumbing through them or mailing off their “bingo cards” for advertiser literature.
MichaelHammer
The same can be said of “scientific” studies and has a lot to do with the reason so many people don’t trust science these days. I always ask who did the research, how long was the study, and most importantly, who funded the study. There is a direct correlation with awards and why few of them mean anything. What’s more is even when the whole process is on the up and up, many awards are so subjective as to be worthless. When it comes to tools the most useful awards are based on value, quality of construction or innovation.
Stuart
I don’t know what you’re talking about.
Real scientific studies are peer-reviewed, and as far as I am aware, grant money or other funding is usually acknowledged at the end.
A lot of scientific studies are misinterpreted by mass media, and then other media channels take things further, like a game of telephone.
fred
BTW – when I was talking about trade magazines I did not wish to imply that scientific journals that are refereed by peer review were in the same category. Can a “scientific” study be biased or flawed by poor methodology, bad sampling etc.? Sure but then the results are not likely to be repeatable in a peer review. This notion that all science is fake – IMO – is being promoted by folks that may have their own agenda to do so. A healthy dose of skepticism is always good – as is the curiosity that underlies the scientific method. The critical nature of peer review – that rigorously analyzes the data to see if it supports the hypothesis – should not be confused with the notion that drawing conclusions from statistically significant data is never possible. We live in an egalitarian society – where our opinions should all be valued – but I’m reminded that my opinions about lets say linguistics are not worth anything because I never studied the subject. I would, therefore, prefer to defer to a linguistics expert should I have such a need.
AdamM
There’s a reason ‘replication crisis’ is a household term now, why double-blind studies are the new norm (although still rare) and you’ll often see comments about small and/or non-diverse sample size
(just like with the awards here, the way you choose your test subjects can significantly skew the study results).
For further reading, I’d suggest:
U.S. Science Officials Take Aim at Shoddy Studies
How Many Scientists Fabricate and Falsify Research?
Stanford Researchers Uncover Patterns in how Scientists Lie About Their Data
Stuart
Oh, there are bad eggs and bogus studies too.
But for the most part, the scientific community polices itself quite well.
Dale
I don’t know about that, check out this video by Veritasium https://www.youtube.com/watch?v=42QuXLucH3Q
TonyT
The area probably makes a difference e.g. hard sciences versus health/diet, social “sciences” etc
Framer joe
Really? Like in the 70’s when global cooling ,a age of frozen tundra was widely embraced in the scientific community.
Or the latest Global Warming ,which has been shown that there has been no change in 19 yrs , so of course they change the name to…. Climate Change because it didn’t fit their model….now every single tragic weather event is climate change.
In the end it’s about …money…,getting funded for some cause..
Like local channels over exaggerating storms , hurricanes,for days ,etc..why? For ratings ,which translates into money…
A W
As a consumer, I really don’t care about the best product in narrow categories released within the last 12 months. I care about the best product available right now.
That’s why tool shootouts are far more valuable to me as a reader than tool awards.
fred
I agree in part – that is I’d like to know what’s the best tool for me right now.
I used to look at tool shootouts in amateur woodworking magazines. Sometime they were informative – but other times I sort of laughed – having had experience with one or more of the tested tools. It all depends on whether the tester(s) hit the right “buttons” that are important to you and the way that you work and/or plan to use the tested tool. Reading between the lines of the tests is important – but sometimes challenging. Even when tests are done (like on automobiles) by Consumers Union – what they test and report on may not be what’s important to you.
Mattd
I second this. The only reviews I really put any stock in are multi tool head to heads and people I know were not compensated and paid for the tools them selves. Most other sites I check out just to see what new stuff is coming out, or whar good deals.
Framer joe
Tool shootouts by most YT are just as bad.
Guys that know nothing about tools or how-to use them having at it…real tool reviews and workshop addict are the only “real” reviews
Stuart
Lol, that’s a good one. You need to add a (sarcasm) tag so that others know you’re joking.
Stuart
These days, everyone wants to be an influencer, and some will do anything to get there.
Corey
I was just explaining this crap to the Mrs regarding cars lol Ever notice that just about every car/truck add boasts “best in class” awards for random crap? Not so impressive when just about all those vehicles are somehow identified in their own particular class.
Benjamen
A while ago, I was listening to a guy who used to do phone surveys. .
Just to make it easy, let’s pick a Honda CRV (that’s my wife’s car) I’m not saying Honda does or doesn’t do this, but it’s just a name everybody recognizes.
Now Honda hires a polling company to do a survey. They create the survey with 20 questions like: which car is the safest in it’s class? which car is the roomiest in it’s class? etc and list out the cars in that class and ask people to choose.
Statistically people are going to pick the Honda CRV for a few of those questions, so then Honda would take those questions where they were choosen as “best in class” and tout them in their ads.
Because these are real surveys, they can’t get called out on false advertising, even though they are meaningless.
JoeM
That anyone even bothers with “Awards” anymore is beyond me. Even if something wins an award of some sort, it says nothing about its usability to the individual, or its availability to a key target group.
Let’s face it… Narrow the field all you want, if no one can buy it, and it doesn’t do the job people paid for, then it’s not a tool at all, let alone an award-winning tool. Awards genuinely mean squat in the world.
Apollo 13, starring Tom Hanks, won awards of all sorts when it was released. Is it a movie worth watching? Honestly, I couldn’t tell you. I’ve hit play on that movie via VHS, DVD, BluRay, Torrent Download, AND Streaming services since it was released, and I STILL have never made it past a half hour in without falling asleep. I’ve tried watching with friends, and family, and I’ve never physically owned the thing, and it never fails. I go night-night when that thing plays. So, DESPITE the awards, it’s not a good movie FOR ME. And so, I shouldn’t waste any more time on it. Same goes for Tools, Cars, and all the rest. The Awards mean NOTHING. What matters is the CONSUMER buying the item. Most importantly, is it right FOR THAT CONSUMER?
chancla
I used to write reviews for a computer hardware site. MFRs from Asus to AMD to the little guys would mail me stuff to overclock, stress test, etc. This was a little less than 10 years ago. It wasn’t the biggest site, but we were pretty decent. My AMD rep said I was probably the best review site overclocker (not pro overclocker)[cred for those who don’t care]. We were trying to pick up Intel and other MFRs to send us review samples.
I’d write what I thought were honest reviews. My negative reviews were always changed around by the editor (and part owner) to be neutral at worst [he’d change my summaries and opinions, but not the data], and I even saw our “stamp of approval” or whatever on retail packaging, for stuff I had reviewed, that sucked! It drove me nuts, but there was nothing I could do. My name’s been scrubbed from the site (they updated the site and the old author names were replaced with the site name, last I looked), but from my experience, a site will do anything to make the paying and giving manufacturer good cred, especially if it gives more exposure. After all, who wants to send review samples to someone who might be negative? Despite having “unconditional review sample” written on everything, we somehow wound up with entirely almost all positive reviews.
I’d imagine the “best drill” categories are either from sites trying to get more manufacturers on or title like that because it matches google searches and they can get affiliate links and ads in.
After that experience, I have a very jaded view of reviews and a lot of product based sites in general. Outside Milwaukee, which I know to be a gravy train of freebies, this is the least Stankiest of review sites for tools, and I don’t mean to call the proprietors purveyors of the unethical. I’ve worked with Milwaukee, and those who have worked at Milwaukee, and I know how much they give away.
all this rambling is to say – toolguyd is a little jaded with respect to the Brookfield, WI tool developer, but is seriously the most well rounded, least biased, and seriously lacks-an-obnoxious-style-of-writing review site. I majored in english before getting talked out of it, dabbled in linguistics, and I’m anal about language like that english accent dude on the cooking show is about his food.
hope this gives a little insight to those not in the know on review sites, and give some cred to the toolguyd crew – love what you guys do. and stu- you from WI?
Stuart
Thank you for the kind words, I appreciate it!
Regarding “best drills” types of posts, that’s something I’ve done in the past, and something I might do again in the future. Google rankings aren’t really a factor, since Google search results are full of spammy Amazon affiliate-focused sites. For me, it was to answer a popular question and emails that start with “I’m looking to buy a new cordless drill, which ones should I look at?” This was intended to be a “tool guide” after all.
While surprised about what you said about computer hardware reviews, I guess it shouldn’t be too much of a shock. There have been plenty of tool reviews and recommendations that I’ve read or watched that invoked “are you kidding me?!” reactions.
JoeM
What chancla says sends chills down my spine. I was raised with linguistics in the house, was groomed to be a writer, doctor, or lawyer by my Dad’s Jewish side of the family, and instead of a paper route or grocery store job, I got a job, at 14, as a full time Computer Tech.
I’m not positive, but I’m pretty sure at least SOME of his work filtered up my way at some point or another. It may have been 1998-2001 when I was in the field, but it totally rings true.
If you think the pay-to-play system is bad for TOOLS, just multiply it by a thousand, and you have the Computer Component market of the Millennium Barrier Bubble. Boxes of components stamped with review site awards, quotes, star ratings… it was a crazy time to be a builder back then. Asus, ABit, Antec, Vantec, Intel, ATI (Now owned by AMD.), AOpen, Panasonic, Plextor, IOmega… A little store would leave or shut down around 7, you’d do some final overnight stuff until 9 or 10, then go home and sleep. By 8:30 AM, you go to prepare to open and the Fax Machine, your E-Mail, and a hundred messages over the Phone or your other IM systems, would be waiting for you. Out of every 2000 messages you’d get a day, soliciting you buy X from company Y instead of Z, there would be all of 4 or 5 TOTAL that would be an important update on the current prices of the things you already buy or stock, and MAYBE one that was from a customer wanting to talk about a purchase.
All these companies were RABID for the services of people like chancla here. Their fliers and E-Mails were FLOODED with quotes from them, saying “This product does A and B better than that other company, just ask (Insert Reviewer Company) if you don’t believe us, try for yourself! (Limited Time Offer)”
Sends SHIVERS down my spine, even now!
Chancla
I didnt mean for that to be applied to you. I dont think I’ve ever seen anything clickbait here.
And re reading what I wrote, I really need to proof read!!
Stuart
Oh, I know. But, I do have some “Best…” posts linked to in the top menu, and it’s something I’d like to do again (hopefully with lists of all the products tested if I can manage it).
Sometimes I answer comments for one person’s benefit, other times it’s for everyone else who’s reading the discussion.
csroc
Reminds me of a lot of product certifications. Granted, some of them aren’t as bad. The companies do have to pay to have their product certified but as a consumer you can at least know that means it meant a certain level of performance. Unfortunately not all certifications mean much and even when it is a meaningful certification, just because a product is certified doesn’t mean it’s better than a product that didn’t even pay to go through the certification process.
Evadman
I don’t care one bit about awards. I read reviews from actual users, check testing blogs and review sites like this one. The only challenge is filtering out the paid for reviews.
I will not buy something nowadays without research, which means I almost never buy anything as soon as it comes out.
Doresoom
I’ve also seen “pay-to-win” awards with super narrow categories that may only have had one release within the year that fits.
I guess it’s a lucrative income for whichever organization is giving the awards, but personally think it compromises credibility. Maybe not enough people pay attention to how the candidates are selected.
Stuart
Some awards processes seem so unbelievably lucrative that it just boggles my mind.
But with respect to the tool world, my frustration comes from the brands and marketers who boast about the awards, and how they use them in advertising.
“Why did your product win an award? How many competing products were there, IF there any others? Would there have been any reason or scenario where the product would not have been given an award?” Marketer need to be able to answer this about any “earned” awards they are promoting, especially in advertising materials. In my experience, they cannot.
An industry friend was at a trade show a few years ago, talking to a product manager, engineer, or presenter who was enthusiastically telling them about their new tool having won an award. The rep didn’t know that the awards program was “pay to enter” and might not have looked at any other products in the same sub-category. I don’t remember many other details, but I remember being told that it took some wind out of the rep’s sails because they couldn’t answer any questions about how or why the award was given to them.
In elementary school, I got into trouble for not showing my work on a math test. In college I had to resubmit a history paper and was docked a grade because my professor wanted to see footnotes for every single piece of information that was referenced. While I don’t always show my work now, I’m always prepared to answer questions about how I get from point A to B. I have found that product brands, at least ALL of those that I’ve asked, are unable to explain to me – or anyone – the context behind the awards they won and are advertising and telling customers about.
How can products that aren’t tested earn awards? If products are tested, was this done before production runs start shipping? Were the products cherry-picked or modified in any way?
JoeM
Don’t take this wrong, Stuart, but this is EXACTLY why we trust ToolGuyd over other sites. You don’t tell us who won, you tell us what the tools DO for us.
Stuart
Thanks, I appreciate it!
That has always been my goal – information and insights.
It’s not my aim to tell you what a tool will do for you, but to help you determine that for yourself.
fred
Stuart:
Thanks for what you do to keep us informed and entertained. Your humble and open-process attitude goes a long way to maintaining the credibility of your blog.
Regarding how to pick the best tool, that can be challenging even if one knows lots about its proposed use. I think that if I had gathered all our plumbers together at one of our workout centers and asked for purchase recommendations for a new batch of say: drum augers – I probably would have gotten lots of different opinions to consider.
Digging into the reasons for such recommendations might have been useful – but more challenging. We did something akin to that from time to time when faced with some larger new purchasing decisions. Often we’d then buy a few new tools to pass around and try out – before buying in bulk.
Interestingly, I think that if I had asked the same drum-auger question of the carpenters in the remodeling business – I would have gotten opinions too – even though they might have had little or no experience with clearing drain or sewer lines.
Jim Felt
YES!
Kent
“Award Winning” is something I ignore.
Ages ago, I became cynical enough to ignore most advertising claims – for example, Sears has done nothing but tell lies about how many HP their tools have.
So, “award winning”, “class leading”, “best in class”, etc all get ignored.
Framer joe
Rep..it’s all about reputation. In work,in life. You walk a fine line between receiving free tools and recommending them. I think you do an excellent job of keeping bias out.
It’s difficult because your sponsored so where is your loyalty, to your readers or your income ?
I think you balance everything well for the most part. I trust your articles, although sometimes I may criticize them.
My company depends on tool innovations, faster,better,safer….that’s why I read. It’s about providing more jobs for families.
With so many news sources about tools or nightly news stories, being skewed,to far left or to far right,or this color or that color is the best ,with thier own agenda not reporting actual facts.Its refreshing to have ToolGuyd staying right in the center of it all..
I appreciate your blog and all your hard work you put into it.I always wish you and your family the best the life has to offer.
RC WARD
What a shock huh? That tool companies are corrupt and lack morals in this world filled with a lack of morals and corruption .
ToolOfTheTrade
There could never be an honest award recipient of a pay to play competition. I think it’s obvious that whoever spends the most or offers the most incentives is the winner. Reviews are even worse. How can a brand new product that nobody knows about or has yet to be purchased have 5 star ratings on Amazon(well-known for getting paid for & buying their reviews) eBay, etc? It’s pretty messed up that product reputation is bought instead of earned.
Stuart
There’s a chain effect.
A few years ago, someone complained to me that another tool review site bought 5,000 (maybe 10,000?) “likes” and “followers” on Facebook and Twitter. The site in question saw massive increases in “likes” overnight, and with not any more “engagement.” Other site seem to have done the same later on, presumably because “everyone else was doing it” and because they didn’t want to seem lesser in comparison in advertisers’ eyes.
I’ve seen brands and sketchy retailers do this as well, to make them seem more legitimate.
Someone I know shops on Amazon, using the number of reviews as a deciding factor. I’m sure there are others like them.
Corey
[redacted]
And then there’s crap like this that reads like an assault to your intelligence. Grammar and sentence structure aside, it’s painfully clear that the author (if it’s even a person) has kess than zero idea about the subject matter. Impact bits made of hss for drilling? Cons being incompatible with non 1/4″ chucks? In just the first paragraph they switch to referencing an actual impact being good for use for years. It’s this clickbait revenue world we live in that’s demeaning and working against consumers. Rant over. Just found that gem searching “strong impact bits 2018.”
Stuart
That’s an Amazon affiliate “niche” site. They’re quickly built using templates, and many appear to scrape straight from Amazon, or are bot-written. They use tricks to cram keywords and titles into valueless pages, and Google can’t seem to learn how to differentiate between that kind of garbage and actual content.
Corey
That actually relieves me to learn it’s at least mass generated, and the product of someone’s actual effort or intent.
Stuart
That’s my understanding at least. I can’t imagine that any of it is actually seriously written, despite what any “about” pages might say.
TonyT
My feeling is that Google search has been getting progressively worse over the last few years
Adam
I hate these things. They’re everywhere, and making it increasingly hard to find actual reviews of tools.
OhioHead
Great post Stuart, I typical “shop & buy” brands know for historical innovation/created the segment they are known for……this is why I own DW/MKE/Bosch/Skil wormdrive/Metabo/P/C/B & D Industrial.
This article made me think of the potential pay to play toolblogs & “best of categories awards” and I do not purchase tools on a daily basis, but frequent & read a few blogs to help “educate” me on my historical knowledge and what is actually taking place in the real world!
Keep up the good work!
Scott K
Most of these awards are meaningless to me and in now way influence my purchasing habits. I know that some of the “award” companies charge for entry/consideration and then more for the rights to use said awards in ad campaigns.
Nathan
Very pointed topic. It’s why I don’t trust anything printed by consumer reports or JD Power.
amongst many other “systems”. Back in the 90’s when I found out Consumer reports either got paid or sent items to review I knew then I could stop reading them.
One of the things I like about this site and what lead me here. Hands on reviews of items – comparison testing with real activites – and up front honestly about how items are represented.
Redcastle
Several years ago one of the junior administrators in our office saw a category in a fairly prestigious architectural magazine of their annual awards which he thought we might just quality for. He called, asked how the nomination process worked, was sent a form, he completed it and a week later was advised that we had won (turned out it was a one horse race). The cynical part of me says that the price of a table at the awards dinner might have been part of it.
On a very serious note comments on here relating to forbidding negative reviews remind me that at least two of the major financial crises since 1987 can be substantially attributed to misrepresentation and manipulation of research reporting so as not to upset a client.
The confirmation that the dumbing down and hyperbole in what I have seen described as main stream media is driven by the (insert appropriate name) of this drivel is because they are paid by the click does leave me wondering where you can find outside of personal experience advice which is rational.
Rodger
The Pro Tool Innovation awards are the biggest, bogus thing I have ever seen. They gave an award to a RYOBI backpack blower for the most powerful? I have the same unit and my biggest complaint is how under-powered it is. My 10 year old plug in Toro has more power.
It sucks when consumers are the ones that lose out. These Pro Tool Guys probably got tons of money to promote these awards and mislead the consumer. Shame on them.