It has been nearly 2 years since I ranted about how I felt about disingenuous tool “awards.” And you know what? It still really bugs me off that some magazines and online publications require payment before they will consider a product for an award.
Some – but far from all magazines and websites that do this – have official “rules and regulations” that describe how awards selection is basically run like a contest, but where brands and companies that want to be considered must pay an entry fee. But they don’t elaborate upon the “odds of winning,” and unsurprisingly they don’t mention the entry fee or payment requirements when discussing award “winners.”
Most that I have seen do not openly disclose the fact that an entry fee is required.
Advertisement
Why don’t these magazines and websites mention that there is a mandatory entry fee when discussing award winners? I cannot think of a good answer to this question. Why bury or flat out hide the existence of entry fees, indeed?
I cannot shake the feeling that magazines and websites that charge entry fees simply give “awards” or other recognition to every single paying entrant. Maybe I’m wrong, but it’s hard to imagine otherwise.
One magazine that requires entry fees for their awards process has an initial entry round, after which an award winner is named. When there are award finalists and a final award winner, even if all paying entrants are named as finalists, maybe there is at least the resemblance of a selection process, and hopefully one that is based on merit.
Let’s say that there’s a website or magazine accepting entries for their “best review site” awards. ToolGuyd is the only tool-related site that enters, and is given title of Best Tool Review Site. Let’s say 2 tool related sites enter. Then, ToolGuyd might win an award “Best Wide-Spectrum Tool Review Site,” and the other site might win a slightly different award. All the site has to do is create slightly different awards category for every entrant.
Would companies keep paying for awards entry every year if they weren’t guaranteed a commensurate level of exposure?
So why bring this up again?
Advertisement
I regularly receive press releases from companies that boast about the various “awards” they won. Oh really? Did your company pay a fee for consideration, and was it the only product in that particular category? Is your product even out yet?
In one instance, a brand showed off a product prototype at a show, but it wasn’t fully functional. A part crucial to the functionality of the device was not yet available, and so there was a substitute not at all representative of what would be in the final product. The product looked like it came off the production line, until you turned it on. So how the heck could this product have won top accolades as anything but a paperweight?
Another product was only in the “proof of concept” stage in the development process, months away from production, but it too won an “award”.
I can understand why magazines and websites might have a paid entry awards program. I don’t agree with it, but I can understand it.
It was an email exchange with a brand representative earlier today that added to my disappointment and brought all this back to the surface.
Brand: Yay, our products are awesome, they won some awards, you should post about it and tell the world.
ToolGuyd: Thanks, but I’m not interested in hearing about paid entry awards.
Brand: We pay to enter, no guarantee we will win.
ToolGuyd: Let me guess, every “entry” you paid for resulted in an award, right?
Brand: You got that right.
They then went on to say things about brand exposure and marketing and what-not which only further hurt my impression of the brand they represent.
Some brands even have an “awards” page, where they list badges and various awards that they “won.” Some of those awards seem to be genuine, while others look to be completely bogus. How are readers supposed to tell the difference?
I don’t trust product awards at all anymore. Heck, I can’t even trust a lot of tool or product reviews anymore either, as too many seem to come straight from a brand’s PR or marketing contact. There are some “reviewers” that won’t take a look at a product unless it arrives with a check.
When you see a brand boasting about an award their product won, or they put an award badge or label on their website, or worse – they put it on product packaging, ask yourself:
Did they pay for consideration?
What was the criteria for winning?
How many products were in the running for that award?
And if you’re not sure of the answer, ask them.
In a perfect world, you should have all the information you need to answer all 3 of these questions. Even 2 out of 3 answers might suffice. But as it turns out, transparency and paid entry awards “contests” don’t go hand in hand.
Some magazines and websites even have companies sponsoring their annual awards, and no I’m not kidding. So they have advertisers paying for them to accept mandatory entry fees as part of an awards selection process. I’ve mainly seen these in more specialty product areas, but it’s nonetheless ridiculous.
It’s not what is being done that bugs me, but that it’s all being done behind curtains. Sometimes I wish that my naivety remained intact.
Oh, and if you clicked through from the front page, that’s an unrelated image from a post sharing a satirical video about a fake firewood-making business.
KL
In what I consider a similar vein, some tool reviewers only post positive reviews. If they did not like a product, or it failed during testing, we never hear of it. The reasoning is similar – if a reviewer posts a negative (but honest) review, Company X will simply no longer send them tools to test. In a sense, the reviewer gets “paid” in free product with the (wink, wink) understanding that a positive review will be forthcoming, which is kinda like advertising, eh?
mizzourob
and then they stock pile all of them for a super duper end of the year Christmas tool bag giveaway
Dan
What?? So let me get this straight. They keep all the new tools until the end of the year and give them away to their readers? What aholes! That is just so insane, how dare they screw the reader like that. Let me guess they even pay the shipping?? The last thing I want is to go to a website and get a free tool,especially for just subscribing. This is just egregious and will not stand. What idiots, they should keep the tools for themselves, the nerve of them to give them to readers. Mizzourob hopefully you don’t subscribe those types of sites. You better unsubscribe before you get sent a free tool.
mizzourob
I wonder if this is Dan from TIA fame… love the website and the show especially that you go out of the way to include panosonic and metabo tools that no one else will include. My only thought is that having more true pros use the tools to get their real feedback just like what toolselect used to do.
Stuart
In addition to the time, money, and effort it costs to build and manage a team of professional users that provide long-term feedback, there’s also consistency. One electrician might love their Hole Hawg, another might find such a tool unfathomable (true story). Because of how users in the same trade can have different needs and preferences, maintaining a team of testers usually loses a lot of its value.
I sometimes send tools out for secondary or professional opinions. A lot of those times, I won’t hear anything back unless I nag for feedback. That can be a good thing, but also means wasted time, effort, and expense on my part.
With ToolSelect, they had a huge budget and the staff to do what they did. But consistency was also lacking. Did any particular DIYer or pro review more than 1 tool each?
Stuart
In the nearly 7 years since I started ToolGuyd, I don’t remember ever haven’t been in a single *wink, wink, nudge, nudge* situation.
There have been some instances every now and then where a PR rep, marketing exec, or brand manager tried to influence how I do things or what is said about a brand, product, or product line, but those incidents have been very infrequent.
Some companies do treat tool and product samples as compensation, but this too happens very infrequently.
I don’t know how rampant paid reviews are in magazines, but when new brands reach out they sometimes ask about fees and the “submission process”. I do know that on some sites – and YouTube channels – some reviewers won’t talk about a tool unless there’s payment in-hand. A “pay me first” advocate once asked me why I continue to review tools for free. The answer’s simple – because that ensures the review is done with readers’ interests at heart.
Brian
I’ve done several reviews for hockey equipment from big and small companies, they’ve always asked me to give them an honest assessment. I’ve always warned them that I will be honest and I may be harsh, they’ve pretty much all responded with “That’s why we’re asking for your review.” I believe that most companies truly do believe they have the best product or at least best in category.
I think this “award” thing is simply a proven advertising method, it’s been proven in the auto industry so it’s trickled into other categories. I wouldn’t think tool reviewers and award programs fit in the same category…that being said, you should always question your source for a review. There are a lot of people that will just say nice things about products, those are the guys to avoid. Then there are people that assess the product giving no real opinion but listing facts, those are more trust worthy. Lastly are the ones that can distinguish personal preference from quality, I don’t like using Klein screwdrivers but they are very good screwdrivers and you can see why others swear by them…in my opinion, those are the reviewers to seek because they try to present the product without a personal bias. These are the guys that will list a nice set of pros and cons.
mizzourob
I was just thinking about then when I read a different website Las week that listed 2015 award winners in a bizarre array of categories that defied logic, e.g. Winner – Cordless Angle Grinders: 18V/20V Max, and Winner – Cordless Angle Grinder: Braking, 18V/20V Max. They also had a Winner – Cordless Hammer Drill: 12V but did not have a category for 18/20v cordless hammer drills. There was also an award winning cordless pluming tool that has not been released yet either.
Diplomatic Immunity
Lol protoolreviews.
RocketTech
I think much of this is the marketing director justifying their budget and salary. unless you use only objective tests or blind tests on subjective, you are conducting an opinionated review.
Even Consumer Reports if falling down on this, after some of their review results have been called into question by empirical data. They are clearly trying to pick winners to influence the marketplace- maybe they have altruistic reasons, maybe they have fat checks.
Stuart
Every review is going to be somewhat opinionated. As objective as I try to be, some things simply come down to preference – it’s unavoidable. The important thing is an attempt to remain objective, fair, and consistent.
Eric
Very enlighting and why I appreciate the open and honest way you run this site!
Gary F
That’s one reason why I love your site. I never knew people paid for the awards. Personally I think that’s pretty shady. If they told us upfront, I wouldn’t mind. If you have to paid for an award or a review, what else is there that they aren’t telling us, such as if a tool is really worth the price. Who is really coming first, the consumer or the website or magazine. I understand you have to pay bills, but obviously you don’t care about the readers. If a manufacturer has to pay for an award, how good can their tools really be?
So I read about a tool that won an award, I go purchase the tool because according to them, it’s worthy of an award. Only to find out it really isn’t the best tool available. So I spent my hard earn money on a tool that is probably not the best tool. OK got it.
Sean
Speaking of getting “paid” for a product review, I wonder if these new customer review programs make any of you as sick as they’re making me (Home Depot has done it for awhile and now Amazon has the “Vine” program.)
You send me a $200 product for free and you’re damn right I’m probably going to give it a star or two more than if I’d been questioning that recent hole in my pocket. I’m not saying you must pay for a product to review it objectively or, at least fairly, but it is certainly an influential factor in many direct customers reviews. Furthermore, these reviewers get labeled “Top X Contributer” and have incentives to get more and more free crap sent to them, and how do you get free crap sent to you? Make sure to give the last product or last brand a 5-star review.
It’s basically the same incentive program they try to push on these “put a paycheck in the box with the tool” reviewers that you’re discussing. The difference is these companies now have an army of hardworking internet-surfers who love free stuff writing the best damn objective 5 out of 5 star opinions they can.
Diplomatic Immunity
That’s why I usually look at the negative, detailed reviews first. Sometimes you can get a better gauge of a products quality from it. Then I look at positive, detailed reviews. I’ll avoid the positive rah rah shish boom bah reviews and the one sentence negative reviews. I want objective reviews with pros AND cons in detail.
Pete
Partially why i dont like ‘Real Tool Reviews’ youtube channel- He never says his opinion or even reviews any tools he simply give a long detailed explanation on how it works then says how awesome everything is. I as a consumer want to know your “Review” not a preview!
But yes i do hate that some companies pay for awards, i think is much more prevenlent in the car/truck world which is why i dont read any articles from any magazines on anything anymore. While they
BikerDad
I like the RTR channel precisely because he gives a long (admittedly sometimes too long), detailed explanation, looks at features and points out shortcomings. He doesn’t read off the press release as I’ve seen some others do.
One problem with the great majority of tool reviews is they simply don’t have enough time with the tool. This is especially true with power tools. How well do the batteries take and hold a charge after 6 months of use? Is the cord to tool connection prone to cracking and breaking? Do adjustments easily jam up from dust and the like? Do the hinges on the case fail after a year? How easy is it to actually put the damn tool back into it’s case when you’re done working with it? How difficult it is to actually dial the tool in, and does it keep adjustments/alignments well? If there is something that’s just a little off, does the tool break in to you, you to the tool, or do the twain never meet over time? (This is primarily a concern of ergonomics and fit.) Unfortunately, most of the “professional” tool reviewers don’t seem to have the real opportunity to use the tools enough. Take, as an example, the new DeWalt 20Max XR Drywall Screwdriver. The best review is going to come from some guys who’ve been hanging drywall for 6 months. Does anybody know of a reviewer who’s also a professional drywaller? The experience of such a person working with the tool is going to be much deeper than the guy who rocked his garage. Yet, generally, our reviews of actual use of the tool in question tend to come from the “rocked his garage” school.
I’m not terribly concerned about the “paying for awards” thing, mostly because “professional” tool reviewers are only one source of info for me. It would be better if everything was up front and disclosed, but it’s not something I’m going to break a sweat over.
Stuart
I used to test tools long-term before formally reviewing them. What I found was that 1) I ended up with a pile of tools that I wanted to start using, but couldn’t because I had to put more time in with a particular tool, and 2) It doesn’t take 6 months of constant use to know what I need to about most tools.
The beauty of an online review format is that readers and users can chime in when something goes wrong. I’ve tested Ryobi’s cordless nailers quite a bit (one of these days I’ll finish the full review), but hadn’t experienced a jam. Visitors and readers commented about their jams, and then one reader provided a solution that seems to work for everyone.
Your wants are reasonable, but require far deeper pockets that most independent reviewers and magazines have. And even, then there’s consistency.
Just yesterday you were saying that cordless screwdrivers were useless and that brands should kill them off. If I have a professional user who happens to share your view, and ask them to review a new cordless screwdriver, how will their reviewing that driver help anyone? It won’t, so I would ideally have to find someone who uses a tool like that, and ask them to do the review. Or, there’s me, who uses a tool just like that, and who understands what it’s for and who might want to use it.
Professional user reviews are great sources of information. I’m not looking to replace that. ToolGuyd is a starting point, or ending point, for your tool purchasing decisions research.
Chase
The real tool review guy is the biggest fake on the net. He thinks he is God. I commented one time about a tool on his channel and he deleted my comment. I guess I challenged his ego. He got kicked off the garage gazette and spams pretty much every forum i visit. He is all about the manufacturers and getting money from them.
He also bad mouths you a lot Stuart, claims you live in an apartment and never test the tools. Don’t you guys wonder how some of his videos get a massive amount of traffic right away? He pays for it. His website is garbage, his wife basically posts press releases o yea, real informative! If you can’t see through that guy I have a bridge to sell you cheap in Brooklyn. He is from my area and i always see his review samples for sale in Craigslist! you can tell from the pics it’s his shop background he asks the same price as new, idiot. I bet he’s making a ton of money from that racket.
Stuart
I hear a lot of what he says about me and ToolGuyd. Selling tool samples? That’s a HUGE no-no. There’s a lot more I have heard and know about the guy, but he’s not worth the time and energy to talk about.
One thing is true – I do live in an apartment (not for much longer, as my son is now walking and starting to run and needs more space). So what? Sometimes I test tools, most of the time I use tools. I’ve always been transparent about the tools I’m proficient and experienced with, and those I’m using for the first time.
A while back some readers were disappointed when I told them I usually don’t do my own auto repair or anything more than basic maintenance. Having a garage or basement to work out of wouldn’t change that, at least not for a long time, as I have too many other projects I’d rather give priority to.
Dan
You are not alone Stuart, we used to talk to him and helped promote him. Then we find out from multiple people/brands he is bad mouthing us also. He is all about himself, people are starting to find out what he is all about. He is burning bridges faster then he gets traffic. If you ever meet him you will see how thinks he is above everyone. Somehow media events are no more fun with him around.
James
The fact that so many of the brands you prefer (Wera, Facom, etc) are brands that I otherwise would not have heard of is a big part of the reason I use this site. I’m 99% sure that the 18-20V DeWalt/Makita/Ridgid/Porter Cable/other big box store brand’s comparable tools are all comparable. I find value in seeing tools I’ll have to seek out but may well appreciate longer term.
Derek
I enjoy this tool site the most for many of the same reasons written above. I feel like I am getting your honest opinion. I do wish that you had more empirical information like I put x amount of pressure on socket a before it broke vs socket b, but still I think you do a great job!
Stuart
It’s a good idea, and one I’ll keep in mind, thanks! It’s not really possible right now, though.
While I would love to collect and provide empirical data and information like that, the time and effort requirement would be enormous. I would need multiple samples, multiple tests, and setting up tests for proper control and consistency conditions can be difficult.
I couldn’t just grab a bench vise, some hex strock, a breaker bar, and sockets to test for socket failure.
I have seem amateur testing on some other sites and YouTube, but they often lack control and repeatability. The conclusions are also usually filled with holes.
Brand A might fail a few ft-lbs sooner than Brand B, but does that really matter if both failure points are well above practical user conditions? Is Brand A tougher and Brand B stronger? Is Brand B better because it requires more force to fail? Will Brand A always fail sooner than Brand B, or can different test conditions change the outcome?
Hugh
I use to purchase an end or beginning of the year tool review magazine that always seemed to have a certain brand on top. The other thing that was annoying was there wasn’t always representation from all the well known brands sold in the US . It’s not surprising that they pay for their awards.
Although I think even accurate and precise reviews won’t sway people to purchase tools based off of color of tool or battery platform.
MikeIt
When you say they have to pay to enter, how much are we talking about? I can understand charging a small fee to cover time for a tech to test out the equipment, maybe two or three hours worth. Maybe they factor the cost of the tool as a fee too? I can also understand a fee to cover some contest related expenses, maybe a tropy or plaque, or a photographer and web or printing costs. So there will be some costs that need to be paid upfront. However, I would like to know how much we are talking about. Is it $500, $5000, $50,000? Give us an idea. I think our outrage may be proportional to the fee, lol.
Just Mark
I was approached several times by industry rags offering to write feature articles on my business/service/product if i would take out a full page add in their magazine or buy add space on their web site. Depending on your business, this can be a good way to spend your marketing dollars.
The problem for me is that it has made me suspect of all articles/reviews I read with the exception of Consumer Reports. And of course your fine site, Stuart. If I want to know how good/bad drill model x or toaster model y is, I am likely to go to forums that have a bunch of people that are using the product talking about it and then still wonder how many people in the forum are plants.
Farid
A friend of mine worked at one of those infamous “Consumer Choice Awards” that you see local businesses boast about, and It is all pay for play. Same thing with the Builders associations “Parade of Homes” awards. The builders themselves rotate as leaders of the group and they rotate the awards among themselves.
Kudos to Stuart for holding the line .
JeffD
It is my opinion that all reviews should be unbiased and truthful. Had I not been forthright ToolGuyd wouldn’t have published a couple of my guest reviews many years ago.
As a member of Amazon’s Vine program I still hold these virtues true. Sending me a free product doesn’t sway my review or add stars.
Robert
http://homefixated.com/craftsman-dog-bone-wrench-review/#more-28115
I haven’t been back to this site since this review. Not helpful, just a love fest for a gimmick tool. Having a hard time believing it was written accurately.
mizzourob
What I think is a good test for a site’s value is the communist around it. Several of what I suspect are the pay-for-play award sites never have any comments on their articles. The comments on ToolGuyd are just as useful as the articles themselves.
mizzourob
*comments (not communist)…. silly auto correct
Jerry
Spot on article. One reason I like ToolGuyd is that the reviews are honest. I’ve quit even bothering to read sites (or magazines) that will say things because of kickbacks (advertising dollars). There was a time, when I could tell you what brand was going to be top rated, by what blog or mag did the test. I can even recall one test, where the product being tested was in the bottom 3 in every category they measured, yet they gave it the top honors because the company that made it had a history of making good products. HUH??? Reading between the lines, I think they meant that the company that made that product had several full page ads in their rag.
Keep up the good work, Stuart! I like your honest reviews/comments, and like that you aren’t afraid to call it like you see it.
Paulc
That real tool review guy is a joke. Never has anything bad to say, everything is awesome. That guy is a marketers dream. Unfortunately it’s us who pays the price of his bs. And paying for tool awards is wrong. Stuart you are always straight forward. There are a couple good sites, I like the tools in action channel and workshop addict. That’s about all i trust. Workshop addict seems honest and tools in action just say what they want. I like AVE also except he thinks everything is crap and had a hissy when he didn’t get a CNBC machine sent to him. Still I like his content.