ChatGPT is an AI chatbot that is designed to write natural-sounding conversational dialog.
It has been in the news a lot recently, with many people marveling at what it can do.
I was curious, and so I signed up for an account and gave it a try. I asked the following question:
Advertisement
Can You Recommend a Cordless Drill for Professional Users in Less than 400 Words.
Here’s what ChatGPT wrote in response:
1. DeWalt DCD999B 20V MAX XR Brushless 3-Speed Hammerdrill: This drill is part of DeWalt’s line of heavy-duty cordless tools, and it’s a popular choice among professionals. It has a powerful brushless motor that delivers up to 42,000 beats per minute, making it ideal for drilling through tough materials like concrete and steel. It also has a 3-speed transmission, so you can adjust the speed according to the task at hand. The drill comes with a 20V MAX XR lithium-ion battery, which provides long run time and fast charging.
2. Milwaukee 2801-22CT M18 Brushless Hammer Drill Kit: This drill is another popular choice among professionals. It has a brushless motor that delivers up to 1,200 in-lbs of torque, making it ideal for heavy-duty drilling tasks. It also has a 2-speed transmission and an LED light for illuminating the work area. The drill comes with a M18 REDLITHIUM XC5.0 battery, which provides long run time and fast charging.
3. Makita XPH102 18V LXT Cordless Hammer Driver-Drill: This drill is a versatile tool that can be used for drilling and driving applications. It has a powerful motor that delivers up to 480 in-lbs of torque, making it suitable for a wide range of tasks. It also has a 2-speed transmission and an LED light for illuminating the work area. The drill comes with an 18V LXT lithium-ion battery, which provides long run time and fast charging.
When choosing a cordless drill, it’s important to consider the size and weight of the tool, as well as the battery type and run time. You’ll also want to consider the drill’s features, such as the chuck size and type, the number of speed settings, and the presence of additional features like an LED light or a belt clip. Finally, be sure to read reviews and compare prices to find the best value for your money.
Did the AI Get it Right?
Kind of, but with many inaccuracies.
Advertisement
Dewalt DCD999B is a 20V Max FlexVolt Advantage model, and it’s indeed a great option for pro users.
ChatGPT says the Dewalt model delivers up to 42,000 BPM. That is incorrect. The specs are hard to find, but sales pages – such as Home Depot’s – list its impact rate as 38,250 BPM.
ChatGPT says it comes with “a 20V Max XR lithium-ion battery, which provides long run time and fast charging.” This is incorrect. The B in the SKU indicates it’s sold as a bare tool, meaning a battery is not included.
Also, the hammer mode is only relevant for masonry drilling, whereas the AI suggests it helps with drilling steel.
The Milwaukee 2801-22CT M18 cordless drill is also a decent compact model.
ChatGPT says this is a hammer drill, but it is not; the 2801 is a non-hammer cordless drill.
ChatGPT says that it delivers up to 1,200 in-lbs of torque, and this is also incorrect. The 2801 cordless drill delivers up to 500 in-lbs max torque.
The model number, 2801-22CT, specifically the xxxx-x2CT part indicates that the kit comes with 2 compact-size batteries. ChatGPT says it comes with an XC5Ah battery, which is wrong.
ChatGPT recommended the Makita XPH102, an outdated and long-discontinued model.
Their general advice is sound, but very generic. It reads as if it the AI trained itself by drawing from the countless number of poorly-written or auto-generated junk review content that plagues the internet these days.
Bonus: Are 20V Cordless Power Tools Better than 18V Tools?
I asked the ChatGPT AI a different and more nuanced question. Here is where it actually surprised me.
The response:
In general, a cordless power tool with a higher voltage rating will be able to deliver more power and potentially perform more demanding tasks than a tool with a lower voltage rating. However, there are many other factors that can affect a tool’s performance, such as the type of motor it uses, the size and efficiency of the battery, and the overall design and construction of the tool.
So, it is not necessarily the case that 20V tools are always better than 18V tools. It really depends on what you need the tool for and how it compares to other options in terms of its features and specifications. It might be helpful to do some research and compare different models to find the one that is the best fit for your needs.
Did the AI Get it Right?
Tool users more familiar with cordless power too tech will know that 20V Max and 18V are actually the same thing, a fact that ChatGPT doesn’t seem to pick up on.
However, the “they can be, but it depends” spirit of answer is pretty spot-on.
One Thing to Remember About ChatGPT
News articles and commentators have been claiming that ChatGPT will “change the world,” “change everything in social media and content creation,” and “change the face of digital marketing forever.”
The key point to remember is that garbage in will mean garbage out. I believe that’s why the AI chatbot got so many things wrong about the 3 drills it recommended. There’s also no indication why they chose those 3 models.
I had to contact two brands recently. One didn’t provide public CFM specs for their blower fans, but they were able to provide specs that were provided in private customer service documents. Another didn’t provide load rating specs for casters, and again they were able to pull it up from private tech documentation.
Those brands might be able to use an AI chatbot to parse private tech specs and documentation, saving employee time for higher level discussions, such as where a customer might need help finding a product matched to their needs.
It will be interesting to see what comes of this.
Collin
Now get DallE to draw what a M18 snowblower, a MX Fuel 0 turn, and a M18 pouch cell battery would look like.
MM
The first one sounds like the AI learned by imitating those crappy “review” sites that just regurgitate specifications back, only with more errors than usual.
The second one I think the AI nailed with an eerie degree of accuracy. Note that the question you asked it did NOT mention the word “MAX”, or any other keyword to indicate that you were talking about the marketing names for batteries. In my opinion the question was correctly interpreted as one about nominal voltage only. I am very curious what it would say if you asked it specifically about “20V Max” vs “18V”. Or perhaps find out if it knows that US-spec “60V Flexvolt” is the same as Euro spec 54V?
Bonnie
A lot of those parked/SEO-abusing review sites were already using AI systems to generate content.
It’s also not new to journalism. The Washington Post wrote their own AI reporter 5-ish years ago, and you’ve been able to get this stuff off-the-shelf for at least as long.
Matthias Thimm
This is because it “remembers” what you and it where talking about earlier. So it already mentioned the 20V MAX and it will continue to do so in later conversations.
If you’re getting into detail, these AIs are nailing it sometimes. But other times and I found out for myself most of the times it is quite not right. I for example was searching the difference between two electronic components. They’re identical but ONE thing is different and the AI treated them as the same and replaceable among themselves. Which is not the case. So yeah, if you know what you are doing it can help decide, but for a complete newbie in some areas it could lead to frustration I think.
Robert
So this experiment by Stuart Labs Limited somewhat confirms what I had suspect about what come up as the top search results when I search for “Best XXX.”
Usually a “Best 10”… A mix of sensible words and some jarring mistakes. Which led me to suspect it was either AI or offshore paid influencers.
I’m really getting to dislike commercial use of AI. Recently a chat fooled me into thinking it was passing the Turing test for a while. But a pattern emerged that it was giving the same set of answers. Which was a waste of my time because I had an atypical problem that needed a human with more flexibility to answer.
Ted
I hope you clear your browser history after searching for “best XXX”
TonyT
I think of lot of companies would be better off providing parametric search, with accurate information, like Mouser, Digikey, ADI, TI, Microchip, etc do, than feeding their datasheets to ChatGPT, which very well might inaccurately regurgitate the information.
Also notice how general your question was. You get a much better answer (at least from people or parametric search, not sure about from ChatGPT) when you’re more specific about your requirements. After all, the best drill for a professional could be a installation driver, hammer drill, SDS drill, magnetic drill, etc
Stacey Jones
Well whe AI Chatbots lie, these are less accountable than humans. Perhaps, these would make great politicians?😂😂😂
Suzy
Since when have politicians been held accountable? And where?
Bill Hall
I read somewhere on the ChatGPT site that it was trained on 2021 data. So it wouldn’t know about any of the latest release tools.
Ct451
I remember I tried to sign up for this when I saw it on tv but they wanted to verify my phone number and gave up. It is the sort of over-hyped tech “news” that plays well because it has microsoft and elon but it is just one step. It may lead to something useful down the road but that’s many steps away. It’s not for anything important. Plus there are other players in this game.
Michael F
I’m a software developer working daily with OpenAI’s beta API using the text-davinci-003 model (what ChatGPT uses).
One thing to keep in mind is that it’s a language model, not a search engine. It’s been trained on data from the past few years with the focus on communicating as close to a human as possible in language, not necessarily being factually correct. If your question relies on up to date information (say, news from the past week) the model won’t understand. So in this case it could be using old or outdated data about tools from up to 3 years ago.
Understanding the limitations of software always helps!
Stuart
That makes a lot of sense, thank you!!
Still, the way the mass media has been talking about how this AI is going to upset entire content and creative industries, I wanted a quick test to see just what it could do.
I’m glad to see I’m not going obsolete anytime soon.
CountyCork
I don’t think ever obsolete. AI will have its place I just hope not too big of one.
Badger12345
+1. We need Stuart and Toolguyd to sort through and curate the numerous future articles we’ll see generated by AI. The level of “noise” is going to increase dramatically.
TomD
ChatGPT is also well known for giving the “this but also maybe that it depends” answers because those are most often “correct” for some value of correct.
Go not to the ChatGPT for counsel for it will answer both no and yes.
Jack Smith
I think they should feed this thing all the data from your website and then see how it does. It would have to be better. There are more really horrible tool review sites than good ones, so I’m not surprised by the results. Most of the top results are wildly inaccurate.
Frank D
Sounds exactly like most of those top best review sites … loaded with amazon referral links … that Google loves to rank so highly
Stuart
Agreed. Many of those “reviews” like to use the word “transmission” when talking about gear or speed range selection.
Jared
Your job is safe for awhile from the sounds of it!
While the second answer was more compelling, it doesn’t work because it’s a good answer – it works because it mixes in enough factors that it seems like a reasoned response. The problem is, a person who isn’t well versed in the subject matter doesn’t know the correct answer to compare. I could see it fooling someone into thinking they got a good answer and have information they can rely on, repeat and share.
That’s alarming at least as much as it is impressive. E.g. if the bot’s purpose is primarily to offer a convincing response rather than a correct one.
Kevin McKenzie
Please, please, please note that, in a very real sense, the results you get from ChatGPT and the like are bullshit. This is a concept that the philosopher Harry Frankfurt wrote about, where the person/entity speaking isn’t telling the truth or lying, they just don’t care about whether what they’re saying is true or false.
ChatGPT/AI in general right now has no conception of anything; they’re just trying to come up with a plausible answer to your question. They’re not trying to answer your question correctly; they have no understanding of what that even means. They’re amusing, but dangerous, as everything they return seems true, until you look at it closely, assuming you have the knowledge to do so.
Rcward
Very true, sorry to see this guy even playing with this BS
Dude
That is no different than any human entities on the internet. The internet mostly BS that gets repeated and repeated until people assume the BS is common knowledge. When you seek enlightenment caveat emptor.
MtnRanch
Apples and Oranges. In one “opinion” ChatGPT talks about battery voltage and, in another, battery current. No mention of torque in the DeWalt opinion. No mention of beats/min in the Milwaukee and Makita opinions.
The lesson here is that the ChatGPT is as trustworthy as a politician, they both spew out words that sound good but have no meaning and no facts to back them up.
I’ll stick with ToolGuyd opinions.
Steve
What’s eerie to me are the pictures that AI creates!
SamR
Those are not pure AI creations. They are just blends of other human-made pictures/creations.
RCWARD
Not a good idea to even give people a sniff that you are using one of these “apps” to write your reviews. Not a good look.
David Z
Stuart didn’t imply that he might do that at all. In fact, he clearly showed that the tool isn’t capable.
Stuart
Thanks! Mass media sites keeps saying the AI will make content creators and reviewers like me obsolete, and so I was curious.
I was a little worried that this new one will flood Google with more garbage “reviews” and “expert” buying guides, but it’s clearly deeply flawed.
CosmicFurFace
… and therefore your worries are well-founded, as it’ll fit right in with what’s there now…
Stuart
I would never use anything like this for actual content; my curiosity is of academic curiosity.
It seemed interesting given how much hype I’ve been hearing about it. I also felt this deflated a lot of the claims.
I wrote about Wolfram Alpha a few years ago – that is something I do use on occasion, for unit conversions and similar.
https://14cyiuhvcgv.com/wolfram-alpha/%3C/a%3E%3C/p%3E
Jack Smith
Do you really think this article indicates that he would use it to write articles?
How does an article that is clearly written to show how poor and inaccurate the computer is, make people believe that he is using it to write tool articles?
I’m just curious about how you worked that out in your head. Did you actually read it?
Dud
Excellent article with understandable explanations. An expert/enthusiast would know the mistakes but they wouldn’t be the ones to ask and the person who asks… Well, let’s hope they don’t ask for medical advice.
Mxx
Dollars in, dollars out heuristic.
Jack Smith
I agree with the recommendation of the DeWalt 999 drill and the kit actually comes with 6 Ah Flexvolt battery.
It’s a little heavy but it’s a pretty fantastic drill. I’ve used it to drill 1 inch holes through a concrete block wall and it was very fast with a high quality masonry bit.
All tool guys know DeWalt models ending in a B are bare tools.
I think if the computer was fed good information that it would do a decent job at this. Like he said, garbage in garbage out.
John G
I asked the new, Jan 9th, version of ChatGPT, the same question “Can You Recommend a Cordless Drill for Professional Users in Less than 400 Words”. Notably, it selects different drills. Only two this time. Neither are hammer drills. It appears to get all the details correct on the DeWalt, but not the Makita.
I then asked the original 20V versus 18V question. It gave me essentially the same answer. But then I asked tweaked the question to include 20V Max – “Is a 20V Max cordless power tool better than an 18V tool?” It did much better:
“It’s not a straightforward comparison to say that a 20V Max cordless power tool is better than an 18V tool. It depends on the specific tool and how it is designed.
20V Max and 18V are both marketing terms used by different manufacturers to indicate the maximum voltage a battery can reach, it doesn’t necessarily mean that the 20V tool will be more powerful than the 18V tool. It’s important to compare the specifications of the tools, such as the motor power, speed and torque, to determine which tool will be better suited for your needs.”
Ct451
As it turns out the AI still relied on humans for filters because by it self had no concept of anything. Cheap labor from Kenya will train your AI for $2 an hour.
Jared
Have you revisited AI recently Stuart? It’s changing.
A year ago I thought it was barely useful. It could sometimes make a rough draft I could use, but it was a technically lousy and generally verbose writer.
I would occasionally ask Chat GPT to write or brainstorm something. I always (heavily) edited what it wrote, or would only use it after I’d done the work to see if it came up with something I didn’t. It was useful, but only with a knowledgeable person’s oversight.
I noticed an improvement about 6 months ago, when its “knowledge” was becoming more specific. It wasn’t just giving generic answers anymore. It could give information specific to a certain geographic region or based on particular resources. Its writing seemed to be improving too – I could ask it to copy my style for example, and it would write something I edited less.
I think it has gone even further now. I can train it on specific documents and resources and it will come up with answers that rival what I can do. I suppose it’s still doing that by interacting with someone who is technically competent, who directs it where to look and fact-checks the work product – but there’s no denying it speeds up what I do.
When they talk about AI replacing jobs, I think it’s happening – faster than I’m comfortable with. It’s not that my job will disappear (at least for the time being!), but that I’m noticeably more efficient. E.g. how many people will my organization need to do work like mine if I’m doing it twice as fast? I think that’s a realistic possibility and not far removed.
Stuart
I played with Chat GPT the other day. I asked it about ToolGuyd and it kept getting some things wrong.
I read today that AI can help you get to easy answers quicker, but doesn’t know the answers to questions you didn’t think to ask.
It cannot scan websites, or extract from source code, and so there’s not much it can do for me when it comes to extracting links or descriptions for images and things like that.
I have not found any way for it to speed up anything I do here for ToolGuyd.
I’m redesigning ToolGuyd’s design and layout soon – maybe it can help with trickier coding tasks.
In another post, a reader is having difficulty understanding the concept of “tabs” in a cylindrical Li-ion cell.
Chat GPT can spit out reasonable-sounding definitions, but there’s no easy way to build that into understanding. It’s best to start from scratch manually.
I asked it why some slotted screwdrivers are flared outwards, and most of the bullet points were wrong.
For example: “Better Access to Recessed Screws: The wider tip helps reach screws in slots that might be recessed or slightly countersunk without the shaft interfering.”
That’s a property of cabinet-tip screwdrivers where the tip is the same width as the shaft, as opposed to being wider.
It also said: “Avoiding Damage to Surrounding Material: In applications where the screw is in a tight or awkward spot, the flared tip can help avoid contact with surrounding material, reducing the risk of damage.”
That’s also inaccurate, as the greater width of the tip can increase the potential for damage to surrounding materials.
Jared
I’m surprised a bit by that, but I suppose I’m not using it in a tool-context. I haven’t really thought about how it could apply there.
Part of the improvement for me is training it on specific source material. I don’t mean just a few select manuals or webpages – I’ve “fed” it entire text books, numerous webpages, articles, my own writing, etc. then asked it to respond to a specific question based on those materials.
I don’t trust it to do my work for me. It will still hallucinate. But sometimes… it’s shocking how good it is and synthesizing information and even doing some analysis. Plus, it does that in seconds and I can ask it for sources or coach it on what kind of answer I want.
One of the ways I first starting using it was to tell it the question, give it my answer, and ask it if there was anything I hadn’t thought of.
Stuart
That’s my favorite question to ask product managers and engineers – “is there anything I haven’t asked you?”
I am all for finding time-saving shortcuts, but have not seen where AI can do that for me yet.
Even when creating more complex posts and pages, it’s time-consuming but easier to do it manually than any auto-generating means.