I’ve said it time and time again: AIs aren’t trained to produce correct answers, but seemingly correct answers. That’s an important distinction and exactly what makes AIs so dangerous to use. You will typically ask the AI about something you yourself are not an expert on, so you can’t easily verify the answer. But it seems plausible so you assume it to be correct.
Thankfully, AI is bad at maths for exactly this reason. You don’t have to be an expert on a very specific topic to be able to verify a proof and - spoiler alert - most of the proofs ChatGPT 5 has given me are plain incorrect, despite OpenSlop’s claims that it is vastly superior to previous models.
most of the proofs ChatGPT 5 has given me are plain incorrect, despite OpenSlop’s claims that it is vastly superior to previous models
Both of those can be true.
I mean yeah, but they specifically mentioned its amazing performance in tasks requiring reasoning
I suspect this will happen all over with in a few years, AI was good enough at first, but over time reality and the AI started drifting apart
And then, the very same CEOs that demanded the use of AI in decision making will be the ones that blame it for bad decisions.
while also blaming employees
Probably more this, the idea to use LLMs is good, but the employees just didn’t know how to use it right. To say the LLMs are the problem is to admit being wrong, or worse, being gullible in the face of marketing material. The one thing that is drilled in as a first principle of business leadership is to never ever look weak by being wrong or tricked.
What employees?
Of course, it is the employees who used it. /s
AI is literally trained to get the right answer but not actually perform the steps to get to the answer. It’s like those people that trained dogs to carry explosives and run under tanks, they thought they were doing great until the first battle they used them in they realized that the dogs would run under their own tanks instead of the enemy ones, because that’s what they were trained with.
Holy shit, that’s what they get for being so evil that they trained dogs as suicide bombers.
I raised this as a concern at the corporate role I work in when an AI tool that was being distributed and encouraged for usage showed two hallucinated data points that were cited in a large group setting. I happened to know my area well, the data was not just marginally wrong but way off, and I was able to quickly check the figures. I corrected it in the room after verifying on my laptop and the reaction in the room was sort of a harmless whoops. The rest of the presentation continued without a seeming acknowledgement that the rest of the figures should be checked.
When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.
I suspect stories like this are happening across my industry. Meanwhile, the company put out a press release about our AI efforts (literally using Gemini’s Gem tool and custom ChatGPTs seeded with Google Drive) as something investors should be very excited about.
When I approached the head of the team that constructed the tool after the meeting and shared the inaccuracies and my concerns, he told me that he’d rather have more data fluency through the ease of the tool and that inaccuracies were acceptable because of the convenience and widespread usage.
“I prefer more data that’s completely made up over less data that is actually accurate.”
This tells you everything you need to know about your company’s marketing and data analysis department and the whole corporate leadership.
Potemkin leadership.
This is why AI will only be used in subjective things that dont need to be perfect, like art. You cant create intelligence by regurgitating reddit posts via an LLM.
And with art it’s more or
lassless (stupid mobile typing!) blatant plagiarism.True, though its still extremely useful, and its impossible to determine where its stealing from for the average consumer. Its objectively useful.
Your high tech tool produces wrong data?
You cant not use it, your competitor will.
So you must do something else to make it more reliable, something small that the Firm can do.
And if you only caught it 3 months in, that firm is incompetent as fuck.
But its probably not even a real post.






