studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software
One study found a significant correlation between confidence in AI output and belief in the paranormal.
💀 💀 💀
That’s HILLARIOUS
I could’ve told them this, tbh.
Anyone can tell anyone anything, it doesn’t make it true or validated.
output is up (more code, more commits, bigger diffs)
We’ve known measuring output by lines of code is counterproductive for a long time.
I contributed -13,000 lines of code this week. Very proud
Has management known that?
A lot of problems seem to be downstream from “management are idiots and assholes”
I’m sure there were pockets but it legitimately seemed like that dragon had been slain until recently. Trying to assign more meaning to scrum points has been in vogue for a while though.
My team assigns both hours and points to tasks. I’ve never seen the points used for anything but they still spend time on it.
Can you retell my company that?
For a small exorbitant consulting fee that legitimizes me, I can tell them that.
studies show a clear trend – output is up (more code, more commits, bigger diffs), but outcomes don’t reflect that trend. If anything, the average team is taking longer to ship worse software
&
burning out sooner,
suffocating high pace churn,
debugging bad code.
There are some cardinal sins within the annals of software development in the workplace. The relevant two here are:
- Do not build a scoreboard for productivity
- Do not equate keystrokes with effort or value rendered
These create perverse incentives that can really screw everything up, including creating permanent damage to company culture, products, and productivity. Anytime you see these things done, it’s because you have (or are) lazy-ass management.
Edit: I also just learned that this is an application of Goodhart’s Law.
Oh god a former company had the burndown chart on a big plasma screen at all times
LLMs cannot distinguish between recent and out-of-date information in the context, and information in the model itself, learned during training (“dominant priors”), can often “outweigh” information we give it
LLM inference is more accurate when we give them examples (demonstrations) rather than just describing what we want.
Deep neural networks, including LLMs, struggle to learn patterns with long-range dependencies, at any scale of model. They will always be “driving in fog”, with local, short-range probabilities crowding out long-range ones. In case you were wondering why they suck at the “big picture” – probabilistically, it’s a blur.
I’d guess that what all this stuff adds up to is, sustainable use of LLMs as a coding tool for nontrivial projects calls for an entirely reworked set of software development practices to conform to its limitations effectively, but the people in charge really really want and believe it to be a drop-in efficiency boost, and a big mess results. This reminds me a lot of the articles and arguments I’ve read over the years about low level vs high level programming languages and frameworks. Probably will play out a similar way.
Great summary.
It’s obvious if you actually use software beyond the average literacy of a talking chimp. I use hundreds of apps across iphone, mac, and linux os’s. Literally none of them have noticeably increased in quality, stability, or feature-set beyond their average between 1-5 years ago.
Mac and iphone appear to have more bugs and shittier quality control than at any other point in the last decade.
Quality software is getting harder and harder to find thanks to all the slop-abandonware being promoted by slop-content and slop-SEO on slop-enshittified search engines.
I notice far more idiocracy-grade errors in digital content, cx, business processes, product listings, etc than ever before.
Weather forecasts have gone to dogshit in the last 2 years. Even same-day forecasts can shift on a dime unpredictably. It’s at the point where I check 3 apps. Until this year, I never had a time where I woke up to 0% chance of rain and sunny, then looked outside to see rain. Not a sun shower. A rainy day hour-long downpour.
Auto-generated subtitles are great for content that was never going to receive human attention, but they’re clearly being used to replace humans. At least once a week I notice a major contextual error that completely alters the perception of the line/scene, and there’s no way to submit corrections.
Art, culture, and knowledge are being actively corrupted, bastardized, and destroyed.
The future simultaneously sucks while being dumb as all fuck. Complete clown show run by the idiot criminals, rapists, pedophiles, scammers, and thieves.
Yes it blows my mind everyone can’t see all the historically perfectly fine software starting to crumble to dust
The big one for me was rsync, maintainer started vibecoding it and the first release with those changes had a huge amount of regressions.
Yet it keeps getting held up as an example of “using AI tools properly”.
This doesn’t seem to cover there is also no LLM that doesn’t plagiarize, or where the training data appears to be compatible with such behavior (e.g. CC0). Now I don’t know what that means legally, but morally it seems to be tossing away other project’s licensing and I think for FOSS as a whole that’s no good.
Also something worth reiterating: https://machinelearning.apple.com/research/illusion-of-thinking LLMs apparently can’t do basic logical reasoning. Even a junior coder can do that. I’m always surprised anybody would let LLMs near their code, at all.
Of course it didnt cover that. This was an article on the efficacy of AI in writing software, not a treatise on ethical or legal concerns. I would completely lose trust in the author if they started piling on every reason why “AI Bad”, because it would be clear that they have an agenda
In my opinion that’s not the framing the article uses. It just says “I’m currently pulling together a bunch of sources – that are mostly recent – on the topic of LLMs and their use in software development.”.
Leaving out any ethical concerns for that basic framing seems like a pretty notable choice, so I thought it was worth pointing that out.
