We’re about to learn a painful lesson about delayed gratification in software engineering.
New data from China, 26,811 students tracked January 2023 through June 2025. Students using AI for homework saw their scores jump 20 percent. Completion time dropped nearly half. They aced the assignments.
Then exam season came. Those same students scored 20 to 40 percent worse when they couldn’t use the tool.
The homework phase is over. The exam phase is coming.
We’re doing this in software right now. Vibe coding feels incredible. Features ship fast. Nobody’s asking what happens in Month 18 when the original dev has left and nobody understands the codebase.
Commercial pilots fly with autopilot for most of every flight. They’re required to maintain manual flying proficiency regardless. If the system fails mid-air and the pilot can’t take over, people die.
Most teams using AI right now have forgotten how to fly manually. They’ve become passengers in their own systems. The autopilot flies, nobody checks instruments, and the first sign of trouble will be a breach notice or outage.
Three rules:
- Command the mission. Define architecture before prompting. Ambiguity kills in code and in flight. Delegate selectively. Offload mechanical work. Keep design and security reviews human. Verify everything. Audit before production.
- Never trust the automation without checking instruments.
- Quick wins feel good. Sustainable engineering feels boring. Boring keeps systems standing.
Organisations surviving the next two years won’t ship the fastest. They’ll be the ones who remember how to fly without the aids.
people insisting that you actually be skilled, independently of your tools, doesn’t make them Luddites. Rather, being unable to do so makes you a phony.
The chart they didn’t post showed that time spent on homework correlated well with exam score regardless of AI use.
Turns out using AI to avoid homework (in this scenario, replace work) might be the issue. Using it to improve the quality of work (e.g. when you’re done coding a block, getting an AI to help fuzz your inputs to test that block) is likely to be beneficial.
This is kind of like the “you won’t have a calculator with you all the time” argument by your 8th grade math teacher in 2003, except instead of making yourself dependent on a little solar powered device (that you could have with you all the time if you really wanted to, even before smartphones) you’re becoming dependent on a subscription service offered by a trillion dollar company that can increase the price tenfold at will, or cut you from the platform entirely, stripping you of all your professional expertise at once.
that can increase the price tenfold at will
That will have to increase the price ten fold to stop bleeding billions of dollars every month.
The American taxpayers will save them, whether they want to or not.
This is kind of like the “you won’t have a calculator with you all the time” argument by your 8th grade math teacher in 2003
Which is grand, given we’ve had researchers agreeing effectively incorporating calculators into math education improves everyone’s ability to do math without calculators - Since the 90s
There’s pretty good arguments to include calculators from a really young age, to improve the understanding of numbers
There is no good argument for including calculators before the pupils haven’t mastered basic calculations in their head, there are good arguments against it.
You’re ending up with a larger group of pupils “mastering” basic calculations through counting, when you remove the calculator as a tool for learning. Using the strategy of “counting all”, is one of the clearest signs a 1st grade pupil will be behind their peers in 4th grade.
Broken calculator (just search for it), is one way to include calculators in a didactic way. But using calculators in general, improves a kids ability to estimate results - Which is an ability that helps anyone out tremendously, as it’s giving you a clear idea if something went completely wrong in a calculation
A “broken calculator” isn’t a calculator. It requires you to calculate in your head to be effective, which is exactly what I was talking about.
First you need to solidify this ability, before it makes sense to introduce reliance on calculators.
It’s still a calculator, it just misses some features. It’s one didactical way to include calculators.
and also the calculator is actively destroying the planet.
There’s models that run on your phone.
“You won’t have a tiny pocket computer with you all the time” really is the exact same argument with different specs. And frankly I’m a lot more likely to do math without a computer than write code without a computer.
when the original dev leaves and nobody understands the code
In my experience at work, no one cares about this. They assume the AI will be able to get there with enough tokens/time with the help of the human at the other end. They have done this and decided it was good enough.
Meanwhile, people like me who don’t use AI coding agents and are PRing and modifying the code are having a hell of a time.
They assume the AI will be able to get there with enough tokens/time with the help of the human at the other end. They have done this and decided it was good enough.
Or if it does break, just hit it with the AI stick until it works again.
The luddites had a point.
They weren’t opposed to technology, but the use of technology to remove skilled labor to produce inferior products.
Also, the pilot/autopilot argument is, kinda not applicable. The pilots aren’t just fucking around- autopilot systems only manage some aspects of flying, the pilots are still fully engaged.
I feel like their point was that the gains of technology were not distributed fairly, which is largely the concern today. It’s not “technology is bad”, or “technology make bad thing”; rather, “the owners of the machine undercut our labor, and we lose our jobs”, which I think was true. In the modern world, there are likely less tailors, as a percentage of the working population, certainly in first world countries. Sweatshops still exist, and it’s people’s labor competing against a production line, in terms of ROI for a business.
I get the sense we all know AI isn’t going to bring about a uptopia for all people, because the economic systems don’t incentivize that. There’s some hypothetical world where this sort of technological advance could benefit all people, but it’s not the one we live in. That’s a reality & major problem for the technology (among other issues).
I certainly think the cost of labor for tech development is going to drop significantly due to the market pressures; look at all the tech layoffs. At the same time, the upside is these tools make certain projects accessible & possible, because prior, would have needed significant funding, amazing solo skills, etc. There’s new opportunities available, but it’ll come at the cost of cratering tech position compensation & job security.
I don’t know anyone in my field right now who feels safe (game dev), and I have many talented friends out of work.
The pilots absolutely can just fuck around for most of the flight, and even during the critical moments of takeoff and landing, the plane can do a lot of it automatically these days.
Pilots need to actively practice flying without autopilot to keep those skills up in case the autopilot fails.
The pilots job is specifically to prevent to finding out phase.
the plane can do a lot of it automatically these days.
It can but nobody is out there flying CAT IIIc autoland approaches because its ‘easier’.
They did have a point. The printing press still changed the way information works
Luddites weren’t concerned about the printing press. Their fight was over mechanized looms and knitting frames.
Their concerns were mostly about losing their jobs, as well as the absolute dog shit quality the mechanical machines produced at the time.
In 1814, printers in London did have similar conniptions over the steam press, though.
Til I always assumed they were the dudes who drew the fancy letters
the luddites had a point… in the moment… kinda… but i think have largely been pretty conclusively proven wrong… clothing is now plentiful, and i doubt anyone would want to go back to paying taylors to make all their clothes for 10x more, even if the clothing would be much higher quality, which i even doubt: clothing quality doesn’t come down to artisan skill any more, it comes down to choices you make, and an artisan can still make cost cutting decisions on fabric, construction, etc
So…have you not heard the damage fast fashion did to the planet? Sure, it make making cloth quicker and cheaper and widely available, but in the end it’s a huge source of chemical waste, landfill waste, and microplastic. Because clothing are so much cheaper to buy new than fixing it, we lost a lot of skilled labour to automation.
The same thing happened everywhere, automation take away human agency, and that take away skill learning. In the end humanity will suffer because less and less people gonna learn how to fix the shit that they rely on. That didn’t happen on accident, that’s by design. We all thought AI gonna upskill a lot of people, but i feels like instead of making a nation of artisan it gonna make a nation of 7/11. Just look at Destin’s struggle to make a fucking bbq brush fully American, the nation of 300millions people can’t even provide parts that fully made within the nation.
Maybe. I think a sizable group of people would rather put fast fashion back in the bag.
Fast fashion is a different topic from the machine produced vs tailor point they’re making.
Made to order is getting more and more traction, because we have machines able to fully automate it. Making higher quality clothing more available than ever
No. They weren’t proven wrong, pauperism was the consequence and it was horrible. Luckily we shifted that back to regimes that don’t like to report much on terrible living and environmental conditions so we don’t have the filth at our doorstep.
I agree. A lot of blame that should be squarely laid at the feet of bad distribution of wealth via capitalism, is being put onto technological advances making things more efficient. The fact is, even in a utopian society where every person has access to any material needs they could want, we’d want to be able to produce those things more efficiently to get them into the hands of more people at a lower cost to society as a whole.
If we had a fair system - firms having to be run as worker cooperatives, wealth going into the hands of the working class instead of a few capitalists, etc, none of the Luddite-esque complaints would hold. In this example, yes, cost of clothing would go up if we fairly paid all textile workers and if they owned part of the company they work for. It would still be orders of magnitude cheaper than bespoke, hand made clothing requiring one skilled worker tens or hundreds of man hours of work.
And as hard as it is to change that system, I actually think that’s easier than trying to put the genie that is technology back in the bottle.
This is largely an oversimplification because it depends on how you use AI in the first place. Are you building modularly and incrementally? Or are you letting it ride on black?
ive personally found LLM coding to be pretty effective if i adhere to very strict “engineering best practices” like TDD, good documentation, thorough API design, systems design, etc.
im very afraid that the next generation of software engineers will never develop the skills needed to make the most out of it because they skipped the rudiments of good software engineering.
to me it feels like playing jazz without learning scales. it’s possible to be an incredible jazz musician without formal training if you’re a savant… but you’re probably not.
All it takes is two developers who let it ride and approve each other’s PRs.
I have a hard time telling AI authored LinkedIn thought leadership apart from formulaic LinkedIn thought leadership sometimes, but this feels AI written, which makes it fairly amusing considering the point it’s making.
(Does vibe coding feel incredible? Also, “Nobody’s asking what happens in Month 18 when the original dev has left and nobody understands the codebase” implies that the vibe coder original dev understood the codebase at any point, which isn’t an argument I’d feel comfortable making for any of the ones I’ve worked with)
Does vibe coding feel incredible?
There’s definitely a magic in watching it print code with almost no input. Like watching git history with Gource.
It’s not good code, but it is cool to see.
Compilers would look the same. You give some people a tool that turns fiddly low-level repetition into relatively painless double-checking. Then you yank that tool away, and oh no, they’re not as good at the fiddly low-level repetition.
What matters is - are these students worse-off, with the tool? It’s not going away. Foreverafter, a couple gigs of linear algebra will turn your to-do list into an actual running project. We have to ask if teaching the fiddly low-level repetition is the point… or just a means to an end.
You can scoff that compilers are objectively better, but they weren’t! Sometimes they still aren’t! We were teaching kids to consider inline assembly well into the Windows Vista era. Even now, compilers with decades of active development can have bizarre corner cases, which any hardened bit-banger of machine code would never write. But they’re all good enough that plaintext assembly has become an arcane third-year topic, and hexadecimal machine code is the realm of bare-metal maniacs.
Teaching math, kids have to know what numbers are, and how base ten works, and why the calculator says 11-3=8. But as you go up to college math it matters a lot less whether people can do shit in their heads… or even remember how. Open-book and open-calculator exams are still fuckin’ hard. They’re just hard because you’re trying to apply some obscure proof to a rat’s-nest of swooping symbols, instead of because you’re worried about having enough time to write out long division to umpteen places. The latter can be sufficiently approximated with two wooden sticks. We can assume anyone doing math professionally has two sticks. Continuously! As a given, for their job. Probably at home as well.
Teaching programming, you have to break people a certain way. No gentle and charming syntax can disguise the mechanized nastiness of what code is. Whatever semi-readable text you’re looking at will be squinched through a flake of tortured sand in a bazillionth of a second. But only after getting transformed at least two times in processes you kinda have to trust will be computationally equivalent, unless you want to minor in a branch of mathematics haunted by Stephen Wolfram. Ultimately there is a shockingly small state machine designed to trick you into off-by-one errors.
But once people can knock out fizzbuzz on a paper 6502… they’ve kinda got it. They’ve been inculcated. They will soon apply arcane patterns to a rat’s-nest of obtuse variable names. Ssshould they have to do that by hand? They’re not doing the assembly by hand. They’re not even looking at the assembly. They might never need to think about assembly again. So long as they’re playing spot-the-bollocks for a flurry of pull requests, that’s arguably a more useful skill than writing clean toy code from scratch. Especially in pen and ink.
That’s what they’re doing for my SO’s big introductory class in the humanities. No homework, quizzes, or papers. Short answer exams done in a special testing center.
Really sucks for people who were decent at essays and turning in assignments on time but not great at exams.
yeah, but it’s great for those that hated busywork and were good at exams
I wasn’t great at exams, but I think if I re-did university today I’d still do much better because my personal learning style is best when I talk through the material with someone. Sitting in a lecture listening to someone monologue at me, reading a static textbook, those weren’t great. But with LLMs every student now has full-time access to a “live tutor” with infinite patience and no sense of social risk from asking it dumb questions. It’s absolutely my perfect learning environment.
I’m the “good at exams” type, I find paper writing (or novel writing… 😢) unpleasantly tedious. If we lose the “good at structuring complex thoughts in a way that can be coherently decoded by other humans” types, the “good at exams” types will suffer. We need people who can write good papers and people who can recall diverse facts and arrange them into simple shapes quickly.
It wouldn’t be hard to change things for essay-types to still get to shine. We’ve been moving away from actual home work anyway, for good reason imo (brief aside, I also think homework sends the wrong message for healthy separation between work and life in the working world). Give in class time for handwriting essays, either completed in one session or over the course of some days. In class books or e-readers for research.
If we’re being honest here, we all kind of acknowledged already that education has been in dire need of reform for a long time now, we were talking about the need for radical education reform over a decade ago… I think AI is highlighting these problems and making them worse more than they are really a root cause.
we were talking about the need for radical education reform over a decade ago
Education, and especially critical thinking, has been systematically gutted by conservative governments for at least 4 decades in the Western world. The goal is the absolute minimum knowledge and thinking skills (preferably without learning skills) necessary to complete the wage slavery tasks.
Yeah I would have done much better in high school like this, though I don’t think AI will eliminate the poster projects unfortunately.
same, it almost makes me want to go back to uni
Fuck software engineers
Is that an offer?
It’s an order.
The vilification of the Luddites as cantankerous tech haters is a really good example of how malicious falsehoods can succeed, I think. I’d guess that most English speakers who know the word would define Luddite as “someone who hates technology”, and I think it resonates on some level because we all hate technology some of the time. The fact that Luddites didn’t indiscriminately hate technology, they just destroyed the means by which their oppressors oppressed them, is handily swept aside. And I never even questioned that narrative on Luddites until I happened to read up on them.
It’s sneaky, because before then “Luddite” didn’t feel like an important or politically charged term at all. It was simply a slang word for people who grump about technology. Calling people haters just short-circuits any examination of the so-called haters’ reasons. It’s exactly what people say about terrorists, except instead of technology its freedom. They just hate freedom. I wonder how many people have “terrorist = freedom hater” embedded so deeply they don’t even think about it.
One persons terrorist is another persons freedom fighter
they were absolutely not right… the modern world of plentiful clothing simply wouldn’t be possible without industrial clothing machinery (and i’m not talking about fast fashion and waste - though industrial clothing also makes those things possible, i think that’s an outcome of the plenty rather than an inevitable outcome of industrialised clothing)
artisans don’t magically make better clothing… in fact, modern machinery can and does produce much higher quality, complex clothing than a human alone would, and in much higher volumes
they had a moment when they had a point, when the machinery just wasn’t very good (and of course the child labour concerns)
clothing quality on the modern era comes down to choices far far more than the skill of the person making the clothing… artisans are still able to make cost cutting decisions
the point we should take away from the luddites is more about retraining than about their core point… this applies to coal miners and the green transition, and will at some point in the future apply to the logistics industry when driverless trucks become reliable: we can’t just toss an entire industry out of work and expect them to be okay with it (or for that matter expect the economy to absorb them)… but it’s a much better world when fewer people are required to achieve the same output
What do you think luddites believed and did?..
It’s like you didn’t read the article.
It’s funny watching this type of bellyaching play out like a loop over history.
People went through this EXACT same bellyaching when compilers first started showing up. The machine coders considered themselves superior and their skills indispensable. Now those people are extinct and modern people consider their compiler skills to be elite, not realizing they’re using the baby mode that people railed against in the past.
It turns out that you, in fact, don’t need to know what you don’t need to know, for the most part.
Understanding the layers of abstraction down to the metal is still a useful way of developing an instinct for how compilers, interpreters, languages, and algorithms etc. work and what is possible. If you ever truly need to optimize something you’ll end up descending those layers, it’s why C has inline assembly and with RAMageddon upon us optimization may well return to being a valued skill.
Which is to say you don’t need to know what you don’t need to know, until you do, and you’ll be better anyway if you can.
Did anyone do studies on the homework and exam scores of people using compilers vs people not using compilers?
Did anyone establish that people who used compilers are measurably dumber than the people who didn’t?
No?
Well then obviously it’s not the same situation, now is it?
Dont expect the pro-AI crowd to have any reasoning capacity. The only argument they have for AI is “I’m lazy”.
Nobody’s asking what happens in Month 18 when the original dev has left and nobody understands the codebase.
Throughout my career, I’ve often had to sync a git repository for some random project that I’ve never worked with before and then had to muddle through whatever was in there hoping that the documentation was vaguely complete and accurate.
Nowadays it’s actually much easier. If the documentation isn’t complete or accurate I just turn my coding agent loose on it and tell it to write the documentation.
I get the impression that the guy writing this hasn’t actually used modern coding agents, this is not a problem.
I see coding AI as being similar to a compiler. Before compilers came along programmers had to work in assembly language. Nowadays very few programmers ever do anything in assembly and that’s perfectly fine.
(talking about non-trivial code bases here)
Not being able to understand a code base and needing an agent to do that is one thing (skill issue I guess), but not knowing the intention behind a piece of program is a complete different thing (the agent can’t read the mind of the original author on why certain business compromise was made, how the design evolved to this point, …).
I get the feeling all these people defending vibe coding never worked on a non-trivial project and something that has to live and serve large volume of requests for 5+ years.
The number of Chesterton’s fences getting bulldozed right now is scary to contemplate.
i’d say it’s not quite like a compiler because you still need to know what the code does… a compiler you largely don’t care, and it’s deterministic… an LLM hallucinates, so you can’t just trust it, which means you have to be able to assess when it’s doing the wrong thing
i agree that setting an agent going helping you learn a codebase is reasonable, largely because it can quickly point out connections in the code quicker than you can understand the architecture, and its guesses are usually just as good as your guesses until you get knowledgeable with the codebase
however, and that’s a very load-bearing however, i don’t think you should just jump into a codebase and just go ahead and tell the LLM to write new code without understanding what it’s done… the understanding is exactly what this graph is trying to show: actually learning is critical, so your review of the code becomes doubly important: you’re gaining institutional knowledge, and you’re making sure the output isn’t slop
an LLM hallucinates, so you can’t just trust it, which means you have to be able to assess when it’s doing the wrong thing
Unless you can define what the right thing is. Then you just have it write proper tests, and it’s able to verify itself.
I’m no dev, but did do my HND in computing some 25 years ago. I have also tinkered with scripts to solve an itch and have has plenty of VBA and of late DAX and power query.
I did a project for myself around a year ago that was much larger than I normally would these days and I used Gemini. It was a constant battle with the LLM (man it’s hard not to gender and personify these LLMs) to keep it on track and not keep breaking shit all the time. It struggled to writ valid rules for it’s own firebase/firestore. It generally felt like a bad time.
This past week end I got codex to migrate the PWA from google’s services and database into a selfhostable docker stack. It was a pleasure, it documented the whole process map, built a full test environment, followed through and imported my data. It the suggested a few improvements. It even rewrote a a complex equation that Gemini could nust not get correct and added manufacture safe/max bounds.
all this to say ‘yes, modern agents do seem to do a good job of a codebase that they have never seen’
Yeah, I’ve been tinkering with coding agents for a while now and it feels like some sort of “phase transition” happened earlier this year - the models and their harnesses went from being the sort of clumsy barely-passable student coders that people make fun of to being actually highly competent. Lately even the small local models have reached that level - I’ve been using Qwen3.8 27B on my own machine and it’s been doing just fine. I suspect a lot of the folks making fun of them tried them early on and wrote them off, not giving the latest versions a second chance.
The compiler comparison works there too, IMO. I remember back when I was learning the tricks of the trade there were all sorts of ways you were supposed to write code to trick compilers into generating more efficient results. Coding standards would say to use ++i instead of i++ for incrementing loop indices, use x*0.1 instead of x/10 because division was way more expensive than multiplication, using memcpy instead of copying values with =, using int instead of bool, and so forth. Lots of stuff. Some of it was never useful, it was just folklore and superstition. Other stuff did make a difference with early compilers but modern ones know these tricks themselves and can often apply them better than the human programmer can. So the old-school greybeards who still insist on doing that stuff can actually make things worse these days (disclaimer: my beard is pretty grey these days too).
I expect so too will this “hah, vibe slop” attitude eventually pass away. AI is getting quite good quite quickly.
I fear for my career in this field.
To be serious for a moment here on programmer_humor, yeah, it’s a real puzzler. For now there’s still a role for programmers like us in all this because we’re still good at knowing how to translate the requirements of non-technical people into the things we actually ask the coding agents to build for us. But AI’s getting good at that too. As might be expected, they’re language models so interpreting what people tell them using language is kind of their jam.
If I were advising someone who’s looking at whether to get into programming as a profession right now, I’d tell them to only go that route if they had the resources to take a mulligan on their education and try something else if a couple of years from now it turns out that the robots have completely taken the job over. I think it’s a real possibility, and will lead to some interesting times indeed.
Until then, I guess we keep making jokes about it? Just make sure not to take them too seriously. AIs can count how many Rs are in “strawberry” now. They can whip up a bit of Javascript to do it for them in their harness’ sandbox.
