Someone recently asked me—as a person in tech that they know—how many new programmers I see being very AI-dependent. Which made me pause, and then start typing, and then stop typing again. Because I’ve been meaning to write a blog about my AI thoughts, and this seemed like as good an excuse as any to actually do it.

Because tech is indisputably the epicentre of AI; while AI art might be the thing most furries associate with it, that is a drop in the bucket next to its use for coding. And the reason is simple: tech companies are the only people paying relevant sums for AI. Nobody is dropping a million dollars a week to make yet another piece of horse art with fifteen medial rings. But the world’s biggest companies are burning tokens as fast as they can to replace their software engineers, gooning out in techbro bliss at the idea of not having to employ humans anymore for the low low price of twice as much as employing the humans in the first place. After all, good software engineers are expensive, and machines are cheap! What could go wrong?

Everything. Everything is going catastrophically wrong.

Before I go on, I want to note that I won’t be digging into the ethical or environmental concerns around the tech. Not because I think they’re irrelevant; quite the opposite. I think they alone are sufficient reason for it to be shunned. But there’s also nothing more to say on that front that isn’t already known and well-covered by the media. The mass and blatant theft of all of human creativity, the psychopathically eager lust from the world’s richest losers to strangle all creative pursuits to death so that they can be replaced with the soulless unblinking eyes of a pay-per-use plagiarism-bot, and the concomitantly thunderous gallop towards forced environmental collapse, are all apocalyptic enough scenarios that anyone with a mind can look at them and comprehend the badness.²

With coding, it’s different. Because unless you are a coder, it’s an impenetrable world. You know that code gets made, but the process is a black box. Code is a shibboleth that goes beyond mere human language, because it is at once the key to a door of another world and the very essence of that other world. Anyone can assume that AIs are writing code, but what that process entails is a big “I dunno”. I will attempt to demystify some of that, and hopefully in the process make it clear why even though tech is the only industry that appears to have found a legitimate use case for AI, it is doomed to fail like all the others.

My Perspective

It’s useful for me to lay out my credentials here. I’m not just “a” coder. I’m the CTO of a company that provides SaaS services to clients around the world. The services are pathologically realtime and available; we have a three-nines guarantee, the broaching of which incurs ridiculously huge payouts to impacted clients as per their SLAs. Delays of just a few minutes in the delivery of any sort of data lead to very angry support messages. People depend on us to provide services that cannot “just wait”. My job is to invest my 30+ years of experience into managing a team and architecting a system that can deliver what is needed, using the most appropriate tools and processes to make things work smoothly. The system itself is highly complex, split across many subsystems in wildly differing geographical regions, and extremely easy to break without immediately realising it, due to the potential for a small reduction in throughput in one area to slowly accumulate in others until abruptly there are bottlenecks leering out of the woodwork miles away from the original problem. All of this to show that I am not talking about these things in a vague world where “maybe when” or “but only if”. I have applied them in serious production systems that cannot tolerate “oops” or “I didn’t know”. The sorts of system where little things, subtle architectural decisions, actually matter—as contrasted with someone’s dipshit little website that works fine even if the database latency jumps from 50ms to 5000ms and the code size quadruples overnight.

The Experiment

I, like almost all programmers I’m sure, have had my eye on AI since it first showed up with the ability to work on code. But being grizzled enough to know that anyone trying to sell you something always starts by telling you the same lie³, I was wary of it from the start. I won’t recapitulate the entire history of LLMs in coding, but suffice to say that the demos got more extravagant over time until it became apparent that, yes, these models can write code. Then once they got sufficiently integrated into common IDEs, it became the work of “yes, turn on X” to get access to them in an extremely convenient form factor. Heavily aided by the extremely strict nature of programming languages, they could be force-fed enough real code to replicate the patterns into new code of their own which can be compiled and will execute.

I do not believe that AI is a positive development for humanity, but that sometimes feels like a minority opinion in tech (I don’t think it is, I think astroturfing is absolutely rampant, but regardless). Especially 12-18 months ago, the hype that “AI will replace coders” was at frothing-mouth, eyes-rolling, possession-spinning-in-place-vomiting-copium frenzy pitch. I had to have answers for my CEO and the board (the company is privately held, but even those still have boards) as to what value AI could give us, because they are not technical people and their LinkedIn timelines were full of non-stop miracle assertions from pundits that (by design) tried to make us feel like we had to get on the AI hype train right this very millisecond or else we’ll become instantly irrelevant⁴. A “nah” from my gut feel wasn’t gonna cut it.

So, I paid the blood money for access to the two largest models, and gave them a test run for a while. I also asked if any of my employees were interested in trying it out, and got feedback from them as well. Together, we used the models for a few programmer-years of time, several months each, in several contexts, to see exactly what they can and can’t do. I interrogated them weekly based on my own findings to see whether they were a common thing or something unique to my workflow.

The Results

I used the tools in as many contexts as I could. I used them for bootstrapping greenfield systems from nothing, for day-to-day debugging, for hands-off “ok do everything for a bit” for new features on existing systems, and for pure planning with no coding involved. They had the context of the codebase, and I used the best models with the biggest contexts. I gave it the benefit of the doubt and the room to prove me wrong.

Nothing I experienced from using the very best these models have to offer made me change my mind. In fact, it made me certain that using these models at scale is an intellectual disaster.

I said earlier that, yes, these models can write code. My testing didn’t disprove that (nor did I expect it to). They can write code. They can write a lot of it, very fast, and it’s at least 90% accurate. Perhaps 95%. Amazing? Well, no. Impressive perhaps, from a technical standpoint, but in many other ways, it’s a worst-case scenario. And not for the poor humble programmer job, as many might think. It’s a disaster in ways that proponents of the systems will choose to ignore (because it’s easy and it serves their purposes) and which consumers of the systems, even those who are perhaps not really fans of it, will not necessarily realise because of how the issues mostly don’t manifest in the code at all, but in the processes and metacognition around coding.

The AIs wrote serviceable code for me most of the time. They excelled (relatively speaking) at the simplest work, where complex requirements could be elided behind a single websocket or abstracted away to “a wizard did it in Rust”. By “excelled” I am not referring to quality; in all respects, the code they produced was unsatisfying complicated and too often solved problems that would never realistically occur. I spent weeks of man-hours redoing its code to meet my own standards. When I applied them to complex problems, they failed. They consistently mis-identified the real issues until I pushed their noses into it like dogs. They would confidently diagnose non-existent issues or randomly pull in ancient context to support assumptions that had no basis. This was the most unusable code, because it operated on the most sensitive portions of the code. The AI can write infinite tests to validate its changes, but that doesn’t mean anything if the change is wrong.

Did they save me time? Sometimes—but always at the cost of comprehension. I had to choose between a full understanding of the change, and it taking less time when I used the machine. Both were never an option together. And of course, quite often the machine was just flat-out slower overall than doing it myself.

So after a few months of this, I have a lot of Thoughts. I’ll discuss my conclusions below.

Build a tower, make no mistakes

“Make no mistakes” is a bit of a meme with AI in tech now, because of how models had to be explicitly told to not fuck up or they’d fuck up. The success rate of that trick is arguable; they still fuck up. And that’s the first key conclusion I came to. AI can write code, 100%. I cannot deny that. But it cannot for the life (hah) of it architect a damn thing.

As an example, consider a toddler. Can a toddler build something? Yes, it can. It can slap some Duplo blocks together. Tada: a build. Would you use that toddler to architect a dam? No, of course not, you’d ask the lowest bidder to do that to make sure it’s as safe as possible. Point being, building something is not the same as engineering it. And programming, the skill, is subsumed into the greater field called software engineering. Not software building.

Sometimes, I just build software. I slap together some stuff to make a quick hack to automate something for me. I don’t put a lot of work into it, and I rely on the incredible power of modern hardware to make it so I don’t have to think about optimising it or making it pretty or maintainable or crash-free or usable by others. It’s just a piece of flint that I slapped against a rock until it had one vaguely sharp edge, because I know I only need to cut something a few times.

But when it matters, then I engineer software. I sit with my tools and my hard-won experience and I knap, knap, knap, carefully crafting something that is worthy and that will make my life easier for years to come, because of all the up-front work I put into making it properly.

Engineering is a skill. It is the skill. We engineer things with our mastery of abstract thought, our mastery of the physical world, our mastery of our own minds and of goals that can stretch even beyond our own lifetimes and of social structures that we value because they are an intrinsic part of our species. An AI has none of that. It cannot; it does not think, or care, or hope, or dream. It is nothing more than a bigger version of a Markov chain generator, a blind engine that shuffles probabilities to vomit out a number. It has no concept of anything. It cannot even conceptualise a concept, because it is not a mind or a person or a soul, it is a rock with some electricity inside of it. The innumerable subtleties that emerge from the confluence of past experience, future desire, and present realities which must be perpetually held in balance to produce an outcome that not only works now, but into the future as well—an engineered solution, in short—is beyond it. And, I believe, always will be.

The fundamental risk here is that if all we do is build software from now on, if we no longer engineer it because we let the machine do the work, even though the machine is shit, just because the machine is fast and we are lazy, then we will build edifices that appear magnificent but collapse in the first storm. The storms take time to show up, they’re not constant, so there’s lots of time for other people to look at the glittering tower you put up so fast, and feel jealous or inadequate or scared of not getting their own time, and end up asking the same machine to make one for them too. And how glorious they will all look, glittering in the sunlight! How masterfully made, with their flying buttresses that almost all connect to something and very rarely clip through each other! How airy their suspiciously under-reinforced foundations!

One storm will put paid to them all regardless, and nobody will know what to do to fix them, because the machine that built them made no plans and kept no notes.

But I will give this to the models: there’s one sort of engineering that they are consistently amazing at. Overengineering. Because the models aren’t deterministic, because they fuck up so consistently, their training heavily incentivises them to generate code that best resembles a Java engineer’s wet dream. Code is exploded into the most verbose, overdecomposed versions of itself wherever possible. It validates to excess. It assumes insane edge cases and pours out hundreds of lines of code to make them work. It doesn’t take long to become an unmaintainable mess—which is fine if your only view of the system is through the AI itself. It can ingest all that code anytime it needs to remind itself what’s going on (with a small failure rate, naturally; it’s not like it wrote that code, it will make it clear that YOU wrote all the garbage it’s now trying to make sense of). In addition, that assumes it doesn’t forget a key thing during a random compaction cycle, and that it doesn’t confuse itself with its own context and quietly solve the wrong problem somewhere, or you run out of tokens for a week and suddenly have a mountain of code you don’t understand at all. And of course, the big companies losing money every second to operate will definitely never go out of business.

The system works! Stop asking questions, everyone! Keep throwing money at Copilot, it’s so worth, wow!

Suffocation of the subconscious

I consider this the most dangerous outcome of using these tools at length. It was one that took me quite a few weeks to make sense of, because the effect was very quickly apparent but nailing down what was going on was not.

At their core, AIs are basically superfast juniors. They’re constantly surfacing things that you have to review because they cannot proceed until you do. They won’t “find something else to do” like a human, until you get to them, they’ll just stop dead⁵. So there’s strong pressure to attend to them as soon as possible, which means you’re either constantly interrupting your own work, or not doing other work at all and just waiting for the AI’s next request. And as the amount of code grows, it easily exceeds the point where a single human can understand it or the changes it makes at a glance. Soon I’m not taking in, nor applying, any of my knowledge. I’m just a meat-switch that the machine activates to be allowed to move on. All of which means two things:

• I cannot properly mentally leverage my expertise. All the mental shortcuts my brain has learned over the decades to be able to look at code and “see” a solution in an energy-efficient manner is rendered moot when I’m not looking at code, I’m only looking at a Markov chain’s idea of a solution. I know the machine isn’t perfect, so I cannot take the final step and offload all decisions to it. I have to engage my brain just enough to read what it tells me, parse it from first principles, and then try to frame it in terms of what I know. And this takes a lot more mental bandwidth than coding does, because coding intrinsically has lots of downtime where you sit and stare and think. You’re never writing code for 8 hours solid a day—but with AI, you could. Not write it yourself, of course, but unblock it doing that. No breaks, no delays. You become the limit, and you know that, which subtly pushes you to not be that limit so that you can “do more”.
• Ego depletion. That amorphous pool of “stuff” that makes it possible to make good decisions is drained so much faster by these things. Which just means you hit decision fatigue that much faster, which in turn means you’re so much more likely to make a bad call or to simply trust anything the AI says because you simply can’t make your 75th decision of the day to tell it what to do.

The result is exhaustion. High-level mental effort is very draining, but so is mundane repetition. AIs trigger both these states alternately, which is just about the worst thing imaginable. After a couple weeks, I was realising I ended my days noticeably more tired than usual, and I couldn’t figure out why if all I’d been doing is approving a machine. It took some deep introspection to realise what the machine was doing to me.

There’s an excellent video about how the hardest problems in coding are best solved by using your subconscious. Letting the smartest part of your brain quietly but efficiently sort through everything you know and synthesise a solution. It takes time, but it is very effective; I personally consider it the source of all my best ideas. But AI subverts that by never letting the subconscious do any work. Firstly, it hoards all the context by making far too much code for a human to reasonably review. Secondly, it keeps you busy with endless little bites of attention to make sure that even when your day does end, you’re too tired to even have the capacity for subconscious work. And the result is that you never really get any time to come up with truly great solutions. You just go with what the machine did because why not. So your solutions gradually, but inevitably, turn to shit. And with it, your ability to even generate solutions yourself.

The mind is a tool like any other, and must be kept sharp. If allowed to dull through underuse, it stops working nearly as well. And let me tell you, it is far quicker than you think for your coding skills to atrophy even with decades of experience. I only did this for a few months and I already don’t like what it did to me. How are people going to look after 5 years of only working through the portal of these machines? They’ll be useless at actual engineering by then. The saying is garbage in, garbage out, but in this case it’s more fitting to say void in, void out.

What if slot machine but it’s your career?

So let’s imagine a pro-AI person has managed to read this far without going to post a snarky message about me on Friendslop or whatever fucking platform people without a brain use. They might reasonably say, in answer to the exhaustion problem: “don’t try to maximise things then”. Let them sit idle for a few hours if needed and focus on whatever you’re doing. But here’s the really insidious thing about the way AI companies dole out their resources: they’ve gamified productivity.

Unless you work for a company that’s fine just shovelling money directly into a fire, you’ll be on a plan that rate limits your access for coding purposes. There are two windows on these plans: a 5 hour one, and a 7-day one. You don’t get unlimited use of the system, you get a certain number of tokens and when either of your windows are used up, you are locked out until it resets. You can get more at any time by paying the actual price of tokens, which is some 10x the heavily subsidised cost of the tokens you get for your base subscription cost (wow, such business, very money). Due to the huge cost jump, nobody’s really doing that all the time if it’s their own money they’re spending.

Those fucking rolling windows of tokens, dovetailed with the feeling of productivity when using them (it’s basically like terminals full of scrolling code in movies), means that when a new tranche of tokens has unlocked, you feel highly incentivised to use them. Because it doesn’t take long to get to the point where you can eat up a full tranche in an hour, easily. Even more so on the new models that are so dangerous expensive that they need to cost 2x as much as the old one. So each new 5-hour block means “yay, I can do the next part of this long task I’m having it do”. But that just feeds the attention problems I previously mentioned, either because your attention is being artificially engaged for a much longer period than is suitable, or because you’re losing deep-hack time on problems to answering the plagiarism-machine’s questions.

Any system whose default state is “oops, all burnout” is a shit system.

Regression to the meaningless

These tools are trained on vast quantities of pre-existing code. Therefore, they naturally produce similar code. But most code is, by definition, pretty average. Which means the code the AIs produce is also pretty average.

And this has some interesting effects. Because it means that if you write some bad code, it can almost always improve it. Which feels incredibly empowering. “Hell yeah, a better version of the thing I wanted”. But it also means that if you write some brilliant code, it will drag it down. It will not surround it with equally brilliant code. AIs do not produce magnificent, Carmack-esque code. They can’t. You can spend ages bullying one into producing one brilliant thing, but at scale it’s gonna be your standard mess of shit. It’s particularly fond of adding in backwards compatibility for things that it itself added anew not ten minutes before, for example. Or of extracting single lines of code into huge functions. It produces, in a word, techslop.

There is nothing to learn from this code. Anyone using AI code as a template for their own coding style is going to fail every tech interview out there. It is bad code. It can work at the technical level and still be bad code. That’s the thing that’s not obvious to people, from what I can tell: they are happy to equate “it works” with “it is good”, all the while slowly filling up their codebases with a time-bomb of code debt, indirection hell, unnecessary layers and bugs so subtle only a machine could have invented them. It would literally be better to have this code, and any features it enables, not exist than to have this techslop infecting your codebases with it’s aggressive mediocrity.

Systems-driven design

I don’t have anything specific to discuss here, I just wanted to give it a mention as I could say: go fuck yourself. We all tried waterfall and it fucking sucked, don’t peddle me your new stepladder in a different colour and try to tell me it’s totally different when I can fucking see you hiding the soppy rag behind your back and wiping down the steps when you think nobody’s looking.

And so

The question that started this article was whether I see lots of new programmers being very AI dependent. The simple answer is: yes. Because they are entering the workforce at a time where the companies that control the world are all pushing the AI button as hard and as fast as possible, because they are fucking desperate to be at the head of the next gold rush that they think they’re about to unlock. If you’re not in tech, I don’t know if I can convey to you how relentlessly it is pushed on programmers. Of course it’s pushed in countless fields, forced into literally any hole that can be envisioned to be even vaguely AI-shaped—but programming is the only one that’s bringing in meaningful money⁶, so that’s where the focus is moving. Every SaaS tool has slapped “for AI” in their hero, every coding tool now tries to cram agents down your throat like they’re all Dwight K. Schrute at a beet-eating competition, and every social medium is awash with people claiming that AI is so good that you have to use it “or else”.

What the “or else” exactly is, is variable, but it’s always, without exception, in the future. I am perpetually reminded of the old joke about fusion being only ten years away—and always will be. We’ve had widespread LLMs for half a decade now, and yet somehow all the magic that was promised has failed to materialise. “Soon”, we’re told. The next model will be the one that’s so good that it’s obvious that it has to be used exclusively. Just like the one before it, and the one before that.

We are all being told that this is inevitable. That is a story. Trust me, I know a good story when I see one, and this is a fucking great one, because it comes with a promise: “believe the story, and your life will become easier”. And who doesn’t want an easier life?

I feel for any juniors wading into this sea of hijacked priorities, because without experience I’m sure it does look like AI is the thing you need to know to get anywhere going forward. But all that’s actually happening is that they are being robbed of the chance to learn anything. Learning is practical. Nobody learns anything by being told a fact; at most, they memorise a thing. Learning happens when you apply knowledge, most especially when you fail and try again, because the dissonance is unpleasant and the act of overcoming it is highly memorable. Watching a machine produce code is not an engineering discipline. It’s a babysitting job. You’re just making sure the immature thing doesn’t suddenly, for no reason you can fathom, do something that endangers itself or others.

For those juniors using or planning to use these tools extensively, I summarise my argument to you as such: if you use these tools, as a junior with at most a few years’ experience in the field, you will produce better code. You will produce code faster. You will probably produce code that survives production better than your own efforts. You believe all of these things are desirable, and in the end, they might be—but I just have one follow-up question.

What then?

Are you going to go and read all the thousands of lines of code the machine spat out and understand them? How are you going to contextualise everything going on? How are you going to know that the approach it took is the best one for the constraints you’re facing? How, in short, are you going to learn to be an engineer?

Writing prompts is not coding, just like writing prompts is not art (hello to any “AI artists” reading this, I hope terrible things happen to you). If you want to be an engineer, you need to learn to solve problems, and you will not learn that by telling a machine what you want for the rest of your career. Be very, very wary of anyone company or person that tells you otherwise. They do not have your best interest at heart; all they want is to maximise your unit of productivity for their own benefit alone, and you are just the disposable vehicle of their own financial satisfaction.

A lot of people in tech, particularly non-engineers like the blood-suckers at the top, think code is the perfect target for their machines because it’s so rigid. They have no fucking idea what coding is, then. Because as both a very experienced coder and writer (lapsed; derogatory), I can tell you that they are far more alike than dissimilar. Code is not written by sitting down and mechanically outputting the commands in order until the end, and then it’s done. Coding is not an analytical process. Engineering is; a good engineer is so deeply comfortable with analysis that he’ll find all twenty problems with something before even considering why none of them may be relevant right now. But coding is a creative activity, just like writing. Both are abstract processes that creates something from nothing in just the right way to communicate a meaning. At a serious level, coding involves long stretches of not touching the keyboard, simply thinking deeply about the problem and looking for the solution that actually works. Sometimes that solution is removing something, not adding it. Sometimes it’s not doing the original idea at all. And AI can’t not do something. AIs cannot be trained on negative cases. They must do something, and that relentless outputting of slop just for the sake of it can obscure the simple truth that the only thing the techbros have is a hammer, so they are doing everything in their power to convince you that everything is a nail.

The magnificent television show “Hacks” (if you’ve not seen it, stop reading this, go watch it all, then come back) has a superb scene that encapsulates exactly why anyone in the creative arts should be opposed to these technologies. Usually that just refers to art, writing, etc…but coding is also creative. Yet tech companies are pelting full-tilt towards an AI-only future, mostly because there’s not enough pushback from tech people. That can change. Don’t ever feel like any of this is inevitable; that’s a lie you’re being fed to make you give up. The fucking Metaverse was also apparently inevitable, and yet we killed it by the simple system of not using something that is shit no matter how much a billionaire lies to us about it. AI, too, has no legs, and we can send it to the same trash heap.

Conclusion

At my company, after the lengthy experiment and discussions with my colleagues, we have opted to ban the use of AI for coding. The “benefits” it offers are too shallow and far too often poison-pilled, let alone the ethical and environmental concerns. The junior employees in particular voiced serious concerns about how quickly they found themselves effectively excised from growing understanding of their own codebases.

It is a wolf in sheep’s clothing, a hungry money-pit that only grows deeper, an intellectual shortcut that erases what it bypasses. It is a clever scam being pushed by greedy psychopaths to make as much money as they can before the entire house of cards inevitably collapses. And nothing of value will be lost.

Epilogue

enter room

You come to a door. It is flanked by two great stone statues, who stare upon you impassively. “One of us always lies,” one intones, “while the other always tells the truth. Ask your question.”

is ai the future?

The statues glance at one another. “OK, so,” the one says, “I’m the one who lies usually, but…nah, man. Just no.”
“What he said,” the other adds. “That shit’s fucked up.”

And now, if you’ll excuse me, I’m going to go write some code.

---

Footnotes

1. And they’re certainly not AGI. If you think they are or might be, I implore you to grow two fucking braincells and them make them fight one another so that you don’t have enough time left over to make further inane statements about shit you clearly don’t understand. ⤴

2. And if you can’t, then this article isn’t for you. Feel free to post your own hot take in the comments below, I will read every single one. ⤴

3. “You definitely need this.” ⤴

4. Preying on ideas of AI that people have internalised from decades of science fiction and hoping it covers up the crevasses that dominate their actual implementations. ⤴

5. “AGENTS WILL—!” Shut the fuck up, you Butlerian traitor. ⤴

6. Still not even close to enough to make this entire shitfest profitable for anyone except Nvidia, though. ⤴