■ Vibe coding is no threat to humanity — yet

ChatGPT and assorted Large Language Models (LLMs) aren't just detrimental to school homework assignments. If we are to believe the hype, automated coding agents will deprecate us programmers too. We will join the ranks of horse saddle-makers, sea-sponge harvesters, human computers and the like obsolete trades. Being a conservative person by nature I side with sensible voices that dismiss vibe coding as a trendy folly. However facing extinction isn't a matter to be left to opinion, especially with children's education to plan for the future. It must be tested.

My experience with generic LLMs (Copilot etc) is that whenever you ask them something about programming, unless they can find a github project with the answer, they reply quickly, "confidently" but wrongly — for any "difficult" programming question at least. Of course there are specialized coding agents like Opus and Claude (even free ones like google Gemini) that have many supporters including my college-era friend Dennis. He is excited with AI coding tools, praising the increased productivity they offered him in real life programming. So I gave him a simple problem to play with.

target tab behavior

For the latest xplorer² version 6.2 I added a dummy [+] tab that when clicked opens a new folder tab. I am programming in C++/WTL and the custom tab control I am using is a typical piece of source code that results after 20 years of tinkering: working but rather messy. I added a new extended style for the management of this special tab, and the whole revision took 75 new lines of code (out of 2615 lines in the header file).

For Google Gemini CLI (v2.5) I made a simple WTL project, using a modified tab control header removing some xplorer² "complications", that just creates empty text editor tabbed views. You can download the project to see for yourself:

Click to download tab control project (400KB)

Visual Studio 2010 project files, including WTL9.1 if required

Many of the vibe evangelists "succeed" with completely new projects, built from scratch using AI. A more realistic trial must test the AI against modifying existing code, and that's what this experiment is all about. I say the task requested from Gemini was quite simple (it took 2% code modifications when I did it manually). Only one control is affected without any complex cross interactions. Walk in the park? This is what we asked from Gemini CLI:

I have a custom tab control written in WTL/ATL (C++). I want to add a dummy [+] tab at the right hand side of the control, which when clicked will open a new tab, like any internet browser. The plus character should be painted with a bigger font. Be careful of drag-drop rearranging the tab buttons; the dummy button should stay last. Also normal buttons support a context menu when right clicked, but the dummy [+] button shouldn't show anything.

Note this context includes the objective and some points that should be taken into account like drag/drop and context menu, that shouldn't apply for the dummy button. We wanted to aid the AI to succeed.

Google Gemini came back with a detailed analysis and code change recommendations. You can read the details in TABBEDAI.DOC file in the source code package above. All LLMs excel at talking, but I was amazed by how precise the actual code changes were too! It aped the coding style and even named variables similar to what I did (e.g. added a state flag called TCIS_PLUSBUTTON with the correct value 0x4000 (the next available state after those already defined). It created a new font for the big + button as required and initialized it and used it correctly (both for calculating the tab rectangles and painting them). This was an experienced programmer at work! We are all doomed! <g>

The initial code spurt was the closest we got to the target though. It all went downhill from that point. As we passed more feedback about glitches and small errors, each successive round of code updates did more harm than good. If it was a human I would say that "he" lacked concentration and focus and reacted to our latest remark without considering the previous history. "He" was also behaving greedily, going for the quick fix without considering the bigger picture. This is a tab control object whose internal state and methods must stay consistent under the added feature. For example the method GetItemCount that returns the number of tabs needed to be reconsidered; sometimes it should return all the tabs (including the dummy), sometimes only "real" tabs. The AI couldn't get the overview correctly and half of the uses of GetItemCount were wrong (just as if it tossed a coin to decide). Close but no cigar.

If you want to peer into the source code project (above), the main file modified is CoolTabCtrl.h, that has the following versions (use a diff tool to compare them). Search for "@@@" for some inline comments I added to the source code, remarking about good and bad AI modifications.

CoolTabCtrl-original.h is the original tab control code (mine).
CoolTabCtrl-intermediate.h contains some of the accepted modifications, that sort of half-worked but with several glitches (e.g. the + button was placed first instead of last, text not aligned properly etc)
CoolTabCtrl-final.h is the last set of recommendations trying to solve the problem of the dummy being last, but that went haywire, with wrong code, syntax errors, even hallucinations and senseless destruction of good code. That's when we called it a day.

Conclusions

We are experiencing a paradigm shift. The LLM revolution is pervasive. With a few billion adjustable parameters, neural networks acquire an apparent logic and intelligence spontaneously. This is amazing, but it isn't our intelligence, it is just a glorified parrot that sees eyes where there are none to find. It doesn't understand anything. It can get away with easy problems but crumbles under complexity. Everyone could build a shed but you need an architect to make a skyscraper.

Testing the current state of the art in AI coding agents (Google Gemini 2.5 pro) with a real world source code modification (relatively simple and self-contained), we can say that human programmers aren't going anywhere soon. AI is at best a bright junior programmer that needs constant supervision. Here are the main issues:

It needs detailed guidance (context). It isn't enough to say "give me a dummy tab to add new tabs". One needs to point out all the things that can go wrong and need to be changed in minute detail. Otherwise the AI just greedily "corrects" the minimum amount of lines without considering the overall object state.
One step forward, two steps back. If it doesn't get it right first time (a virtual impossibility), it gets worse with each round of revisions. One needs to monitor closely the alterations to check for subtle logical and syntax errors. You can't get a goat-herder to guide the AI, only an experienced programmer can do it.

I can see how a programmer can be assisted by coding agents to rapidly develop completely new apps or add completely new (unconnected) features to existing programs, under careful supervision. Independent and unsupervised AI programming is a distant fiction that will only generate cars with 3 wheels. Perhaps vibe coding is just for simple JS websites; programming involves too many interconnected details.

Remember how once we thought chess was an impregnable human intelligence fortress? Will AI eventually eliminate computer programmers like it did with chess players? Not anytime soon, but possibly. As for Vanessa's recommended academic planning... I am not so sure :(

ps. A big thanks to my friend Dennis for executing the above experiment on Gemini.

Any vibe coders that have access to more advanced paid AI tools like Claude, that feel they can do better in this code modification challenge, please try it out and tell us of the outcome.