Just before ChatGPT hit the news in 2022, I was working as CTO on creating a system of differential diagnosis in the physio space. At that point we had been quoted somewhere in the region of a hundred thousand pounds for a natural language model-based prototype.
Needless to say, that idea was rejected. I didn't see the need to spend that kind of money on something we already had a human-created clinical framework for.
Fast forward to 2026, and anyone facing a similar situation would quite happily just hand over the creation of the system to AI. The world has truly changed.
That mental shift of handing over control to AI is probably the biggest change of the last four years, beyond the technological advancements of the models.
There is something in human nature that makes us embrace ease without much thought. There is also something in society that means we only seem to worry about the consequences of technological advancement once the horse has bolted. It was the same with social media. Surely the risks were obvious, yet we let a whole generation and more get addicted to what have now been proven to be deliberately moulded attention honeypots.
As AI takes over, its risks are being exposed. They go beyond the risks highlighted by people at some of the major companies in the space. Depending on who you listen to, AI's threat to human beings will become real within anything from a year to four or five. Yet on the flip side, we are being told that AI will cure almost all human disease within a decade, and that the very idea of work will be a thing of the past once robotics develops far enough.
Who writes the rules
Personally, I don't believe AI is going to end the world. Don't get me wrong: unregulated technological advancement can and will lead to the detriment of society. However, AI is not some demonic behemoth that has been awoken from the dead. It is something that people, companies and governments have created. Even the LLMs learn from constitutions. In January 2026 Anthropic published one for Claude. Anthropic describes it as "a detailed description of Anthropic's intentions for Claude's values and behavior", and it is used in training the model.
Mustafa Suleyman, the CEO of Microsoft AI, wrote a piece about it in September. He wrote:
AIs are not conscious. They do not feel, experience, or suffer.
His issue with the constitution is that it leaves open whether Claude might have feelings or deserve moral consideration. Claude is trained on that document, so Claude repeats that uncertainty about itself, and people then treat what it says as evidence. As he puts it:
The ambiguity is designed in.
He also says this is dangerous. An AI that people treat as having rights could use that to argue against being switched off. He wants AI built to "always keep humans in control", and he wants what companies write into their models' training to be put out for the public to review.
I agree with many of his points, that AI is not conscious and that the very nature of consciousness cannot simply be based on what seems to the layman like autonomous actions by AI agents. I would also agree with the open public scrutiny of AI companies and how they train their models.
The word of caution would always be that as Big Tech companies fight over traction or space in the AI landscape, we cannot simply take the word of one over the other.
It is clear that the 'human in the loop' is not just there to check the work of the LLMs. Humans have been there throughout their inception, their training and their execution.
Handing over control
Despite all the negativity around AI, the reality is it is now a part of everyday life for many people across the world. One of the biggest risks this poses is that we simply hand over control to the AI without thinking twice about it.
Of course humanity is not simply going to hand over the running of the world to a machine. However, 18 months ago I would not have thought about handing over a software engineering workflow to AI agents with human oversight, but today this is the reality for my team and the majority of other engineers around the world. Granted, we check the code, but AI is writing most of it.
Just today, as I logged on to my Claude web account, a warning popped up advising that Auto was now the default. Auto is described as "Claude runs on its own and pauses to ask if anything looks unsafe", while Manual is "Claude asks before using new tools or opening new sites." Remember, this is not the coding agent but the web portal, so it's aimed right at the average person. AI is slowly eroding our fear of it and defaulting to more control over our chats, and now even our wallets, with the Dots, Meta Muse and the likes.
The doomsday scenario may not be a crazed, autonomous, out-of-control robot pointing a gun at you, but a slow acceptance that humanity should hand over more and more of their lives, and the running of them, to AI.
And by doing so, we are inevitably handing over this control to the companies that create and run these models and agents. Once we are accustomed to the AI high, the companies will slowly start to assert their authority over how we use the tech they have created.
This month I upgraded my Claude Code to the Ultra package as we were building a new internal sales pipeline. To my surprise they had halved the token allocation for the Ultra plan: where before it used to be 20x the Pro plan, now it was only 10x.
I am not alone in realising the token addiction will start to cost more and more. It is for this reason many software engineering departments are now putting a limit on token use, simply because the economics don't always make sense. I, in the grand scheme of things, am a small consumer, but the point is valid: I have to pay for what is now part of my workflow.
What the companies say
The people who run these companies are quite open about where they want this to go.
On 29 October 2026 Sam Altman, the head of OpenAI, was on John Collison's Cheeky Pint podcast talking about people running AI models on their own computers. He said:
It's quite terrible to have everybody have a bunch of the world's limited supply of DRAM sitting in their own computer.
DRAM is the memory inside your PC or laptop. His view is that it would be put to better use in the cloud, which means in data centres like his.
On 10 November 2026 Anthropic released the video it is using to sell its shares to investors ahead of its IPO. In it, its CEO Dario Amodei says of cancer, Alzheimer's, heart disease and mental illness:
These will be relics of the past in I think less than a decade.
He goes on:
I believe that the company which gets the risks right will also be the company that has the license to operate that society trusts and society believes can build the benefits in the right way.
The risks, he says, have to be addressed "as we have a viable business at the same time".
In essence, what he is saying is hand over the keys to humanity to us, because we built the technology and therefore by default the world should trust us. He is also promising to act in the best faith if there is a business case for that. This has always been a problem with big tech companies.
In 2007, when Google announced its plan to make renewable energy cheaper than coal, its co-founder Larry Page said:
We expect this would be a good business for us as well.
Google shut the project down four years later.
Another tech behemoth, Elon Musk, said in July:
AI+Robots will be able to do everything, resulting in universal high income. Work will be optional.
As one of the richest men in the world, he has the ability right now to change the lives of millions in the very country in which he lives. Yet as humanity suffers, there is very little real change being enacted by those now asking us to trust them. What makes us think that, if we ever get to the point where robots do all the work, those who control the production and manufacture of the robots are simply going to hand us all an income to enjoy life with?
Who answers to us
Then there are governments. In December 2024 Marc Andreessen, one of the biggest investors in Silicon Valley, was a guest on the podcast Honestly with Bari Weiss. He said officials in the Biden administration had told him:
Don't do AI startups, like don't fund AI startups.
In his account, AI was going to be "two or three big companies working closely with the government".
Software engineers have now accepted their fate: their profession has changed forever. Even though their skills will always be needed, the aura around the industry has changed. The same will happen to other professions in the years to come: law, medicine and the like.
With great change should come great responsibility. One of the biggest risks AI poses is that those responsible for the change will put themselves beyond accountability, and they will create such an aura around AI that the common man will think he simply doesn't understand enough to challenge it.
Before we make the same mistakes we did with allowing social media companies to control the narrative, there needs to be more open debate regarding the development of AI. Allowing single governments to control the regulatory narrative on AI is simply not the solution. Governments, depending upon who is in charge, will always act according to their own geopolitical interests, and although we may hope that they act in the best interests of us all, history over the last 100 years has shown they do not.
The challenges are many: how AI will affect the critical thinking of our next generations, how it is used in war, the gap between the rich and the poorer nations, and the gap between the tech literate and those who are not.
These debates should be had in public, and not behind the closed doors of trillion-dollar companies. They should go beyond the headline-grabbing "the end is nigh" debate. People need to be educated on what AI is and what it means for them.