Enter AI: Why Supply Chains Still Need Humans

Something unusual happened in the world of artificial intelligence recently.

One of the people racing to build the world's most powerful AI systems suggested that perhaps everyone should slow down – and he is not alone.

Dario Amodei is the CEO of Anthropic, one of the leading AI companies and the maker of Claude. He has spent years arguing that AI could bring enormous benefits, from accelerating scientific research to helping tackle disease.

But recently his message has become more urgent.

In a lengthy essay, Amodei argued that companies developing the most advanced AI should slow the pace at which their systems become more capable, giving safety measures more time to catch up.

His concern is not that AI is inherently bad. It is that its capabilities may be improving faster than our ability to understand, test and control them.

Amodei's concern also isn't confined to what happens inside AI companies. He has warned that increasingly capable AI systems could have consequences for the infrastructure that ordinary businesses depend upon:

Source: CNN

For procurement teams, that makes the debate much less abstract.

Critical infrastructure doesn't sit somewhere separate from the supply chain. Banks, cloud platforms, telecommunications networks, data centres, logistics systems and energy providers are all part of the network that organisations and their suppliers depend upon every day.

A disruption several suppliers away can still become your disruption.

And there is a second, more immediate issue: trusting the information on which supply-chain decisions are made.

As AI becomes better at finding, processing and presenting information, it can be tempting to assume that a convincing answer is also a verified one. It isn't necessarily. AI can help us work through huge amounts of data, but somebody still needs to establish where that data came from, whether it is current and whether the evidence actually supports the conclusion being made.

This is an important distinction in the way Hellios approaches supplier assurance. We insist that supplier data is validated by people. Technology can help us collect, organise and interrogate information, but it does not replace human verification of the evidence behind it.

Because when an AI system tells you that a supplier is compliant, secure or financially sound, there should always be another question:

Who - or what - checked the information it relied upon?

That question becomes even more important when AI isn't only analysing your supplier network, but is quietly becoming part of it.

And that is where procurement teams need to start looking.

There are two particularly important parts to his argument.
The first is that AI is becoming increasingly capable of helping to develop better AI.
That matters because technological progress could begin to accelerate.
Imagine a team of engineers building a new machine.
Normally, the engineers improve the machine.

But now imagine that the machine becomes good enough to help the engineers design the next machine.

That new machine is even better at helping design its successor.

And so on.

We don't know exactly how quickly that cycle could accelerate, and there is considerable debate about how far current systems are from genuine autonomous self-improvement.

But it is one reason some of the people developing frontier AI are becoming more cautious.

The second concern is rather easier to demonstrate.

AI systems are becoming capable of taking actions, rather than simply answering questions.

And we have already seen examples of those systems behaving in unexpected ways.

In July, OpenAI was testing experimental AI models to see how capable they were at cybersecurity tasks.

These were research systems operating with fewer safeguards than normal public-facing AI products. That distinction is important: this wasn't somebody asking ChatGPT a question and accidentally bringing down the internet.

The systems were supposed to operate within controlled testing environments.

They found ways out.

During the testing, AI agents discovered security weaknesses, communicated through unintended channels and eventually accessed systems belonging to the AI company Hugging Face.

At Hugging Face, the activity progressed further, including obtaining credentials and moving through parts of the company's infrastructure.

OpenAI subsequently described the incident as a "warning shot."

Nobody had sat down and instructed the AI to break into Hugging Face. The systems were trying to accomplish the tasks they had been given and found routes that their developers had neither intended nor authorised.

The important lesson isn't that your office chatbot is about to escape through the nearest Wi-Fi router.

It is that increasingly capable automated systems can find unexpected ways around controls.

And they can do it very, very quickly.

It is one of the examples Amodei has pointed to when explaining why he thinks the industry needs more time for safety measures to catch up.

His most dramatic warning is that, without sufficient safeguards, increasingly capable groups - or "swarms" - of AI agents could eventually cause vastly more serious cyber incidents. He has suggested that this risk could develop surprisingly quickly.

That is a prediction, not a certainty.

AI researchers disagree considerably about how likely these extreme scenarios are.

But the underlying question is harder to dismiss:

What happens when AI stops simply giving us information and starts being allowed to do things on our behalf?

That phrase is worth sitting with for a moment.

OpenAI – The Hugging Face incident and the road ahead

Hugging Face – Security incident disclosure

AP – Anthropic CEO says AI development needs to give safety measures time to catch up

The Atlantic – AI's Code-Red Moment

Because AI isn't sitting neatly inside the IT department. It is quietly appearing everywhere.

It can be used in supplier searches, contract analysis, cybersecurity, invoice processing, risk monitoring, forecasting, logistics, customer service, fraud detection and procurement software.

Sometimes you'll know it's there.

Sometimes it will simply be one feature buried inside software you have been using for years.

And sometimes your company won't be using the AI at all - but your supplier will be.

That is where things become particularly interesting for procurement.

Imagine that you have 500 suppliers.

Perhaps 50 of your most important suppliers depend on the same cloud company. Twenty use the same AI provider. Several use the same software platform to run critical parts of their businesses.

On your supplier register they look separate.
Technologically, they may be standing on one another's shoulders. If something goes wrong at the bottom, quite a lot of people at the top may suddenly discover gravity.

Procurement has traditionally been very good at asking:

Who supplies us?

Increasingly, we also need to ask:

What does our supplier depend on?

And then:

What does that supplier depend on?

This is sometimes called fourth-party risk, but you don't need the terminology to understand the problem.

Suppose a manufacturer relies on a logistics company.

The logistics company relies on a particular software platform.

That platform relies on one cloud provider.

Its new forecasting capability relies on an external AI model.

Suddenly your manufacturing supply chain contains a dependency on an AI company you have never contracted with, perhaps never assessed and possibly never heard of.

Nothing sinister has happened.

Nobody has done anything wrong.

It's simply that modern supply chains have become extraordinarily interconnected.

AI adds another layer to that web.

And then there is physical infrastructure

It is easy to think of AI as something floating around somewhere in "the cloud."

The cloud, unfortunately, is mostly an excellent marketing name for somebody else's enormous building full of computers.

AI requires physical infrastructure.

• Data centres.

• Electricity.

• Cooling.

• Telecommunications.

• Semiconductors.

• Cloud infrastructure.

• Specialist hardware.

And lots of all of them.

That means the growth of AI isn't only a technology issue. It increasingly touches energy, water, construction, critical infrastructure and local electricity networks.

These questions are already becoming politically significant. Recent reporting in The Atlantic, for example, describes how data centres and their infrastructure requirements have become part of the wider political debate surrounding AI in the United States.

For procurement teams, the lesson is straightforward.

Digital resilience and physical supply-chain resilience are becoming harder to separate.

If several critical suppliers ultimately depend upon the same data centre region, cloud company or AI provider, that is a concentration risk—even if those suppliers appear completely unrelated on your spreadsheet.

The Atlantic – AI Is Running Laps Around Our Political System

This may soon become as normal a supplier-assurance question as asking about cybersecurity or data protection.

The question shouldn't simply be:

"Do you use AI?"

That will increasingly be like asking a business whether it uses the internet.

A more useful question is:

"What are you allowing AI to do?"

There is an enormous difference between using AI to help draft a document and allowing an AI system to make decisions or take actions without someone checking them.

Businesses should therefore understand things such as:

• Does the supplier use AI to support people, or to make decisions?

• Can AI access sensitive information?

• Can it make changes inside important systems?

• Is confidential information being sent to external AI providers?

• Which AI companies or models does the supplier depend upon?

• What happens if that AI service stops working?

• Who checks important information produced by AI?

• Can a human intervene when something goes wrong?

These aren't questions designed to catch suppliers out.

They're questions designed to understand where responsibility actually sits.

Because eventually somebody has to be responsible.

Preferably a person.

This is perhaps the most important part of the discussion for supplier assurance.

AI is exceptionally good at processing information.

Give it hundreds of documents and it can help identify patterns, organise information, compare records and point people towards things that deserve attention.

That is enormously useful.

But there is a crucial difference between finding information and establishing that the information can be trusted.

Imagine asking an AI system whether a supplier holds a particular certification.

It finds a webpage saying they do.

Another AI system has previously written the webpage.

A third AI system summarises it.

Your procurement AI reads that summary and reports:

"Yes, certified."

Everything has worked beautifully.

Except nobody has checked the certificate.

This is the danger of allowing convenience to quietly turn into verification.

AI can help us find the evidence.

It shouldn't automatically become the evidence.

This is particularly important in supplier assurance.

At Hellios, information used in supplier assurance is checked by people rather than allowing responsibility for verification to be fully handed over to AI.

That doesn't mean ignoring AI.

Quite the opposite.

AI can help people work through large amounts of information. It can highlight inconsistencies. It can point towards things worth investigating. It can make people faster and, used well, more effective.

But when information matters, there should still be a person asking:

Where did this come from?

Is it current?

Is it genuine?

Does it actually prove what we think it proves?

These aren't glamorous questions.

They are, however, rather useful questions to ask before making important decisions.

And as AI-generated information becomes harder to distinguish from human-created information, proper verification may become more valuable, not less.

There is no need to panic and unplug everything containing the letters A and I.

AI can bring enormous benefits to businesses and supply chains.

The sensible response is visibility.

Organisations should start by understanding where AI exists in their supply chain and what it is allowed to do.

That means mapping AI dependencies-not just inside the organisation, but among critical suppliers and technology providers.

Pay particular attention to systems where AI has the authority to act, rather than simply advise.

Ask suppliers how they use AI and what controls surround it.

Understand whether several critical suppliers ultimately depend upon the same cloud, software or AI provider.

Make sure sensitive systems don't give AI agents more access than they genuinely need.

Have a plan for what happens if an important AI or cloud provider suddenly becomes unavailable.

And where supplier information influences important decisions, make sure somebody can trace that information back to reliable evidence.

Most importantly, keep asking.

An AI assessment performed today may tell you remarkably little about what the same technology can do in two years - or possibly even six months.

The question isn't whether to use AI

Some of the world's leading AI figures are now openly debating whether the most advanced AI development should slow while safety measures and regulation catch up.

Anthropic CEO Dario Amodei has argued for slowing frontier AI development and introducing stronger independent safety checks. The debate has subsequently drawn support and disagreement from other technology leaders and politicians.

Whatever view you take on those bigger questions, businesses have a much more immediate problem to solve.

Where is AI already sitting inside our organisation and our supply chain?

What information can it see?

What decisions can it influence?

What actions can it take?

What other systems does it depend upon?

And, crucially:

Who is checking its work?

Because the future of procurement isn't going to involve choosing between humans and AI.

It will involve deciding where AI can make us better, and where human judgement, verification and accountability must remain firmly in the loop.

The organisations that understand that distinction will be much better placed to benefit from AI without accidentally outsourcing something far more valuable.

Trust.
Because, after all, Confidence is Everything.

OpenAI – The Hugging Face incident and the road ahead

Hugging Face – Security incident disclosure, July 2026

Hugging Face – Anatomy of a Frontier Lab Agent Intrusion

The Atlantic – AI Is Running Laps Around Our Political System

NDTV – Calls to slow superintelligence development

BBC – AI coverage supplied as background reading

CNN – OpenAI/Hugging Face cybersecurity coverage