Enterprise IT risks of AI and agentic AI

Everyone involved in IT needs to be focused on the implications of artificial intelligence (AI) on enterprise cybersecurity. Yet precious little energy is being expended on making sure AI is secure.

The industry is hell-bent on ramming AI into every corner of society: it’s a network effect. The more people hear about AI, the more they feel a need to have it in their own organisations. And there is certainly pressure from the top, where executives are sold on the benefits of AI-powered efficiency gains. Who is worrying about the IT security implications? Ask an industry expert and the canned answer is “guardrails”, but this is surely open to interpretation.

Speaking at the UN’s AI for Good summit, Meredith Whittaker, president of secure message app, Signal, warned about how the security of Signal and other applications would be compromised by agentic AI. It is worth considering how the tech sector positions agentic AI. It is being seen as a helper, an AI-based personal assistant and a tool that automates all manner of workflows and processes.

But consider what it would need to do, as Whittaker points out in her discussion, in order to book and coordinate a restaurant booking with a group of friends. Access to the browser to perform a restaurant search, the user’s calendar and messaging app are all needed to make the process fully automated. The industry recommends keeping a human in the loop for critical decisions, which begs the question what can agentic AI do and what shouldn’t it do.

Best practice in cybersecurity recommends that applications run at the so-called “application layer”, which, from an operating system topology perspective, means they are provided with the least amount of security access needed to run correctly. But AI agents need access to everything; they need to be in “God Mode”. They may indeed run at the application layer, but they require the highest level of security access: they need full access to applications in order to act as an intermediary that is able to automate workflows and processes on the user’s behalf.

Hold that thought….for decades, operating systems have been engineered to offer application isolation, so that one piece of code does not impact any other code running on the same IT infrastructure.

Now along comes agentic AI, and the foundation of application integrity are undermined.

Along with the topic of application security, business and IT leaders should also consider the commercial reality. AI is being pushed to consumers and businesses as a free or comparatively low-cost service, compared to the exhorbitant costs of a home-brew option. As AI providers pool data to deploy increasingly sophisticated models of human behaviour, how will that impact society and non-AI businesses?