Secure Code Warrior CEO on surviving the AI ‘vulnerability apocalypse’

How can we ensure the quality of code produced using AI tools?

Pieter Danhieux: I don’t think AI is capable today of generating code that is both secure and of high quality. It builds code fast, but the quality of that code, in my opinion, is not there yet. We have some facts to prove that. Secure Code Warrior will soon release a paper that we've written in conjunction with the University of Melbourne where we benchmarked the ability of the frontier models to generate secure code in 16 different languages.

What comes out is that none of them writes flawless code. There are some big differences. Every model has strengths and weaknesses. Some models are better in Java and worse in C++, or vice versa. This extends to different versions of the same model. It’s going to be incredibly confusing for developers selecting models and knowing whether they are going to get secure code or not.

Our plan is to monitor the situation and benchmark every model as they come out, because I want to know when they are generating secure code. For now, we need to rely on humans and the methods we already have for producing secure code.

Does this present a particular problem for citizen developers and vibe coding?

Danhieux: It is a problem for anyone who’s building software, whether they are vibe coders or professional developers. These models generate secure code in some languages and not in others, but nobody tells you: “Be careful because you’re using Java, and the model you’ve selected is not great at producing secure code in that language”. That’s one of the things we’re going to build into our product.

The only thing that’s changing with AI is that we're going to pay Anthropic instead of people to generate code, find any vulnerabilities and then patch them

If the code produced by citizen developers using AI tools such as Claude and Gemini is not secure, it might be possible to deploy it in a way that is secure and fits into a broader architecture. I hope somebody will create technology like the iPhone App Store or the Google Play store where the hardware and the operating system is hardened and you can deploy those apps in containers. But as far as I’m aware, nobody has that technology for enterprises at the moment.

A lot has been said recently about the ability of Claude Mythos to find vulnerabilities in code. Could that capability be added to models that generate code, so that they detect and fix vulnerabilities before they release the code they are working on?

Danhieux: I don’t have access to Mythos so I can’t really speak about that. In the past, we paid people to develop software, we paid people to find holes in it, and then we paid people to fix those holes. We paid three times, but the software should have been secure in the first place. The only thing that’s changing with AI is that we're going to pay Anthropic instead of people to generate code, find any vulnerabilities and then patch them. We’re replacing paying humans with paying a vendor. But the models that are generating this code will not generate secure code from the start.

Mythos, from what I have read, is great at finding vulnerabilities and can also fix them. That means we’re going to be faced with a storm of vulnerabilities that are going be detected. Organisations like the Commonwealth Bank have access to that model through their partners and they’re quickly patching all their systems. But what about all the other businesses in Australia? Anthropic has now expanded access to Australia organisations, but we don’t know anyone who has access.

I don't really understand why this is being limited to certain banks and not the rest of the economy, because I think the rest are going to suffer when that model comes out, and models are already being leaked and copied. I suspect within the next three to six months, that vulnerability apocalypse is going to begin. I told our product security guy, “we need to stop thinking about ‘let's patch this vulnerability within 24 hours or 48 hours,’ we need to build a system that can patch it within minutes.”

So, they're trying to figure out how can we use AI to learn about vulnerabilities and get a patch into production within minutes? We’ll have to use AI because no human or no team will be able to work at that speed. We’ll have to use AI for defence.

It’s great that they have systems like Mythos, but we need a lot more focus on reacting, defending and helping developers build and patch these things, even in legacy code, because some of this code will be in Cobol, Java, and very old languages.

Last year, you released AI Security Rules for free on GitHub. How’s that going?

Danhieux: The idea is to provide some rules that can be applied to weaker models to bring the security of the code they output up to the level of more expensive models. The rules we released last year were an experiment for us and we’re using the data we have collected to target specific models generating specific languages. For a particular model, we might have five rules to apply when generating Java, so the output is more secure. That’s not in the product today, but we hope to have that by the end of the year.

The reason for having a small number of rules is that the more context you provide a model, the more it ignores some of that context. If you include 300 rules, it might ignore 100 of them. You can't take every single security rule and give it to the model – you need to be specific. So, we plan to have rules that specifically patch the gaps we've identified in particular models.

Some people talk about firing most of their developers, but I think that if you have a thousand developers using agentic AI, why would you fire half of them and slow your speed of development?

Then there’s the SCW Trust Agent that you released a few months ago. What role does that play?

Danhieux: I predict that some models will be compromised. Whether they are Chinese models or European models or whatever, at some point in time, we’re going to discover that a model has been compromised for months. Then every organisation will be asking, did we use this model? Where was it used? What parts of our code have been influenced by it? How do you know that nobody used it?

That's the capability we're providing with Trust Agent: you can go back in time and see, for example, that DeepSeek was used by these agents or these developers, and it was used on these applications and these lines of code so there's full traceability.

Additionally, it goes a step further by providing visibility of your AI tooling. Are they AI-assisted? Are they AI-native? Are they agentic? Are they using Claude Code or Google Antigravity? Which MCP [model context protocol] servers are being used? Trust Agent maps all that out, then you can have a governance policy in place to stop tools and models from being used where you do not consider them fit for purpose. Trust Agent provides visibility and governance for software development.

If you look at the last 60 days, we’ve had two or three massive breaches in the AI ecosystem that nobody saw coming. GitHub was one of them, and that’s a company owned by Microsoft that takes security seriously. We see different types of automated attacks compared with a year ago, and that's going to accelerate and get worse until we get hold of this ecosystem and put the right visibility and controls in place.

What short-term advice would you give people responsible for software development within their organisation?

Danhieux: Move fast with AI, but don’t move too fast. You need to adopt it in a controlled way and not give everyone the freedom to use whatever vibe coding tool they want to write code and publish it as fast as possible to production. Unfortunately, I see a lot of my peer CEOs telling their people to just go for it because we need to reinvent ourselves, and to publish code as fast as possible to show the market that we are on the AI train. That is risky.

Do you think developers will still exist in a couple of years’ time?

Danhieux: When I use AI in my life, it gets me 80% of the way there, and I’m a fairly technical person. For that last 20%, I have to keep debugging, fixing and re-running. I think that must be the same in software development. It can get you to 80% very quickly, but to make sure that it works fully developers have to do debugging and problem solving and fitting it into the right architecture.

So, I do think developers will still exist. Some people talk about firing most of their developers, but I think that if you have a thousand developers using agentic AI, why would you fire half of them and slow your speed of development? I think there are other economic drivers for firing developers rather than replacing them with by AI. I'm convinced developers will continue to exist but probably doing different things than they did in the past.