Podcast: Key hurdles in AI from proof of concept to production

What are the key technical hurdles to putting AI into production?

So, what we’ve seen with customers is that they typically start with a very small POC. POCs are very easy to set up. There is a large number of vendors you can select to get something going around AI, but where we see the biggest struggle is when people try to take things from POC or test level into full production. This is where lots of challenges come up for them.

Challenges can be around performance – the systems they choose to go in production just don’t perform [well] enough. There are [also] typically issues around scalability if you test something at small scale.

Things are much easier if you’re going to go to 100s, 1,000s or tens of 1,000s of GPUs like some customers do to get full production scale. You see lots of different issues at large scale, so a key point is to pick a vendor that actually has successful deployed technology at very large scale because, typically, when you go in production, this is where the real scale happens and that’s where the typical problems show up. 

What are the key organisational hurdles to putting AI into production?

If you take AI to production, the organisational hurdles are typically that a lot of different things come together. It is different to IT projects where the consumer or the end user says, “I want to onboard this new application” or “I want to get this additional business suite working for me.”

AI has more requirements [that need] very tight integration of all the infrastructure in terms of hardware as well as ecosystems integration. [It is] different to traditional IT projects – an AI project needs much tighter integration between its various aspects.

You need people from the networking side; people who have something to do with the storage side as well as the compute side of the infrastructure. And then you also typically need the end users, the data scientists or the people who write the application that uses the AI infrastructure, and you need to bring them all together.

So, one of the big organisational hurdles we see is that there are established boundaries within companies where they have segmented areas of the infrastructure on the hardware and software side. For AI projects, it’s absolutely instrumental to bring all these people together on one table in order to have a successful deployment.

How can we summarise the key differences between AI pilot projects and AI in production?

The key thing really is things look much simpler at POC or pilot stage than they actually are in real full-blown production.

So, if you start a project, you should from the beginning figure out what it would look like in full production. [You should] make sure that when you do a pilot it aims towards this, that the architecture is scalable and what you deploy will be able work at a small scale but also at very large scale. 

Is there a technical or operational template that customers can use to ensure they successfully transition to operation in AI projects?

The key is to have a focal point for the project [that can] pull in resources and leads from various areas. If you can’t form one holistic team that does it, you at least need to have somebody who is in charge that organises and brings all the right people to the table.

AI typically touches a lot of different areas of infrastructure. This is not a traditional IT project, and so you need a much tighter integration between the various teams and organisations to have a successful outcome. 

What role in the organisation or in the IT organisation would typically take that on? Or are new roles being formed?

Well, there are clearly some new roles popping up. But what we see is that projects that are the most successful are the ones being driven by business value creation.

So, typically this [would be] a C-level executive sponsor who says, “We want to leverage AI to create real business value.” These are the ones that are typically the most successful because they’re driven from a revenue profitability perspective and that gives it the right focus, the right level of investment and also the right executive sponsorship to ensure it is done with priority.

You can very easily overcome hurdles within organisations or roadblocks that are much harder to solve at a lower level.