bluebay2014 - Fotolia

How ANZ firms are driving automation and AIOps

Tech leaders from Westpac, NAB, Telstra and ACC New Zealand share their automation journeys, from overcoming cultural resistance to the cautious adoption of AI

A fireside chat at the recent Ansible Automates Sydney event gave attendees insights into how major organisations across Australia and New Zealand are using automation, offering valuable lessons from their experiences.

According to Sean Dudding, service owner of patterns automation at Westpac, there was no single factor that sparked Westpac’s automation journey. “We stepped back and looked at how people were interacting with infrastructure, how people were provisioning infrastructure, and why new projects were taking so long to build servers,” he said.

Internal customers, such as project and support teams, were having a poor experience, meaning Westpac’s end-customers weren’t benefiting from new services as quickly as they should.

“That’s when we decided to go on the automation journey,” said Dudding. Fortunately, there were no objections: all the bank’s project and support teams, alongside management, were on board and could see the benefits of automation.

It’s a different story at National Australia Bank (NAB). “One of the biggest things that we've got is fear… You've got a lot of people thinking, ‘if I adopt this, I'm actually going to write myself out of a job’,” said Brett Stevens, the bank’s automation engineering manager.

Stevens and his team invested significant time in convincing staff that they would actually be writing themselves into a job, because failing to embrace this industry trend would ultimately lead to job losses. “In hindsight, it's a bit harsh, but it's basically where we're going,” he said.

The next step was to convince people that their workloads were appropriate for automation and to show them how best to use it to get work done.

“One of the more interesting things is people that say, ‘You write me the code, and I'll just execute it in the order that you asked me for, I don’t want to know about it’. Those people have written themselves out of a job,” he observed.

But it is not all stick and no carrot. “We're all in change management these days and it's a horrible process for most people, [so] how can we do that better?” Doing a workflow that includes some service management alongside the automation of change management is a major draw.

If a process can be automated, should it be? According to Kenny Cheng, senior chapter lead at Telstra, the telco takes a customer-first approach. “What are we automating, and does that improve the customer experience?”

Reaching 70% or 80% automation delivers true value, he noted, because it reduces manual effort and frees the team to serve the customer. There are diminishing returns after that, but if 80% automation means customers are still queuing, there is justification for pushing for 100%.

Infrastructure as code

The Accident Compensation Corporation (ACC) in New Zealand has broadly adopted infrastructure as code and documentation as code. Like many organisations, it had hundreds of documents telling engineers how to build and maintain various systems. These documents were largely written by individuals and were not subject to a formal review process.

Consequently, if a new team started work on an existing system, they often had to debug or reverse-engineer the documentation and carry out testing to determine how the system should be built. “So, we started doing complete infrastructure as code for everything,” said Phillip White, DevOps and automation engineer at ACC.

“We try to do a pull request behind everything that we do, even for documentation, so that the engineers all understand how the platform should run.” That way, they don’t have to maintain Word documents, as the information is embedded in the code.

“When you get fresh people coming in, you can say, ‘Do you know Ansible? Do you know the scripting?’ They can just learn that instead of learning the platform. Through the code, you can actually teach the engineers how to manage the platforms because all they have to do is learn the code.”

Paving the way for AIOps

Asked whether automation with Ansible Automation Platform (AAP) could serve as a foundation for artificial intelligence for IT operations (AIOps), Stevens said: “We're a little early still. I think a lot more adoption of AAP and event-driven [automation] needs to be built first [to] get the foundation absolutely rock solid.”

“We still have a lot of teams that want to do it the old way, so I think with AIOps you need to be a little bit more holistic about your approach,” he added.

Another consideration is policy-as-code, and AAP 2.6 now includes policy-as-code interfacing, Stevens noted. “It's new, it's fresh, but it's well worth investigating.”

His advice was to “get your policy right, get your practices right, get your pipelines right, get everything right first, and then you can start adopting [AIOps] on a small scale.”

While people are talking about a future where AI can autonomously write, deploy and run code, Stevens believes the industry is not there yet. “That's the dream, and I think we're a long way from that... There still has to be a human at this point to say, ‘Yes, this is good, run it’.”

NAB is aiming to automate everything possible with current tools before moving to a stage of allowing changes to be made autonomously. Initially, such changes are likely to focus on predictive maintenance, such as detecting that a memory leak will cause a system failure and taking pre-emptive action.

However, Stevens noted that this is more event-driven automation than true AIOps, suggesting human intervention is still needed to avoid disrupting critical activities like end-of-year processing. “Then later on, we might inject some AI workloads or AIOps logic into those workflows as well.”

Westpac’s Dudding agreed that caution is needed, suggesting a key strategy is to place automation between observability and incident management. Westpac has positioned AAP between its observability tooling and ServiceNow. If there is an automation for a particular event, it is sent via Event-Driven Ansible (EDA). “We've also built in a way that we can query AI via a number of models that we've distributed on our OpenShift AI platform to also interact with that workflow,” he said.

For example, previously, a trouble ticket would be raised when a CPU or memory alert occurred, and an hour might pass before someone looked at it. The first stage of automation was to pass the alert to EDA first, triggering a comprehensive health check of the endpoint device. Within seconds, that detailed information could be added to the newly raised ServiceNow ticket.

Recently, this process was enhanced by sending the health check results to the bank’s AI platform, which generates advice based on the data and appends it to the ticket.

Ticket enrichment is an amazing yet non-intrusive use case, Dudding suggested, calling it “a great place for us to start with AIOps.”

The AI coding wave

However, Dudding is not convinced that AI coding assistants such as Ansible Lightspeed fully democratise automation development.

“We have to look at what a developer does. A lot of people imagine they just write code all day, but they do a lot more than that. They write requirements, they document test cases, they write documentation for the release, and they write documentation for people to consume the code,” he pointed out.

“You really need to integrate AI into all those processes. It's not just about writing the code. A lot of friction is in all that documentation and testing. To get those efficiencies, you have to have AI across the whole spectrum of work,” he added.

That said, such tools do enable people with little coding experience to deliver small pieces of well-defined automation, provided they are given the right guidelines and frameworks.

Enterprise-wide deployment of agentic AI and autonomous automation requires trust, according to Stevens. This involves getting the platform and the intent right, deciding what should be automated, and setting the limits of that automation. After getting comfortable with the necessary AI components, organisations can then move towards agentic and autonomous operations.

Given Telstra’s maturity with Ansible Automation, Cheng was asked where he wants to be in 18 months’ time. Interestingly, he highlighted talent development. Junior and graduate recruits have a great opportunity to learn these new skills, eventually becoming mid-level and senior experts in the future. But it’s not just the technical side: it is important to give these juniors the right mentors to maintain the culture.

The session closed by asking the panellists for their biggest piece of advice for someone starting their automation journey today.

Cheng advised exploring the available automation technologies. This can provide leadership opportunities within a team and open doors for career progression.

Stevens recommended developing a roadmap: decide whether the goal is to automate everything or focus on specific infrastructure types or business processes. Either way, prioritising different projects is important.

White noted the importance of adopting automation in conjunction with other teams, suggesting that interacting with them to understand their pain points is far more beneficial than writing an automation just for yourself.

Dudding chose to focus on the AI side of automation. “With AI and AIOps, be suspicious, be cautious,” he counselled. “Each of the LLMs out there has its own personality. Get to know how it's going to respond. Avoid hallucination. And once you're happy, then go for it.”

Read more about IT in ANZ

Read more on Artificial intelligence, automation and robotics