monsitj - Fotolia

Unprecedented hardware problem brings down Australian Stock Exchange

Financial regulator launches probe after hardware failure in main database led to problems for stock exchange market system

The Australian Stock Exchange (ASX) said an “unprecedented” hardware problem caused the outage that forced the exchange to open late and shut down trading early on Monday 19 September.

The ASX’s early closure drew widespread criticism from brokers and has resulted in an investigation by the financial regulator, the Australian Securities and Investments Commission (ASIC).

ASX managing director and CEO Dominic Stevens issued a statement on Friday 23 September apologising for the incident and tabled a report about it.

According to the report, a hardware failure in the main database used to operate the ASX market began the sequence of events that led to a number of market interruptions.

Normally the hardware problem would have failed over to the ASX disaster recovery site (DRS) to avoid interrupting normal operations.

But the 19 September hardware failure was “unprecedented”, according to the ASX, and caused the entire database to fail over to the DRS.

But not all the parts of the trading system connected to the replicated database at the DRS and stayed connected to the original database, thinking it was working correctly.

Action to find and rectify the problem caused the market to open late at 11.30am. At 1.42 pm, the ASX found a reference data inconsistency across a range of securities that had been caused by the initial hardware failure and improper rollover.

The ASX then shut down trading and the market did not reopen again that day.

Read more about IT in stock exchanges

“The ASX does not decide to close the market lightly,” said Stevens, who stood by the ASX’s decision to shut down the exchange.

“Understandably, concerns have been raised on various fronts. However, we make no apologies for following our long-established and well understood protocols, or for exercising our judgement to ensure the market was fair, orderly and transparent.

“The outage to ASX trade was unacceptable to the market and unacceptable to ASX. I again offer my apologies for the disruption caused.”

Stevens highlighted the ASX’s previous reliability record, pointing out that in seven of the previous 10 years, the exchange had experienced 100% system uptime, with 99.7% uptime in each of the other three years.

“We are also co-operating closely with ASIC to help with its review,” he said.

The ASX has been undergoing a $50m technology transformation, using the US Nasdaq firm for its clearing platforms and risk management system, Swedish firm Cinnober Financial Technology for its equity and derivatives trading platform, and US-based Tibco for middleware in the market monitoring system.

Stevens’ report also noted a period of “not orderly” trading when the market opened for securities in the alphabet range N-R, known as Group 4 securities. Those securities opened 20 minutes ahead of the announced 11.30am late opening on 19 September due to an operator error. Trades went through for one minute before that group was shut down and all those trades were later cancelled.

The ASX reopened on Tuesday 20 September and has operated normally since. .............................................................................................

Read more on IT for financial services