For a real-time payments disruption that occurred in October, the Reserve Bank of Australia has released a post-incident report.
Hundreds of thousands of payments were delayed during the outage, which lasted from 7 p.m. on October 12 until the early hours of October 13, and some of them took more than five days to clear.
The bank said a “operational issue” occurred when the software that provisions its virtual servers underwent a modification.
The study noted that “control deficiencies connected with the virtual server solution design contributed to the rapid propagation of the problem,” which in turn “exacerbated the scope of the outage due to a failure to comply with the RBA’s technological change management policy.”
Some systems continued to function thanks to the Reserve Bank Information Transfer System’s (RITS) and the Fast Settlement Service’s (FSS) redundancy features, but others were rendered inoperable, and the system’s resilience was significantly compromised.”
According to the study, “the volume and haphazard pattern of interruption greatly hindered the incident response.”
The audit stated that there is insufficient real-time monitoring of the execution of FSS transactions, adding that “it took the RBA too long to establish the magnitude of settlement aborts occurring.”
According to the study, it would have been more “timely” and “assertive” to restart a component in charge of FSS settlement notifications.
The visibility issue was raised once more: “A more constructive approach to system recovery may have been adopted earlier if additional information on the severity of the damage had been available.”
The FSS [payment gateways] may have self-suspended to stop NPP payments, obviating the need to coordinate subsequent retries, manage aborts, and perform manual reconciliation.
