July 25, 2012 7:35 pm
.
Banking: Finance’s fifth column
.
The RBS software meltdown is a wake-up call for a sector that has long neglected its IT systems
.
©FT Graphic: Lloyd ThatcherA month after a disastrous IT failure halted the routine processing of payments at Royal Bank of Scotland, a few thousand of its Irish customers are still experiencing problems.
Message boards are littered with irate Ulster Bank account holders complaining that their pay cheques have not appeared, pushing them further into overdraft, while withdrawals have often been charged twice.
They are just a tiny proportion of the millions of customers across RBS’s three main brands – RBS, NatWest and Ulster – plunged into chaos after a supposedly straightforward software update malfunctioned, triggering one of the most devastating systems failures in banking history.
After upgrading software that processes payments, RBS’s system crashed, leaving it with a rapidly snowballing backlog of transactions. The bank says most of the problems are now fixed, but the debacle has been a wake-up call for lenders around the world, many of which are running dated, clunky systems that are hard to upgrade – and for regulators for whom IT has long been a low priority.
At a time when goodwill towards the industry has been shattered by a string of scandals – from the rigging of borrowing rates to the £8bn bill for mis-selling loan insurance in the UK – a computer glitch in Edinburgh has revealed another potentially cataclysmic risk to the global banking system.
“If people can’t access their money, the ripple effect for consumers and businesses is ginormous,” says a former UK industry regulator. “It’s not just the bank’s customers who are affected – but who they employ and trade with. It can quickly send the economy into a tailspin.”
The problem for many of the biggest banks, in the US and Europe as well as in the UK, is that they have expanded rapidly in the past few decades, largely through acquisitions. Many opted to bolt new IT systems on to existing platforms rather than the more laborious – and potentially disruptive – process of transferring new customers on to their existing network. The legacy is a complex web of thousands of programmes linked by fragile connections.
“Banks have layered systems on systems on systems, and they have not had time to look back and see how they all work together,” says Al-Noor Ramji of Misys, the UK banking software company, who previously worked at Credit Suisse First Boston and Dresdner banks.
For RBS, which is 82 per cent owned by UK taxpayers, the debacle is expected to result in a hefty fine – potentially tens of millions of pounds – and a large compensation bill. It has stressed that no one affected – whether or not they are a customer – will be out of pocket as a result of the problems.
.
For the industry as a whole, it has laid bare the consequences of a decades-long lack of investment in IT for basic retail activities, a problem compounded by a focus on more profitable investment banking activities in the run-up to the crisis.
As banking grows more sophisticated – and lenders from emerging markets with more reliable, modern systems threaten to upstage western rivals – it is increasingly clear that European banks will have to fix their technology, and fast. Indeed, some have already spotted the opportunities presented by upgrading.
But while the RBS meltdown was confined to the British Isles, the ramifications will be felt worldwide. Among myriad problems, from tougher regulation and weaker profits to sluggish economic growth, the effectiveness of IT systems is about to take centre stage.
“If banks don’t upgrade within the next decade, they will see things start to go increasingly wrong,” says Dan Mayo of Ovum, a technology consultancy. With a full overhaul likely to cost billions, however, institutions are reluctant to take the plunge before they really have to.
Big European and US banks typically built their systems in the 1970s or early 1980s, when most customers were satisfied with a local branch and a cheque book. But in the past 40 years the landscape has changed dramatically – first with the introduction of cash machines in the 1980s, then telephone and internet banking in the 1990s and early 2000s and, more recently, mobile applications that allow customers to access accounts any time, anywhere.
The chief IT architect at one of the UK’s biggest banks compares this gradual evolution to extending a two-bedroom house into a 12-bedroom mansion without altering the foundations. “They have thought about knocking [their networks] down and starting again – but they can’t do it,” he says. “The time, risk and complexity of rebuilding is too much. Instead banks have invested in modernising the ‘house’ – making sure it is as resilient as possible.”
But the original networks are buckling under the increasing volume and complexity of payments. “Banks are notorious for having antiquated, brittle systems tied to more modern systems,” says Bradley Wood, co-founder of GreySpark Partners, a London-based capital markets consultancy.
.
José María Fuster, global head of IT at Spanish bank Santander, describes this jumbling of old and new as the “spaghetti phenomenon”. “This is very common in the banking industry and presents a huge inefficiency,” he says.
To make matters worse, many of the people who know the systems well have retired or been made redundant, while lenders have increasingly moved IT functions to cheaper offshore locations or external companies.
“In many cases it is a workforce issue, with banks having a relatively small pool of people who really know the nuts and bolts of these systems,” says Mr Mayo of Ovum. Like other big global lenders, RBS has pushed through sweeping cost cuts since the financial crisis, which triggered a £45bn bailout. In the wake of the meltdown, it is investigating whether these cuts jeopardised its systems.
“When banks are pressed on cost, an obvious area to cut is IT,” says Mr Wood at GreySpark. “They would rather spend on activities that have a material contribution to their top line.” He says severe outages are rare – there have been cases of customers being unable to withdraw money or use internet services for a few hours but no one is aware of failures of a similar magnitude to that at RBS.
As a result, he says, some have become complacent. “It is perfectly possible to build systems that don’t fail,” adds Mr Wood. “Air traffic control systems don’t fail – neither do life support machines. There is a spectrum of risk versus investment that needs to be weighed up, and banks have been cutting corners.”
Mr Ramji says the average large bank would spend up to 7 to 10 per cent of turnover on IT systems annually. He believes RBS was at the low end of that range.
The bank says it has spent £1.5bn each year for the past two years on IT. However, senior RBS executives admit the retail business suffered under-investment in the years leading up to the credit crunch, when the former management under Fred Goodwin, chief executive, was fixated on expanding the investment bank.
.
Other banks have paid closer attention to their technology. For example, Santander, which has grown rapidly in recent years, uses one system in nine of the 10 main countries in which it operates. “If, when we launch a new product, it’s a trade-off between time to market or operational risk, risk is more important to us,” says Mr Fuster. However, the bank has suffered service disruption in new markets such as the UK.
Many established western banks face the danger of being leapfrogged. In emerging markets such as Kenya, unified systems are being built from scratch using the latest technology. Smaller institutions such as the UK’s Nationwide Building Society and the Co-operative Bank are building more reliable software and could soon start taking business.
For bigger global banks, often with up to 20m retail customers, a full upgrade is a vast project. One IT designer at a big European lender says it could easily take three years and cost billions – so reluctance is unsurprising.
Nevertheless some experts expect the next five to 10 years to be a tipping point for banks to switch to better platforms. Some institutions are starting to acknowledge that slicker systems could be a “differentiating and cost-reducing factor” as profit margins contract, according to a senior retail banker at one of Europe’s largest lenders.
Deutsche Bank is investing €1bn a year until 2015 in a platform that will replace the disparate systems it inherited after buying Germany’s Postbank a few years ago. It says the new infrastructure has slashed costs by €200m this year already. Santander says it spends about €1.2bn a year on IT – but benefits from savings of double that because its systems are simpler to use and maintain.
Generally, specialists say global regulators have taken a hands-off approach to policing and auditing day-to-day IT procedures. The UK’s Financial Services Authority has never fined a bank for a specific IT failure. The closest it came was a series of large penalties on lenders including Barclays, Credit Suisse and Société Générale for failing to provide accurate reports on daily transactions, largely due to weaknesses in their systems.
The RBS incident will heighten its focus. “The worst thing they can do is find the problem but not put further controls in place to make sure it doesn’t happen again,” says a former regulator.
The extra demands could hardly come at a worse time for the industry. It is having to adapt to expensive regulatory reforms. Customer goodwill has ebbed away following recent scandals. Most institutions are likely to be wary of jeopardising their reputations further by embarking on ambitious IT upgrades unlikely to succeed without at least a few glitches.
Aware that the reputational damage of another meltdown would be devastating, Stephen Hester, RBS chief executive, recently wrote in a letter to MPs that a “substantial programme of activity will be implemented to reduce the potential of comparable incidents recurring”.
While it is unsurprising that some RBS customers are tempted to move their accounts, they may find they are just as vulnerable at rival banks.
“Once banks catch a bullet, they tend to respond well,” says Paul Mee, a partner at Oliver Wyman, the global consultancy. “The problem is that the bullets are different each time.”
Additional reporting by Daniel Schäfer
Last month’s computer problems at Royal Bank of Scotland plunged the company into a Kafkaesque tangle of unmanageable bureaucracy, writes Sharlene Goff.
RBS said the initial problem was rectified within a couple of days but it lost track of which payments had been processed and had to draft in a team to check manually. With a backlog of about 20m transactions building up every day, it was drowning. At one point the bank had 100m unprocessed payments.
These difficulties had started with a fairly minor change. On Sunday June 17 the bank launched a routine upgrade of the “batch processing” software that allows it to process the millions of ordinary payments and cash withdrawals that customers make everyday. The following night when its system started working through that day’s transactions it became clear something was wrong. The process was slow, eating through too much memory. So 24 hours later, RBS attempted to unwind the upgrade – and the technology collapsed.
IT specialists who know RBS’s systems well believe the glitch was probably the result of a simple human error – the wrong button being pressed at the wrong time. They have questioned whether RBS had the appropriate disciplines in place to ensure actions were properly supervised and approved by senior IT staff.
“When things went wrong there was no single person that could take responsibility for restarting the system,” says Al-Noor Ramji, general manager of Misys’ banking business.
The bank is due to provide a full account of what happened to regulators in the coming weeks. The extent of the operational failings – and the fact that this is one of the first incidents to occur under tough new rules that link penalties to the affected business’s revenues – mean it is likely to receive a hefty fine that could run into tens of millions of pounds.
Such severe outages are rare at banks, although a number have suffered less extreme IT failures, largely as a result of bedding down acquisitions. Santander encountered problems when it integrated Abbey, the UK building society, for example. Bank of Scotland customers also suffered disruption when they joined the Lloyds TSB platform last year.
Other industries – particularly telecoms companies – are finding they have to make amends for their technical failings.
O2, the mobile phone operator owned by Spain’s Telefónica, was this month forced to give almost 8m UK customers several days of free service after a network failure.
Earlier this year France Telecom’s Orange gave users a free day of phone use and cinema tickets following a nine-hour network failure.
The RBS incident will heighten its focus. “The worst thing they can do is find the problem but not put further controls in place to make sure it doesn’t happen again,” says a former regulator.
The extra demands could hardly come at a worse time for the industry. It is having to adapt to expensive regulatory reforms. Customer goodwill has ebbed away following recent scandals. Most institutions are likely to be wary of jeopardising their reputations further by embarking on ambitious IT upgrades unlikely to succeed without at least a few glitches.
Aware that the reputational damage of another meltdown would be devastating, Stephen Hester, RBS chief executive, recently wrote in a letter to MPs that a “substantial programme of activity will be implemented to reduce the potential of comparable incidents recurring”.
While it is unsurprising that some RBS customers are tempted to move their accounts, they may find they are just as vulnerable at rival banks.
“Once banks catch a bullet, they tend to respond well,” says Paul Mee, a partner at Oliver Wyman, the global consultancy. “The problem is that the bullets are different each time.”
Additional reporting by Daniel Schäfer
RBS systems failure: Machines that can become banks’ enemies within
Last month’s computer problems at Royal Bank of Scotland plunged the company into a Kafkaesque tangle of unmanageable bureaucracy, writes Sharlene Goff.
RBS said the initial problem was rectified within a couple of days but it lost track of which payments had been processed and had to draft in a team to check manually. With a backlog of about 20m transactions building up every day, it was drowning. At one point the bank had 100m unprocessed payments.
These difficulties had started with a fairly minor change. On Sunday June 17 the bank launched a routine upgrade of the “batch processing” software that allows it to process the millions of ordinary payments and cash withdrawals that customers make everyday. The following night when its system started working through that day’s transactions it became clear something was wrong. The process was slow, eating through too much memory. So 24 hours later, RBS attempted to unwind the upgrade – and the technology collapsed.
IT specialists who know RBS’s systems well believe the glitch was probably the result of a simple human error – the wrong button being pressed at the wrong time. They have questioned whether RBS had the appropriate disciplines in place to ensure actions were properly supervised and approved by senior IT staff.
“When things went wrong there was no single person that could take responsibility for restarting the system,” says Al-Noor Ramji, general manager of Misys’ banking business.
The bank is due to provide a full account of what happened to regulators in the coming weeks. The extent of the operational failings – and the fact that this is one of the first incidents to occur under tough new rules that link penalties to the affected business’s revenues – mean it is likely to receive a hefty fine that could run into tens of millions of pounds.
Such severe outages are rare at banks, although a number have suffered less extreme IT failures, largely as a result of bedding down acquisitions. Santander encountered problems when it integrated Abbey, the UK building society, for example. Bank of Scotland customers also suffered disruption when they joined the Lloyds TSB platform last year.
Other industries – particularly telecoms companies – are finding they have to make amends for their technical failings.
O2, the mobile phone operator owned by Spain’s Telefónica, was this month forced to give almost 8m UK customers several days of free service after a network failure.
Earlier this year France Telecom’s Orange gave users a free day of phone use and cinema tickets following a nine-hour network failure.
Copyright The Financial Times Limited 2012.
0 comments:
Publicar un comentario