By Muzammil Shabudin, Risk Advisory Lead for SAS UK & Ireland.

News that The Bank of England had initiated an external review of its forecasting models, to ensure that it was doing everything possible to better respond to economic disruption, was welcomed by many back in June.

The review followed months of uncertainty and criticism from politicians accusing the Bank of repeatedly failing to predict the rise and persistence of UK inflation. The Bank of England Governor, Andrew Bailey, admitted that it would take “a lot longer than we expected” for inflation to come down. This has left economists continuing to warn of further interest rate rises and mortgage lenders rushing to reprice loans, meaning the problems facing the UK economy are clearly not going away.

Further pressure was applied when a cross-party group of MPs called for an overhaul of forecasting processes, deeming that the Bank of England’s modelling was not producing accurate results.

This situation brings to the fore the importance of good model management, especially given the economic turbulence witnessed in recent years. If the models relied upon by the financial services sector are no longer able to accurately forecast events - such as interest rate rises – then economic stability becomes much harder to maintain.

For this independent review to be deemed a success, a structured approach must be taken, using a risk and control audit methodology and the use of consistent, robust and scalable analytics techniques. Here are some of the key elements that should be considered.

Ensuring good governance

It’s important to point out that forecasting and risk models are only as good as the governance framework in which they operate. No matter the quality of the data that goes in, if organisations are not continually reviewing their processes around model development, usage and reporting, there is a chance that these models become unfit for purpose.

A clear governance framework will also help to ensure that any models requiring amendments or recalibration are easily and quickly identified. With automated modelling techniques now becoming far more common, organisations such as the Bank of England need to ensure that their forecasting and risk models are fully explainable.

This becomes all the more important when faced with criticism or scrutiny from regulators or MPs.

Of course, the Bank of England has thousands of models in place so questions will need to be asked around how broadly they want to consider their models, how in-depth they want to go and whether or not they want to review or rebuild every model. Similarly, there is a question around how far back into the data management space the review ought to go.

When looking at risk mitigation, the auditors will also be focused on the controls in place to mitigate risk, whether or not they have been effective to date and if they remain fit for purpose. If the risk mitigation process is found to be overly manual or overly automated, this will raise questions about its effectiveness.

All of these questions need to be considered before the review begins, to ensure that the outcome is satisfactory.

Advances in technology and the need for greater regulation

Aside from the external factors that have made forecasting more challenging, namely the global pandemic and war in Ukraine, rapid advances in technology have also raised questions.

The increased adoption of artificial intelligence (AI) and machine learning (ML) means that forecasting and risk models are now able to evolve much faster than they previously would have done. Without the right technology in place, this can soon start to create challenges.

In fact, regulation around model risk management processes is already becoming more stringent, with the Prudential Regulation Authority (PRA) having recently directed UK banks to improve model and data governance processes through the introduction of new model risk regulation.

The Supervisory Statement SS1/23 highlighted the fact that UK banks were lagging behind international peers when it came to ‘effective and robust’ model risk management (MRM). Not only did this leave them open to damaging losses, inaccuracies could have an impact on the overall stability of the UK economy.

With this in mind, the new proposed standards contain five key principles that have been designed to reduce the probability and severity of future crises in the financial sector. Covering model identification and model risk classification, firms must have an established definition of a model that sets the scope for MRM, a model inventory, and a risk-based tiering approach to categorise models to help identify and manage model risk.

There is also a focus on good governance, with firms required to promote good MRM culture from the top down, setting clear model risk appetite, approving the MRM policy and appointing an accountable individual to be responsible for implementing a sound MRM framework.

Alongside this, firms must have a robust model development process with clear standards for model design, implementation, selection and performance measurement.

Given the volatility of the market and challenging economic backdrop, firms will also be required to regularly test their data, model construct, assumptions and outcomes - key processes that will help to identify, monitor, record, and remediate any limitations and weaknesses within the models.

In addition, the PRA has introduced independent model validation to ensure that recommendations for remediation or redevelopment are actioned as quickly as possible so that models are suitable for their intended purpose. Should models be under-performing, firms also need to take quick action, often in the form of an independent review to ensure that they are working effectively.

Taking action

SAS works with organisations across all aspects of the financial services sector, having partnered with over 80 banks to implement robust MRM processes. Given the rapidly changing environmental and digital landscapes, as well as the aforementioned increasing use of AI and sophisticated modelling techniques, now is undoubtedly the time for firms to adopt a more strategic approach not only to MRM but all model management.

As we have seen recently with The Bank of England coming under fire, inadequate or flawed design and implementation of models can lead to adverse consequences that pose significant risks to both their own financial stability and the overall economic stability of the UK economy.