Banking on model risk management

Modeling is a common approach for predicting results based on data inputs. But what is model risk management (MRM) and how does it influence a financial institution’s ability to evaluate risk?

MRM helps ensure that the models a financial institution employs are developed and used appropriately and that outputs are accurate and reliable. Model risk has emerged as a key concern for financial institutions and industry regulators since models play a significant role in a modern financial institution’s decision-making processes.

A comprehensive MRM framework, and a strong focus on MRM testing and monitoring, can go a long way toward implementing a successful MRM program. It makes sense that the effectiveness of the models is tested and adjusted on an ongoing basis to guarantee the veracity of their outputs. But with regulators continuing to push financial institutions to elevate their model risk capabilities, there are even more reasons to prioritize MRM and look for ways to stay ahead of rising expectations.

The goal of the following five practices is to ensure that your MRM strategy delivers on its full potential.

1.    Implement monitoring processes that enable continuous, ongoing, and comprehensive insights into model performance, relevance, and reliability.

Monitoring is one of the most important elements of an MRM framework. Financial institutions rely on monitoring to assess a model’s basic performance and reliability, to track and report changes in a model’s parameters, and to test and benchmark models against new data sets.

Financial institutions typically have a wide range of model risk monitoring options at their disposal, including:

• Monitoring to assess model risks identified during development and validation
  
• Input monitoring to ensure that models are working with complete, accurate, and timely data sets
• Output monitoring to assess a model’s real-world value and reliability, incorporating user feedback, changing economic and regulatory circumstances, a model’s own objectives and key assumptions, and prior validation exercises, among many other options
• Monitoring sources of model process risk—for example, issues with data quality, computer code, and system integration
• Monitoring environmental changes (i.e., products, exposures, activities, clients, or market conditions) that may impact model risk
• Regular model performance monitoring via back-testing, benchmarking, sensitivity analysis, and stress-testing exercises

Model risk monitoring is especially valuable when it’s implemented as a continuous process. These capabilities make it easier for financial institutions to understand how model performance changes over time and to identify model performance issues in real time.

Financial institutions sometimes struggle to maximize the value of their monitoring capabilities—for example, not knowing how to select KPIs or interpret benchmarking results. Working with a team of MRM advisors allows your institution to close these gaps and maximize the value of their model risk monitoring capabilities.

2.    Master the relationship between model assumptions and model performance.

Even the most basic models build on a set of underlying assumptions—the ground rules, limitations, overlays, and adjustments that enable useful questions about an uncertain future. Model assumptions can define radically different versions of reality; you can’t get much value from a model unless you first understand how these assumptions shape a model’s relevance and the accuracy of its predictions.

Learning to recognize model assumptions is an important skill, but it’s not enough to enable a coherent MRM strategy. Sensitivity analysis, stress testing, and other methods allow model owners to test, interrogate and challenge model assumptions. They also allow for an understanding of how changing model assumptions will affect a model’s output.

It’s also important to develop the ability to assess and rank model assumptions based on their relative impact and importance within an institution’s risk modeling framework. This is a critical step for focusing model development and redevelopment resource spend—in terms of dollars as well as headcount—on projects that support an institution’s most critical assumptions.

Model risk monitoring is especially valuable when it’s implemented as a continuous process. 

3.    Take advantage of simple, proven methods to build a more robust model development process.

The model development process looks similar to software development in a number of ways. This includes a shared emphasis on:

• Structured and well-documented processes across the development life cycle
• Robust quality assurance processes designed to find and fix problems before a model deploys into a production environment
• The use of version control and changelogs to give developers a way to roll back changes that behave in unexpected or unproductive ways

In fact, these capabilities can be a starting point for a much bigger conversation about a financial institution’s model life cycle. A third-party advisor can help your institution explore workflow capabilities, model inventory databases, centralized document storage, and other technology capabilities that enable a robust and highly scalable model development process.

4.    Recognize and understand the risks and responsibilities of relying on third-party models.

Many financial institutions rely on third-party vendors to supply key risk management and decision support models. These can be a real source of value, but they are also an often-overlooked source of model risk—and a potential compliance headache.

In fact, banks are fully accountable for any compliance issues or risk associated with models supplied by third-party vendors. Model users should insist upon reviewing model design, testing, and monitoring data to verify a vendor’s performance claims. Institutions should also expect to validate their own use of third-party models, to confirm they are using the model in a way that is consistent with its intended purpose, and to conduct ongoing testing and monitoring. Access to third-party vendors’ own validation reports (whether performed internally or by one of their own vendors), model certifications, and SOC reports should be a routine step in this process.                                                                         

The reality, however, is that many institutions rely on third-party models that are virtually a black box to the people using them. Even when validations are performed to verify models are working correctly, the controls at the third-party model vendor often remain a mystery to many institutions. It’s also commonplace for financial institutions to use third-party models without contingency plans, should the model become unavailable or fail.

This is an area where a trusted partner can play an immediate and valuable role in solving an urgent MRM challenge. An advisor can provide independent oversight to assess, test, and validate third-party models and to conduct ongoing testing as well. In cases where a third party can’t or won’t offer a SOC report, an advisor can lead an effort to investigate and determine how the third party receives, manipulates, processes, and reports an institution’s data. All these tasks roll up into a win-win approach: model users stay focused on strategic tasks, and financial institutions stay off the compliance hot seat.

A holistic approach to MRM also shields model owners from a compliance environment where regulators are quick to criticize, hard to please, and impossible to satisfy. Model owners who work within an institution’s MRM process benefit from leaving compliance issues to the specialists and focusing on model quality and effectiveness.

5. See the big picture: Embrace and engage with your institution’s MRM strategy and processes.

A typical financial institution may use dozens of models that address every category of risk, offer decision support, provide valuation and pricing guidance and even address marketing and insurance needs. Over time, the teams that build and maintain these models may lose sight of this model ecosystem and begin viewing themselves as an island—one with unique needs and requiring unique solutions.

The most important thing these teams can do to support effective MRM is to look up and recognize the big picture. MRM begins with a shared, standardized, comprehensive approach to governance, life cycle management, risk management, model control, and other key capabilities. It’s an approach that avoids wasteful redundancies, leverages economies of scale, and lifts a significant overhead burden off the shoulders of individual model owners and developers.

A holistic approach to MRM also shields model owners from a compliance environment where regulators are quick to criticize, hard to please, and impossible to satisfy. Model owners who work within an institution’s MRM process benefit from leaving compliance issues to the specialists and focusing on model quality and effectiveness.

The ongoing process of modeling success

Keep in mind that MRM isn’t a task or a discrete goal; it’s a continuous process that reaches across your functions, organizational roles, and risk management needs. Look for ways to keep improving and innovating with your MRM processes, and you’ll be well-positioned for long-term success.