Model validation a key risk control for insurers
Helps manage credit, insurance, market and other risks
Are you using the right model validation approach?
Model validation touches many types of risks, including, but not limited to, credit risk, operational risk, market risk, insurance risk and economic capital risk. Understanding the model functionality is the first step in analyzing the associated risk and how one might test or validate such a model. The process of model validation commonly involves a defined scope of objectives that review the mathematical and theoretical soundness of functionality and use. Typically, the model validation team within an organization works with its various business units to establish policies that govern the development and associated documentation.
A closer look at model validations
This requires an understanding of the model's use, limitations, policies and procedures, all of which are acquired through a review of model documentation and discussions with model developers. Understanding the product (swaps, credit derivatives, fixed income bonds, etc.) is critical to a successful validation.
Model reviews should follow a four-step process:
1. Review of assumptions and inputs
An organization must ensure that certain model assumptions are based on analysis that is reflective of current market conditions and activity. Determining the source of inputs and assumptions is critical. Do they come from a reliable source? Does the process of updating and vetting the assumptions occur within a controlled environment? Do the inputs come from an internal model or an external source? What are the controls surrounding this source? Inputs will vary by model. For example, a credit risk model will use financial ratios, while a default model might use macroeconomic inputs.
We often work with our clients during this review by:
- Researching appropriate benchmarks applicable to the model assumptions
- Creating a historical trend analysis of assumptions and inputs
- Determining appropriate calibration of certain assumptions, have the inputs been properly calibrated to observable market data?
- Determining the sensitivity of certain assumption changes. Which inputs create more variability?
2. Theory – is the logic accepted and can it be supported?
The creation of the model is based upon the theory of a particular financial product. Does the developer of the financial model have the proper background?
We assist clients by:
- Providing guidance and input on financial theory of related products
- Verifying the theoretical soundness (as reflected in recent financial literature and evidence) of the pricing and risk relationship of various products.
3. Testing of the code – does the math work?
The testing of the code and mathematics behind the financial theory often requires detailed line-by-line procedures. For many spreadsheet models, this occurs through a replication of calculations and formulas, given the same set of assumptions. Spreadsheet models should always be tested using a full replication of the workbook calculations, due to the lack of security and keystroke errors, which often go undetected. Many debt-like features of certain bonds have detailed pay down logic or waterfall rules that can be easily replicated within a spreadsheet environment.
We often work with clients to:
- Provide an independent recalculation of financial product amortization and expected cash flows, including performance measurement statistics, such as yield, duration and weighted average life.
- Test option pricing models and application of Value at Risk (VAR) assumptions and assessment.
4. Reporting and model output
The analysis of model reporting is critical to the decisions made by senior management. Robust reporting functions allow users to backtest and benchmark more efficiently and accurately. The reports should also be used for clearly outlining the assumptions and associated results based on the given set of inputs. Many companies also use reporting to provide outside parties with a trail of evidence and support for audit-related activities.
McGladrey has helped clients:
- Provide a thorough review of the reporting mechanism and associated accuracy of disclosed items
- Backtest the model and provide feedback on predicted outcomes versus actual outcomes
- Generate scenarios that would potentially cause the reporting functionality to be stressed. Provide example of industry-reporting requirements and associated model-reporting capabilities.