Validation
CA-5.8.140
Banks must have a robust system in place to validate the accuracy and consistency of their internal models and modelling processes. A bank must demonstrate to CBB that the internal validation process enables it to assess the performance of its internal model and processes consistently and meaningfully.
Apr 08CA-5.8.141
Banks must regularly compare actual return performance (computed using realised and unrealised gains and losses) with modelled estimates and be able to demonstrate that such returns are within the expected range for the portfolio and individual holdings. Such comparisons must make use of historical data that are over as long a period as possible. The methods and data used in such comparisons must be clearly documented by the bank. This analysis and documentation should be updated at least annually.
Apr 08CA-5.8.142
Banks should make use of other quantitative validation tools and comparisons with external data sources. The analysis must be based on data that are appropriate to the portfolio, are updated regularly, and cover a relevant observation period. Banks' internal assessments of the performance of their own model must be based on long data histories, covering a range of economic conditions, and ideally one or more complete business cycles.
Apr 08CA-5.8.143
Banks must demonstrate that quantitative validation methods and data are consistent through time. Changes in estimation methods and data (both data sources and periods covered) must be clearly and thoroughly documented.
Apr 08CA-5.8.144
Since the evaluation of actual performance to expected performance over time provides a basis for banks to refine and adjust internal models on an ongoing basis, it is expected that banks using internal models will have established well-articulated model review standards. These standards are especially important for situations where actual results significantly deviate from expectations and where the validity of the internal model is called into question. These standards must take account of business cycles and similar systematic variability in equity returns. All adjustments made to internal models in response to model reviews must be well documented and consistent with the bank's model review standards.
Apr 08CA-5.8.145
To facilitate model validation through backtesting on an ongoing basis, institutions using the internal model approach must construct and maintain appropriate databases on the actual quarterly performance of their equity investments as well on the estimates derived using their internal models. Institutions should also backtest the volatility estimates used within their internal models and the appropriateness of the proxies used in the model. CBB may ask banks to scale their quarterly forecasts to a different, in particular shorter, time horizon, store performance data for this time horizon and perform backtests on this basis.
Apr 08
