Back Custom print Link Text Only Rich Text Print

  • (iv) Validation and Documentation

    • CA-5.8.139

      Institutions employing internal models for regulatory capital purposes are expected to have in place a robust system to validate the accuracy and consistency of the model and its inputs. They must also fully document all material elements of their internal models and modelling process. The modelling process itself as well as the systems used to validate internal models including all supporting documentation, validation results, and the findings of internal and external reviews are subject to oversight and review by the CBB.

      Apr 08

    • Validation

      • CA-5.8.140

        Banks must have a robust system in place to validate the accuracy and consistency of their internal models and modelling processes. A bank must demonstrate to CBB that the internal validation process enables it to assess the performance of its internal model and processes consistently and meaningfully.

        Apr 08

      • CA-5.8.141

        Banks must regularly compare actual return performance (computed using realised and unrealised gains and losses) with modelled estimates and be able to demonstrate that such returns are within the expected range for the portfolio and individual holdings. Such comparisons must make use of historical data that are over as long a period as possible. The methods and data used in such comparisons must be clearly documented by the bank. This analysis and documentation should be updated at least annually.

        Apr 08

      • CA-5.8.142

        Banks should make use of other quantitative validation tools and comparisons with external data sources. The analysis must be based on data that are appropriate to the portfolio, are updated regularly, and cover a relevant observation period. Banks' internal assessments of the performance of their own model must be based on long data histories, covering a range of economic conditions, and ideally one or more complete business cycles.

        Apr 08

      • CA-5.8.143

        Banks must demonstrate that quantitative validation methods and data are consistent through time. Changes in estimation methods and data (both data sources and periods covered) must be clearly and thoroughly documented.

        Apr 08

      • CA-5.8.144

        Since the evaluation of actual performance to expected performance over time provides a basis for banks to refine and adjust internal models on an ongoing basis, it is expected that banks using internal models will have established well-articulated model review standards. These standards are especially important for situations where actual results significantly deviate from expectations and where the validity of the internal model is called into question. These standards must take account of business cycles and similar systematic variability in equity returns. All adjustments made to internal models in response to model reviews must be well documented and consistent with the bank's model review standards.

        Apr 08

      • CA-5.8.145

        To facilitate model validation through backtesting on an ongoing basis, institutions using the internal model approach must construct and maintain appropriate databases on the actual quarterly performance of their equity investments as well on the estimates derived using their internal models. Institutions should also backtest the volatility estimates used within their internal models and the appropriateness of the proxies used in the model. CBB may ask banks to scale their quarterly forecasts to a different, in particular shorter, time horizon, store performance data for this time horizon and perform backtests on this basis.

        Apr 08

    • Documentation

      • CA-5.8.146

        The burden is on the bank to satisfy CBB that a model has good predictive power and that regulatory capital requirements will not be distorted as a result of its use. Accordingly, all critical elements of an internal model and the modelling process should be fully and adequately documented. Banks must document in writing their internal model's design and operational details. The documentation should demonstrate banks' compliance with the minimum quantitative and qualitative standards, and should address topics such as the application of the model to different segments of the portfolio, estimation methodologies, responsibilities of parties involved in the modelling, and the model approval and model review processes. In particular, the documentation should address the following points:

        (a) A bank must document the rationale for its choice of internal modelling methodology and must be able to provide analyses demonstrating that the model and modelling procedures are likely to result in estimates that meaningfully identify the risk of the bank's equity holdings. Internal models and procedures must be periodically reviewed to determine whether they remain fully applicable to the current portfolio and to external conditions. In addition, a bank must document a history of major changes in the model over time and changes made to the modelling process subsequent to the last supervisory review. If changes have been made in response to the bank's internal review standards, the bank must document that these changes are consistent with its internal model review standards;
        (b) In documenting their internal models banks must:
        •   provide a detailed outline of the theory, assumptions and/or mathematical and empirical basis of the parameters, variables, and data source(s) used to estimate the model;
        •   establish a rigorous statistical process (including out-of-time and out-of-sample performance tests) for validating the selection of explanatory variables; and
        •   indicate circumstances under which the model does not work effectively.
        (c) Where proxies and mapping are employed, institutions must have performed and documented rigorous analysis demonstrating that all chosen proxies and mappings are sufficiently representative of the risk of the equity holdings to which they correspond. The documentation should show, for instance, the relevant and material factors (e.g. business lines, balance sheet characteristics, geographic location, company age, industry sector and subsector, operating characteristics) used in mapping individual investments into proxies. In summary, institutions must demonstrate that the proxies and mappings employed:
        •   are adequately comparable to the underlying holding or portfolio;
        •   are derived using historical economic and market conditions that are relevant and material to the underlying holdings or, where not, that an appropriate adjustment has been made; and,
        •   are robust estimates of the potential risk of the underlying holding.
        Amended: April 2011
        Apr 08