SR 11-7

What is SR 11-7?

In 2011, the United States Federal Reserve and the Office of the Comptroller introduced Supervisory Regulation 11-7 (SR 11-7).  The introduction of this guidance resulted from the recognition that quantitative analysis and models were playing an ever more important role in the decision making of financial institutions.  
‍
The use of these types of analysis methods by banks has become routine and is now used to support a broad range of activities including credit underwriting, valuing investments, managing risk, safeguarding client assets and determining the adequacy of reserves.
‍
SR 11-7 was introduced to provide banks comprehensive guidance on effective model risk management.  The intent of the regulation is to provide financial institutions with guidance around how to mitigate potential adverse financial consequences that might arise from decisions that are based on quantitative methods and models that are either incorrect or improperly used.

What is Considered a Model?

According to the regulation, the term model refers to a quantitative method, system, or approach that applies statistical, economic, financial, or mathematical theories, techniques, and assumptions to process input data into quantitative estimates.

The definition of model also covers quantitative approaches whose inputs are partially or wholly qualitative or based on expert judgment, provided that the output is quantitative in nature.

A model consists of three components:

1. An information input component, which delivers assumptions and data to the model.
2. A processing component, which transforms inputs into estimates.
3. A reporting component, which translates the estimates into useful business information.

‍

Model Risk Management

The use of models introduces model risk which the regulation defines as the potential for negative outcomes due to flawed or misused model outputs. This risk can result in financial loss, poor decision-making, or reputational harm, and highlights the critical need for proper model evaluation and application.  

Model risk occurs primarily for two reasons – fundamental errors or model misuse.  

Fundamental Errors

These may arise at any stage of the modeling process, from initial design to implementation.  The mathematical calculations and quantifications underlying models rely on theoretical frameworks, sample designs, numerical methods, and chosen inputs.  Conceptual and logical errors along with incorrect or inappropriate design choices can have downstream consequences that undermine the validity of a model’s outputs.

Model Misuse

A fundamentally valid model producing accurate outputs consistent with its design objectives may still exhibit a high degree of risk if it is misused.  Decision makers should understand the limitations of every model under their control and avoid using models in ways that are not consistent with their original intent.

Assessing Model Risk

Model risk should be managed like other types of risk. Banks need to identify the sources of risk and assess the magnitude of the risk posed. In doing so, they should consider both individual model risk and in the aggregate risk of all models in use.  
‍
The use of "effective challenge" is critical in assessing the risk posed by an individual model.  This entails the critical analysis of a model against its objectives by informed, technically competent parties who can identify model limitations and assumptions.  These individuals are often in the best position to suggest appropriate changes that will mitigate any risks identified as part of an effective challenge process.

Three Areas of Risk Management

There are three specific aspects of model risk management that the guidance covers.

1. Model Development, Implementation and Use
2. Model Validation
3. Model Governance

Model Development, Implementation & Use

Documenting The Model

Documentation should include a broad range of model elements including its purpose; explorations and comparisons of possible different approaches; design theory, logic and methodologies used; pre and post model testing and information on the data used to train the model.

Ensuring The Use of Quality Data

Model creators should be able to demonstrate that the data and information used are suitable for the model.  As part of this demonstration there should be a rigorous assessment of data quality and relevance.

Pre-Release Testing

Pre-release testing includes creating and executing test plans, summarizing results with commentary, and analyzing representative samples, and finally ensuring that all activities are well-documented.  

Post-Release Testing

Real world use provides fresh opportunities to identify situations where a model may perform poorly or become unreliable in its predictions.  A plan for post-release testing should be developed and implemented to help safeguard the organization from risks that may not have been exposed during the pre-release period.  

Model Validation

Model validation involves verifying that models perform as expected, aligning with their design objectives and business uses. It ensures model soundness, identifies potential limitations and assumptions, and assesses their impact.
‍
The individuals responsible for conducting model validation should have the requisite knowledge and technical skills necessary to review model performance and the explicit authority to require changes to models when issues are identified.  

How much or how little validation is required is directly proportional to the importance of the model to the bank’s operations and the level of risk posed by the model.

An effective validation framework should include three core elements:

1. Evaluation of conceptual soundness, including developmental evidence.
2. Ongoing monitoring, including process verification and benchmarking.
3. Outcomes analysis, including back-testing.

The need to validate the models used by any bank includes the validation of 3^rd party models.  Third party models pose unique challenges for validation and other model risk management activities because the modeling expertise may be external to the bank and because some components of the third-party model may be proprietary.  

Irrespective of the level of difficulty involved, vendor models need to be incorporated into a bank's broader model risk management framework.  Though the process may need to be modified, third-party model validation should follow the same general principles that are applied to in-house models.

Model Governance

Strong governance, policies, and controls are essential for the effectiveness of the model risk management framework.  Weak governance undermines the effectiveness of model risk management, even if development, implementation, use, and validation are satisfactory.

While an organization’s board of directors is ultimately responsible for good governance, senior management is tasked with executing and maintaining the appropriate governance framework.  

Key deliverables that senior management are responsible for ensuring exist include:

• Appropriate policies and procedures exist and are effectively communicated
• Roles and responsibilities of individuals involved in developing, implementing and running models are defined and assigned including the role of internal audit and external resources
• The creation and maintenance of a comprehensive of information for models implemented for use, under development for implementation, or recently retired.
• Documentation of model development and validation activities sufficiently complete that hat parties unfamiliar with a model can understand how the model operates, its limitations, and its key assumptions

Compliance Framework

The above steps layout the steps required to achieve compliance with SR 11-7.  But many organizations struggle to lay down an appropriate framework that allows them to easily meet the requirements of multiple related but often overlapping AI governance guidance and regulations.  This is where the concept of Minimum Viable Governance (MVG) comes in.  
‍
The MVG approach to governance focuses on right sizing the effort involved in establishing an AI governance program - not too much, not too little, but just enough to protect the organization while maintaining AI innovation cycles.

MVG involves three core facets:

1. Establishing a governance inventory to ensure visibility into all AI usage.
2. Applying lightweight controls to manage verification, evidence, and approvals without overwhelming innovation.
3. Implementing streamlined reporting to achieve transparency and understand how AI is being used.

‍