Stu is a technologist and entrepreneur who has been innovating and commercializing large scale model intensive production systems for over two decades.
Forbes – November 18, 2020
For some industries, the use of AI and machine learning models is novel, but several industries—consumer finance and insurance in particular—have been building, using and governing models for decades. These industries have well-developed governance practices built largely around algorithmic, rule-based and other model technologies and regulations that predate AI models. But AI/ML models have unique technical and operational characteristics that compound current governance challenges:
- Adoption of AI models is proliferating rapidly, across business lines, leading to incomplete or inaccurate model inventories.
- The “black box” characteristics of AI/ML algorithms limit insight into the predictive factors, which is incompatible with model governance requirements that demand interpretability and explainability.
- The use of real-time data to make real-time decisions requires a new level of model performance monitoring.
- Increased algorithmic complexity and data use is driving increased value from models, but also greatly increases business risk and regulatory exposure, especially as the scale of model usage exceeds the capability of humans to track them.
Many of the enterprises I talk to are revisiting their model operationalization and governance processes and strengthening them with new capabilities to accommodate the increased use of AI/ML technologies. There are four key areas that enterprises are focusing on to ensure proper governance and risk management for their AI/ML models:
1. Enterprise-Level Model Inventory
You can’t govern what you can’t see, so every model risk management (MRM) program must start with a centralized model inventory that includes all the metadata associated with every model throughout its life cycle, from development to deployment, modification and retirement.
This model metadata, which documents the model’s complete history and lineage, captures a broad range of elements including the specific software and libraries used in its development, the data used to train the model, the people involved in the model’s development and maintenance and what they created or changed, the model’s intended business use and KPIs, an explanation of the key influencing factors behind the model’s decision-making, etc. This information isn’t static and needs to be continually updated—and readily accessible to satisfy audit requirements—throughout the model’s life cycle.
2. Enterprise-Level And Standardized Model Life Cycle (MLC) Management
Every model is unique and needs to be managed according to its own model life cycle. But effective governance of models in an enterprise requires a consistent approach to defining, implementing, monitoring and reporting on model life cycles. Standardization doesn’t enforce a single technical or operational approach, but rather enables the unique development, deployment, operationalization and governance aspects for each model and its MLC to be fully captured and automated in a consistent, efficient and transparent manner.
The need for standardized MLCs is becoming more acute given the increasing variety of development platforms and model factories available from software vendors, open source projects and cloud services operators. As data scientists create models faster and citizen data scientists participate in creating business unit-specific models, it’s increasingly easy for different teams to have different processes for operationalizing models—which makes the governance challenge that much harder.
Standardizing MLCs has benefits beyond governance: It reduces deployment delays and gets models into production faster. Having an enterprise-wide approach to MLC definition and management is also essential to maintaining security as enterprises increasingly embrace cloud services. When managing processes and practices that use on-premises tools, the IT team has typically controlled access and usage, providing a line of defense for enforcing consistency. Given the need for greater security and access control in the use of cloud services, having well-defined model life cycles and established processes around them is even more important.
3. Enterprise-Level Production Model Monitoring And Model Operations
Monitoring of AI/ML models is growing beyond human scale in most enterprises. Monitoring begins when a model is first implemented in production systems for actual business use and continues through retirement (and beyond for historical archive purposes). Monitoring includes verifying internal and external data inputs, recording schema changes, tracking statistical performance and data drift, and ensuring the model performs within the operational and business parameters set for it.
Since each model is unique, monitoring frequency generally varies for each model. For monitoring to be most effective, it needs to include alerts and notifications of potential upcoming performance issues, and it needs to track and log the remediation steps until model health and performance is reinstated.
Monitoring by itself is “one-hand clapping” and needs to be tightly coupled with remediation processes. For example, detecting model drift is not enough. Monitoring workflows needs to connect with workflows for retraining, retesting or other corrective actions as required, initiating change requests and gating activities that need approvals.
4. Model Life Cycle Automation
Automation is critical to successful model operations and governance given the increased complexity and volume of AI/ML models. Automation provides the means to orchestrate and dynamically manage and enforce every step in each model’s life cycle, providing the management oversight needed for ongoing operations and good governance and risk management.
A well-designed model life cycle will leverage, not duplicate, the capabilities of the business and IT systems involved in developing models and maintaining model health and reliability. This includes integrating with model development platforms, change management systems, source code management systems, data management systems, infrastructure management systems and model risk management systems. Duplicating any of the work of these systems introduces unnecessary effort, errors and risk.
In Summary
Enterprise-level model governance means maximizing model value while minimizing business risk and regulatory exposure. The payoff is significant: Enterprises that have effective model operations and governance can experience top-line increases and also reduce operating expenses by 20% or more, according to “The evolution of model risk management” report from McKinsey & Co.
Good governance requires enterprise-wide visibility to all models, well-designed and standardized model life cycles, real-time monitoring, and automation to enforce business and technical processes across data scientists, model engineers, AI enterprise architects and business stakeholders. Enterprises need AI-ready model governance to drive business value and protect the enterprise against massive regulatory and brand risks.
