💡 Did you know? Enterprises face a staggering 42% gap between expected and actual AI usage. For organizations investing heavily in AI, understanding how to track and optimize these initiatives is critical to realizing their full potential.
That’s where AI Portfolio Intelligence comes in.
In this webinar, we explore how Fortune 500 AI leaders are:
- Pinpointing high-value use cases that align with strategic goals
- Ensuring ROI transparency across their AI initiatives
- Balancing innovation with compliance through effective governance
We also talk about the real costs of AI—training, infrastructure, and operational expenses—and how a strategic portfolio approach can prevent overspending on low-impact initiatives.
Don’t miss this opportunity to gain actionable insights and accelerate your AI strategy in 2025!
Transcript
Introduction to AI Portfolio Intelligence
Welcome to the first webinar of the year in our Good Decisions monthly series. Today, we’ll be discussing AI Portfolio Intelligence—the key to tracking enterprise AI value. I'm Jay Combs, VP of Marketing, and with me is Dave Trier, VP of Product.
We have about 30 minutes to cover this important topic. If 2023 was the year of AI hype and 2024 saw growth in AI pilots—especially in generative AI—then 2025 will be the year enterprises expect to see ROI, real value, and business growth from their AI initiatives.
The Shift Towards AI as an Investment Portfolio
An emerging trend we’re seeing this year is enterprises treating AI initiatives like an investment portfolio—analyzing benefits, costs, and value to make strategic decisions. This means identifying which initiatives are worth continued investment and which underperforming projects should be phased out.
Because of the sheer number of AI use cases—and the real costs associated with them—AI Portfolio Intelligence has become a necessity. In this session, we’ll dive into what AI Portfolio Intelligence is, why it matters, and how you can get started.
The Value of Generative AI in Enterprises
I think most people are fairly comfortable with this idea by now, but models and AI—especially generative AI—have become an enterprise’s most valuable assets. In the past, value was centered on real estate, manufacturing plants, software, and then data. Now, with generative AI, the opportunities around models and AI are tremendous. This isn’t a controversial statement—it simply highlights how critical AI has become to enterprise strategy.
We’ve seen a gold rush in generative AI use cases across industries, including financial services, banking, CPG, retail, and pharmaceuticals. Many enterprises are already seeing early results—not just from generative AI, but also from traditional models.
The point is, the opportunity is massive. It’s great to see select enterprises like Prudential and Procter & Gamble making these investments and achieving strong returns.
Challenges in AI Implementation and ROI Expectations
Despite the excitement around AI, many unknowns remain—especially for organizations still investing in new use cases and operating in the pilot phase. We saw this at the end of last year through a few interesting studies.
One report from Accenture highlighted massive growth in AI-related investments, particularly in defining AI strategies. They projected about a billion dollars in revenue just from helping organizations determine their AI approach and launch pilots. However, these weren’t production-ready models—just initial AI pilot programs. The early investment in AI strategy alone is staggering.
Another report from Battery Ventures on the state of enterprise tech revealed a widening gap between expected AI adoption and actual deployment. By mid-2024, AI production rates were 42% below what enterprises had projected. In other words, organizations expected certain AI initiatives to be fully deployed and generating business value, but those expectations fell significantly short.
So, what’s causing this gap? What’s preventing AI use cases from moving into full production? And why are so many companies struggling to realize AI’s potential?
Understanding Costs in AI Initiatives
This brings up an important question—not just about AI’s value, but about its costs. The expenses behind AI initiatives are very real, and without a clear understanding of them, managing investments becomes incredibly challenging.
Dave will now break down the actual costs associated with AI initiatives and introduce a new approach—AI Portfolio Intelligence—to help organizations move pilots into production while effectively managing investment and business impact.
Expectations for AI ROI in the Coming Years
Thanks, Jay, and welcome, everyone! Good morning, good evening—wherever you’re joining from.
As Jay mentioned, the opportunity around AI is huge. But 2024 didn’t see as many AI use cases reach production as expected. That means in 2025, we need to reassess: What are the real investments being made in AI?
This slide provides a representative sample—not a complete list—of what goes into an AI use case’s development, deployment, and ongoing usage. It’s meant to get you thinking about what’s truly involved in bringing an AI model into production.
AI has enormous potential benefits, but let’s examine the costs that come with it—some of which organizations may not have considered.
- Infrastructure Costs – Whether you're training your own generative AI models or using existing solutions, infrastructure costs can be substantial. Even if you’re not building a model from scratch, AI requires resources for development, experimentation, and production environments. Hosting applications and maintaining supporting AI models all come with real expenses.
- Development Costs – AI talent is expensive. Organizations need data scientists, ML engineers, domain experts, and business stakeholders to develop, test, and integrate AI into business processes. Even for simple use cases, significant time and effort go into ensuring models are functional and valuable.
- Data Preparation & Labeling Costs – Many AI initiatives require structured and labeled data, which can involve expensive tooling, manual effort, and external services.
- Operational & Maintenance Costs – Once an AI system is deployed, someone needs to monitor, maintain, and govern it. Even if you’re using vendor models, there are costs for oversight, compliance, and ensuring the AI continues operating as expected.
The takeaway? AI is a long-term investment, and organizations need a structured way to manage costs while maximizing value. That’s where AI Portfolio Intelligence comes in.
The Concept of AI Portfolio Intelligence
This brings us to what we call AI Portfolio Intelligence.Think of it like managing a portfolio of stocks. Imagine a surge of new IPOs—just like the current wave of AI innovations. You wouldn’t blindly invest in every IPO that comes out. Instead, you’d do your due diligence—analyzing strategy, growth potential, and fit within your overall portfolio before making investment decisions.AI should be approached the same way. Just because there are exciting new AI tools—whether off-the-shelf solutions or custom-built models—doesn’t mean enterprises should jump in without a plan. Organizations need to ask:
- How does this AI initiative align with our business goals?
- Do we have transparency into how success will be measured?
- Are we investing in AI projects that truly deliver value?
By applying AI Portfolio Intelligence, businesses can rationalize their AI investments—focusing on initiatives that drive measurable outcomes while deprioritizing those that don’t. Instead of the Wild West approach, where everyone is experimenting and trying different things, this method introduces structure, visibility, and smart investment strategies.That doesn’t mean companies shouldn’t experiment with AI. Experimentation is critical. The key is ensuring resources are allocated wisely—failing fast where necessary and doubling down where AI has the most impact.So, how do we implement AI Portfolio Intelligence?
Implementing AI Portfolio Intelligence
This approach is something we’ve developed by working with customers and observing best practices across industries. Think of it as applying rigor to AI investments—similar to managing a stock portfolio.
- Intake Process – The first step is a lightweight intake process. This doesn’t need to be overly complex—just enough to categorize AI initiatives based on expected business impact. For example:
- Is this AI project expected to drive top-line revenue, cost savings, or productivity improvements?
- What is the estimated financial impact? Even a rough projection—like a potential $500K to $1M increase in revenue—helps establish a baseline.
- Rapid Prototyping – Once an AI use case is identified, rapid prototyping is essential. The goal isn’t just to validate the technology, but to ensure it fits into daily business operations.
- Does this AI tool integrate seamlessly into workflows?
- Can it deliver the expected impact in practice?
- Investment Assessment – Before moving to full production, it’s important to estimate the financial investment required.
- What are the infrastructure and resource costs?
- Will external expertise or consulting be needed?
- Is the timing right, or should this initiative be revisited later?
Sometimes, an AI project shows promise but isn’t cost-effective at the moment due to internal skill gaps or technological immaturity. That’s fine—it can be shelved and revisited when conditions improve.This structured approach ensures AI initiatives are backed by thoughtful decision-making, rather than just excitement over new technology.
Tracking Costs and Measuring ROI
Once a decision is made to invest in AI and move it to production, tracking costs becomes critical.AI initiatives involve real costs:
- Infrastructure expenses (cloud computing, storage, hosting)
- AI and engineering resources (development, maintenance, monitoring)
- Software licensing fees and compliance costs
To maximize AI’s impact, organizations must track these expenses alongside the projected benefits.A structured tracking process ensures that AI investments are continuously evaluated. This allows businesses to compare actual performance against expectations—confirming whether an AI initiative is delivering value or if adjustments are needed.By implementing AI Portfolio Intelligence, organizations can manage AI like an investment portfolio—investing in projects that provide measurable ROI while deprioritizing those that don’t.
Measuring ROI and Performance of AI Systems
Now that we’re tracking costs, the next step is measuring performance against those costs to understand the return on investment (ROI) for AI initiatives.Some AI use cases have quantifiable benefits. For example, in marketing and advertising, AI might recommend a product, and if a customer purchases it, we can directly attribute that sale to the AI model.However, not all AI benefits are easy to measure in dollars and cents. In many cases, organizations need feedback loops—whether through user surveys, operational insights, or other mechanisms—to assess impact.The key is having a system in place to systematically collect performance data, associate it with AI use cases, and compare results against costs. By doing so, organizations can evaluate whether AI initiatives are generating real business value.This process also enables aggregate reporting—providing a high-level view of which AI use cases are delivering the most impact and which may need to be adjusted or phased out.
Strategic Investment in AI Portfolios
At the end of the day, this is about doing what’s best for the organization.
- If an AI initiative is showing strong benefits, it may make sense to increase investment in that area.
- If a project isn’t delivering the expected impact, it may be time to scale back or discontinue it.
By applying this structured approach, enterprises move away from experimenting blindly and toward managing AI strategically—just like a financial portfolio.To illustrate this, let’s look at an example from ModelOp Center, our AI Portfolio Intelligence software.With ModelOp Center, enterprises can:
- Visualize their AI portfolio – Understand where AI is being used and which business objectives it supports (e.g., revenue growth, cost savings, innovation).
- Analyze projected vs. actual benefits – Track whether AI investments are delivering the expected impact.
- Monitor AI lifecycle stages – See how many AI use cases are in development, piloting, or full production.
This level of visibility and tracking transforms AI from a cool technology into a structured investment strategy, ensuring AI initiatives drive measurable value.
Getting Started with AI Portfolio Management
So, how do you get started? We recommend following the Rule of Three:
- Make AI Portfolio Intelligence a central focus – Shift your AI program’s mindset from running pilots to actively tracking AI’s impact.
- Integrate portfolio management into your AI charter – Whether you have an existing AI governance framework or are just getting started, bake AI Portfolio Intelligence into your program. This will help transition from experimentation to real business value tracking.
- Leverage technology to implement AI Portfolio Intelligence – Use software to systematically track AI investments, costs, and benefits. This ensures different teams—data science, business leaders, and compliance—have the insights needed to make informed decisions.
By taking these steps, organizations can move from AI hype to AI value, ensuring their investments are strategic, measurable, and aligned with business goals.We’ve only scratched the surface today, but hopefully, this gives you a clear direction on how AI Portfolio Intelligence will be a key focus in 2025.
Key Takeaways for AI Management
To wrap things up, let’s go over some key takeaways.First, it’s time to shift how we manage AI. We need to adopt an AI Portfolio Intelligence mindset—tracking AI value and measuring ROI, just like managing a portfolio of stocks.In past webinars, we’ve discussed the idea of minimum viable governance—doing just enough governance to get started without slowing innovation. The first step in that approach is visibility—getting a clear picture of all AI initiatives across the enterprise. Once you have visibility, you can start managing AI like an investment portfolio.Finally, putting this into action requires mobilizing your people, processes, and technology. By aligning these three areas, organizations can effectively track AI’s value and make data-driven investment decisions.These simple but powerful steps will set you up for success in managing AI strategically.Now, before we wrap up, we have time for a quick audience question.
The Role of ModelOp in AI Governance
One of our audience members asks:"Does ModelOp primarily help centralize AI portfolio assets, or does it focus on cross-referencing features and security requirements across models to ensure compliance and alignment?"Great question! As Jay and I have discussed, ModelOp is focused on overarching AI governance. But governance, as we define it, isn’t just about compliance—it’s also about ensuring AI investments align with business strategy and drive value.So, to answer your question, it’s both.
- We provide visibility into AI investments – ModelOp helps organizations maintain an inventory of AI models, tracking business value, limitations, and anticipated ROI.
- We enable compliance and security alignment – AI governance isn’t just about tracking costs and benefits; it’s also about ensuring AI initiatives meet security, risk, and compliance standards.
At ModelOp, we don’t see governance as just policing AI—we see it as enabling organizations to make the right AI investments while ensuring those investments are secure, compliant, and valuable.