Achieving Minimum Viable Governance

Chet Kapoor: Jim, welcome to the Inspired Execution Podcast.

Jim Olsen: Thank you. I'm glad to be here and join you today.

Chet Kapoor: So, you've been programming for 35 plus years, starting at 13 years. So, two questions. What led the 13-year-old Jim to programming? And the second question is, what's the first thing you programmed?

Jim Olsen: Okay, yeah. Well, 13 was my first job programming, not just my first programming. I started to teach myself at nine. My dad worked for Bell Laboratories on that kind of thing. Yeah. So, I had some access to some high-tech computer equipment, being one of those old thermal acoustic coupler. You plug the phone inside and you play with it. And you print them out. If you leave your print out in the sun, it turns black. It's very early stuff.

Chet Kapoor: It's been a long time since somebody has said that story. That is great. Yeah, I know. It was a while ago.

Jim Olsen: Yeah. So, that's what I started with. And I really just started playing with it. I love the logic behind programming and problem-solving and that aspect of it. And, to actually be able to do that and experiment and get real-time feedback. You know, a lot of science is a lot slower paced. Well, computers are not. Yeah. Even back then, on 110 baud, you got your answer pretty quick. That's the fun part of it. And that's what attracted me to it. I started getting books and learning and magazines back then. And I kind of picked everything up and started playing and went from there.

Chet Kapoor: That's awesome. So, you've been a founder and architect and now CTO at ModelOp. How did you find the transition to CTO and any advice that you can share with others?

Jim Olsen: Yeah, sure. One of the biggest things was learning to step back a little more. In that, you want to set out leading and guiding principles. I'm still very hands-on. I still take on bugs and fix things and things like that. But, learning to let people find their own direction as well and basically looking at your foundation and building really great things upon that and willing to let it go maybe a little different direction than you thought you might. Whereas, more when you're like VP engineering or director of engineering, you're very hands-on. You're keeping track. You're running sprints. You're all that kind of things. CTO gives you that luxury to instead to try to help educate, but then give them the freedom to run as well.

Chet Kapoor: There is a transition, right? Especially the hands-on CTOs, right? Because there's a tendency to say, you're obviously not getting it. Let me show you. And you do it yourself. And by the way, that applies to engineers and tasks. But it also applies to how people deliver projects. So is that because there's a subtlety, right? Of saying, let me show you versus let me teach you.

Jim Olsen: Yeah, absolutely.

Chet Kapoor: And how has that transition been?

Jim Olsen: It was definitely a change. Because I actually started originally for a brief period of time as VP of engineering here. And then switched over. So it was definitely a little switch of like, okay, I've got to let the new VP of engineering run things their way. And step back from that role and realize, okay, maybe I see a mistake being made, or maybe it's not the way I would approach it, although it's even perfectly okay. And learning to do that and say, hey, no, they've got to let them find their own way of doing it. But still making sure, you're providing your role as the architect and setting the good foundation and overall directions at the highest level, but not sweating the details, if that makes sense.

Chet Kapoor: Yeah. No, for sure. For sure. All right, let's talk about Modelop and AI governance. Can you give us, you know, AI governance, as I talk to CIOs and customers, AI governance shows up in discussions. And we touch on it lightly, but we don't go deep into it because it's not something that we deeply understand. I obviously, you know, it's one that is intriguing for me personally because I think it's about at the individual level, it's at a group level, it's at a company level, but it's also at a national level, right? There's governance across the board. So give us a broad perspective on the current state of the market, on what you're seeing, what's working, what are some immediate problems we can solve? What are some things that we'll have to think about longer term? And you can skate between, you can scale, you can go from individuals all the way to countries, right?

Jim Olsen: Yeah. Yeah, exactly. I mean, it said literally the EU regulations on AI just took effect, I think it was two or three days ago. Yeah. So now technically everybody's compliant to what those are asking of you. And if you're not, there's different businesses. Now what we've really seen evolve is because of generative AI, you know, it's really gotten out there. People are aware of it. It's in their hands and that, the average person, if you have Windows, you've got Copilot, you can play with it for free kind of things on that. Whereas before, initially when we started this journey, we were more involved with like banks and things like that, a few things, because there's been regulation around AI or ML models for a long time, or in fact, any kind of a model, even an Excel spreadsheet making a prediction was under something called SR-117. Yes. And if you didn't follow that, big fines, billions of dollars of fines if you're a big bank. So those were some of the earliest customers we worked with. But there were other kind of more forward-thinking companies too that just realized you make a bad business decision with these, that costs you money.

Chet Kapoor: Yeah. Especially when you get into the... You're liable for it, right? Especially banks. All regulated industries, actually. It's true for insurance companies and everybody else.

Jim Olsen: Yeah, but even if you're not regulated, for instance, a while back I worked with a manufacturer and they have hundreds of models on the floors building these devices. And one of these things goes wrong and gets out to the customer, they've got SLAs and everything to get the stuff back. So there's real business costs beyond just regulatory costs that you still want to provide governance to make sure you're putting out quality and you're maintaining quality and you have the ability to understand what's actually going on and who's using what. So what's really changed that we've seen is now we're, as opposed to being in the banking industry where a lot of our initial clients were and understood the real risks, we're starting to see consumer companies and healthcare industry and everybody's coming on because there's these new regulations that pop out like in healthcare. If they use any AI ML model at all, in several states, they have to disclose that. And if they don't, they're under regulations that don't. So more and more companies are kind of coming online. We're seeing this explosive effect of starting to understand. Now, a lot of people still just sit back and they're like, I'll just wait and see.

Chet Kapoor: So it's really interesting though, Jim, right? So you have models like Llama that are completely open source, right? And you can take a look at them, but you can look at OpenAI or even Gemini, right? They are not, right? But if I think about building a RAG app, right? I may go a little deeper into this. I think about RAG app. I can build all of the entire RAG app with all open source transparent components where you can actually traceability of what happened when, all that stuff in an app, except for two things, right? There's proprietary data about the individual, which has PCI ramifications, which can't be shared, which by the way is not a big deal. But you don't have transparency on how the model came up with its recommendations, right? When it provides content, how does it do that? How do we solve for that in the enterprise space as well as in society in general?

Jim Olsen: Yeah, interpretability is a big issue with models. Not all models are inherently interpretable as to why things, you can do techniques like Shapley and Lime that basically cast logistic regressions back on to guess why they made these decisions. But when we get into natural language models, those don't really work. So there's not really a great solution out there for understanding truly why they're making the statements or et cetera they are. So instead you have to control it from a business perspective and start to look at what's the business outcome? Is it in line with what I want? And more like, I mean, guardrails is overused as a term because obviously guardrails AI. But yeah, putting guardrails around what the thing is doing, how it's doing, keeping track of that, that's just general model monitoring. And we've come up with a few unique methodologies for being able to do that and detect model drift. Because if it's performing okay initially, what kinds of characteristics about the things it's putting out could potentially drift? And we look more at that. Interpretability is impossible with a real LLM. You look at vector clustering and some things like that, better for developing the model, not so much for looking at the business outcome.

Chet Kapoor: For sure. Continuing that part of the conversation, when you are talking to enterprises that are looking at governance, do you land up telling them the difference between your approach and what other ML Ops companies who have reinvented themselves to Gen AI? And you have to tell them the difference or do you think they intuitively get how, if I may, I'll just put a label, it could be wrong. Gen AI Ops or AI Ops is different from ML Ops. Do you think they get it or you have to spend some time?

Jim Olsen: Some do, some don't, a lot don't. Especially efforts that are pushed by the data scientists, they're more likely from, not unselfishly, their perspective where ML Ops is designed around the idea of how do you develop the model and do basically an agile process on break, retrain, break, retrain, break, retrain an approach. Whereas AI governance is separate from the development process. You can still use ML Ops to develop your model and break it, they're complementary. But how do you get your attestations that, these things, what data sets were used, who approved it, what were the reviews, how do we snapshot this thing so we know the exact version that was deployed. These are more of the things you have to do in governance flows, whereas ML Ops is very targeted at the data scientist flow of basically hyper-parameter tuning and those kinds of things.

Chet Kapoor: Who is typically, who is your customer? Who is the one that uses your product?

Jim Olsen: Well, using, it's an enterprise solution. You can't run governance in isolation. You can try things. Matter of fact, if you're following something like SR117, you have to, the developers cannot sell for you. You have to have an independent organization that does that. So this is some kind of governance organization that they might have already? Yes and no, because it expands further than that because really you want to even tie in your CICD and your DevOps platforms too. So for instance, like one of the features we do, we can kick off Jenkins jobs for deployment and tie it back to the exact version of the model that was deployed. So you have that whole single pane of glass view of what happened to that model and how it got to production. So it spans organization. So typically that's why we see like the CDIO or something like that, where it's a much higher level organization that owns this because it's true enterprise software. You can start in a group, but ideally if you really want to do it right, you're spanning all of the functions of your organization and pulling it together because you start with a use case. You don't even build anything and you bring it all the way to ultimate retirement. That's real governance.

Chet Kapoor: Generally, and then I want to talk a little bit about responsible and governance, but generally this is obviously a board level concern for most companies, right? Not just the CIO or CEO level. What is the deployment time? Like what does zero to MVP look like? .. I know it's a very open-ended question, right? Because how large the enterprise is, what they do. But generally, how long is this if they say go?

Jim Olsen: Yeah, generally, I mean, it depends on the organization. We've had ones that have been just a few weeks. I mean, installing the software can take... I can install it in an hour. It's all Helm. It's containerized, etc. That kind of thing. But the key is more like any... It's like coding software is relatively easy. The coding part. The hard part is figuring out what does success look like? And that's often for an organization is pulling together the different pieces and how do we make these people talk and what does your governance process look like? So we usually try to start out with what we call minimum viable governance, where we just cover the very, very basis and we can get that up and running fast. But we have other organizations like large financial Fortune 500 industries we work with who know they need the full thing. And that takes obviously a lot longer to get all the players to talk together and agree on the process. Not so much the installation of software, but that process and being in agreement.

Chet Kapoor: I agree. No, for sure. So I'm a CIO. I asked the question, Jim, a lot of people are talking to me about why I need governance. I understand why I need it. But a lot of people are also talking to me about responsible AI. How do you differentiate the two?

Jim Olsen: You got the governance, which almost has the angle of kind of CYA. That's why I asked your question, right? Yeah. But then there's like making sure it really is doing the right thing on that. So for instance, we've developed some, again, I didn't find a lot of tests out there. So I had to kind of invent some of my own. And one of the ones we went down the path of is detecting bias in responses based on protected classes. And we developed some techniques and using prompts and et cetera to make sure that they can get, like I changed from, male to female or vice versa, but everything else remains the same. I should be getting a darn similar answer. If I'm not, why not? And like you could see this in early versions of chat GPT-4, you would get very different answers. So we introduced some SBIRT cosine similarity to responses, et cetera, to make determinations there and make sure that the quality of the product you're putting out is okay and comparing response between different LLMs and how they perform that way.

Chet Kapoor: That's awesome. So, two concrete steps for enterprises to get started.

Jim Olsen: Well, the first is you can't know if you're governing AI without knowing what you have. So the first step is getting an inventory and amazing, most people have little to none or an Excel spreadsheet. And that's one of our foundation of capabilities because of that is a very detailed inventory that can auto-populate itself based on information and through the natural flow of the process of doing governments to make it easier to get all the information. So you've got to have that inventory. And then you've got to have, in my opinion anyway, monitoring of understanding how are these things, establish a baseline because natural language, you can't get every under the curve and say this was this accurate. But you have to establish a baseline early on before you deploy it and then continue to see against that baseline is it drifting? Is it responding differently? Is this emotional response different? Is it whatever? So that you can understand if you do have a problem because it isn't simple. There's no ground truth.

Chet Kapoor: Yeah, no, for sure. I love that. Those are two simple things that people can get started on, right? So I recently read your blog and I love this quote, which is real AI has awareness of causality leading to answering questions you haven't dreamt of yet. Yeah. I just love that. Just love the quote, right? I don't mind if I use it and give you credit, but I think it's really awesome. What's your, what did you, was there something specific you were referring to there? And then I have a follow up question.

Jim Olsen: Okay, yeah. Well, a lot of people, there's a lot of kind of doomsday predictions of LLMs are going to create the end of the world and things like that out there. And it's a big lack of understanding of how they semantically work under the covers and what they're doing. They don't have that ability to basically dream up something new. They can weave together what they have and determine through vectorization, et cetera, what makes sense to respond with, but they don't really understand why they're saying that. And that's why the same people who fear AI is going to take over their coding jobs and all that. It's like, no, the hard part of coding, as we already mentioned, is figuring out the problem and the solution. And these things can help you figure out the syntax and proper patterns and way of doing it, which is absolutely helpful and can be a time saver. But the hard part of programming is figuring out what the heck to do.

Chet Kapoor: No, for sure. So what's your bold prediction for the future of AI?

Jim Olsen: I think we'll start to see some things like, I mean, I think to address that, we're going to have to start to see more of a long-lived feedback mechanism somehow where it can learn from its mistakes. So how do we basically create loops within the network that actually things and then persist that over time? Because right now, it's like, you can start with the same seed, you'll always get the same response. It can increase the temperature and change it randomly. You'll get a slightly different response. But, it's still pretty consistent in what it's doing. So how do we make it learn from its mistakes and grow over things? And I also think one of the fundamental things for true AI is having needs. If you don't have needs, wants, feel pain, et cetera, how do you learn? I think my greatest lesson is that it's all the mistakes I've made, not all my successes.

Chet Kapoor: Yeah, for sure. That's a really, really good way to put it. We were talking before we started about you spending the summer in the Rockies and you're off the grid, right? And you're obviously doing the podcast from there. What was and continues to be the hardest part of being an off-the-grid CTO?

Jim Olsen: The hardest part is, I would say, kind of, there's always some connectivity challenges. It's gotten much, much better on that kind of things. But also, the idea of going out to travel and business, it involves more effort. Or if there is a problem, you've got to fix it and you've got to get it done before the next meeting. It's like, oops, the power went out. Better go figure it out now. I've got the system pretty good, so it doesn't happen. You have things like, oh, the water froze. So these kinds of things can pop up in the middle of when you're trying to do business and stuff. There's not heat unless I throw a wood in the stove. There's not there. Very self-sufficient.

Chet Kapoor: So, Jim, what is the benefit of doing this? I mean, obviously, it helps your state of mind. You're in the wilderness. You're alone. All those things can be very – but from your perspective, you're obviously doing it because it actually helps you up your game. How does that happen?

Jim Olsen: There's two radically different aspects of it. One, as I mentioned earlier, coding is problem solving. What's the ultimate problem solving than basically doing what I call bush fixes, where it's like, oh, something broke. It's an hour to town. I'm going to figure it out and how to make it work with what the parts I have on hand are. That's not that different than coding when you get down to it. But the second more important part is you are on the computer all day. You're doing code. You're reading blogs. You're joining webinars. You're whatever. To be able to walk away at the end of the day, walk outside, and look at peace and quiet and relax and let your mind reset so that you have energy to do it again the next day is, I think, vital to humans as a whole to have that ability to step back and walk away from home when you need to. That's true.

Chet Kapoor: That's awesome. That's great. I'm glad you're enjoying it. Okay, we're at the rapid-fire section of the podcast. So I'll ask you five different questions, quick responses. So what's your favorite programming language?

Jim Olsen: Java is a core for enterprise software. It's just so ubiquitous.

Chet Kapoor: Yeah. Favorite LLM to use at work?

Jim Olsen: Locally, I've really enjoyed playing with Orca 2B lately. It was slightly different. And remotely, chat GPT 4.0 is kind of a standard.

Chet Kapoor: Yeah. One thing in your life you want AI to automate?

Jim Olsen: My laundry.

Chet Kapoor: That is such a consistent answer, right?

Jim Olsen: Who wants to do laundry?

Chet Kapoor: I know. That's such a consistent answer. If you could live off the grid anywhere in the world, where would you go?

Jim Olsen: I think I would enjoy northern Norway.

Chet Kapoor: Wow. Wow. You do like the cold. One AI governance trend you are most excited about?

Jim Olsen: The general recognition by the industry that governance is important and can help us. It's not meant to be punitive. It's meant to help your business.

Chet Kapoor: Correct. Correct. So you can get some agility, get going, find out what the problems are. And this is something, Jim, I actually talked to a bunch of folks about, right? When we were doing this on the Internet, when the web came around, no one thought that you'd be able to cash checks on HTTP, right? And guess what? We're doing that all the time, right? And so people just have to realize that this is a journey. But while you're on the journey, magic is not going to happen. You will start something. You will be successful. But sometimes you'll fail. And you have to learn from those. And especially because AI is so human-like, right? You will have to learn from it. And then there's just a gradual iterative process, is my point.

Jim Olsen: Yeah, absolutely. I mean, that's been the way since I've been doing this for a long journey, and we see cycles, and it gets better. And sometimes the things we think are going to be big, not so big. Sometimes the things we thought were like, oh, that's cute. They become mainstay.

Chet Kapoor:  No, for sure. For sure. Jim, this has been awesome. Thank you very much. It was great catching up with you. And the conversation we had before about you being off the grid as well as on the podcast itself, I really, really enjoyed it. We appreciate the time. Thank you.

Jim Olsen: Oh, thank you as well. I really enjoyed it as well.

‍