A featured contribution from Leadership Perspectives, a curated forum for enterprise technology leaders, nominated by our subscribers and vetted by the CIOApplications Editorial Board.
The Kraft Heinz Company
Setting MLOps for financial success
Jorge Balestra
2023 is shaping up to be a tough economic year for the world economy, so the number of hard discussions will increase. ML is a growing and exciting area, but it will not be immune to the tightening belt on the horizon. Increased questions directed to the ML leadership change and start to head into spend reduction territory —“How much is the cost to run X model? Can you reduce your ongoing cost by Y %?.”
As the proverb says, “A journey of a thousand miles begins with a single step,” but what should be our first step? In my experience, the immediate steps can be grouped into three primary areas:
• Financial Architecture and deployment
• Standards and Processes
• Routines
Financial architecture and deployment
A clear technical architecture is mandatory and commonplace in most organizations, and the challenge is how the technical architecture is aligned with key financial principles. Financial principles require that technical components can scale, report, be replaced, and change according to the now-needed directives and not optional.
• Can the technical architecture easily change to accommodate financial variability? Will you cost scale up and down based on usage7
• Is a clear cost model built in? Is every cost element identified?
• Is your ML foundation able to change to facilitate cost arbitrage if economic conditions (funding and cost) change? Have you considered the pros and cons of native vs. agnostic cloud tools?
Standards and Processes
Standards that are fully documented and enforced are the best practices that most organizations follow. However, standards that support and enable quick evolution faced with a changing financial environment are not as common. Standards that align with healthy financial management can address questions like:
• Are the most cost-efficient methods used consistently?
• Are standards on modularity clear so loads can be sent to the most cost-efficient resource?
• Is waste caused by duplication prevented by existing standards and processes?
Routines
Consistent financial performance rests on the ability to monitor and take corrective action in a timely fashion continuously. Success depends on the right people looking at the right indicators at the right time. Key areas to address are:
• Is reporting at an adequate level of granularity available without manual intervention?
ML is a growing and exciting area, but it will not be immune to the tightening belt on the horizon.
• Are actual results vs. budget reviewed regularly, and are action items reviewed for progress?
• Do continuous improvement techniques work, so costs are optimized over time in every process? Have you enabled real-time cost alerts so you can proactively act?
The points above are not meant to be all-encompassing but should be used to ask the right questions to set up MLOps for success. A solid foundation, especially when setting up the ML capabilities, will pay off in the long run. Failure to set up correctly leads to progressive worst outcomes. You may need to retrofit—always a costly and extensive alternative. In the worst-case scenario, you will face a financial crisis that will force a reactive mindset.
