The FinOps Code for AI Cost Control

As organizations increasingly adopt Large Language Models (LLMs), AI agents, and multi-agent systems, managing operational costs becomes significantly more challenging. Traditional FinOps practices were designed around infrastructure resources such as virtual machines, containers, storage, and network traffic. However, AI workloads introduce a new cost driver: tokens. This repository accompanies the presentation "The FinOps Code for AI Cost Control", which explores how organizations can move beyond infrastructure-level cost allocation and gain visibility into AI consumption at the application level using OpenTelemetry. The project demonstrates practical techniques for tracing, monitoring, and allocating AI costs by tracking token usage across applications, services, workflows, and autonomous agents.

From Resource Tagging to Token Tracing:

The FinOps Code for AI Cost Control
Shifting from Infrastructure Tagging to Application-Level Telemetry
FinOps Evolution - The 7 Phases
The public cloud has existed for more than two decades, since AWS launched its first services in 2002.  
FinOps = the discipline and practice of managing cloud spend
Phase 1: Observational FinOps
The Infancy
Focus: Accessing and collecting cost and usage data.
Goal: Gaining a clear picture of consumption and cost before receiving the cloud bill.
Challenges: Early cloud providers did not expose cost data effectively, and reporting formats varied by vendor.
Improvements: FOCUS (FinOps Open Cost and Usage Specification) provides uniform cost and usage datasets.
Status: Observational FinOps is a necessary, foundational component.

Phase 2: Analytical FinOps
The Childhood
Focus: Analyzing collected data to understand the underlying drivers of cost.
Challenge: It is often hard to understand where the money is, as managed services include costs for compute, network, and storage resources, and idle resources still contribute to cost.
Outcome:
    - Extracting meaning from data is vital for actual optimization.
    - Leads to identifying potential waste, detecting anomalies, and defining
      automated  guardrails.

Phase 3: Attributional FinOps
The Adolescence
Focus: Attributing the undifferentiated cost of resources to specific services to manage infrastructure costs.
Process: Starts with foundational practices like resource tagging.
Complexity: Gets complicated with shared resources (load balancers, Kubernetes).
Impact: Closes the FinOps feedback loop by providing financial data back to engineers, allowing them to evaluate how their components impact the overall system cost.
↑Back to Top