Controlling the cost of AI

I don’t think there’s a single leader we talk to that isn’t worried about AI spend. It’s budgeting and OKR-setting season, and so everyone is trying to answer two messy questions: how much are we going to spend on AI next year, and what do we measure to prove it’s working?

We’ve discussed how to measure AI impact previously, and I’m sure we will again very soon. But we find that we’re usually pushing measurement on orgs, while everyone is proactively asking us about controlling spend (perhaps because it feels more tangible; perhaps because the CFO won’t take “I don’t know” for an answer).

If you want predictable spend, it’s tempting to reach for hard individual caps. That makes it straightforward to calculate the upper bound. But hard caps don’t only limit spend; they also limit your potential return. Based on talking to and working with dozens of companies, we think there’s a better way.

You can’t optimize what you can’t see

Not only do you as leaders need granular visibility into AI spend, but your front-line managers and individual contributors need visibility as well. You can’t expect them to optimize and control something they can’t easily see. The more transparent you can be, the better. Give your teams the information they need to track, compare, and educate themselves.

We’re also big fans of capturing and using session data for good. If you can do that, it’s a huge advantage. If you can’t do that, you should think hard about why not and what it would take to get there. This allows you to optimize intelligently at the org level through model defaults, model routing, eliminating invisible token waste, and more.

Once you’ve identified, educate and learn

Once you know who your top and bottom spenders are, go talk to them. Top spenders typically fall into two categories: the first is people who are doing innovative, amazing things that you and others in the org can learn from; the second is people who are doing naively wasteful things (“Fable for everything!”) and simply need a little education. And don’t forget to talk to your bottom spenders too. These are people who haven’t figured out how to get value out of the compute yet, and they offer their own opportunities for both providing and receiving insights.

Reward, don’t punish

Hard caps will punish your top performers and discourage experimentation. We often see a small subset of builders who are getting considerably more leverage out of AI than their peers. Hard caps may rein in the wasteful spenders, but they will also prevent your best from producing their best. Those who are demonstrably getting more leverage out of AI should be given more token spend, not less. They’ve earned it and should be rewarded.

Soft caps paired with visibility can set expectations and reinforce incentive structures in a healthy way -- assuming your culture is already healthy enough that people won’t treat a soft cap like a non-negotiable hard cap. Soft caps should be a guardrail to keep a system of trust on track, not an enforcement tool. Whatever energy you were planning to exert on enforcement and exception handling is better redirected somewhere else, like measurement, education, or widespread optimization.

Here’s what we’re reading this week.

Apple cuts AI usage budgets

Bloomberg reported this week that Apple cut daily AI spend limits for some engineers from $300 to $200 a day, with a further reduction to $100 a day planned. There’s plenty to unpack here, but let’s be clear that $100 a day is still a pretty healthy budget. While we still don’t believe in hard caps, if you need an upper bound, at least make it an upper bound that’s in line with realistic usage. Also, based on the report, the cap doesn’t seem to apply evenly to all engineers, so it’s not a true global hard cap.

Accenture sets $29 per month individual Claude budget

This is from a few weeks ago, but 404 Media reported that Accenture was limiting centrally funded Claude use for many employees to $29 for an entire month. That is roughly 1/70th of Apple’s planned $100-a-day engineering allowance over 20 working days. Beyond the $29, employees need to tie their work to a specific team or project to get additional usage. This sets expectations that low usage is the baseline and puts the onus on individuals to prove the value of their usage. We’d rather see leadership redirect that energy into measuring, defining, and optimizing ROI in ways that impact broad swaths of the company.

Cloudflare’s soft cap

Cloudflare doesn’t hard cap, but they do lightly disincentivize spend by escalating who gets notified. Similar to Apple, the limits are high enough that there’s actually not much to see here (hundreds of dollars per day). In contrast to Accenture, where employees need to justify the spend before it happens and for very low amounts, it’s not unreasonable to think you might need to explain to your manager or a leader what you’re doing with $1,000 a day.

How software engineering is changing: a tale of two eras

Neil Kakkar is a software engineer at AI-native startup Tano, with previous stints at PostHog (pre-AI) and Bloomberg. In this guest post on the Pragmatic Engineer newsletter, he walks through how his own workflows and output have changed. He leads with an impressive stat: an 11x increase in his monthly PR output at Tano compared to PostHog. But that didn’t come “for free”; much of the surrounding work had to change to make that possible. Individual engineers own more, human code reviews become optional after onboarding, and mistakes get fixed forward.

He’s also candid about what gets harder. More parallel work means more context switching, more mental fatigue, and less shared understanding of what everyone else is building. We appreciate an account that includes both the impressive output and the part where you’re exhausted by midafternoon. There’s plenty here for leaders to think about beyond whether their engineers are shipping enough PRs.

Your productivity gains are limited if the rest of the company is not onboard with AI. It’s clear that AI is necessary, but it’s not sufficient.

Guillermo Rauch on what AI could improve in software

Guillermo Rauch, CEO of Vercel, isn’t afraid of a bold prediction. Here he argues that AI will (eventually? soon?) rewrite most human-authored software, and that software will be better in almost all ways as a result.

In many ways, we are witnessing the true birth of software engineering.

You don’t have to agree. But it’s worth thinking about what you will wish you had done differently if it turns out he’s right.

One thing to try this week

Talk to your team! Find out who your top spenders are, and ask one or more of them to walk you through how they’re using AI.

Be clear that you’re coming from a place of curiosity and learning, not judgement or wrist-slapping. Can you tell if they’re being innovative or wasteful? If not, you’ve got your own upskilling to do. But if you can, you’ve just learned something valuable.

Maybe even something worth sharing with the rest of the team as well.

As long as the agents do their chores, they deserve an allowance.

— Claire + Zach