Real change in, real change out

We like to talk about the need for AI "transformation" rather than AI "adoption". Transformation conveys a degree of change that adoption alone does not. Far and away the most common pitfall we see is teams adopting AI tooling but not changing the way they work. It's hard to get real change out if you're not putting real change in.

In a recent post on X, Aaron Levie uses the term "process reengineering" to describe changing workflows to take advantage of AI rather than "just layering on agents into the existing flow". It's a great name, and it's worth examining whether your approach to AI fits this moniker.

We believe that the engineering function is a great place to start AI transformation. Because of the highly verifiable nature of software development, as well as the fact that models are very good at writing code, if you can't transform the way your engineering team works with the current tools, you're going to have a hard time doing it anywhere else. And also, you're going to learn a ton from making agents successful in your software development lifecycle (SDLC), or perhaps making your SDLC successful with agents, that can be applied to other functions as well.

That last point, that maybe you need to adapt your SDLC to agents instead of the other way around, that's process reengineering. Other functions within your company probably don't have anything nearly as well defined or obsessed over as the SDLC. But I assure you there is a process lurking in there, waiting to be reengineered, and that's where you're going to find real leverage.

The most AI-forward companies aren't just adopting tools broadly, they're using what they learned from reengineering the SDLC to rethink processes and workflows across the organization around the joint capabilities of humans and agents.

Here's what we're reading this week:

Gabriella Garcia on running non-engineering teams like engineering teams

I run six teams at Warp. Not one of them is engineering. All of them run like engineering teams

Gabriella, COO at Warp.co, lists out 5 ways those teams have changed the way they operate, from tracking all work in Linear to "shifting left" and investing in the system when something breaks. She doesn't call it process reengineering but she says directly "I have changed how my teams operate".

Inside OpenAI’s agentic software factory

Gergely Orosz pulls back the curtain for a peek inside how OpenAI operates. You need to be a paid subscriber to see the full thing, and Gergely has one of the best engineering newsletters out there so it's worth it, but there's also lots of great content in the abridged free version. In addition to fascinating insight about how OpenAI uses custom agents to "hand-hold" a change as it's deployed to production, there's a section on internal adoption of Codex:

In a four-month period, non-engineering orgs like finance, recruitment, and legal went from ~0% usage of Codex to 90% usage. Now, almost all OpenAI employees use Codex and ChatGPT Work weekly.

So we get concrete examples of how OpenAI is reengineering their software processes, alongside proof that it's not just engineering and agentic workflows have spread to many other functions as well.

Aaron Levie’s enterprise-agent field notes

Aaron Levie lists out some of the trends he's seeing recently from conversations with other leaders in the technology space. Our two favorites are process reengineering (no surprise) and a trend reversal away from consolidating on a single model vendor:

Most companies are deploying multiple frontier models within their enterprise. Too hard to standardize on anything and seeing different preferences across their teams and use cases.

Adopting the software factory model: crawl, walk, run

Zach Lloyd, CEO of Warp.dev, is back with another software factory banger. Still wrapping your head around software factories, or don't know where to get started? Zach lays out a crawl-walk-run strategy to make it easier to reason about.

• Start with cloud agents and triggers to automate certain classes of tasks
• Layer on things like tool integrations, shared context, and governance to solve common problems
• Flesh out (or buy off the shelf) the rest of the "factory stack" only when you're ready or consistently running into friction

Building a "software factory" can be daunting. But starting to understand how they work by putting some building blocks like cloud agents and triggers into place shouldn't be, and you might just be amazed by what you can do with those building blocks.

Inside Vera, DoorDash’s Data Agent

The DoorDash data team gave us a detailed explanation of why and how they built a custom agent to handle data questions from across the org. They created an evals framework that allowed for contribution from domain experts, and then focused on improving performance against those evals even as more evals were added:

As we developed Vera, our pass rate rose from 43% to 90% while the evaluation set doubled in size twice. Better models helped, but gains from new releases grew smaller. Our biggest improvements have come from the harness, especially business context, knowledge retrieval, and data modeling.

Literally every data team we talk to is dealing with this same problem (how to enable agentic access to data while maintaining a high quality bar), so we love seeing learnings from a team that has developed a solution.

One thing to try this week

Read about Jev, TypeSafe AI's probabilistic decision model. It's worth understanding how it differs from the LLMs that have transformed the industry.

Then give it a try. It's available on Vercel's AI Gateway and is way way cheaper than LLMs, so there's really no excuse not to.

We already replaced our PR risk classification with Jev. What will you do?

Are you having fun? We're still having fun.

— Claire + Zach