I was working on an ROI model for AI agents when the first version started to feel wrong.

The model itself was straightforward. We could see an agent session, calculate what it cost, associate some value with it, and produce an ROI number. Technically, it was clean.

The trouble appeared when I stopped thinking about the session and started thinking about the work.

A customer request might involve several sessions. One agent might hand something to another. A workflow could pause and continue later. The business might see all of that as one outcome, while the system sees five separate executions.

The calculation was accurate. The unit was wrong.

That distinction changed the problem for me.

We often design products around whatever the underlying system already gives us. Observability systems give us traces. APIs give us requests. LLMs give us tokens. Agent systems give us sessions and runs. These are convenient because they already exist and can be measured precisely.

But the easiest thing to measure is not always the thing the customer is trying to understand.

In this case, the more useful question was not, “What was the ROI of this session?” It was closer to, “What did it cost to get this piece of work done?”

Once I framed it that way, some decisions became easier. Multiple executions could contribute to one business result. Technical telemetry could remain technical telemetry instead of being forced to carry business meaning. We could also avoid introducing another object just because it would make the internal model look neater.

That last part mattered.

One possible solution was to create a new object above the session and make everything attach to it. It would have produced a clean hierarchy. I kept asking myself a simple question: does the customer need this object, or do we need it because our architecture does?

I could not justify it yet, so I preferred not to create it.

Since then, I have started asking a different question earlier in product discovery: “What is the thing the customer is actually trying to count?”

Sometimes the answer really is a request, a session, or a server. There is nothing wrong with technical units when they match the decision the user needs to make.

Problems begin when we inherit the unit without questioning it.

That mistake rarely stays contained. The product starts needing more filters, grouping rules, correlation fields, exceptions, and additional concepts to explain why the numbers do not quite match how the customer thinks about the work.

By then it looks like UX debt or data-model complexity.

Often the original mistake happened much earlier.

We made the wrong thing first-class.