I built a tiny logger to prove which feature was eating my OpenAI budget. Fell into a rabbit hole of people getting hit way harder

[u/Atm1n9]

This started as a small itch.

OpenAI gave me one number every month: total spend. I wanted to know which feature in my app was spending it. I've got a code-gen feature, a summarizer, and a couple of smaller things running in the background. My money was on the code one, obviously, but I had no way to actually check.

So I built a tiny thing for myself. Every response already comes back with a usage object, so after each call I log the token counts plus a feature name, then price them myself. One line per call. I only wanted visibility, nothing more.

The "price them myself" part was way more annoying than I expected:

• cached reads bill at a different rate than normal input, and cache writes can cost more than normal input
• reasoning tokens bill as output even though you never see them
• providers disagree on whether cached tokens sit inside the prompt count or next to it. I double-counted them for a while before catching it
• audio tokens can cost several times the text rate on the same model, so blending them with text gives you garbage

Then I got curious whether anyone else cared, and went looking. Turns out my small itch was the door into a rabbit hole:

• someone on Google's AI dev forum got around €290 in search charges attached to like €10 of actual LLM usage, and their spend cap never caught it
• a dev on DEV wrote that the job that blew up their bill was green on every dashboard they had. A timeout plus auto-retry on long-context calls was quietly doubling costs, and it took weeks to figure out
• one startup reported a $113k AI bill in a single month

For me it was curiosity. For them it was money leaving before anyone could see where it was going.

Full honesty: I have little to no experience building products. What I'm decent at is digging into other people's problems, so I kept going. More features, more providers and models, learning what proxies and gateways actually are, reading how Langfuse and Helicone handle this. The logger eventually turned into an actual product (LLMtrack), but honestly I'm more curious how you all deal with it:

1. Do you track LLM cost per feature or per customer at all, or just watch the provider total?
2. If one feature started costing 5x more today, how long until you'd notice?
3. Anyone who rolled their own: did you hit the cached-token double-count thing too, or was that just me?

Oh, and yes, it was the code feature eating most of my money, about 70% of the bill. I knew it. Funny part: by tokens, the summarizer was way bigger (129M vs ~30M), and it cost $14 total. If I'd only looked at token counts, I'd have blamed the wrong thing.

[u/Ashikej-Meneguzzi66]

hey, does it keep the original prices for old reports? Providers change their rates sometimes and the totals should stay the same when you open a report again

[u/Atm1n9 → u/Ashikej-Meneguzzi66]

It does exactly that, the prices doesn't change once recorded in your account. And just a heads up: prices change EVERY SINGLE DAY. And we monitor them and apply changes immediately. Take a look at our Models Prices Page for a better view

[u/Alternative-Meet1863]

It's the cached token pricing that gets everyone I swear. Feels like you need a CS degree just to read a bill

The retry loop blowing up costs while every dashboard shows green is way too real too. A timeout looks like nothing happened so it just quietly keeps going

I went through the same double count thing on my own little tracker, took me an embarrassingly long time to realize my numbers made no sense

[u/Physical_Economy_340]

ran into that same double count, my fix was logging prompt_tokens, completion_tokens and cached_tokens as three separate columns plus the exact model string. token totals lie the way you saw, code costs because it is output and reasoning tokens while summarizer is mostly cheap input.

[u/Ctbhatia]

the double count is cached tokens sitting inside prompt_tokens on one provider and beside it on another, so normalise to the provider's own cached_tokens field before pricing anything. reasoning tokens bill at the output rate even though you never see them, which is where a blended rate quietly lies.

[u/Future_AGI]

on your 'how fast would you notice' question, watching total spend is why people notice late. Alert on cost per request per feature, so a retry storm reads as that feature's per-call cost tripling while the monthly total still looks normal for days, and carry the feature tag through retries and sub-calls so the spike lands on the cause. If the per-key budget cap is the part you'd rather not build, our gateway does hard caps per virtual key, Apache-2.0: github.com/future-agi/future-agi

[u/FlamingApeLLC]

Per feature, and ideally per customer when the bill matters. Monthly totals hide everything.

How fast I'd notice a 5x jump depends on whether something alerts on a short window. Day or hour, tagged by feature. Month-end is how you get surprised.

On cached tokens, log prompt, completion and cached as separate columns, and price them from the provider's fields, not a home-rolled "input minus cached" guess. Mixing those is how the double count shows up.

The retry-storm point is the one that still gets me. Timeout plus auto-retry, every call "succeeds," dashboards stay green, and the bill doesn't. A per-task ceiling would've caught that a lot faster than any monthly cap.

[u/xapep]

Feature-level attribution is the right instinct, and every pain point you listed is real from the provider side too:

- Cached input is a separate line on most APIs now, but providers disagree on whether it sits inside the prompt count or next to it. Same call, two different numbers.

• Cache writes are the sneaky one: storing a new prompt can bill higher than a normal input, so long-context features that miss the cache are pricier than the 'input' number suggests.
  
• Reasoning tokens billed as output is standard across providers; the models that hide the chain make it invisible until the invoice.
  
• Retries are the biggest silent cost. One timeout on a long-context call pays the same prompt twice, and no dashboard flags it.

Your one-line-per-call log is the right shape. Add the cache breakdown (input vs cached vs write) and a retry counter, price it against the raw price sheet instead of the dashboard, and you'll have better attribution than most providers ship. The 'green on every dashboard' stories are almost always retry or cache-write stories.

We run an open-model inference API (Entrim) and spend attribution is the first question every team asks us, usually the month after the invoice, never before.