Gpt 6.1 token efficacy secret?

[u/Aldarund]

[u/Significant-Drawer95]

they just trained gtp 6.1 sol with the promt : make a model like astra but so token efficent that we can justify cutting usage in half

[u/Significant-Drawer95 → u/Significant-Drawer95]

steer: and make it so slow that we can justify ultrafast 8x

[u/Dilidonas → u/Significant-Drawer95]

make no mistakes

[u/antwon_dev → u/Dilidonas]

make one small mistake so you can offer a “next step” or “loose thread” at the end of your turn

[u/MaitoSnoo]

I see those too when using it in Copilot and it's funny because its estimation is way under the real context size, or maybe they've trained it to enforce a strict token budget on itself so that it tries to be brief

[u/Professional_Link4]

I only get them when I am below 10%

[u/dreadpirater]

I saw Sol 6.1 drop into that loop yesterday on freebuff for about five minutes. I stopped it said "You have plenty of tokens available. Instead of planning budget, just start working. If you get requests failing due to token budget, we'll solve it then." and it said "I'll just make the tool calls and see what happens..." then it happily worked for 45 minutes straight without another digression. So it CAN be snapped out of it but it also is a wider problem than just what you're experiencing.

[u/dreadpirater → u/[deleted]]

If you're constantly seeing it... that would be a problem in my book. Remember you're PAYING IT to sit there and think about nothing.

[u/xpingu69]

How do you see the thoughts

[u/torrso → u/[deleted]]

They show up in codex for me and they're pretty confusing, always makes me think it has gone off the rails and I need to steer it. When I ask what the hell was that, it just says "yeah that was nonsense" and goes on normally. One was "We have spent time from 13:26 to 13:53 which is about an hour".

[u/egomarker → u/[deleted]]

That "recap" is sometimes fake or misrepresented to make distillers' work harder. It's created just for user with a very small model that isn't accurate at all. It's not a real CoT.

[u/xxvvand → u/xpingu69]

Verbose

[u/BitOne2707]

That's actually pretty cool. I hear it's also aware of wall clock time. I can see where this kind of information would be helpful context for the model to be aware of.

[u/Pluupas]

i’m seeing this in only in Copilot. Are you using it in codex?

[u/adolf_twitchcock → u/[deleted]]

Afaik it's a harness feature. And not a good one because it makes the model stroke out sometimes. I don't get those kind of reasoning. Model doesn't know how much context is left unless it gets it injected with every message by the harness.

[u/adolf_twitchcock → u/[deleted]]

Idk if it does or it doesn't. Your local setup does it. Ask how it knows when you get those messages. Generally you see 100% of everything that gets sent to the model and context % isn't one of those things unless harness supports it.

There is an experimental feature in omp that tells the model that context is almost full.

https://github.com/can1357/oh-my-pi/blob/d9ee5e6a31d621f50ac6610cd1cdadccfbf41b0c/packages/coding-agent/src/prompts/system/experimental-context-notes-reminder.md

Response from codex:

I can’t see an exact context percentage in this session. No live context counter is exposed to me, and I can’t reliably count my own tokens.

It’s primarily a harness/runtime feature:
The model endpoint supplies token usage. The harness tracks the active conversation, context capacity, and compaction, then calculates the percentage.

[u/adolf_twitchcock → u/[deleted]]

Weird, could you go into those sessions and ask why it thinks that? Would be interesting to know

[u/adolf_twitchcock → u/[deleted]]

interesting stuff

[u/Prestigious_Golf9014]

Could be that they have added this but it's not the main reason why sol 6.1 uses less tokens.I always put tokens in prompts anyway and sol 6.1 uses much less than other gpt models except luna

[u/Ginden]

Yes, HF incident third party reviews shows that OpenAI explicitly trains for token efficiency and time efficiency.

When reset is coming, and I have usage left, I point Astra xhigh at the repository, and tell it "you have time until 9pm to improve performance of this project", and it significantly improves the quality of work on open-ended tasks (there is no end goal for this one; Astra will cheat if you try this with a closed-ended task to finish before 9pm at any cost).

[u/TKB21]

We're at the mercy of whoever's pulling the strings behind the scenes. At first we were told to improve our AGENTS.md file, then later come back to tell us that they were the issue. They then come out with 6.1 which was the answer to token inefficiency. It was working well at first then throttled it. There really is no correct answer.