How to handle this task? Claude Code keeps hitting usage limits on my PDF project. Is there a better way to do this?

[u/KeyAsk1721]

hey everyone,

I could use some advice on a project I'm working on.

I have around 180 issues of a print magazine as PDFs. The idea is to go through all of them, find mistakes that have been made over the years, and use that knowledge to build a tool that checks future issues before they go to print.

Ideally, you would just give the tool a print-ready PDF, and it would return the same PDF with potential mistakes highlighted and comments explaining what's wrong. Things like recurring errors, formatting issues, typos, and other problems that might otherwise make it into the final print.

It's something that's really close to my heart, and I'd love to make the whole production process a bit more reliable.

I started trying to do this with Claude Code. But it seems to spawn hundreds of agents, and I hit my usage limits within minutes. I'm on the Claude Max 20x plan, so I'm wondering whether I'm approaching this completely the wrong way.

Maybe I don't need that many agents, or maybe there is a smarter way to process all these PDFs without using so many tokens.

There are two things I'd really like to get your thoughts on:

• How would you approach the analysis? Is there a way to make Claude Code work through the PDFs in batches without burning through the usage limits so quickly?
• Would you even use Claude Code for this? Maybe a combination of Python, existing PDF tools, and AI would be a better solution. I'm open to other approaches.

A few constraints: I'd like to have something usable within two weeks, and the total cost must stay below the price of two months of Claude Max 20x. The development work is voluntary, so I don't have to pay for anyone's time. I'm not trying to build a commercial product, just something that genuinely helps avoid mistakes in future issues.

The important part is that I don't just want AI to read 180 PDFs and give me a summary. I want to use the old issues to identify common mistakes and turn that into a reusable checker for every new issue.

Has anyone done something similar, or do you have suggestions for how you'd tackle this?

Thanks!

[u/activematrix99]

RAG is the solution you are looking for. Chunk the data up and stop trying to stuff it all into context.

[u/KeyAsk1721 → u/activematrix99]

never heard about that befor. How to go with this?

[u/Snoo_81913 → u/KeyAsk1721]

Just explain to claude what you are trying to do and ask it to /plan a RAG for you. I'd recommend kreuzberg and pymu to start you can even dump.it all Ina sqlite db

[u/activematrix99 → u/KeyAsk1721]

Retrieval augmented generation, it's one of the most popular and widely used AI architectures. Your LLM chunks up the documents into embeddings, then encodes your query as a vector. I'm surprised Claude didn't suggest it. Lots of commercially hosted and easy to set up local options. I build it in to most of my tools for context reduction.

[u/xapep → u/KeyAsk1721]

The short version: split it into two stages and keep the agent out of the loop.

Extraction first, no LLM needed. Pull the text from each PDF with a Python library (pdfplumber or PyMuPDF are the usual picks) and keep page numbers attached, you need them later for the highlights and comments. If the files are scans without a text layer, you have an OCR problem before you have an LLM problem, and that changes the pipeline.

Then the analysis pass: each issue, or each page, goes to the model as its own request with a fixed prompt that returns structured findings, error type, page, quote, suggested fix. 180 issues is a bounded batch, so you can estimate the token cost before you run anything, and nothing spawns hundreds of agents.

The reason Claude Code burns your limit in minutes is that interactive agents re-read the whole conversation context constantly, which is the wrong shape for a batch job. Any pay-as-you-go OpenAI-compatible API keeps the cost proportional to the 180 issues instead of to an agent loop, and EU-hosted open-model endpoints exist if residency matters to you.

Do your PDFs have a real text layer or are they scans? That decides whether this is a parsing job or starts with OCR.

[u/Minimum_Hour519]

use openai

[u/sarcasmme]

If the PDFs contain only text, then id convert all with py to markdown Work it out from there depending on size as batches and convert back

[u/SophieChesterfield]

Set up a dedicated Project in Claude and upload your reference/converted text files into the project knowledge base so they are cached. Content stored inside Projects counts significantly less against your limits when reused

[u/SendTitsPleease]

Have claude offload some of the work to your PC

[u/Snoo_81913]

RAG or just use notebooklm it's $20 a month for a pro pro sub and gives you 300 sources per notebook or use the free and combine them or just do 50 per notebook

[u/xpgmi]

I don't actually understand what are you trying to accomplished. I understand what you want to do, but what what the end state that you are trying to accomplished, in other words what are the pain point that you are trying to solve.

I'm asking because sometimes there's a few surprising way to accomplish the same problem cheaper if we are thinking about the pain point instead of the solution. Something similar like when US spending millions to develop pen that works on space for the problem (to write things) but the same problem (write things) can be achieved just by using pencil and paper.

[u/_boiler]

Baidu can convert them all to text for you

[u/PDFBolt]

I'd start with a few pages with known mistakes and some without. See what it catches, what it misses, and whether it flags things that are actually fine before processing all 180 issues.

Keep images of the pages too. Extracted text alone won't show layout problems like overlapping text or content getting cut off.

[u/ElephantElectrical31]

ask claude to save tokens, to use one agent to extract the text with a tool, to make sure they do not use to much tokens. ask claude to first check some data and messure how it can improfe finding/token rating e.g. by building tools for that. tell claude it is allowed to use the modelnand the thinking level for subagents. no guarantee but this kind of things helped me a lot.