Every token you send to an LLM costs money, consumes context window, and potentially dilutes the quality of the response. After building six AI-powered products, I got obsessive about token efficiency — and built LCC (Local Context Compiler) to formalize that obsession into a tool.
The problem is easy to see in practice. A typical support chatbot prompt includes: the system instruction (200 tokens), conversation history (500-2000 tokens), retrieved context from a knowledge base (1000-3000 tokens), and the user's latest message (50-200 tokens). At scale, you're sending 4000+ tokens per request when 60% of that is redundant boilerplate, repeated headers, or irrelevant context chunks.
LCC attacks this at four levels. First, whitespace normalization — collapsing multiple newlines, stripping trailing spaces, removing empty lines. This alone saves 5-10% on messy inputs. Second, structural deduplication — if the same paragraph appears twice in a retrieved context block (common with overlapping RAG chunks), LCC keeps one copy and removes the rest. Third, abbreviation compression — replacing repeated long phrases with shorter tokens where the LLM can still understand the meaning. Fourth, token measurement — counting the exact token cost of any text block using tiktoken, so you can make informed decisions about what to include.
The results on real production prompts surprised me. On a legal intake system processing Portuguese-language case descriptions, LCC reduced average prompt size by 42% without any measurable degradation in response quality. On an English-language customer support system, the reduction was 35%. The savings compound: lower token costs, faster response times (fewer tokens to process), and more room in the context window for actually relevant information.
One technique that works remarkably well: context windowing with priority scoring. Instead of stuffing the entire conversation history into every prompt, score each message by relevance (recency + semantic similarity to the current query) and only include the top N messages that fit within a defined token budget. This is how OmniAtende CRM handles long customer conversations — the bot always has the most relevant context without blowing past the context window limit.
A common mistake I see in AI projects: treating the context window as infinite. GPT-4 Turbo gives you 128k tokens, so developers dump everything in. But LLM attention degrades over long contexts. A focused 4000-token prompt with high-relevance context consistently outperforms a 40,000-token prompt with everything including the kitchen sink. Less is more, but only if the 'less' is carefully selected.
LCC is open source and designed to run locally with zero API calls. It's a deterministic text processing pipeline — no neural network, no external service, no latency. You pipe text in, you get optimized text out. I use it as a pre-processing step before every LLM call in my production systems.
The roadmap includes integration with the Agentic Prompt Intake protocol, so agents can clarify requirements and compress context in a single pipeline before execution. Token efficiency isn't just about cost — it's about signal quality.