VocabX cuts SLM inference cost with a domain-specific vocabulary

What is changing?

Dharma AI introduced VocabX on October 21, a method that rebuilds a small language model’s vocabulary around task cost. In its production Portuguese essay grader, the Qwen3‑4B base, data, training recipe, and seed were held constant while only the tokenizer changed. The company says these results come from the current version of a paper still under review and may change before publication.

Instead of selecting only frequent pieces, VocabX accounts for the measured cost of input and output tokens. It rebuilds subwords while turning repeated long expressions and JSON structures into single “macro tokens.” In the test, vocabulary size fell from 151,669 to 64,767 and average output length dropped from 350 to 101 tokens.

Across the same 500 requests on an RTX 5090 with vLLM, Dharma AI reports a 77% reduction in measured inference cost and a task score increase from 0.703 to 0.810. This is a provider measurement from one production setup; other hardware, stacks, and tasks require independent validation.