Role: Developer Org: Open Science Laboratory (OpenSciLab) Distribution: Python package, on GitHub and PyPI
Summary
Estimates token counts for LLM input using rule-based and ML methods. Built for prompt analysis, token budgeting, and optimizing interactions with token-metered systems.
The design point is the cost of getting that number. An exact count means loading a whole tokenizer; ToCount is for the case where you need the figure quickly — sizing a prompt, staying inside a token budget — and don’t want to pull in a tokenizer just to get it. It reaches that estimate by rule-based and ML methods.
It is one of the natural-language and LLM tools in the OpenSciLab portfolio, sitting next to Memor and XNum.
Links
Related
Memor — Conversational Memory Across LLMs — sibling LLM tooling. XNum — Universal Numeral System Converter — the other text-preprocessing utility in the same OpenSciLab NLP group. Open Source Philosophy — why these tools get written and released at all.