2 Commits

Author	SHA1	Message	Date
Salman Mohammed	75454b96da	feat: use tiktoken-rs instead of tokenizers, single global tokenizer (#3115) ... Co-authored-by: jack <>	2025-07-01 14:42:51 -04:00
jack	495cdfb33c	feat: implement async token counter with network resilience and performance optimizations (#3111) ... Co-authored-by: jack <> Co-authored-by: Salman Mohammed <smohammed@squareup.com>	2025-06-30 08:45:17 -04:00