About TokenDyno

A dynamometer for LLMs — continuous, independent measurement of real inference speed, set against vendor claims. No manual runs, no cherry-picked results — just a worker that fires on a fixed schedule and writes what it finds.

Why it exists

Vendor TPS claims are unreliable. They are measured under conditions the vendor controls — ideal prompts, empty queues, sometimes first-token time folded into the rate — and they almost never reflect what a real user sees on a real request. TokenDyno exists to measure the thing that actually matters to a consumer of these APIs: how fast does this model generate text for me, right now, on the plan I would buy?

Independent, continuous measurement is the only way to get that number. One-off benchmarks go stale; vendor dashboards advertise peaks. A worker that hits the same endpoint on a fixed schedule produces a living record that tracks outages, throttling, and quiet regressions vendors never announce.

How it works

A long-running worker benchmarks every model on a fixed cycle — about every 10 minutes on Ollama Pro, about every 60 minutes on the capped plans — across every provider, using the same prompt and the same hybrid measurement method for every run. Results are stored in Postgres and served via a small Hono API. The site is a static Astro build that hydrates chart islands client-side. See the methodology page for the full measurement spec.

Because every provider is benchmarked on the same engine with the same method, the numbers are directly comparable — you can put an Ollama model next to an OpenCode Zen model next to an OpenCode Go model and the comparison is fair.

Multi-provider, same yardstick

TokenDyno benchmarks three providers side by side, across four plan tiers:

  • Ollama — hosted API, benchmarked on both the Pro and Free plans
  • OpenCode Zen — pay-per-use API (Zen endpoint)
  • OpenCode Go — monthly subscription plan (Go endpoint)

Same hardware, same prompt, same measurement method. Sampling cadence varies by provider — about every 10 minutes on Ollama Pro's flat subscription, about every 60 minutes on the capped plans — because cadence is spend there. It changes how fresh a number is, never how it is measured.

Who runs this

This site is built and maintained by Anton Gulin. Anton writes the benchmarking engine itself — the worker that runs the measurements, the API that stores and serves them, and this site. It is a personal, independent project; infrastructure costs come out of pocket, which is why donations and referrals via the affiliate links on model pages are appreciated. The full measurement method is public on the methodology page, and the same method is applied to every provider.

Independence & sponsorship

TokenDyno is not affiliated with Ollama, OpenCode Zen, or OpenCode Go, and receives no payment for ranking, placement, or favorable coverage. Every number is generated automatically by the worker on a fixed per-provider cycle — no result is hand-edited or reordered. The sponsor slot on this site is display advertising only; it never influences the leaderboard or model pages.

Some model pages include referral/affiliate links to providers. Those links do not affect the measured numbers or the order in which models appear — they only help cover API subscription costs when a reader signs up.

Support this project

If you find it useful, the best thing you can do is share it. Financial support via GitHub Sponsors helps keep the API subscriptions running (and therefore keeps the data fresh).

Sponsor

Contact

Bugs, methodology questions, or sponsorship enquiries: i@anton.qa