Benchmarks

Documentation / Reference

Benchmarks

How the numbers on the home page were measured: the repository, the runs, the dates, and the run a competitor finished first.

Time until dependencies are ready

This is the 5.3 seconds on the home page.

RunnerDependencies readyWhole job through typecheckRuns
Our machine5.3 s46 s5
Jog Max46 s85 s3
Jog standard76 s137 s3
  • Repository: our own monorepo, 3,677 packages, the same commit on every runner.
  • Cache state: warm. The commit had already run on each runner, so its cache was filled before the timed runs.
  • Timed interval: reading or downloading the cache, up to the point where dependencies are ready to use. The second column is the whole job to the end of typecheck.
  • Figures: medians of the warm runs.
  • Size: 16 vCPU and 64 GB on every runner.
  • Date: 17 Nov 2026.

The difference is where the cache was. Our machine read its dependencies from its own disk. The Jog runners downloaded their caches first.

With the tests included, Jog Max finished first

On 18 Nov 2026 we ran the same repository to the end of its full test suite. One warm run on each runner:

RunnerWhole job with tests
Jog Max538 s
Our machine567 s
Jog standard830 s

The cache on local disk won the set-up. Jog Max ran the tests themselves faster and finished first. A job that spends most of its time in tests gains little from a faster set-up.

Time for a free machine to pick up a job

The 2 to 3 seconds on the home page is the time from a job being queued to its start when a pre-connected standby runner was idle. We measured it with jobs run one after another against a single standby runner. Without a standby runner, a job waits for a new virtual machine: 33–37 seconds from queued to started in live runs on our own pool. How quickly a job starts has the conditions.

What these runs do not show

  • The dependency benchmark ran on our own CI server, with an ordinary GitHub Actions runner and a cache on its local disk. It did not use the Smart Workspaces action.
  • The ship classes on the pricing page have not run this benchmark. We publish no speed for them.
  • One repository was measured. Yours will differ: time a job of your own before you rely on a figure here.
  • We do not claim faster cores, a guaranteed job time or a guaranteed start time.
  • The run with tests is one run per runner, not a median.

The prices in the cost comparison

The home page and the pricing page compare a Frigate, 8 cores · 16 threads · 64 GB at $144 a month, with metered runners of the like-for-like size, 16 vCPU and 64 GB. These are the list prices per minute, read on 17 Nov 2026, in US dollars:

RunnerPer minute at 16 vCPU
GitHub-hosted$0.042
Blacksmith$0.032
Jog Max$0.0176, billed by the second
Jog standard$0.0088, billed by the second

A metered cost is hours × 60 × the rate. The comparison counts the hours your CI runs at that size. It is a list-price estimate, not an invoice, and a full month of continuous use is a ceiling, not a typical bill.

A meter costs less in a quiet month. The Frigate costs the same as GitHub-hosted at about 8% of the month in use, as Blacksmith at 10%, as Jog Max at 19% and as Jog standard at 37%. Below those, the metered runner is cheaper.

The source repository is private. Source access is by request.

Search concepts, guides and command examples.