Benchmarks
How the numbers on the home page were measured: the repository, the runs, the dates, and the run a competitor finished first.
Time until dependencies are ready
This is the 5.3 seconds on the home page.
| Runner | Dependencies ready | Whole job through typecheck | Runs |
|---|---|---|---|
| Our machine | 5.3 s | 46 s | 5 |
| Jog Max | 46 s | 85 s | 3 |
| Jog standard | 76 s | 137 s | 3 |
- Repository: our own monorepo, 3,677 packages, the same commit on every runner.
- Cache state: warm. The commit had already run on each runner, so its cache was filled before the timed runs.
- Timed interval: reading or downloading the cache, up to the point where dependencies are ready to use. The second column is the whole job to the end of typecheck.
- Figures: medians of the warm runs.
- Size: 16 vCPU and 64 GB on every runner.
- Date: 17 Nov 2026.
The difference is where the cache was. Our machine read its dependencies from its own disk. The Jog runners downloaded their caches first.
With the tests included, Jog Max finished first
On 18 Nov 2026 we ran the same repository to the end of its full test suite. One warm run on each runner:
| Runner | Whole job with tests |
|---|---|
| Jog Max | 538 s |
| Our machine | 567 s |
| Jog standard | 830 s |
The cache on local disk won the set-up. Jog Max ran the tests themselves faster and finished first. A job that spends most of its time in tests gains little from a faster set-up.
Time for a free machine to pick up a job
The 2 to 3 seconds on the home page is the time from a job being queued to its start when a pre-connected standby runner was idle. We measured it with jobs run one after another against a single standby runner. Without a standby runner, a job waits for a new virtual machine: 33–37 seconds from queued to started in live runs on our own pool. How quickly a job starts has the conditions.
What these runs do not show
- The dependency benchmark ran on our own CI server, with an ordinary GitHub Actions runner and a cache on its local disk. It did not use the Smart Workspaces action.
- The ship classes on the pricing page have not run this benchmark. We publish no speed for them.
- One repository was measured. Yours will differ: time a job of your own before you rely on a figure here.
- We do not claim faster cores, a guaranteed job time or a guaranteed start time.
- The run with tests is one run per runner, not a median.
The prices in the cost comparison
The home page and the pricing page compare a Frigate, 8 cores · 16 threads · 64 GB at $144 a month, with metered runners of the like-for-like size, 16 vCPU and 64 GB. These are the list prices per minute, read on 17 Nov 2026, in US dollars:
| Runner | Per minute at 16 vCPU |
|---|---|
| GitHub-hosted | $0.042 |
| Blacksmith | $0.032 |
| Jog Max | $0.0176, billed by the second |
| Jog standard | $0.0088, billed by the second |
A metered cost is hours × 60 × the rate. The comparison counts the hours your CI runs at that size. It is a list-price estimate, not an invoice, and a full month of continuous use is a ceiling, not a typical bill.
A meter costs less in a quiet month. The Frigate costs the same as GitHub-hosted at about 8% of the month in use, as Blacksmith at 10%, as Jog Max at 19% and as Jog standard at 37%. Below those, the metered runner is cheaper.