Kimai: 350 test cases, measured*
Test automation case study: 6 Claude Code agents against Kimai, an open-source time tracker. Report of 5 October 2026. The hours and cost table below works as a test automation ROI calculator, with measured numbers.
* Measured run: 55 landed; 11 needed a human due to tested app errors, time included in totals.
The totals for 350 test cases are estimated from the 66 measured ones, by resampling them 20,000 times: 43.3 hours in all, of which 11.1 hours of agent work and 32.1 hours of human work, with a 95% confidence interval of 42.2 to 44.4 hours.
Hours and cost
| Measure | qa-signoff Pro (agents and humans) | QA engineer (Germany), €54.71/h |
|---|---|---|
| Hours | 43.3 | 190.9 |
| Cost | €2,155 | €10,445 |
| of which tokens, agents running 11.1 h | €396 | |
| of which human prompting and review, 0.33 h | €18 | |
| of which human unblocking of blocked test cases, 31.8 h | €1,741 | |
| Range, 10th to 90th percentile | €8,090 to €13,491 | |
| Faster | 4.4× | 1× |
| Cheaper | 4.8× | 1× |
What this proves and what it does not
One measured run on one app. The 66 measured test cases were the most important checks, the ones a release must pass; the remaining 284 are estimated from them. The saving comes from the whole pipeline, the free package included, not from Pro alone.