Python type checker benchmark

Weekly cross-checker performance on pinned open-source projects.

Run summary

Generated 2026-09-14T19:17:05.763862+00:00 · Python 3.14.6 · Linux-6.17.0-1022-azure-x86_64-with-glibc2.39

CheckerMeasuredFailedTotal timeMedian timeMedian RSS
pyright1.1.41490
1413.6s
36.3s1608 MB
pyrefly1.3.090
40.8s
2.7s325 MB
ty0.0.8090
58.1s
6.7s357 MB
mypy2.3.181
389.3s
46.6s588 MB
zuban0.9.390
40.7s
3.6s150 MB

Package comparison

Each checker runs on a separate hosted runner, so timings are independent trends rather than a fastest-checker ranking. Cells show wall time and peak RSS.

Packagepyrightpyreflytymypyzuban
ansible19.34s1098 MB1.09s223 MB1.66s307 MB10.61s290 MB1.32s132 MB
click2.86s358 MB0.25s112 MB0.15s105 MB2.10s120 MB0.15s45 MB
homeassistant142.72s6150 MB9.38s979 MB10.26s1737 MB100.44s1680 MB12.85s876 MB
numpy36.27s1608 MB2.68s325 MB6.72s357 MBNot measured3.63s150 MB
pandas953.56s4481 MB9.66s473 MB16.84s894 MB87.92s886 MB5.07s454 MB
pytest13.00s881 MB0.74s167 MB0.73s211 MB7.91s245 MB0.81s99 MB
requests2.52s346 MB0.21s116 MB0.11s103 MB2.40s126 MB0.15s46 MB
torch132.39s4967 MB8.22s666 MB9.76s1458 MB95.31s1462 MB8.36s606 MB
transformers110.90s5312 MB8.61s887 MB11.91s1523 MB82.58s1489 MB8.38s641 MB

Pyright analysis stats

File counts and phase timings reported by Pyright --stats.

PackageParsed/boundCheckedFindReadTokenizeParseImportsBindCheckCycles
ansible9895830.040s0.090s0.427s0.790s0.230s1.023s16.120s0.000s
click111170.000s0.017s0.113s0.203s0.033s0.263s1.910s0.000s
homeassistant1114798500.417s0.960s3.023s5.397s2.347s7.267s119.790s0.000s
numpy6033560.010s0.067s0.357s0.740s0.260s0.943s33.287s0.000s
pandas189514590.060s0.247s1.073s2.443s0.310s2.927s944.397s0.000s
pytest5422430.020s0.060s0.380s0.570s0.120s0.787s10.567s0.000s
requests168190.000s0.020s0.117s0.233s0.067s0.293s1.450s0.000s
torch307924000.133s0.390s2.280s3.960s0.637s4.930s118.583s0.000s
transformers353629010.120s0.437s2.127s3.827s0.910s8.893s97.383s0.000s