Python type checker benchmark

Weekly cross-checker performance on pinned open-source projects.

Run summary

Generated 2026-09-27T05:57:10.461481+00:00 · Python 3.14.6 · Linux-6.17.0-1022-azure-x86_64-with-glibc2.39

CheckerMeasuredFailedTotal timeMedian timeMedian RSS
pyright1.1.41490
426.1s
23.6s1274 MB
pyright (threads)1.1.41490
334.0s
20.6s106 MB
pyrefly1.3.190
43.3s
2.9s317 MB
ty0.0.8490
59.9s
6.7s370 MB
mypy2.3.181
229.1s
27.5s588 MB
zuban0.10.090
39.0s
3.5s152 MB

Package comparison

Different checker jobs run on separate hosted runners, so timings are independent trends rather than a fastest-checker ranking. The two Pyright modes run sequentially in one job. Cells show wall time and peak RSS.

Packagepyrightpyright (threads)pyreflytymypyzuban
ansible13.61s1095 MB14.72s106 MB1.12s220 MB1.63s316 MB5.98s290 MB1.29s132 MB
click2.00s358 MB2.98s88 MB0.23s109 MB0.17s106 MB1.19s120 MB0.15s44 MB
homeassistant100.44s6145 MB82.93s263 MB9.72s979 MB10.83s1744 MB61.10s1677 MB12.43s876 MB
numpy23.62s1274 MB20.65s101 MB2.88s317 MB6.72s370 MBNot measured3.49s152 MB
pandas96.81s3613 MB63.81s125 MB10.48s471 MB16.46s913 MB50.20s886 MB4.77s454 MB
pytest8.10s857 MB9.31s96 MB0.79s166 MB0.85s211 MB4.26s245 MB0.76s99 MB
requests1.56s344 MB2.54s89 MB0.22s116 MB0.13s104 MB1.35s126 MB0.14s45 MB
torch93.95s4965 MB73.16s215 MB8.69s673 MB10.41s1490 MB55.94s1464 MB7.80s607 MB
transformers86.04s5137 MB63.88s218 MB9.15s875 MB12.68s1649 MB49.08s1493 MB8.12s641 MB

Pyright threading comparison

Sequential runs on the same runner compare single-threaded Pyright with --threads, requested without an explicit count on a runner reporting 4 logical CPUs. Bars show wall time on a shared scale; cells also show peak RSS.

PackageSingle-threadMulti-threadSpeedup
ansible
13.61s1095 MB
14.72s106 MB
0.92x
click
2.00s358 MB
2.98s88 MB
0.67x
homeassistant
100.44s6145 MB
82.93s263 MB
1.21x
numpy
23.62s1274 MB
20.65s101 MB
1.14x
pandas
96.81s3613 MB
63.81s125 MB
1.52x
pytest
8.10s857 MB
9.31s96 MB
0.87x
requests
1.56s344 MB
2.54s89 MB
0.61x
torch
93.95s4965 MB
73.16s215 MB
1.28x
transformers
86.04s5137 MB
63.88s218 MB
1.35x

Pyright release history

Track per-package execution time and peak memory across every stable Pyright release from the prior year.

View Pyright package performance by release

Pyright analysis stats

File counts and phase timings reported by Pyright --stats.

PackageParsed/boundCheckedFindReadTokenizeParseImportsBindCheckCycles
ansible9895830.020s0.067s0.277s0.457s0.157s0.750s11.113s0.000s
click111170.000s0.010s0.070s0.130s0.023s0.167s1.380s0.000s
homeassistant1114798500.243s0.610s2.123s3.203s1.573s4.933s85.563s0.000s
numpy6033560.010s0.050s0.237s0.440s0.173s0.550s21.807s0.000s
pandas189514590.037s0.190s0.833s1.580s0.263s1.820s91.223s0.000s
pytest5422430.010s0.043s0.240s0.350s0.080s0.467s6.633s0.000s
requests168190.000s0.017s0.073s0.123s0.033s0.163s0.930s0.000s
torch307924000.073s0.277s1.627s2.473s0.473s3.707s84.333s0.000s
transformers353529010.070s0.330s1.603s2.447s0.703s6.473s75.673s0.000s