Python type checker benchmark

Weekly cross-checker performance on pinned open-source projects.

Run summary

Generated 2026-11-08T06:27:14.496134+00:00 · Python 3.14.6 · Linux-7.0.0-1017-azure-x86_64-with-glibc2.39

CheckerMeasuredFailedTotal timeMedian timeMedian RSS
pyright1.1.41490
666.5s
39.5s1298 MB
pyright (threads)1.1.41490
530.8s
33.4s104 MB
pyrefly1.4.190
52.4s
4.0s307 MB
ty0.0.9990
22.3s
1.5s367 MB
mypy2.4.181
338.0s
39.9s622 MB
zuban0.10.290
39.3s
3.6s147 MB

Package comparison

Different checker jobs run on separate hosted runners, so timings are independent trends rather than a fastest-checker ranking. The two Pyright modes run sequentially in one job. Cells show wall time and peak RSS.

Packagepyrightpyright (threads)pyreflytymypyzuban
ansible22.53s1081 MB23.92s104 MB1.38s205 MB0.71s277 MB8.46s314 MB1.22s131 MB
click3.12s356 MB4.46s87 MB0.30s95 MB0.08s74 MB1.63s130 MB0.15s46 MB
homeassistant155.72s6096 MB128.62s261 MB11.51s959 MB5.19s1678 MB85.69s1872 MB12.97s870 MB
numpy39.50s1298 MB33.35s100 MB3.97s307 MB1.53s367 MBNot measured3.57s147 MB
pandas145.51s3759 MB104.15s117 MB12.74s470 MB4.18s894 MB80.49s932 MB4.82s444 MB
pytest13.74s861 MB14.75s95 MB0.96s148 MB0.38s180 MB6.28s266 MB0.79s100 MB
requests2.41s342 MB3.95s88 MB0.28s99 MB0.07s71 MB1.81s136 MB0.14s47 MB
torch155.88s4859 MB118.83s211 MB10.24s666 MB5.30s1366 MB82.18s1597 MB7.60s594 MB
transformers128.04s5068 MB98.74s212 MB11.04s869 MB4.85s1404 MB71.43s1787 MB8.08s628 MB

Pyright threading comparison

Sequential runs on the same runner compare single-threaded Pyright with --threads, requested without an explicit count on a runner reporting 4 logical CPUs. Bars show wall time on a shared scale; cells also show peak RSS.

PackageSingle-threadMulti-threadSpeedup
ansible
22.53s1081 MB
23.92s104 MB
0.94x
click
3.12s356 MB
4.46s87 MB
0.70x
homeassistant
155.72s6096 MB
128.62s261 MB
1.21x
numpy
39.50s1298 MB
33.35s100 MB
1.18x
pandas
145.51s3759 MB
104.15s117 MB
1.40x
pytest
13.74s861 MB
14.75s95 MB
0.93x
requests
2.41s342 MB
3.95s88 MB
0.61x
torch
155.88s4859 MB
118.83s211 MB
1.31x
transformers
128.04s5068 MB
98.74s212 MB
1.30x

Pyright release history

Track per-package execution time and peak memory across every stable Pyright release from the prior year.

View Pyright package performance by release

Pyright analysis stats

File counts and phase timings reported by Pyright --stats.

PackageParsed/boundCheckedFindReadTokenizeParseImportsBindCheckCycles
ansible9885830.040s0.107s0.403s0.737s0.243s1.010s19.260s0.000s
click111170.000s0.013s0.110s0.207s0.043s0.273s2.150s0.000s
homeassistant1115398500.687s0.943s3.223s5.177s2.177s7.153s133.187s0.000s
numpy6063560.010s0.073s0.400s0.723s0.297s0.863s36.580s0.000s
pandas189914590.060s0.263s1.220s2.437s0.307s2.867s136.420s0.000s
pytest5452430.020s0.063s0.420s0.537s0.120s0.690s11.453s0.000s
requests167190.000s0.020s0.117s0.203s0.057s0.267s1.440s0.000s
torch307824000.167s0.450s2.097s3.477s0.630s4.550s143.050s0.000s
transformers353629010.147s0.470s2.120s3.613s0.947s9.097s113.293s0.000s