Python type checker benchmark

Weekly cross-checker performance on pinned open-source projects.

Run summary

Generated 2026-10-25T06:02:22.807066+00:00 · Python 3.14.6 · Linux-6.17.0-1022-azure-x86_64-with-glibc2.39

CheckerMeasuredFailedTotal timeMedian timeMedian RSS
pyright1.1.41490
463.5s
26.0s1268 MB
pyright (threads)1.1.41490
371.7s
23.1s104 MB
pyrefly1.4.090
45.0s
3.6s330 MB
ty0.0.9190
59.9s
7.0s375 MB
mypy2.4.081
341.5s
41.0s620 MB
zuban0.10.190
40.4s
3.7s145 MB

Package comparison

Different checker jobs run on separate hosted runners, so timings are independent trends rather than a fastest-checker ranking. The two Pyright modes run sequentially in one job. Cells show wall time and peak RSS.

Packagepyrightpyright (threads)pyreflytymypyzuban
ansible15.31s1094 MB17.26s104 MB1.19s226 MB1.76s320 MB8.63s310 MB1.20s130 MB
click2.27s358 MB3.43s89 MB0.27s115 MB0.15s109 MB1.63s128 MB0.17s44 MB
homeassistant108.97s6127 MB92.64s264 MB9.81s998 MB11.78s1755 MB86.41s1870 MB13.38s866 MB
numpy25.98s1268 MB23.12s101 MB3.55s330 MB6.96s375 MBNot measured3.72s145 MB
pandas103.95s3578 MB72.80s124 MB11.01s500 MB15.86s924 MB81.52s930 MB4.97s447 MB
pytest9.31s854 MB10.73s96 MB0.82s168 MB0.90s213 MB6.53s263 MB0.80s98 MB
requests1.79s342 MB2.99s89 MB0.26s116 MB0.12s105 MB1.85s134 MB0.15s45 MB
torch106.49s4942 MB78.04s216 MB8.79s692 MB10.77s1484 MB81.48s1591 MB7.79s591 MB
transformers89.40s5193 MB70.69s222 MB9.30s932 MB11.61s1491 MB73.41s1782 MB8.21s624 MB

Pyright threading comparison

Sequential runs on the same runner compare single-threaded Pyright with --threads, requested without an explicit count on a runner reporting 4 logical CPUs. Bars show wall time on a shared scale; cells also show peak RSS.

PackageSingle-threadMulti-threadSpeedup
ansible
15.31s1094 MB
17.26s104 MB
0.89x
click
2.27s358 MB
3.43s89 MB
0.66x
homeassistant
108.97s6127 MB
92.64s264 MB
1.18x
numpy
25.98s1268 MB
23.12s101 MB
1.12x
pandas
103.95s3578 MB
72.80s124 MB
1.43x
pytest
9.31s854 MB
10.73s96 MB
0.87x
requests
1.79s342 MB
2.99s89 MB
0.60x
torch
106.49s4942 MB
78.04s216 MB
1.36x
transformers
89.40s5193 MB
70.69s222 MB
1.26x

Pyright release history

Track per-package execution time and peak memory across every stable Pyright release from the prior year.

View Pyright package performance by release

Pyright analysis stats

File counts and phase timings reported by Pyright --stats.

PackageParsed/boundCheckedFindReadTokenizeParseImportsBindCheckCycles
ansible9885830.033s0.070s0.313s0.613s0.183s0.783s12.460s0.000s
click111170.000s0.010s0.090s0.167s0.033s0.210s1.500s0.000s
homeassistant1115098500.530s0.650s2.323s3.870s1.767s5.437s91.930s0.000s
numpy6053560.010s0.050s0.273s0.550s0.187s0.700s23.773s0.000s
pandas189814590.050s0.200s0.990s2.067s0.267s2.260s95.907s0.000s
pytest5452430.010s0.050s0.270s0.443s0.080s0.537s7.543s0.000s
requests167190.000s0.010s0.090s0.180s0.040s0.207s1.010s0.000s
torch307824000.137s0.317s1.540s3.433s0.520s3.923s93.087s0.000s
transformers353629010.127s0.337s1.573s2.980s0.800s7.533s78.070s0.000s