Python type checker benchmark

Weekly cross-checker performance on pinned open-source projects.

Run summary

Generated 2026-11-15T05:53:11.436867+00:00 · Python 3.14.6 · Linux-7.0.0-1019-azure-x86_64-with-glibc2.43

CheckerMeasuredFailedTotal timeMedian timeMedian RSS
pyright1.1.41490
382.6s
21.5s1380 MB
pyright (threads)1.1.41490
328.6s
21.5s104 MB
pyrefly1.4.190
36.9s
3.0s315 MB
ty0.0.10490
31.2s
2.0s356 MB
mypy2.4.181
269.3s
32.0s620 MB
zuban0.10.290
39.6s
3.5s147 MB

Package comparison

Different checker jobs run on separate hosted runners, so timings are independent trends rather than a fastest-checker ranking. The two Pyright modes run sequentially in one job. Cells show wall time and peak RSS.

Packagepyrightpyright (threads)pyreflytymypyzuban
ansible13.64s1083 MB17.12s104 MB1.07s206 MB0.99s275 MB6.38s313 MB1.15s132 MB
click1.99s357 MB3.12s88 MB0.24s94 MB0.11s74 MB1.22s130 MB0.15s46 MB
homeassistant91.05s6092 MB78.89s266 MB8.06s958 MB7.43s1681 MB71.10s1873 MB12.90s871 MB
numpy21.45s1380 MB21.46s101 MB2.96s315 MB1.99s356 MBNot measured3.53s147 MB
pandas82.22s3551 MB62.31s122 MB8.71s472 MB6.06s881 MB63.09s928 MB4.83s444 MB
pytest7.98s867 MB9.79s95 MB0.67s147 MB0.56s179 MB4.57s267 MB0.81s102 MB
requests1.52s344 MB2.59s88 MB0.21s99 MB0.10s71 MB1.38s137 MB0.14s46 MB
torch87.48s4888 MB68.28s244 MB7.17s673 MB6.98s1366 MB64.04s1592 MB7.77s598 MB
transformers75.31s5289 MB64.98s221 MB7.79s901 MB7.01s1402 MB57.52s1786 MB8.30s630 MB

Pyright threading comparison

Sequential runs on the same runner compare single-threaded Pyright with --threads, requested without an explicit count on a runner reporting 4 logical CPUs. Bars show wall time on a shared scale; cells also show peak RSS.

PackageSingle-threadMulti-threadSpeedup
ansible
13.64s1083 MB
17.12s104 MB
0.80x
click
1.99s357 MB
3.12s88 MB
0.64x
homeassistant
91.05s6092 MB
78.89s266 MB
1.15x
numpy
21.45s1380 MB
21.46s101 MB
1.00x
pandas
82.22s3551 MB
62.31s122 MB
1.32x
pytest
7.98s867 MB
9.79s95 MB
0.81x
requests
1.52s344 MB
2.59s88 MB
0.59x
torch
87.48s4888 MB
68.28s244 MB
1.28x
transformers
75.31s5289 MB
64.98s221 MB
1.16x

Pyright release history

Track per-package execution time and peak memory across every stable Pyright release from the prior year.

View Pyright package performance by release

Pyright analysis stats

File counts and phase timings reported by Pyright --stats.

PackageParsed/boundCheckedFindReadTokenizeParseImportsBindCheckCycles
ansible9885830.023s0.070s0.277s0.520s0.180s0.740s11.050s0.000s
click111170.000s0.010s0.070s0.133s0.027s0.177s1.347s0.000s
homeassistant1115498500.373s0.550s1.907s4.147s1.460s4.690s75.570s0.000s
numpy6063560.010s0.050s0.223s0.450s0.160s0.593s19.547s0.000s
pandas189914590.037s0.177s0.857s1.863s0.213s2.053s76.053s0.000s
pytest5452430.010s0.043s0.243s0.340s0.080s0.503s6.433s0.000s
requests167190.000s0.010s0.070s0.140s0.040s0.173s0.883s0.000s
torch307824000.097s0.293s1.570s3.080s0.493s4.200s75.810s0.000s
transformers353729010.090s0.280s1.417s2.533s0.697s6.130s65.623s0.000s