This living report provides a reproducible computational cost reference.
People implementing Skala can use the report to validate their implementations against our (gpu4)pyscf reference, and
people using Skala can use it to form expectations about the computational cost of their calculations.
The report covers molecular sizes from H2 to ~400 non-hydrogen atoms, and hardware from a 2-thread CPU to a 16-thread CPU to an NVIDIA A100 GPU.
All measurements are done using PySCF and GPU4PySCF and the official skala package.
Comparisons are made to r2SCAN (meta-GGA non-hybrid), B3LYP (GGA hybrid), and M06-2X (meta-GGA hybrid), called via libxc.
In summary, Skala's computational on GPUs is similar to meta-GGA non-hybrids.
On a CPU, it is roughly 10x as expensive as a meta-GGA non-hybrid for small molecules, but that overhead disappears for molecules with ~15–20 non-hydrogen atoms or ~500 atomic orbitals.
A neural-network XC functional introduces an entirely new component to a traditional XC calculation, and brings with it several new software engineering challenges. It is therefore all the more important to understand how this affects the computational cost of DFT calculations, both for users and for developers of DFT codes. This report shows how the computational cost of Skala behaves along three independent axes—molecular size, basis set size, and compute environment—and compares it to the cost of traditional XC functionals—r2SCAN, B3LYP, and M06-2X. The compute environment axis covers both the hardware type (CPU vs GPU) and the degree of parallelism (number of CPU threads), both of which can be conceptually understood along the lines of strong scaling.
For insight into the computational cost, the report breaks down the total cost per SCF cycle into four components: the evaluation of the XC functional itself, the rest of the XC quadrature (includes evaluation of the density on the grid, and contraction of the XC potential back to the AO basis), the Coulomb and exchange build, and the Fock diagonalization.
Three environments are captured here: GPU4PySCF on an NVIDIA A100 GPU, and PySCF on 2 and 16 CPU threads.
Points are converged calculations. Dashed lines are piecewise log–log. The fits are piecewise, with transitions selected by cross-validation. This allows us to capture regimes where different computational components dominate.
Evaluation of the XC functional on the grid
For a traditional functional this is libxc's combined energy-and-derivative call on the grid, which cannot be split further. For Skala it is the network's forward pass alone: its backward differentiates all the way to the density matrix and so also performs the contraction that pyscf does outside the timed call for libxc.
X
XC quadrature per iteration: grid and AO evaluation, density construction, functional evaluation, and potential assembly
XC quadrature evaluates the density and exchange-correlation model on the quadrature grid, then contracts the resulting potential back into the AO basis. In essence, it is the cost of the map from the density matrix to the XC energy and its derivative with respect to the density matrix.
X
On a GPU, all functionals scale the same way, with the meta-GGAs being ~2x as expensive as the GGA. On a CPU, the relative cost of Skala dominates for small molecules, introducing a ~10x overhead for the smallest molecules, which disappears by ~15–20 non-hydrogen atoms. The slightly better asymptotic scaling of Skala is the result of a marginally more efficient contraction of the XC potential in the Skala implementation. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
Coulomb and exchange matrices, the other half of the Fock build
The effective potential is the J/K build plus the exchange-correlation quadrature, so subtracting numerical integration from it leaves J/K on its own. It is a difference of two measured quantities, and it is the part of an iteration that depends only on whether a functional is a hybrid or a non-hybrid.
Hybrid functionals use DF-JK with precomputed 3-center integrals, while non-hybrids use direct DF-J. This results in a lower cost of DF-JK for the smallest molecules, while the asymptotic scaling of Coulomb only is much better than that of Coulomb and exchange. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
One whole SCF iteration. Median over ‘steady-state’ cycles, excluding the first few that are affected by startup overhead like JIT compilation.
The outermost of the three nested layers. Numerical integration sits inside it, and the neural forward pass inside that.
X
Combining all above together, as a GGA hybrid, B3LYP is 1.5–2x cheaper for small molecules, while as non-hybrids, Skala and r2SCAN get cheaper for large molecules. In addition, Skala features the same overhead on CPU for small molecules as seen in the XC quadrature. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
End-to-end SCF time depends on the work performed in each iteration and on the number of cycles required to reach convergence.
The measurements here are noisier than the per-iteration costs, because they combine the per-iteration cost with the iteration count. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
For all functionals we observe a very slow growth of the iteration count with system size. For all functionals, all basis sets, and all environments we observe outliers for the largest systems, which took several times more iterations to converge. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
Work before the SCF loop: grid construction, one-electron integrals, the initial guess, the density-fitting integral build, and the Fock build on that guess. That last one is a full effective-potential evaluation, so setup also contains one exchange-correlation quadrature.
Containing both one XC quadrature and the 3-center integral build, the setup cost shows similar characteristics to the SCF iteration cost. Use toggles on the top of the page to switch between GPU/CPU and basis sets.
The stacked bands partition a steady-state SCF iteration into four directly measured, nested layers: the exchange-correlation functional itself, the rest of the quadrature around it, the J/K build, and everything else in the iteration.
The same calculation end to end.
Three costs are paid before a calculation can start, and none of them appears in the timings above. The first is the machine cold start. The first calculation on a fresh machine compiles the CUDA kernels into the CuPy cache on disk and reads the two gigabytes of PyTorch shared libraries into the operating system's page cache; every later process finds both already there. On this sweep, with GPUs enabled, that costs ~13 s. Neither is specific to quantum chemistry: the libraries are PyTorch's own, and all CuPy kernels an SCF compiles are CuPy's generic array ops.
The second is process start-up: creating the CUDA context and its library handles, executing the Python imports, and moving the functional onto the device. On the A100 it is ~8 s for all functionals. On a CPU the traditional functionals pay less than a second, while Skala pays ~7 s on 16 threads and ~13 s on two, because it is the only one of the four that loads PyTorch.
The third is the first-evaluation overhead, paid the first time the exchange-correlation functional runs on a given system. This includes TorchScript compilation, roughly fixed at 1–2 s, compilation of some GPU4PySCF custom kernels, and GPU memory allocation.
Follow the benchmark documentation to run the same benchmark sweep on your machine or cluster and combine your timings with these reference results.
In all environments and for all functionals, the cost scales roughly linearly with an initial onset, and Skala's evaluation is ~1.5 orders of magnitude more expensive than the traditional functionals. As shown below, this much higher cost doesn't translate into a higher total cost, because the XC grid evaluation is only a small part of the total SCF cycle. The relative onset positions of Skala and the traditional functionals on CPU vs GPU differ, driving Skala's total SCF overhead on CPU for small molecules (see below). Use toggles on the top of the page to switch between GPU/CPU and basis sets.