Skip to main content

Make ESLint Fast

version 2

ESLint is a wonderful piece of software that catches many software bugs and code quality issues before you check them into the shared codebase in a repository. It comes at the price of speed: ESLint is a single-threaded application that can be difficult to speed up.

There are several approaches to solving the limitation of the single core issues. One can conceivably create several ESLint processes with different segments or shards of the files to be linted. This could work, but since the division or segmentation changes depending on the number of files at different points in time, it makes the other methods of acceleration difficult to achieve: remote caching, scoping, and pipelining.

To combat this, a typical task runner spawns a new ESLint process per package. With a sufficiently high number of CPU cores, one can recover the inefficiency of creating new ESLint processes by amortization through parallelization. This, however, has its drawbacks as well! If there happens to be a set of very slow-booting plugins for ESLint, then this bootstrap cost becomes the dominant perf bottleneck.

How can we have the cake and eat it too? Easy! Use dedicated workers!

Worker Runner​

lage solves this with dedicated workers. The idea is simple: if all of the packages in your monorepo use the same ESLint configuration, then you can save the bootstrap time by creating a pool of ESLint instances (say, one per CPU core minus one). And have lage feed it files to lint per package. lage scheduler feeds these to the worker if work is needed. Caching, scoping, and pipeline all continue to work the same way!

Let's see some code!​

First, let's change our lint task configuration in the pipeline to a "worker" type:

/lage.config.js
js
module.exports = {
  pipeline: {
    lint: {
      type: "worker",
      options: {
        worker: "scripts/workers/eslint-worker.js"
      }
    }
  }
};
/lage.config.js
js
module.exports = {
  pipeline: {
    lint: {
      type: "worker",
      options: {
        worker: "scripts/workers/eslint-worker.js"
      }
    }
  }
};

Then, we implement an eslint-worker.js such as this:

/scripts/workers/eslint-worker.js
js
const path = require("path");
 
const { ESLint } = require("eslint");
const { readFile } = require("fs/promises");
 
/** this is the workspace root - find it however you want! */
const PROJECT_ROOT = path.resolve(__dirname, "../..");
 
/** @type {ESLint} */
let eslintInstance = null;
 
/** caches an ESLint instance for the worker */
function getEslintInstance() {
  if (!eslintInstance) {
    const baseConfig = require(path.join(
      PROJECT_ROOT,
      "scripts/config/eslintrc.js"
    ));
    baseConfig.parserOptions.project = path.join(target.cwd, "tsconfig.json");
 
    eslintInstance = new ESLint({
      reportUnusedDisableDirectives: "error",
      baseConfig,
      fix: false,
      cache: false,
      cwd: target.cwd
    });
  }
 
  return eslintInstance;
}
 
/** Workers should have a run function that gets called per package task */
async function run(data) {
  const { target } = data;
 
  const eslint = getEslintInstance();
 
  // You can also use "options" to pass different files pattern to lint
  // e.g. data.options.files; you'll need to then configure this inside
  // lage.config.js's pipeline
  const files = "src/**/*.ts";
 
  const results = await eslint.lintFiles(files);
 
  const formatter = await eslint.loadFormatter("stylish");
 
  const resultText = formatter.format(results);
 
  // Output results to stdout
  process.stdout.write(resultText + "\n");
 
  if (results.some((r) => r.errorCount > 0)) {
    // throw an error to indicate that this task has failed
    throw new Error(`Linting failed with errors`);
  }
}
 
// The module export is picked up by `lage` to run inside a worker, and the
// module's state is preserved from target run to target run.
module.exports = run;
/scripts/workers/eslint-worker.js
js
const path = require("path");
 
const { ESLint } = require("eslint");
const { readFile } = require("fs/promises");
 
/** this is the workspace root - find it however you want! */
const PROJECT_ROOT = path.resolve(__dirname, "../..");
 
/** @type {ESLint} */
let eslintInstance = null;
 
/** caches an ESLint instance for the worker */
function getEslintInstance() {
  if (!eslintInstance) {
    const baseConfig = require(path.join(
      PROJECT_ROOT,
      "scripts/config/eslintrc.js"
    ));
    baseConfig.parserOptions.project = path.join(target.cwd, "tsconfig.json");
 
    eslintInstance = new ESLint({
      reportUnusedDisableDirectives: "error",
      baseConfig,
      fix: false,
      cache: false,
      cwd: target.cwd
    });
  }
 
  return eslintInstance;
}
 
/** Workers should have a run function that gets called per package task */
async function run(data) {
  const { target } = data;
 
  const eslint = getEslintInstance();
 
  // You can also use "options" to pass different files pattern to lint
  // e.g. data.options.files; you'll need to then configure this inside
  // lage.config.js's pipeline
  const files = "src/**/*.ts";
 
  const results = await eslint.lintFiles(files);
 
  const formatter = await eslint.loadFormatter("stylish");
 
  const resultText = formatter.format(results);
 
  // Output results to stdout
  process.stdout.write(resultText + "\n");
 
  if (results.some((r) => r.errorCount > 0)) {
    // throw an error to indicate that this task has failed
    throw new Error(`Linting failed with errors`);
  }
}
 
// The module export is picked up by `lage` to run inside a worker, and the
// module's state is preserved from target run to target run.
module.exports = run;