Transformer.js

HuggingFace Transformers.js is a JavaScript library that lets you run pretrained models locally on your machine. The library uses onnxruntime to leverage the CPU/GPU capabilities of your hardware.

In this guide, we will show how to create summaries using the Transformers.js library.

Import the pipeline

The snippet below imports the Transformers.js library and loads the summarizer pipeline and model. You can specify a model name or let the library pick the latest and greatest.

import { pipeline } from "genaiscript/runtime"
const summarizer = await pipeline("summarization")

Allocating and loading the model can take some time, so it’s best to do this at the beginning of your script and only once.

Invoke the pipeline

The summarizer pipeline has a single argument, the content to summarize. It returns an array of summaries which we need to unpack to access the final summary text. This is what we do below and summary_index contains the summary text.

const [summary] = await summarizer(content)
// @ts-ignore
const { summary_text } = summary

Final code

The example below generates a summary of each input file before letting the model generate a full summary.

script({
    title: "summary of summary - transformers.js",
    model: "ollama:llama3.2:1b",
    files: ["src/rag/markdown.md"],
    tests: {
        files: ["src/rag/markdown.md"],
        keywords: ["markdown"],
    },
})

console.log("loading summarizer transformer")
import { pipeline } from "@huggingface/transformers"
const summarizer = await pipeline("summarization")

for (const file of env.files) {
    console.log(`summarizing ${file.filename}`)
    const [summary] = await summarizer(file.content)
    // @ts-ignore
    const { summary_text } = summary
    def("FILE", {
        filename: file.filename,
        // @ts-ignore
        content: summary_text,
    })
}

console.log(`summarize all summaries`)
$`Summarize all the contents in FILE.`