Image Alt Textify

Alt text is essential for making images accessible to everyone, including those with visual impairments. It provides a textual description of the image, allowing screen readers to convey the content to users who can’t see the image. However, writing alt text for images can be time-consuming, especially when dealing with a large number of images. This is where AI can help. By using a language model like OpenAI’s GPT-4, you can generate alt text for images automatically, saving you time and effort.

In this sample, we will build a tool that generates alt text for images in Markdown files.

Configuring the script

This script is composed of TypeScript code, designed to run with the GenAIScript CLI. Let’s break it down:

script({
    title: "Image Alt Textify",
    description: "Generate alt text for images in markdown files",
    parameters: {
        docs: {
            type: "string",
            description: "path to search for markdown files",
            default: "**.{md,mdx}",
        },
        force: {
            type: "boolean",
            description: "regenerate all descriptions",
            default: false,
        },
        assets: {
            type: "string",
            description: "image assets path",
            // change the default path to your assets folder
            default: "./assets/images",
        },
    },
})

Here we declare the script with a title and description, specifying it uses OpenAI’s GPT-4 model. We also set parameters for the file paths, choice to regenerate all descriptions, and the assets path.

Next, we extract environmental variables:

const { docs, force, assets } = env.vars

Searching for images

Following that, we define a regular expression to find images in Markdown:

const rx = force
    ? // match ![alt?](path) with alt text or not
      /!\[[^\]]*\]\(([^\)]+\.(png|jpg))\)/g
    : // match ![](path) without alt text
      /!\[\s*\]\(([^\)]+\.(png|jpg))\)/g

const { files } = await workspace.grep(rx, {
    path: docs,
    glob: "*.{md,mdx}",
    readText: true,
})

The script uses workspace.grep to find all occurrences of the regex pattern in the specified documents.

Generating alt text

For each image URL found, we generate alt text using an inline prompt and defImages.

for (const file of files) {
    const { filename, content } = file

    // map documentation relative path to assets path
    const url = resolveUrl(filename)

    // execute a LLM query to generate alt text
    const { text } = await runPrompt(
        (_) => {
            _.defImages(resolvedUrl)
            _.$`
                You are an expert in assistive technology.

                You will analyze the image
                and generate a description alt text for the image.

                - Do not include alt text in the description.
                - Keep it short but description.
                - Do not generate the [ character.`
        },
        {
            // safety system message to prevent generating harmful text
            system: ["system.safety_harmful_content"],
            // use multi-model model
            model: "openai:gpt-4o",
            ...
        }
    )

    imgs[url] = text
}

Updating files

Finally, we update the Markdown content with the generated alt text:

const newContent = content.replace(
    rx,
    (m, url) => `![${imgs[url] ?? ""}](${url})`
)
if (newContent !== content) await workspace.writeText(filename, newContent)

We replace the placeholder in the original content with the alt text and save the updated file.

💻 How to Run the Script

To run this script, you’ll need the GenAIScript CLI. If you haven’t installed it yet, check out the installation guide.

Once you have the CLI, you can run the script with the following command:

npx genaiscript run iat

Safety

The script imports a default safety system message to prevent generating harmful text.

    // safety system message to prevent generating harmful text
    system: ["system.safety_harmful_content"],

In Azure OpenAI deployments, you can also turn on content filters to prevent accidentally generating harmful content.

Full source (GitHub)

/*
 * Markdown image alt text updater
 */
script({
    title: "Image Alt Textify",
    description: "Generate alt text for images in markdown files",
    parameters: {
        docs: {
            type: "string",
            description: "path to search for markdown files",
            default: "docs",
        },
        force: {
            type: "boolean",
            description: "regenerate all descriptions",
            default: false,
        },
        assets: {
            type: "string",
            description: "image assets path",
            default: "./slides/public",
        },
        dryRun: {
            type: "boolean",
            description: "show matches, do not compute alt text",
            default: false,
        },
    },
})

/** ------------------------------------------------
 * Configuration
 */
const { docs, force, assets, dryRun } = env.vars

/** ------------------------------------------------
 * Helper functions (update as needed)
 */
/**
 *  Return the resolved url for the image
 */
const resolveUrl = (filename: string, url: string) => {
    // ignore external urls
    if (/^http?s:\/\//i.test(url)) return undefined
    // map / to assets
    else if (/^\//.test(url)) return path.join(assets, url.slice(1))
    // resolve local paths
    else return path.join(path.dirname(filename), url)
}

/** ------------------------------------------------
 * Collect files
 */
// search for ![](...) in markdown files and generate alt text for images
const rx = force // upgrade all urls
    ? // match ![alt](url) with any alt
      /!\[[^\]]*\]\(([^\)]+\.(png|jpg))\)/g
    : // match ![alt](url) where alt is empty
      /!\[\s*\]\(([^\)]+\.(png|jpg))\)/g
const { files } = await workspace.grep(rx, {
    path: docs,
    glob: "*.{md,mdx}",
    readText: true,
})

/** ------------------------------------------------
 * Generate alt text for images
 * and update markdown files
 */

// a cache of generated alt text for images
const imgs: Record<string, string> = {}

// process each file
for (const file of files) {
    const { filename, content } = file
    console.log(filename)
    const matches = content.matchAll(rx)
    // pre-compute matches
    for (const match of matches) {
        const url = match[1]
        if (imgs[url]) continue // already processed

        const resolvedUrl = resolveUrl(filename, url)
        if (!resolvedUrl) continue // can't process url
        console.log(`└─ ${resolvedUrl}`)

        if (dryRun) continue

        // execute a LLM query to generate alt text
        const { text, error } = await runPrompt(
            (_) => {
                _.defImages(resolvedUrl)
                _.$`
                You are an expert in assistive technology.

                You will analyze the image
                and generate a description alt text for the image.

                - Do not include alt text in the description.
                - Keep it short but description.
                - Do not generate the [ character.`
            },
            {
                // safety system message to prevent generating harmful text
                system: ["system.safety_harmful_content"],
                maxTokens: 4000,
                temperature: 0.5,
                cache: "alt-text",
                label: `altextify ${resolvedUrl}`,
            }
        )
        if (error) throw error
        else if (!text) console.log(`.. no description generated`)
        else imgs[url] = text.replace(/\[/g, "") // remove [ from alt text
    }
    // apply replacements
    const newContent = content.replace(
        rx,
        (m, url) => `![${imgs[url] ?? ""}](${url})`
    )
    // save updated content
    if (newContent !== content) {
        console.log(`.. updating ${filename}`)
        await workspace.writeText(filename, newContent)
    }
}