Image Alt Textify
Alt text is essential for making images accessible to everyone, including those with visual impairments. It provides a textual description of the image, allowing screen readers to convey the content to users who can’t see the image. However, writing alt text for images can be time-consuming, especially when dealing with a large number of images. This is where AI can help. By using a language model like OpenAI’s GPT-4, you can generate alt text for images automatically, thus saving you time and effort.
In this sample, we will build a tool that generates alt text for images in Markdown files.
Configuring the script
This script is composed of TypeScript code, designed to run with the GenAIScript CLI. Let’s break it down:
script({ title: "Image Alt Textify", description: "Generate alt text for images in markdown files", parameters: { docs: { type: "string", description: "path to search for markdown files", default: "**.{md,mdx}", }, force: { type: "boolean", description: "regenerate all descriptions", default: false, }, assets: { type: "string", description: "image assets path", // change the default path to your assets folder default: "./assets/images", }, },})Here we declare the script with a title and description, specifying it uses OpenAI’s GPT-4 model. We also set parameters for the file paths, choice to regenerate all descriptions, and the assets path.
Next, we extract environmental variables:
const { docs, force, assets } = env.varsSearching for images
Following that, we define a regular expression to find images in Markdown:
const rx = force ? // match  with alt text or not /!\[[^\]]*\]\(([^\)]+\.(png|jpg))\)/g : // match  without alt text /!\[\s*\]\(([^\)]+\.(png|jpg))\)/g
const { files } = await workspace.grep(rx, { path: docs, glob: "*.{md,mdx}", readText: true,})The script uses workspace.grep to find all occurrences of the regex pattern in the specified documents.
Generating alt text
For each image URL found, we generate alt text using an inline prompt and defImages.
for (const file of files) { const { filename, content } = file
// map documentation relative path to assets path const url = resolveUrl(filename)
// execute a LLM query to generate alt text const { text } = await runPrompt( (_) => { _.defImages(resolvedUrl) _.$` You are an expert in assistive technology.
You will analyze the image and generate a description alt text for the image.
- Do not include alt text in the description. - Keep it short but descriptive. - Do not generate the [ character.` }, { // safety system message to prevent generating harmful text system: ["system.safety_harmful_content"], // use multi-model model model: "openai:gpt-4o", ... } )
imgs[url] = text}Updating files
Finally, we update the Markdown content with the generated alt text:
const newContent = content.replace( rx, (m, url) => `![${imgs[url] ?? ""}](${url})`)if (newContent !== content) await workspace.writeText(filename, newContent)We replace the placeholder in the original content with the alt text and save the updated file.
💻 How to Run the Script
To run this script, you’ll need the GenAIScript CLI. If you haven’t installed it yet, check out the installation guide.
Once you have the CLI, you can run the script with the following command:
npx genaiscript run iatSafety
The script imports a default safety system message to prevent generating harmful text.
// safety system message to prevent generating harmful text system: ["system.safety_harmful_content"],In Azure OpenAI deployments, you can also turn on content filters to prevent accidentally generating harmful content.
Full source (GitHub)
/* * Markdown image alt text updater */script({ title: "Image Alt Textify", description: "Generate alt text for images in markdown files", parameters: { docs: { type: "string", description: "path to search for markdown files", default: "docs", }, force: { type: "boolean", description: "regenerate all descriptions", default: false, }, assets: { type: "string", description: "image assets path", default: "./slides/public", }, dryRun: { type: "boolean", description: "show matches, do not compute alt text", default: false, }, },})
/** ------------------------------------------------ * Configuration */const { docs, force, assets, dryRun } = env.vars
/** ------------------------------------------------ * Helper functions (update as needed) *//** * Return the resolved url for the image */const resolveUrl = (filename: string, url: string) => { // ignore external urls if (/^http?s:\/\//i.test(url)) return undefined // map / to assets else if (/^\//.test(url)) return path.join(assets, url.slice(1)) // resolve local paths else return path.join(path.dirname(filename), url)}
/** ------------------------------------------------ * Collect files */// search for  in markdown files and generate alt text for imagesconst rx = force // upgrade all urls ? // match  with any alt /!\[[^\]]*\]\(([^\)]+\.(png|jpg))\)/g : // match  where alt is empty /!\[\s*\]\(([^\)]+\.(png|jpg))\)/gconst { files } = await workspace.grep(rx, { path: docs, glob: "*.{md,mdx}", readText: true,})
/** ------------------------------------------------ * Generate alt text for images * and update markdown files */
// a cache of generated alt text for imagesconst imgs: Record<string, string> = {}
// process each filefor (const file of files) { const { filename, content } = file console.log(filename) const matches = content.matchAll(rx) // pre-compute matches for (const match of matches) { const url = match[1] if (imgs[url]) continue // already processed
const resolvedUrl = resolveUrl(filename, url) if (!resolvedUrl) continue // can't process url console.log(`└─ ${resolvedUrl}`)
if (dryRun) continue
// execute a LLM query to generate alt text const { text, error } = await runPrompt( (_) => { _.defImages(resolvedUrl) _.$` You are an expert in assistive technology.
You will analyze the image and generate a description alt text for the image.
- Do not include alt text in the description. - Keep it short but descriptive. - Do not generate the [ character.` }, { // safety system message to prevent generating harmful text system: [ "system.assistant", "system.safety_jailbreak", "system.safety_harmful_content", "system.safety_validate_harmful_content", ], maxTokens: 4000, temperature: 0.5, cache: "alt-text", label: `altextify ${resolvedUrl}`, } ) if (error) throw error else if (!text) console.log(`.. no description generated`) else imgs[url] = text.replace(/\[/g, "") // remove [ from alt text } // apply replacements const newContent = content.replace( rx, (m, url) => `![${imgs[url] ?? ""}](${url})` ) // save updated content if (newContent !== content) { console.log(`.. updating ${filename}`) await workspace.writeText(filename, newContent) }}Content Safety
The following measures are taken to ensure the safety of the generated content.
- This script includes system prompts to prevent prompt injection and harmful content generation.
- The generated description is saved to a file at a specific path, which allows for a manual review before committing the changes.
Additional measures to further enhance safety would be to run a model with a safety filter or validate the message with a content safety service.
Refer to the Transparency Note for more information on content safety.