Images

Images can be added to the prompt for models that support this feature (like gpt-4o). Use the defImages function to declare the images. Supported images will vary with models but typically include PNG, JPEG, WEBP, and GIF. Both local files and URLs are supported.

defImages(env.files)

Read more about OpenAI Vision.

URLs

Public URLs (that do not require authentication) will be passed directly to OpenAI.

defImages(
    "https://github.com/microsoft/genaiscript/blob/main/docs/public/images/logo.png?raw=true"
)

Local files are loaded and encoded as a data uri.

Buffer, Blob

The defImages function also supports Buffer and Blob.

This example takes a screenshot of bing.com and adds it to the images.

const page = await host.browse("https://bing.com")
const screenshot = await page.screenshot() // returns a node.js Buffer
defImages(screenshot)

Detail

OpenAI supports a “low” / “high” field.

defImages(img, { detail: "low" })

Scaling

You can specify a maximum width, maximum height. GenAIScript will resize the image to fit into the constraints.

defImages(img, { maxWidth: 800 })
// and / or
defImages(img, { maxHeight: 800 })

Auto cropping

You can automatically remove uniform color on the edges of the image.

defImages(img, { autoCrop: true })