Skip to content
GenAIScript
Blog

File Output

Reliable file generation, whether new or updates, is one of the most challenging parts of working with LLMs. The GenAIScript script supports a few approaches and formats to generate files: for small files, regenerating the entire content is typically more efficient. For large files, generating edits is more efficient.

How it works

GenAIScript automatically adds a system message that teaches the LLM how to format the output files.

Let’s start with a script that generates a poem and asks the GenAIScript to save it to a text file.

poet.genai.mjs
$`Generate a 1 sentence poem and save it to a text file.`

Since no system prompt is specified, GenAIScript adds the default set of system prompts, including the system.files prompt. This prompt instructs the LLM to generate a file with the output of the script. The LLM responds with a code section that also mentions a filename. This is the format that GenAIScript can automatically parse out.

FILE ./poem.txt:


```
In twilight's gentle embrace, dreams dance like whispers on the breeze.
```

By default, file edits are not applied automatically. In Visual Studio Code, a refactoring preview is opened and the user can accept or reject the changes.

A screenshot of a text editor interface showing a file named "poem.txt" being created. The text "In the whispering twilight, shadows dance to the melody of stars." is highlighted. There are options to apply or discard changes.

In the CLI, the changes are silently ignored unless the --apply-edits flag is used.

Terminal window
npx genaiscript run poet --apply-edits

Changelog format

The full regeneration of files only works for small files. For large files, GenAIScript uses a custom changelog format that is designed to minimize hallucinations.

commenter.genai.mjs
def("FILE", env.files)
$`Comment every line of code and update the file. Use the changelog format.`

When we run the script on a source file, the LLM generates a changelog that contains the changes to the file. GenAIScript will parse this output and generate a file edit similar to a full file update.\

```changelog
ChangeLog:1@packages/sample/src/greeter.ts
Description: Added comments to each line of code to explain functionality.
OriginalCode@1-6:
[1] class Greeter {
[2]     greeting: string
[3]
[4]     constructor(message: string) {
[5]         this.greeting = message
[6]     }
ChangedCode@1-6:
[1] // Define a class named Greeter
[2] class Greeter {
[3]     // Property to hold the greeting message
[4]     greeting: string
[5]
[6]     // Constructor to initialize the greeting property
[7]     constructor(message: string) {
[8]         // Set the greeting property to the provided message
[9]         this.greeting = message
[10]     }
OriginalCode@7-11:
[7]
[8]     greet() {
[9]         return "Hello, " + this.greeting
[10]     }
[11] }
ChangedCode@7-11:
[7]
[8]     // Method to return a greeting message
[9]     greet() {
[10]         return "Hello, " + this.greeting
[11]     }
[12] }
OriginalCode@12-18:
[12]
[13] interface IGreeter {
[14]     greeting: string
[15]     greet(): string
[16] }
[17]
[18] export function hello() {}
ChangedCode@12-18:
[12]
[13] // Define an interface for a Greeter
[14] interface IGreeter {
[15]     // Property to hold the greeting message
[16]     greeting: string
[17]     // Method to return a greeting message
[18]     greet(): string
[19] }
[20]
[21] // Export an empty function named hello
[22] export function hello() {}
OriginalCode@19-20:
[19]
[20] let greeter = new Greeter("world")
ChangedCode@19-20:
[23]
[24] // Create a new instance of Greeter with the message "world"
[25] let greeter = new Greeter("world")
```

As you can see, the changelog format is much more heavyweight in terms of token; however, it is more reliable at producing edits in large files.

Declaring file outputs

The defFileOutput function lets you declare file output paths and the purpose of those files. This function is used to specify the output files that are generated by the script.

defFileOutput("src/*.md", "Product documentation in markdown format")

In our example, we tell the LLM to produce the poem at poem.txt and it also allows GenAIScript to validate the file location and automatically apply the changes.

$`Generate a 1 sentence poem and save it to a text file.`
defFileOutput("poem.txt", "the generated poem")

In the background, GenAIScript adds a system message that looks like this and tells the LLM where files should be.

## File generation rules


When generating files, use the following rules which are formatted as "file glob: description":


poem.txt: the generated poem

Schema Validation

You can associate a JSON schema with the file output. This schema is used to validate the content of the file before it is written to disk.

const schema = defSchema("KEYWORDS", {
    type: "array",
    items: {
        type: "string",
    },
})
defFileOutput("src/rag/*.keywords.json", "An array of keywords in the file", {
    schema,
})

File output post processing

You can register a callback to programmaticaly manipulate the generate files using defOutputProcessor.

System prompts

The support for generating files is defined in a few system prompts. These prompts are typically automatically added but you may need to add them back if you specify a custom set of system prompts.