Automating scripts

Once you have a script that you are happy with, you can automate it using the command line interface.

Running a script using the CLI

The basic usage of the CLI is to run a script with a tool name and a list of files.

npx --yes genaiscript run <script> <...files>

The CLI will use the secrets in the .env file, populate env.files with <...files>, run the script and emit the output to the standard output.

Tip

npx allows you to run a command from the genaiscript npm package (either one installed locally, or fetched remotely). Add --yes flag to automatically agree to any prompts without confirmation.

You can use the CLI to run your scripts in a CI/CD pipeline. The CLI will return a non-zero exit code if the script fails, which can be used to fail the pipeline.

Apply Edits

Add the --apply-edits flag to the CLI to automatically write the file edits.

npx --yes genaiscript run <script> <...files> --apply-edits

Caution

An LLM may generate arbitrary code that can be harmful to your system. We recommend that you review the modified code before executing it. This could be done through a separate branch and a pull request. You can also use a container to run the script in a sandboxed environment.

Refer to Security and Trust for more discussion.

GitHub Action

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. This section explains how to integrate your GenAIScript in GitHub Actions workflows and pull requests.

Authentication

Configure the secrets and variables on your repository or organization so that GenAIScript can connect to your LLM.

The secrets and variables should match the .env file in your local environment.

Running a script

Use the cli to run the script in a GitHub Action.

• Make sure to pass the secrets and variables to the script to give access to the LLM.
• use the --out <path> flag to store the results in a directory so that you can upload them as an artifact.

    - run: npx --yes genaiscript run <script> <...files> --out genairesults
        env:
            # variables
            OPENAI_API_TYPE: ${{ env.OPENAI_API_TYPE }}
            OPENAI_API_BASE: ${{ env.OPENAI_API_BASE }}
            # secret, redacted
            OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Add the trace to the action summary

Use the out-trace flag to output the trace to the summary file, $GITHUB_STEP_SUMMARY (see example).

- run: npx --yes genaiscript run ... --out-trace $GITHUB_STEP_SUMMARY

Diff

You can use host.exec to execute git command to retrieve changes in the current branch.

const { stdout: changes } = await host.exec("git", [
    "diff",
    "main",
    "--",
    ":!**/genaiscript.d.ts",
    ":!**/jsconfig.json",
    ":!genaisrc/*",
    ":!.github/*",
    ":!.vscode/*",
    ":!yarn.lock",
])

def("GIT_DIFF", changes, {
    language: "diff",
    maxTokens: 20000,
})

Note that you’ll need to pull origin/main branch to make this command work in an action.

- run: git fetch origin && git pull origin main:main

Storing output in artifacts

Ensure that the directory containing the results is uploaded as an artifact. You can review the trace by opening the res.trace.md file. in the zipped artifact.

- uses: actions/upload-artifact@v4
  if: always()
  with:
      path: |
          genairesults/**

GitHub Pull request

If your GitHub Action is triggered by a pull request event, you can use the following flags to add comments: description, conversation and reviews.

In order to create comments, the workflow must have the pull-requests: write permission and the GITHUB_TOKEN secret must be passed to the script.

permissions:
    pull-requests: write
...
    - run: npx --yes genaiscript run ...
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        ...

Update Description

The --pull-request-description inserts the LLM output into the pull request section (see example). The command takes an optional string argument to uniquely identify this comment, it is used to update the comment (default is the script id).

- run: npx --yes genaiscript run --pull-request-description

If you want to run this script when the pull request is ready for review, use the ready_for_review, opened, reopened events.

on:
    pull_request:
        types: [ready_for_review, opened, reopened]

Note

The comment is enclosed in two XML comments (<genaiscript begin [id]>, <genaiscript end [id]>) to allow for easy identification and removal. Make sure to keep those.

Conversation comment

The --pull-request-comment adds the LLM output as a comment to the pull request conversation (see example). The optional argument is an identifier for the comment (default is the script id) so that only one comment appears for the id.

- run: npx --yes genaiscript run --pull-request-comment
  env: ...

Review comments

Use the --pull-request-reviews to convert annotations as review comments to the last commit on the pull request (see example).

- run: npx --yes genaiscript run --pull-request-reviews
  env: ...

GenAIScript will automatically try to ignore duplicate review comments and only create new ones.

To collect the changes of the last commit in the pull request branch (see cool blog post), you can try this git command:

const { stdout: changes } = await host.exec("git", [
    "diff",
    "HEAD^",
    "HEAD",
    "--",
    "**.ts",
])