Automating scripts

Once you have a script that you are happy with, you can automate it using the command line interface.

Running a script using the CLI

The basic usage of the CLI is to run a script with a tool name and a list of files.

npx --yes genaiscript run <script_id> <...files>

where <script_id> is the name of the script (e.g. filename without .genai.mjs) and <...files> is a list of files to run the script on.

The CLI will use the secrets in the .env file, populate env.files with <...files>, run the script and emit the output to the standard output.

Tip

npx allows you to run a command from the genaiscript npm package (either one installed locally, or fetched remotely). Add --yes flag to automatically agree to any prompts without confirmation.

You can use the CLI to run your scripts in a CI/CD pipeline. The CLI will return a non-zero exit code if the script fails, which can be used to fail the pipeline.

Apply Edits

Add the --apply-edits flag to the CLI to automatically write the file edits.

npx --yes genaiscript run <script> <...files> --apply-edits

Caution

An LLM may generate arbitrary code that can be harmful to your system. We recommend that you review the modified code before executing it. This could be done through a separate branch and a pull request. You can also use a container to run the script in a sandboxed environment.

Refer to Security and Trust for more discussion.

GitHub Action

GitHub Actions is a continuous integration and continuous delivery (CI/CD) platform that allows you to automate your build, test, and deployment pipeline. This section explains how to integrate your GenAIScript in GitHub Actions workflows and pull requests.

GitHub Models provide a integrated way to run LLM inference from a GitHub Action.

Configure GitHub Models

To use GitHub Models, you must add the models: read permission to your workflow, pass the GITHUB_TOKEN secret to the cli, and configure the cli to use GitHub Models. This can be done by setting the LLM provider to github in cli.

permissions:
    models: read
...
- run: npx --yes genaiscript run <script> <...files> --provider github
  env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Configure secrets and variables

Configure the secrets and variables on your repository or organization so that GenAIScript can connect to your LLM.

The secrets and variables should match the .env file in your local environment.

Running a script

Use the cli to run the script in a GitHub Action.

• Make sure to pass the secrets and variables to the script to give access to the LLM.
• use the --out <path> flag to store the results in a directory so that you can upload them as an artifact.

- run: npx --yes genaiscript run <script> <...files> --out results
  env:
      # variables
      OPENAI_API_TYPE: ${{ env.OPENAI_API_TYPE }}
      OPENAI_API_BASE: ${{ env.OPENAI_API_BASE }}
      # secret, redacted
      OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}

Add the trace to the action summary

Use the out-trace flag to output the trace to the summary file, $GITHUB_STEP_SUMMARY (see example).

- run: npx --yes genaiscript run ... --out-trace $GITHUB_STEP_SUMMARY

Diff

You can use git.diff to execute a git diff command to retrieve changes in the current branch.

const changes = await git.diff({ base: "main" })
def("GIT_DIFF", changes, {
    language: "diff",
    maxTokens: 20000,
})

Note that you’ll need to pull origin/main branch to make this command work in an action.

- run: git fetch origin && git pull origin main:main

Storing output in artifacts

Ensure that the directory containing the results is uploaded as an artifact. You can review the trace by opening the res.trace.md file. in the zipped artifact.

- uses: actions/upload-artifact@v4
  if: always()
  with:
      path: |
          genairesults/**

Azure OpenAI with a Service Principal

The official documentation of the azure login action contains detailed steps on configure Azure resource access from GitHub Actions.

Note

The login with OpenID Connect (OIDC) is the recommended solution in the Azure documentation pages.

The steps below show how to configure the Azure login action to access the OpenAI resource using a Service Principal client secret.

1. Create a Service Principal following connect from azure secret guide.

2. Assign any role to the service principal (e.g. Reader) in your Azure subscription. You need this to login.

3. Assign the role Cognitive Services OpenAI User to the service principal. You need this so that the service principal can access the OpenAI resource.

4. Configure the AZURE_CREDENTIALS secret in your GitHub repository with the service principal credentials.

   {
       "clientId": "<Client ID>",
       "clientSecret": "<Client Secret>",
       "subscriptionId": "<Subscription ID>",
       "tenantId": "<Tenant ID>"
   }

5. Configure the AZURE_OPENAI_API_ENDPOINT in your repository GitHub Action variables.

6. Add the following step in your workflow to your GitHub action to login to Azure.

   genai.yml

   - name: Azure Login action
     uses: azure/login@v2
     with:
         creds: ${{ secrets.AZURE_CREDENTIALS }}

7. Update each step that invokes the cli to include the AZURE_OPENAI_API_ENDPOINT variable.

   - name: run genai script
     run: npx --yes genaiscript run ...
     env:
         AZURE_OPENAI_API_ENDPOINT: ${{ env.AZURE_OPENAI_API_ENDPOINT }}

GitHub Pull request

If your GitHub Action is triggered by a pull request event, you can use the following flags to add comments: description, conversation and reviews.

In order to create comments, the workflow must have the pull-requests: write permission and the GITHUB_TOKEN secret must be passed to the script.

permissions:
    pull-requests: write
...
    - run: npx --yes genaiscript run ...
      env:
        GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
        ...

Update Description

The --pull-request-description inserts the LLM output into the pull request section (see example). The command takes an optional string argument to uniquely identify this comment, it is used to update the comment (default is the script id).

- run: npx --yes genaiscript run --pull-request-description

If you want to run this script when the pull request is ready for review, use the ready_for_review, opened, reopened events.

on:
    pull_request:
        types: [ready_for_review, opened, reopened]

Note

The comment is enclosed in two XML comments (<genaiscript begin [id]>, <genaiscript end [id]>) to allow for easy identification and removal. Make sure to keep those.

Conversation comment

The --pull-request-comment adds the LLM output as a comment to the pull request conversation (see example). The optional argument is an identifier for the comment (default is the script id) so that only one comment appears for the id.

- run: npx --yes genaiscript run --pull-request-comment
  env: ...

Review comments

Use the --pull-request-reviews to convert annotations as review comments to the last commit on the pull request (see example).

- run: npx --yes genaiscript run --pull-request-reviews
  env: ...

GenAIScript will automatically try to ignore duplicate review comments and only create new ones.

To collect the changes of the last commit in the pull request branch (see cool blog post), you can try this git command:

const { stdout: changes } = await host.exec("git", [
    "diff",
    "HEAD^",
    "HEAD",
    "--",
    "**.ts",
])