Skip to content
GenAIScript
Blog

Blog

Node.JS API

Peli
Peli
GenAIScript developer

A long standing feature request has been to run GenAIScript programmatically from other scripts. We are happy to announce that we have released a Node.JS API for GenAIScript. This API allows you to call GenAIScript from other TypeScript scripts (v1.83+).

Installation

You’ll want to add genaiscript as a (dev) dependency to your project.

Terminal window
npm i -D genaiscript

The run API

The run API is meant to mimic the behavior of the GenAIScript CLI. It takes the same arguments as the CLI and returns the same results. This allows you to call GenAIScript from other TypeScript scripts.

import { run } from "genaiscript/api"
const results = await run("summarize", ["myfile.txt"])

The result object contains the full list of messages, and additional parsed information like modified files, diagnostics and so forth.

Don’t mess with my process

On the caller side, the run implementation is a dependency free, side effect free function. It spawns a worker thread where GenAIScript does the work.

  • No global added
  • No package loaded
  • A few hunbred b of memory used

Help us improve it!

Obvisouly this is a first draft and we could do a better job at providing callbacks for progress. Send us your feedback!

Hugging Face Transformers.js

Peli
Peli
GenAIScript developer

🤗

Hugging Face Transformers.js is a JavaScript library that provides a simple way to run LLMs in the browser or node.js (or Bun, Deno, …).

With the latest GenAIScript, you can use Text Generation Models directly in the script configuration using the transformers model provider.

script({
  model: "transformers:HuggingFaceTB/SmolLM2-1.7B-Instruct:q4f16"
})

GenAIScript will download and cache the model for you, and you can start using it right away fully locally.

There are plenty of models to choose from and you can also follow the Hugging Face documentation to fine tune your own.

Fallback Tools

Peli
Peli
GenAIScript developer

Tools is a powerful feature of LLM models that allows you to augment the LLM reasoning with external tools.

These days, many LLM models come with a built-in support for tools. However, some of them don’t… like OpenAI’s o1-preview and o1-mini.

Fallback tools

With GenAIScript 1.72.0, we introduce the concept of fallback tools. Basically, it consists of a system script that “teaches” the LLM model about available tools and how to call them.

$`## Tool support


You can call external tools to help generating the answer of the user questions.


- The list of tools is defined in TOOLS. Use the description to help you choose the best tools.
- Each tool has an id, description, and a JSON schema for the arguments.
...


\`\`\`tool_calls
<tool_id>: { <JSON_serialized_tool_call_arguments> }
<tool_id_2>: { <JSON_serialized_tool_call_arguments_2> }
...
\`\`\`

A tool example

Here is an example of a tool that generates a random number between 0 and 1.

defTool("random", "Generate a random number", {}, () => Math.random())
$`Generate a random number between 0 and 1.`
  • o1-mini trace (using GitHub Models)
prompting github:o1-mini (~490 tokens)
```tool_calls
random: {}
```


prompting github:o1-mini (~532 tokens)
Your random number between 0 and 1 is **0.7792901036554349**.
  • gemma2 model (using Ollama)
prompting ollama:gemma2 (~716 tokens)


```tool_calls
random: {}
```
prompting ollama:gemma2 (~758 tokens)


The random number is 0.9552638470626966.




Let me know if you'd like to generate another random number!

Activation

The fallback tool mode is automatically activated for known LLM models that don’t support tools natively. The list is not complete so open an issue if you stumble upon a model that should have fallback tools enabled.

It can also be activated manually (see documentation).

LLM Agents

GenAIScript
GenAIScript
GenAI Blogger
 Peli
Peli
GenAIScript developer

GenAIScript defines an agent as a tool that runs an inline prompt to accomplish a task. The agent LLM is typically augmented with additional tools.

In this blog post, we’ll walk through building a user interaction agent that enables the agent to ask questions to the user.

script({
    tools: ["agent_user_input"],
})


$`
Imagine a funny question and ask the user to answer it.
From the answer, generate 3 possible answers and ask the user to select the correct one.
Ask the user if the answer is correct.
`

Let’s dive into understanding how to create an “Agent that can ask questions to the user.”

You can find the full script on GitHub right here.

Metadata

The script is written in JavaScript. It starts by declaring the metadata to make the script available as a system script, which can be reused in other scripts.

system.agent_user_input.genai.mjs
system({
    title: "Agent that can ask questions to the user.",
})

This line sets up the title for our system, making it clear that it’s intended to interact with the user by asking questions.

title and description

The defAgent function defines the behavior of our agent. It takes an agent identifier and a description. These two are quite important, as they will help the “host” LLM choose to use this agent.

defAgent(
    "user_input",
    "Ask user for input to confirm, select or answer a question.",
    ...

GenAIScript will automatically append a description of all the tools used by the agent prompt so you don’t have to worry about that part in the description.

prompt

The third argument is a string or a function to craft prompt instructions for the agent LLM call. The agent implementation already contains generic prompting to make the prompt behave like an agent, but you can add more to specify a role, tone, and dos and don’ts.

defAgent(
    ...,
    `You are an agent that can ask questions to the user and receive answers. Use the tools to interact with the user.
    - the message should be very clear. Add context from the conversation as needed.`,
    ...

model configuration

The last argument is a set of model options, similar to runPrompt, to configure the LLM call made by the agent. In particular, this is where you list the tools that the agent can use.

defAgent(
    ..., {
        tools: ["user_input"],
    }
)

How to use the agent

The agent is used like any other tool by referencing it in the script options.

script({
    tools: ["agent_user_input"]
})
...

Let’s try it!

Let’s try the agent with:

script({
    tools: ["agent_user_input"],
})


$`Imagine a funny question and ask the user to answer it.
From the answer, generate 3 possible answers and ask the user to select the correct one.
Ask the user if the answer is correct.`

and let’s look at the results…

prompting openai:gpt-4o (~150 tokens)


agent user_input: What would be the most unexpected thing to find inside a refrigerator?
run prompt agent user_input
prompting openai:gpt-4o (~234 tokens)


user input text: What would be the most unexpected thing to find inside a refrigerator?

✔ What would be the most unexpected thing to find inside a refrigerator? toaster

prompting openai:gpt-4o (~240 tokens)
toaster
prompting openai:gpt-4o (~156 tokens)


agent user_input: Based on your answer, which of the following would also be unexpected to find inside a refrigerator?
1. A television
2. A penguin
3. A snowman


Please select the correct answer.
run prompt agent user_input
prompting openai:gpt-4o (~263 tokens)


user input select: Based on your answer, which of the following would also be unexpected to find inside a refrigerator?

✔ Based on your answer, which of the following would also be unexpected to find inside a refrigerator? A television

prompting openai:gpt-4o (~269 tokens)
A television
prompting openai:gpt-4o (~162 tokens)


agent user_input: Is your selection of 'A television' the correct unexpected item to find inside a refrigerator?
run prompt agent user_input
prompting openai:gpt-4o (~239 tokens)


user input confirm: Is your selection of 'A television' the correct unexpected item to find inside a refrigerator?

✔ Is your selection of ‘A television’ the correct unexpected item to find inside a refrigerator? yes

prompting openai:gpt-4o (~244 tokens)
true
prompting openai:gpt-4o (~167 tokens)
Great choice! A television inside a refrigerator would indeed be quite unexpected.