Tools
You can register tools (also known as functions) that the LLM may decide to call as part of assembling the answer. See OpenAI functions, Ollama tools, or Anthropic tool use.
Not all LLM models support tools, in those cases, GenAIScript also support a fallback mechanism to implement tool call through system prompts (see Fallback Tools).
defTool
defTool is used to define a tool that can be called by the LLM.
It takes a JSON schema to define the input and expects a string output.
The parameters are defined using the parameters schema.
The LLM decides to call this tool on its own!
defTool( "current_weather", "get the current weather", { city: "", }, (args) => { const { location } = args if (location === "Brussels") return "sunny" else return "variable" })In the example above, we define a tool called current_weather
that takes a location as input and returns the weather.
Weather tool example
This example uses the current_weather tool to get the weather for Brussels.
script({ model: "small", title: "Weather as function", description: "Query the weather for each city using a dummy weather function", temperature: 0.5, files: "src/cities.md", tests: { files: "src/cities.md", keywords: "Brussels", },})
$`Query the weather for each listed city and return the results as a table.`
def("CITIES", env.files)
defTool( "get_current_weather", "get the current weather", { type: "object", properties: { location: { type: "string", description: "The city and state, e.g. San Francisco, CA", }, }, required: ["location"], }, (args) => { const { context, location } = args const { trace } = context
trace.log(`Getting weather for ${location}...`)
let content = "variable" if (location === "Brussels") content = "sunny"
return content })Math tool example
This example uses the math expression evaluator to evaluate a math expression.
script({ title: "math-agent", model: "small", description: "A port of https://ts.llamaindex.ai/examples/agent", parameters: { question: { type: "string", default: "How much is 11 + 4? then divide by 3?", }, }, tests: { description: "Testing the default prompt", keywords: "5", },})
defTool( "sum", "Use this function to sum two numbers", { a: 1, b: 2 }, ({ a, b }) => { console.log(`${a} + ${b}`) return `${a + b}` })
defTool( "divide", "Use this function to divide two numbers", { type: "object", properties: { a: { type: "number", description: "The first number", }, b: { type: "number", description: "The second number", }, }, required: ["a", "b"], }, ({ a, b }) => { console.log(`${a} / ${b}`) return `${a / b}` })
$`Answer the following arithmetic question:
${env.vars.question}`Reusing tools in system scripts
You can define tools in a system script and include them in your main script as any other system script or tool.
system({ description: "Random tools" })
export default function (ctx: ChatGenerationContext) { const { defTool } = ctx defTool("random", "Generate a random number", {}, () => Math.random())}- Make sure to use
systeminstead ofscriptin the system script.
script({ title: "Random number", tools: ["random"],})$`Generate a random number.Multiple instances of the same system script
You can include the same system script multiple times in a script with different parameters.
script({ system: [ "system.agent_git", // git operations on current repository { id: "system.agent_git", // same system script parameters: { repo: "microsoft/genaiscript" } // but with new parameters variant: "genaiscript" // appened to the identifier to keep tool identifiers unique } ]})Model Context Protocol Tools
Model Context Provider (MCP) is an open protocol that enables seamless integration between LLM applications and external data sources and tools.
You can leverage MCP servers to provide tools to your LLM.
defTool({ memory: { command: "npx", args: ["-y", "@modelcontextprotocol/server-memory"], },})See Model Context Protocol Tools for more information.
Agentic Tools
Agentic is
a standard library of AI functions / tools
which are optimized for both normal TS-usage as well as LLM-based usage.
You can register any agentic tool in your script using defTool.
import { calculator } from "@agentic/calculator"defTool(calculator)See Agentic tools for more information.
Fallback Tool Support
Some LLM models do not have built-in model support. For those model, it is possible to enable tool support through system prompts. The performance may be lower than built-in tools, but it is still possible to use tools.
The tool support is implemented in system.tool_calls and “teaches” the LLM how to call tools. When this mode is enabled, you will see the tool call tokens being responded by the LLM.
GenAIScript maintains a list of well-known models that do not support tools so it will happen automatically for those models.
To enable this mode, you can either
- add the
fallbackToolsoption to the script
script({ fallbackTools: true,})- or add the
--fallack-toolsflag to the CLI
npx genaiscript run ... --fallback-toolsPackaging as System scripts
To pick and choose which tools to include in a script,
you can group them in system scripts. For example,
the current_weather tool can be included the system.current_weather.genai.mjs script.
script({ title: "Get the current weather",})defTool("current_weather", ...)then use the tool id in the tools field.
script({ ..., tools: ["current_weather"],})Example
Let’s illustrate how tools come together with a question answering script.
In the script below, we add the retrieval_web_search tool. This tool
will call into retrieval.webSearch as needed.
script({ title: "Answer questions", tool: ["retrieval_web_search"]})
def("FILES", env.files)
$`Answer the questions in FILES using a web search.
- List a summary of the answers and the sources used to create the answers.We can then apply this script to the questions.md file below.
- What is the weather in Seattle?- What laws were voted in the USA congress last week?After the first request, the LLM requests to call the web_search for each questions.
The web search answers are then added to the LLM message history and the request is made again.
The second yields the final result which includes the web search results.