Skip to content

Peli

7 posts by Peli

Fallback Tools

Tools is a powerful feature of LLM models that allows you to augment the LLM reasoning with external tools.

These days, many LLM models come with a built-in support for tools. However, some of them don’t… like OpenAI’s o1-preview and o1-mini.

Fallback tools

With GenAIScript 1.72.0, we introduce the concept of fallback tools. Basically, it consists of a system script that “teaches” the LLM model about available tools and how to call them.

$`## Tool support
You can call external tools to help generating the answer of the user questions.
- The list of tools is defined in TOOLS. Use the description to help you choose the best tools.
- Each tool has an id, description, and a JSON schema for the arguments.
...
\`\`\`tool_calls
<tool_id>: { <JSON_serialized_tool_call_arguments> }
<tool_id_2>: { <JSON_serialized_tool_call_arguments_2> }
...
\`\`\`

A tool example

Here is an example of a tool that generates a random number between 0 and 1.

defTool("random", "Generate a random number", {}, () => Math.random())
$`Generate a random number between 0 and 1.`
  • o1-mini trace (using GitHub Models)
prompting github:o1-mini (~490 tokens)
```tool_calls
random: {}
```
prompting github:o1-mini (~532 tokens)
Your random number between 0 and 1 is **0.7792901036554349**.
  • gemma2 model (using Ollama)
prompting ollama:gemma2 (~716 tokens)
```tool_calls
random: {}
```
prompting ollama:gemma2 (~758 tokens)
The random number is 0.9552638470626966.
Let me know if you'd like to generate another random number!

Activation

The fallback tool mode is automatically activated for known LLM models that don’t support tools natively. The list is not complete so open an issue if you stumble upon a model that should have fallback tools enabled.

It can also be activated manually (see documentation).

LLM Agents

GenAIScript defines an agent as a tool that runs an inline prompt to accomplish a task. The agent LLM is typically augmented with additional tools.

In this blog post, we’ll walk through building a user interaction agent that enables the agent to ask questions to the user.

script({
tools: ["agent_user_input"],
})
$`
Imagine a funny question and ask the user to answer it.
From the answer, generate 3 possible answers and ask the user to select the correct one.
Ask the user if the answer is correct.
`

Let’s dive into understanding how to create an “Agent that can ask questions to the user.”

You can find the full script on GitHub right here.

Metadata

The script is written in JavaScript. It starts by declaring the metadata to make the script available as a system script, which can be reused in other scripts.

system.agent_user_input.genai.mjs
system({
title: "Agent that can ask questions to the user.",
})

This line sets up the title for our system, making it clear that it’s intended to interact with the user by asking questions.

title and description

The defAgent function defines the behavior of our agent. It takes an agent identifier and a description. These two are quite important, as they will help the “host” LLM choose to use this agent.

defAgent(
"user_input",
"Ask user for input to confirm, select or answer a question.",
...

GenAIScript will automatically append a description of all the tools used by the agent prompt so you don’t have to worry about that part in the description.

prompt

The third argument is a string or a function to craft prompt instructions for the agent LLM call. The agent implementation already contains generic prompting to make the prompt behave like an agent, but you can add more to specify a role, tone, and dos and don’ts.

defAgent(
...,
`You are an agent that can ask questions to the user and receive answers. Use the tools to interact with the user.
- the message should be very clear. Add context from the conversation as needed.`,
...

model configuration

The last argument is a set of model options, similar to runPrompt, to configure the LLM call made by the agent. In particular, this is where you list the tools that the agent can use.

defAgent(
..., {
tools: ["user_input"],
}
)

How to use the agent

The agent is used like any other tool by referencing it in the script options.

script({
tools: ["agent_user_input"]
})
...

Let’s try it!

Let’s try the agent with:

script({
tools: ["agent_user_input"],
})
$`Imagine a funny question and ask the user to answer it.
From the answer, generate 3 possible answers and ask the user to select the correct one.
Ask the user if the answer is correct.`

and let’s look at the results…

prompting openai:gpt-4o (~150 tokens)
agent user_input: What would be the most unexpected thing to find inside a refrigerator?
run prompt agent user_input
prompting openai:gpt-4o (~234 tokens)
user input text: What would be the most unexpected thing to find inside a refrigerator?

✔ What would be the most unexpected thing to find inside a refrigerator? toaster

prompting openai:gpt-4o (~240 tokens)
toaster
prompting openai:gpt-4o (~156 tokens)
agent user_input: Based on your answer, which of the following would also be unexpected to find inside a refrigerator?
1. A television
2. A penguin
3. A snowman
Please select the correct answer.
run prompt agent user_input
prompting openai:gpt-4o (~263 tokens)
user input select: Based on your answer, which of the following would also be unexpected to find inside a refrigerator?

✔ Based on your answer, which of the following would also be unexpected to find inside a refrigerator? A television

prompting openai:gpt-4o (~269 tokens)
A television
prompting openai:gpt-4o (~162 tokens)
agent user_input: Is your selection of 'A television' the correct unexpected item to find inside a refrigerator?
run prompt agent user_input
prompting openai:gpt-4o (~239 tokens)
user input confirm: Is your selection of 'A television' the correct unexpected item to find inside a refrigerator?

✔ Is your selection of ‘A television’ the correct unexpected item to find inside a refrigerator? yes

prompting openai:gpt-4o (~244 tokens)
true
prompting openai:gpt-4o (~167 tokens)
Great choice! A television inside a refrigerator would indeed be quite unexpected.

Automatic Web Page Content Analysis

In this blog post, we’ll dive into a practical example showcasing how to leverage GenAIScript for automatic web page content analysis. GenAIScript uses the playwright browser automation library which allows to load, interact and inspect web pages.

Step-by-Step Explanation of the Code

The following snippet provides a concise and effective way to analyze a web page’s content using GenAIScript:

const page = await host.browse("https://bing.com")
const screenshot = await page.screenshot()
defImages(screenshot, { maxWidth: 800 })
const text = parsers.HTMLtoMarkdown(await page.content())
def("PAGE_TEXT", text)
$`Analyze the content of the page and provide insights.`

Let’s break down what each line of this script does:

1. Navigating to a Web Page

const page = await host.browse("https://example.com")

This line automatically navigates to the specified URL (https://example.com). The host.browse function is a powerful feature of GenAIScript that initializes a browser session and returns a page object for further interactions.

2. Taking a Screenshot

const screenshot = await page.screenshot()

Here, the script captures a screenshot of the current view of the page. This is particularly useful for archiving or visual analysis.

3. Defining Images for Analysis

defImages(screenshot, { maxWidth: 800 })

After capturing the screenshot, this line registers the image for further analysis. defImages is a function that makes the screenshot available to subsequent analytical or AI-driven functions in the script.

4. Extracting Text Content

const text = parsers.HTMLtoMarkdown(await page.content())

This command extracts all text content from the page, which can be invaluable for content audits or textual analysis.

5. Storing Text for Further Use

def("PAGE_TEXT", text)

The extracted text is then stored under the identifier PAGE_TEXT, allowing it to be referenced in later parts of the script or for documentation purposes.

6. Analyzing the Content

$`Analyze the content of the page and provide insights.`

Finally, this line represents a call to an AI or script-defined function that analyzes the captured content and provides insights. This is where the real power of automation and AI integration into GenAIScript shines, enabling detailed analysis without manual intervention.

Conclusion

With a simple yet powerful script like the one discussed, GenAIScript makes it feasible to automate the process of web page content analysis. Whether you’re conducting competitive analysis, performing content audits, or simply archiving web pages, GenAIScript offers a scalable and efficient solution.

Support for Agentic tools

Agentic is a standard library of TypeScript AI tools optimized for both TS-usage as well as LLM-based usage, which is really important for testing and debugging.

Agentic brings support for a variety of online APIs, like Bing, Wolfram Alpha, Wikipedia, and more. You can register any Agentic tool in your script using defTool. Here’s an example of how to use the Weather tool:

import { WeatherClient } from "@agentic/weather"
const weather = new WeatherClient()
defTool(weather)

Keeping your README Fresh and Engaging

In the world of open source, a well-maintained README file acts as the front door to your project. It’s often the first thing potential users and contributors see, and as such, it should be both informative and inviting. Today, we’re diving into the GenAIScript that helps keep the README of the GenAI project as fresh as a daisy! 🌼 Check out the actual script file for the details.

This blog post was co-authored with a script.

The Intention Behind the Script

The script we’re analyzing is a maintenance tool designed to import relevant information from documentation and samples into the README to enhance its appeal to users. It ensures that the README is not just a static file but a vibrant, updated document that accurately reflects the features and capabilities of GenAI.

Line-by-Line Explanation

Let’s walk through the script code as if we are crafting it from the ground up:

script({
description:
"Maintenance script for the README that imports information from the documentation and samples to make it more attractive to users.",
tools: ["fs"],
})

Here, we’re defining the script’s metadata, including a description of its purpose and the tools it will utilize. The fs tool indicates file system operations will be involved.

def("README", { filename: "README.md" })
def("FEATURES", { filename: "docs/src/content/docs/index.mdx" })

These lines declare two important files: the README itself and a FEATURES file that contains information to be imported into the README.

$`You are an expert open source maintainer.
...
`

In this template literal, we’re outlining the tasks for the script, including guidelines for updating the README with features, samples, and documentation links while preserving certain sections unchanged.

defFileOutput("README.md")

Finally, we specify that the output of this script will be an updated README.md file.

How to Run the Script

To execute this maintenance script, you’ll need the GenAIScript CLI. If you haven’t installed it yet, head over to the official documentation for installation instructions. Once you have the CLI ready, run the following command in your terminal:

Terminal window
genaiscript run readme-updater

This command will kick off the script and apply the enhancements to your README file, ensuring it’s up-to-date and user-friendly.

Conclusion

A meticulous README is a hallmark of a well-maintained open source project. With this GenAIScript, the GenAI project sets an excellent example of automating the upkeep of project documentation. Embrace the power of automation to keep your project’s welcome mat clean and welcoming. Happy coding! 👨‍💻👩‍💻

Unlocking the Power of Prompts - A Gentle Introduction to GenAIScript 🚀

Ever wondered how to leverage the power of AI and Large Language Models (LLMs) in your projects? Look no further! This post will introduce you to GenAIScript, a tool designed to simplify the creation of prompts and interactions with LLMs. Let’s dive in! 🌊

What is GenAIScript?

GenAIScript uses a stylized version of JavaScript to generate prompts, which are then sent to an LLM. Scripts are stored as files (genaisrc/*.genai.mjs), executed to produce the prompt text and structured results (files, diagnostics) are extracted automatically.

Getting Started

Here’s a simple example to get you started. Create a file named poem.genai.mjs in the genaisrc folder and add the following code:

$`Write a one sentence poem.`

When executed, this script will generate the following prompt:

👤 User
Write a one sentence poem.
🤖 Assistant
Roses bloom, hearts swoon, under the silver moon.

Adding Context

GenAIScript can also use context variables, allowing you to interact with files or other data sources. Let’s see an example where we define a context variable using env.files:

def("FILES", env.files)
$`You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.`

Execute this script to see the generated user message and the assistant’s response. The context variable FILES will contain the list of files in the environment.

👤 User
FILES:
file="src/samples/markdown.md"
What is Markdown?
Markdown is a lightweight markup language that...
You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.
🤖 Assistant
I reviewed the document in "src/samples/markdown.md"
and found the following two important issues:
1. **Missing Consistency in Heading Styles**: ...

Metadata and Script Configuration

You can add metadata to your script using the script function. This helps in organizing and configuring the script, including specifying the model and other parameters. GenAIScript supports various LLM providers, such as OpenAI, Azure OpenAI, GitHub Models, Ollama and more.

script({
title: "Technical proofreading",
description: "Reviews the text as a tech writer.",
model: "openai:gpt-4o",
temperature: 0.1,
})
def("FILES", env.files)
$`You are an expert technical writer and proofreader.
Review the documents in FILES and report the 2 most important issues.`

Next Steps

There you have it! A gentle introduction to GenAIScript to get you started on your prompt engineering journey. Happy scripting! 💻✨