This sample shows how to use OllamaAgent to chat with LLaVA model.

To run this example, you need to have an Ollama server running aside and have llava:latest model installed. For how to setup an Ollama server, please refer to Ollama.

Note

You can find the complete sample code here

Step 1: Install AutoGen.Ollama

First, install the AutoGen.Ollama package using the following command:

dotnet add package AutoGen.Ollama

For how to install from nightly build, please refer to Installation.

Step 2: Add using statement

using AutoGen.Core;
using AutoGen.Ollama.Extension;

Step 3: Create OllamaAgent

using var httpClient = new HttpClient()
{
    BaseAddress = new Uri("http://localhost:11434"),
};

var ollamaAgent = new OllamaAgent(
    httpClient: httpClient,
    name: "ollama",
    modelName: "llava:latest",
    systemMessage: "You are a helpful AI assistant")
    .RegisterMessageConnector()
    .RegisterPrintMessage();

Step 4: Start MultiModal Chat

LLaVA is a multimodal model that supports both text and image inputs. In this step, we create an image message along with a question about the image.

var image = Path.Combine("resource", "images", "background.png");
var binaryData = BinaryData.FromBytes(File.ReadAllBytes(image), "image/png");
var imageMessage = new ImageMessage(Role.User, binaryData);
var textMessage = new TextMessage(Role.User, "what's in this image?");
var reply = await ollamaAgent.SendAsync(chatHistory: [textMessage, imageMessage]);