An overview of built-in IMessage types

Start from 0.0.9, AutoGen introduces the IMessage and IMessage<T> types to provide a unified message interface for different agents. The IMessage is a non-generic interface that represents a message. The IMessage<T> is a generic interface that represents a message with a specific T where T can be any type.

Besides, AutoGen also provides a set of built-in message types that implement the IMessage and IMessage<T> interfaces. These built-in message types are designed to cover different types of messages as much as possilbe. The built-in message types include:

Note

The minimal requirement for an agent to be used as admin in GroupChat is to support TextMessage.

Note

Message will be deprecated in 0.0.14. Please replace it with a more specific message type like TextMessage, ImageMessage, etc.

• TextMessage: A message that contains a piece of text.
• ImageMessage: A message that contains an image.
• MultiModalMessage: A message that contains multiple modalities like text, image, etc.
• ToolCallMessage: A message that represents a function call request.
• ToolCallResultMessage: A message that represents a function call result.
• ToolCallAggregateMessage: A message that contains both ToolCallMessage and ToolCallResultMessage. This type of message is used by FunctionCallMiddleware to aggregate both ToolCallMessage and ToolCallResultMessage into a single message.
• MessageEnvelope<T>: A message that represents an envelope that contains a message of any type.
• Message: The original message type before 0.0.9. This message type is reserved for backward compatibility. It is recommended to replace it with a more specific message type like TextMessage, ImageMessage, etc.

Streaming message support

AutoGen also introduces IStreamingMessage and IStreamingMessage<T> which are used in streaming call api. The following built-in message types implement the IStreamingMessage and IStreamingMessage<T> interfaces:

Note

All IMessage is also a IStreamingMessage. That means you can return an IMessage from a streaming call method. It's also recommended to return the final updated result instead of the last update as the last message in the streaming call method to indicate the end of the stream, which saves caller's effort of assembling the final result from multiple updates.

• TextMessageUpdate: A message that contains a piece of text update.
• ToolCallMessageUpdate: A message that contains a function call request update.

Usage

The below code snippet shows how to print a streaming update to console and update the final result on the caller side.

var helloTextMessage = new TextMessage(Role.User, "Hello");
var reply = agent.GenerateStreamingReplyAsync([helloTextMessage]);
var finalTextMessage = new TextMessage(Role.Assistant, string.Empty, from: agent.Name);
await foreach (var message in reply)
{
    if (message is TextMessageUpdate textMessage)
    {
        Console.Write(textMessage.Content);
        finalTextMessage.Update(textMessage);
    }
}

If the agent returns a final result instead of the last update as the last message in the streaming call method, the caller can directly use the final result without assembling the final result from multiple updates.

reply = agent.GenerateStreamingReplyAsync([helloTextMessage]);
TextMessage finalMessage = null;
await foreach (var message in reply)
{
    if (message is TextMessageUpdate textMessage)
    {
        Console.Write(textMessage.Content);
    }
    else if (message is TextMessage txtMessage)
    {
        finalMessage = txtMessage;
    }
}