| AutoGen for .NET

Middleware is a key feature in AutoGen.Net that enables you to customize the behavior of GenerateReplyAsync. It's similar to the middleware concept in ASP.Net and is widely used in AutoGen.Net for various scenarios, such as function call support, converting message of different types, print message, gather user input, etc.

Here are a few examples of how middleware is used in AutoGen.Net:

AssistantAgent is essentially an agent with FunctionCallMiddleware, HumanInputMiddleware and default reply middleware.
@AutoGen.OpenAI.GPTAgent is essentially an OpenAIChatAgent with FunctionCallMiddleware and OpenAIChatRequestMessageConnector.

Use middleware in an agent

To use middleware in an existing agent, you can either create a MiddlewareAgent on top of the original agent or register middleware functions to the original agent.

Create MiddlewareAgent on top of the original agent

// Create an agent that always replies "Hi!"
IAgent agent = new DefaultReplyAgent(name: "assistant", defaultReply: "Hi!");

// Create a middleware agent on top of default reply agent
var middlewareAgent = new MiddlewareAgent(innerAgent: agent);
middlewareAgent.Use(async (messages, options, agent, ct) =>
{
    if (messages.Last() is TextMessage lastMessage && lastMessage.Content.Contains("Hello World"))
    {
        lastMessage.Content = $"[middleware 0] {lastMessage.Content}";
        return lastMessage;
    }

    return await agent.GenerateReplyAsync(messages, options, ct);
});

var reply = await middlewareAgent.SendAsync("Hello World");
reply.GetContent().Should().Be("[middleware 0] Hello World");
reply = await middlewareAgent.SendAsync("Hello AI!");
reply.GetContent().Should().Be("Hi!");

Register middleware functions to the original agent

middlewareAgent = agent.RegisterMiddleware(async (messages, options, agent, ct) =>
{
    if (messages.Last() is TextMessage lastMessage && lastMessage.Content.Contains("Hello World"))
    {
        lastMessage.Content = $"[middleware 0] {lastMessage.Content}";
        return lastMessage;
    }

    return await agent.GenerateReplyAsync(messages, options, ct);
});

Short-circuit the next agent

The example below shows how to short-circuit the inner agent

// This middleware will short circuit the agent and return a message directly.
middlewareAgent.Use(async (messages, options, agent, ct) =>
{
    return new TextMessage(Role.Assistant, $"[middleware shortcut]");
});

Note

When multiple middleware functions are registered, the order of middleware functions is first registered, last invoked.

Streaming middleware

You can also modify the behavior of GenerateStreamingReplyAsync by registering streaming middleware to it. One example is OpenAIChatRequestMessageConnector which converts StreamingChatCompletionsUpdate to one of AutoGen.Core.TextMessageUpdate or AutoGen.Core.ToolCallMessageUpdate.