Load Balancing
Introduction
Whenever there are multiple healthy destinations available, YARP has to decide which one to use for a given request. YARP ships with built-in load-balancing algorithms, but also offers extensibility for any custom load balancing approach.
Configuration
Services and middleware registration
Load balancing policies are registered in the DI container via the AddLoadBalancingPolicies()
method, which is automatically called by AddReverseProxy()
.
The middleware is added with UseLoadBalancing()
, which is included by default in the parameterless MapReverseProxy
method.
Cluster configuration
The algorithm used to determine the destination can be configured by setting the LoadBalancingPolicy
.
If no policy is specified, PowerOfTwoChoices
will be used.
File example
"ReverseProxy": {
"Clusters": {
"cluster1": {
"LoadBalancingPolicy": "RoundRobin",
"Destinations": {
"cluster1/destination1": {
"Address": "https://localhost:10000/"
},
"cluster1/destination2": {
"Address": "https://localhost:10010/"
}
}
}
}
}
Code example
var clusters = new[]
{
new ClusterConfig()
{
ClusterId = "cluster1",
LoadBalancingPolicy = LoadBalancingPolicies.RoundRobin,
Destinations = new Dictionary<string, DestinationConfig>(StringComparer.OrdinalIgnoreCase)
{
{ "destination1", new DestinationConfig() { Address = "https://localhost:10000" } },
{ "destination2", new DestinationConfig() { Address = "https://localhost:10010" } }
}
}
};
Built-in policies
YARP ships with the following built-in policies:
FirstAlphabetical
Select the alphabetically first available destination without considering load. This is useful for dual destination fail-over systems.
Random
Select a destination randomly.
PowerOfTwoChoices
(default)Select two random destinations and then select the one with the least assigned requests. This avoids the overhead of
LeastRequests
and the worst case forRandom
where it selects a busy destination.RoundRobin
Select a destination by cycling through them in order.
LeastRequests
Select the destination with the least assigned requests. This requires examining all destinations.
Extensibility
ILoadBalancingPolicy
is responsible for picking a destination from a list of available healthy destinations.
A custom implementation can be provided in DI.
// Implement the ILoadBalancingPolicy
public sealed class LastLoadBalancingPolicy : ILoadBalancingPolicy
{
public string Name => "Last";
public DestinationState? PickDestination(HttpContext context, ClusterState cluster, IReadOnlyList<DestinationState> availableDestinations)
{
return availableDestinations[^1];
}
}
// Register it in DI in ConfigureServices method
services.AddSingleton<ILoadBalancingPolicy, LastLoadBalancingPolicy>();
// Set the LoadBalancingPolicy on the cluster
cluster.LoadBalancingPolicy = "Last";