Session 03 - Load Balancing & Rate Limiting

Load Balancing Fundamentals

L4 vs L7 Load Balancing

Azure Load Balancing Services

Khi nào chọn gì?

• Load Balancer - Internal load balancing giữa VMs, non-HTTP traffic (TCP/UDP), cần performance cao nhất.
• Application Gateway - Single-region web app, cần WAF, URL routing, SSL termination.
• Front Door - Multi-region deployment, cần global load balancing + CDN + WAF.

Rate Limiting với ASP.NET Core

ASP.NET Core 7+ cung cấp built-in rate limiting middleware.

Setup cơ bản

csharp
// Program.cs
builder.Services.AddRateLimiter(options =>
{
    // Global limiter cho tất cả requests
    options.GlobalLimiter = PartitionedRateLimiter.Create<HttpContext, string>(context =>
        RateLimitPartition.GetFixedWindowLimiter(
            partitionKey: context.Connection.RemoteIpAddress?.ToString() ?? "unknown",
            factory: _ => new FixedWindowRateLimiterOptions
            {
                PermitLimit = 100,
                Window = TimeSpan.FromMinutes(1),
                QueueLimit = 10,
                QueueProcessingOrder = QueueProcessingOrder.OldestFirst
            }));

    // Custom response khi bị rate limited
    options.OnRejected = async (context, ct) =>
    {
        context.HttpContext.Response.StatusCode = StatusCodes.Status429TooManyRequests;
        context.HttpContext.Response.Headers.RetryAfter =
            context.Lease.TryGetMetadata(MetadataName.RetryAfter, out var retryAfter)
                ? retryAfter.TotalSeconds.ToString("F0")
                : "60";

        await context.HttpContext.Response.WriteAsJsonAsync(new
        {
            error = "Rate limit exceeded",
            retryAfterSeconds = retryAfter?.TotalSeconds ?? 60
        }, ct);
    };
});

app.UseRateLimiter();

Named Policies

csharp
builder.Services.AddRateLimiter(options =>
{
    // Policy cho API thông thường
    options.AddFixedWindowLimiter("standard", opt =>
    {
        opt.PermitLimit = 100;
        opt.Window = TimeSpan.FromMinutes(1);
    });

    // Policy cho endpoint nhạy cảm (login, payment)
    options.AddSlidingWindowLimiter("strict", opt =>
    {
        opt.PermitLimit = 10;
        opt.Window = TimeSpan.FromMinutes(1);
        opt.SegmentsPerWindow = 6; // 6 segments x 10 giây
    });

    // Policy cho endpoint public (search, catalog)
    options.AddTokenBucketLimiter("relaxed", opt =>
    {
        opt.TokenLimit = 200;
        opt.ReplenishmentPeriod = TimeSpan.FromSeconds(10);
        opt.TokensPerPeriod = 20;
    });
});

// Áp dụng policy cho endpoint
app.MapGet("/api/products", GetProducts).RequireRateLimiting("standard");
app.MapPost("/api/auth/login", Login).RequireRateLimiting("strict");
app.MapGet("/api/search", Search).RequireRateLimiting("relaxed");

Các Rate Limiting Algorithms

So sánh

Vấn đề Fixed Window boundary

Window 1: 00:00 - 01:00    Window 2: 01:00 - 02:00
[......90 requests]          [90 requests......]
              ↑ 90 req cuối     ↑ 90 req đầu
              = 180 requests trong 60 giây thực tế (limit 100!)

Sliding Window giải quyết bằng cách chia window thành nhiều segment và tính weighted count.

Reverse Proxy với YARP

YARP (Yet Another Reverse Proxy) là reverse proxy library của Microsoft, xây dựng trên ASP.NET Core.

csharp
// Program.cs
builder.Services.AddReverseProxy()
    .LoadFromConfig(builder.Configuration.GetSection("ReverseProxy"));

app.MapReverseProxy();

json
// appsettings.json
{
  "ReverseProxy": {
    "Routes": {
      "api-route": {
        "ClusterId": "api-cluster",
        "Match": { "Path": "/api/{**catch-all}" },
        "Transforms": [
          { "PathRemovePrefix": "/api" },
          { "RequestHeader": "X-Forwarded-Prefix", "Set": "/api" }
        ]
      },
      "web-route": {
        "ClusterId": "web-cluster",
        "Match": { "Path": "/{**catch-all}" }
      }
    },
    "Clusters": {
      "api-cluster": {
        "LoadBalancingPolicy": "RoundRobin",
        "HealthCheck": {
          "Active": {
            "Enabled": true,
            "Interval": "00:00:10",
            "Timeout": "00:00:05",
            "Path": "/health"
          }
        },
        "Destinations": {
          "api-1": { "Address": "https://api-server-1:5001" },
          "api-2": { "Address": "https://api-server-2:5001" }
        }
      },
      "web-cluster": {
        "LoadBalancingPolicy": "LeastRequests",
        "Destinations": {
          "web-1": { "Address": "https://web-server-1:5000" }
        }
      }
    }
  }
}

Health Probes Configuration

Application Gateway Health Probe

bash
# Custom health probe cho Application Gateway
az network application-gateway probe create \
  --resource-group myRG \
  --gateway-name myAppGateway \
  --name api-health-probe \
  --protocol Https \
  --host-name-from-http-settings true \
  --path "/health/ready" \
  --interval 15 \
  --timeout 10 \
  --threshold 3 \
  --match-status-codes "200"

Health Probe Best Practices

csharp
// Custom health check kiểm tra sâu hơn
public class DeepHealthCheck : IHealthCheck
{
    private readonly IDbConnection _db;
    private readonly IConnectionMultiplexer _redis;

    public async Task<HealthCheckResult> CheckHealthAsync(
        HealthCheckContext context, CancellationToken ct = default)
    {
        var data = new Dictionary<string, object>();
        var isHealthy = true;

        // Check DB connectivity + basic query performance
        try
        {
            var sw = Stopwatch.StartNew();
            await _db.ExecuteScalarAsync("SELECT 1");
            data["db_latency_ms"] = sw.ElapsedMilliseconds;

            if (sw.ElapsedMilliseconds > 1000)
            {
                isHealthy = false;
                data["db_warning"] = "Latency > 1s";
            }
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Unhealthy("DB connection failed", ex, data);
        }

        // Check Redis
        try
        {
            var sw = Stopwatch.StartNew();
            await _redis.GetDatabase().PingAsync();
            data["redis_latency_ms"] = sw.ElapsedMilliseconds;
        }
        catch (Exception ex)
        {
            return HealthCheckResult.Degraded("Redis unavailable", ex, data);
        }

        return isHealthy
            ? HealthCheckResult.Healthy("All systems operational", data)
            : HealthCheckResult.Degraded("Performance degraded", data: data);
    }
}

Trade-off Analysis