How to realize separate flow restriction in multi tenant system

Current limiting is a function that is often integrated in the development of back-end services. It is particularly useful to prevent the system from crashing due to excessive pressure. In a multi tenant system, flow restriction is also particularly useful for limiting the amount of resources used by a single tenant. This article will explore it.

problem

For a multi tenant system, if some tenants use too many resources, it is likely to affect other tenants. For example, for the query of a resource, the maximum capacity of the system is 300qps. Assuming that the query water level of tenants is 10qps under normal circumstances, 30 tenants can be served at the same time; Suddenly, the query water level of a tenant rises to 100qps. If the system has no flow restriction, the system will slow down or even crash. If the system has limited flow, all tenants may be affected by the flow restriction.

The sudden surge in the query level of tenants may be normal business or malicious attack, but either case is unfair to other tenants. At this time, you can limit the flow of different tenants. Let's see how to solve it.

principle

The following figure illustrates the principle of current limiting for multi tenant system:

 

Tenants A, B and C respectively initiate service requests. In the service, first authenticate the user's identity. If the authentication passes, enter the flow limit check phase, and check the flow limit for each tenant. If the flow limit threshold is not reached, you can go to the next step. If the flow limit threshold is reached, terminate the processing and return an error to the tenant.

realization

If you have understood the principle, I believe you are fully capable of writing the corresponding current limiting control logic. However, in order to improve R & D efficiency, we should reuse existing components as much as possible   FireflySoft.RateLimit   To realize this logic, the demonstration code is based on ASP.NET Core. You can also use it in any other. NET network service framework, or even your own handwritten special handler.

Install Nuget package

Using the package manager console:

Install-Package FireflySoft.RateLimit.AspNetCore

 

Or use the. NET CLI:

dotnet add package FireflySoft.RateLimit.AspNetCore

 

Or add it directly to the project file:

<ItemGroup>
<PackageReference Include="FireflySoft.RateLimit.AspNetCore" Version="2.*" />
</ItemGroup>

 

Using middleware

Register the current limiting service in Startup.cs and use the current limiting middleware.

public void ConfigureServices(IServiceCollection services)
{
    ...

    services.AddRateLimit(new InProcessFixedWindowAlgorithm(
        new[] {
            new FixedWindowRule()
            {
                Id = "1",
                ExtractTarget = context =>
                {
                    // It is assumed that the target of current restriction is the user of the tenant Id,It's from HTTP Header Passed from
                    return (context as HttpContext).Request.GetTypedHeaders().Get<string>("userId");
                },
                CheckRuleMatching = context =>
                {
                    // Suppose only for /Weather/GetToday This interface requests current limiting
                    var path = (context as HttpContext).Request.Path.Value;
                    if(path == "/Weather/GetToday"){
                        return true;
                    }
                    return false;
                },
                Name="Multi tenant current limit",
                LimitNumber=10, // Current limiting threshold
                StatWindow=TimeSpan.FromSeconds(1), //Current limiting time window, here is 1 second
                StartTimeType=StartTimeType.FromNaturalPeriodBeign
            }
        })
    );

    ...
}

public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
    ...

    app.UseRateLimit();

    ...
}

 

After these two steps are completed, the multi tenant current limiting function can operate normally.

The same flow limit threshold is used for all tenants here. If the tenant has the difference between privileged tenant and ordinary tenant, you can define different rules for privileged tenant and ordinary tenant in the flow limit rules above. For details, please refer to this article: How to distinguish and restrict different types of users in ASP.NET Core.

Other technical problems

Note that the current limiting algorithm used here is in-process fixed window current limiting. If you need to count and limit current uniformly in a distributed environment, you can use it   FireflySoft.RateLimit   The built-in Redis fixed window current limiting algorithm, but you need to have a Redis service first.

The fixed window algorithm is relatively rigid. In practice, the requests are likely to be uneven, and more will be less. It is difficult for the rigid algorithm to set a reasonable current limiting threshold. If you want to allow certain burst traffic, you can use sliding window, leaky bucket, token bucket and other algorithms, FireflySoft.RateLimit   These algorithms have been integrated in and can be used directly.

If the service capability of the system is enhanced and a tenant is allowed to initiate 20 requests per second, the rules need to be adjusted. The underlying layer of FireflySoft.RateLimit supports dynamic adjustment of rules. For details, see this article: . NET6 runtime dynamically updates current limit .

 

Well, that's the main content of this article. If you are interested, please visit the open source repository: https://github.com/bosima/FireflySoft.RateLimit

Tags: Middleware

Posted on Mon, 29 Nov 2021 20:07:16 -0500 by cspgsl