gateway integrated sentinel flow control downgrade

What is flow control downgrade?

Flow control degradation is a strategy to protect system availability in high concurrency scenarios. It controls and manages system traffic to prevent system resource exhaustion and collapse. When the system faces excessive pressure or abnormal conditions, flow control degradation can temporarily shut down or switch non-critical functions to ensure the normal operation of core services. Several key concepts of flow control degradation are introduced in detail below.

Flow control: Flow control refers to ensuring that the system operates within the tolerance range by limiting the system's request processing capabilities. Prevent the system from being overwhelmed by a large number of requests by setting an upper limit on QPS (requests per second) or the number of concurrent requests. Flow control prevents the system from being overloaded, resulting in extended response times or system crashes.

Degrade processing: When the system encounters abnormalities, insufficient resources, or external dependency failures, in order to protect the availability of core functions, some non-critical or degradable functions can be temporarily turned off or switched. For downgrading, a backup plan can be defined in advance. When the system load is too high or an abnormality occurs, the backup plan can be quickly switched to ensure the stable operation of core functions.

Trigger conditions: The trigger conditions for flow control degradation can be configured based on business needs and system load conditions. Common trigger conditions include request frequency exceeding the threshold, request failure rate exceeding the threshold, system load being too high, etc. Depending on the actual situation, different trigger conditions can be set to implement a refined flow control degradation strategy.

Downgrade policy: The downgrade policy specifies the actions to be performed when flow control downgrade conditions are triggered. Common downgrade strategies include returning default data, returning to static pages, calling alternative interfaces, delaying responses, etc. Choosing an appropriate downgrade strategy can maximize system availability and provide a better user experience.

Dynamic adjustment: The settings of flow control downgrade should have the ability to be dynamically adjusted so that flow control rules and downgrade strategies can be adjusted in a timely manner according to the actual situation. By monitoring the real-time indicators of the system (such as QPS, response time, error rate, etc.), the flow control degradation strategy can be dynamically adjusted according to actual load conditions to improve the flexibility and adaptability of the system.

Why flow control needs to be downgraded?

Protect core services: Flow control degradation can protect the core services of the system from abnormalities or overload. When the system faces excessive pressure, the degradation strategy can ensure the normal operation of core services by shutting down or switching non-critical functions to prevent the system from crashing or becoming unresponsive.

Improve system availability: Through reasonable traffic control and degradation processing, the system can better cope with burst traffic and high concurrent requests, avoiding resource exhaustion and performance degradation. This improves system availability and ensures users can access and use the system properly.

Avoid the avalanche effect: In a distributed system, when one node/service fails or is delayed, other nodes/services may experience cascading failures, forming an avalanche effect. Flow control degradation can limit access to failed nodes and avoid the collapse of the entire system due to the failure of a single node.

Save resource overhead: In high concurrency situations, system resources may be occupied by too many requests, causing other requests to not be processed in time. Through flow control downgrading, the system load can be limited, system resources can be reasonably allocated and utilized, and resource waste and unnecessary overhead can be avoided.

Improve user experience: Flow control downgrade can ensure normal response and fast processing of core functions, improving user access efficiency and experience. Even during peak periods or abnormal situations, users can still obtain stable and acceptable system response, reducing user experience degradation caused by network delays or timeouts.

Prevent malicious attacks: Some malicious attacks may consume system resources through a large number of requests, resulting in service unavailability. By setting traffic control rules, you can limit the frequency of requests from a single IP or user to prevent malicious attacks from affecting the system.

Benefits of flow control downgrading

How gateway integrates sentinel

Install and configure Sentinel: First, download and install Sentinel according to Sentinel's official documentation. Then, configure as needed, including defining rules, setting flow control policies, etc.

Add Sentinel dependencies: In your gateway project, add Sentinel's related dependency libraries. This is usually done by adding the appropriate dependencies in the project's build file (such as pom.xml).

Create a gateway filter: In order to apply Sentinel to a gateway, you need to create a filter that intercepts incoming requests and hands them over to Sentinel for flow control and degradation. This filter can be configured in the gateway's request processing chain.

Configure rules and resources: Define the API or service that requires flow control as a resource in Sentinel, and configure the corresponding flow control rules for it. You can use the annotations or programming methods provided by Sentinel to implement this step according to your needs.

Exception handling and downgrade: When a request is marked as failed by Sentinel, the gateway can perform corresponding exception handling and downgrade operations based on your business needs. This may include returning a custom error message, forwarding to an alternate service, or returning a canned default response.

Monitoring and management: Sentinel provides many monitoring and management functions. You can choose to use Sentinel Dashboard for real-time monitoring, rule configuration, traffic statistics, etc.

Please note that during the implementation process, you need to configure and integrate accordingly according to the specific gateway framework and technology selection. For example, in Spring Cloud Gateway, you can use the Sentinel component provided by Spring Cloud Alibaba to simplify the integration process.

code example

The following is a sample code showing how to integrate Spring Cloud Gateway with Sentinel:

First, make sure you have added the necessary dependencies. Add the following dependencies in the pom.xml file:

<!-- Spring Cloud Gateway -->
<dependency>
    <groupId>org.springframework.cloud</groupId>
    <artifactId>spring-cloud-starter-gateway</artifactId>
</dependency>

<!-- Spring Cloud Alibaba Sentinel -->
<dependency>
    <groupId>com.alibaba.cloud</groupId>
    <artifactId>spring-cloud-alibaba-sentinel-gateway</artifactId>
</dependency>

Next, create a configuration class to configure and integrate the gateway

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.gateway.filter.GlobalFilter;
import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

@Configuration
public class GatewayConfig {
    
    

    @Autowired
    private CustomBlockHandler customBlockHandler; // 自定义的流控异常处理器

    @Value("${backend.service.url}")
    private String backendServiceUrl;

    @Bean
    public RouteLocator customRouteLocator(RouteLocatorBuilder builder) {
    
    
        return builder.routes()
                .route(r -> r.path("/api")
                        .filters(f -> f.stripPrefix(1)
                                .filter(new SentinelGatewayFilter()))
                        .uri(backendServiceUrl))
                .build();
    }

    @Bean
    public GlobalFilter sentinelGlobalFilter() {
    
    
        return new SentinelGatewayFilter();
    }

    @Bean
    public CustomBlockHandler customBlockHandler() {
    
    
        return new CustomBlockHandler(); // 自定义的流控异常处理器
    }
}

In the above example, we configured a route with the processing path /api and applied SentinelGatewayFilter for flow control.

Then, create a custom flow control exception handler:

import java.util.HashMap;
import java.util.Map;

import com.alibaba.csp.sentinel.slots.block.BlockException;
import org.springframework.http.HttpStatus;
import org.springframework.http.MediaType;
import org.springframework.http.server.reactive.ServerHttpResponse;
import org.springframework.web.server.ServerWebExchange;
import org.springframework.web.server.WebExceptionHandler;

public class CustomBlockHandler implements WebExceptionHandler {
    
    

    @Override
    public Mono<Void> handle(ServerWebExchange exchange, Throwable ex) {
    
    
        if (exchange.getResponse().isCommitted()) {
    
    
            return Mono.error(ex);
        }
        ServerHttpResponse response = exchange.getResponse();
        response.getHeaders().setContentType(MediaType.APPLICATION_JSON);
        response.setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
        Map<String, Object> result = new HashMap<>();
        result.put("code", HttpStatus.TOO_MANY_REQUESTS.value());
        result.put("message", "Too many requests");
        return response.writeWith(Mono.just(response.bufferFactory().wrap(result.toString().getBytes())));
    }
}

When a request is flow controlled by Sentinel, a JSON response containing error information will be returned.

Finally, make sure that the backend.service.url value is set in the configuration file to point to your actual backend service address.

Summarize

By integrating Spring Cloud Gateway with Sentinel, traffic control and degradation processing of APIs or services can be achieved. This can improve the stability and reliability of the system and provide real-time monitoring and management capabilities.

During the implementation process, the following steps need to be completed:

Add necessary dependencies: In the project's build file, add the related dependencies of Spring Cloud Gateway and Spring Cloud Alibaba Sentinel.

Create a gateway configuration class: Create a configuration class, define custom routes and filters in it, and configure Sentinel's flow control policy.

Configure routing rules: Use RouteLocatorBuilder to define routing rules for APIs or services that require flow control.

Use SentinelGatewayFilter: In the routing configuration, apply SentinelGatewayFilter to the specified route for flow control and degradation processing.

Custom flow control exception handler: Create a custom flow control exception handler to process flow-controlled requests according to business requirements, such as returning custom error messages.

Monitoring and management: Using the functions provided by Sentinel, such as Sentinel Dashboard, you can realize real-time monitoring, dynamic configuration and traffic statistics of traffic control rules.

Through the above steps, you can integrate Sentinel into Spring Cloud Gateway to implement flow control and downgrade processing of requests flowing through the gateway, thereby ensuring the stability and reliability of the system.

After integration, you can define flow control rules in Sentinel based on business needs, such as restrictions based on QPS, number of threads, etc., and return customized error messages or forward to backup services as needed. At the same time, you can also monitor and manage rules and traffic in real time through Sentinel Dashboard to adjust and optimize system performance.