resilience4j circuit breaker test (with spring boot)

Resilience4j 란?

hystrix로 부터 영감을 받은 경량 fault tolerance library.

다른 외부 라이브러리 종속성이 없기 때문에 가볍게 사용하기 좋다.

 

Circuit Breaker / Rate Limiter / Bulk Head / Retry / Cache / Time Limiter 구현체가 존재.

 

hystrix는 더이상 개발되지 않고 maintenance만 하겠다고 해서 많이 사용하는 추세이다.

 

resilience4j.readme.io/

resilience4j

build.gradle

implementation 'io.github.resilience4j:resilience4j-spring-cloud2:1.6.1'
implementation 'org.springframework.cloud:spring-cloud-starter-circuitbreaker-reactor-resilience4j'
implementation 'io.github.resilience4j:resilience4j-circuitbreaker:1.6.1'
implementation 'org.springframework.cloud:spring-cloud-starter-gateway'
compile('org.springframework.boot:spring-boot-starter-webflux')
testImplementation 'org.springframework.boot:spring-boot-starter-test'
implementation 'org.springframework.boot:spring-boot-starter-actuator'

 

resilience4j 관련 라이브러리들을 추가한다.

 

application.properties

# actuator를 통해 circuitbraker 상태를 확인하기 위해 설정
resilience4j.circuitbreaker.configs.default.register-health-indicator=true
# Circuit Breaker가 에러 비율 또 slow call 비율을 계산하기 전에 요구되는 최소한의 요청 수
resilience4j.circuitbreaker.configs.default.minimum-number-of-calls=5
# 에러 비율 (퍼센트)로 해당 값 이상으로 에러 발생시 서킷이 Open 된다.
resilience4j.circuitbreaker.configs.default.failure-rate-threshold=10
# 서킷의 상태가 Open에서 Half-open으로 변경되기 전에 Circuit Breaker가 기다리는 시간
resilience4j.circuitbreaker.configs.default.wait-duration-in-open-state=10s
resilience4j.circuitbreaker.configs.default.slidingWindowType=TIME_BASED
resilience4j.circuitbreaker.configs.default.sliding-window-size=5
resilience4j.circuitbreaker.configs.default.permitted-number-of-calls-in-half-open-state=3
resilience4j.circuitbreaker.configs.default.automaticTransitionFromOpenHalfOpenEnabled=true
resilience4j.circuitbreaker.configs.default.slowCallDurationThreshold=3000ms
resilience4j.circuitbreaker.configs.default.slowCallRateThreshold=1
resilience4j.circuitbreaker.configs.default.recordExceptions[0]=java.util.concurrent.TimeoutException

resilience4j 설정에서 TimeoutException을 recordExceptions로 등록하면 Timeout이 걸려도 서킷이 동작한다.

timeout으로 테스트할 예정

 

bootstrap.properties

spring.cloud.gateway.routes[0].id=scgapi
spring.cloud.gateway.routes[0].uri=http://localhost:8081
spring.cloud.gateway.routes[0].predicates[0]=Path=/scgapi/**
spring.cloud.gateway.routes[0].filters[0].name=CircuitBreaker
spring.cloud.gateway.routes[0].filters[0].args.name=myCircuitBreaker
spring.cloud.gateway.routes[0].filters[0].args.fallbackUri=forward:/fallback

resilience4j.circuitbreaker.instances.myCircuitBreaker.base-config=default

test api를 만들었다.

timeout이 걸리도록

 

Test ScgApi 

 @GetMapping("/scgapi/getmethod")
    public String getMethod(@RequestParam String key) throws InterruptedException {
        if(key.equals("1")){
            return "success";
        } else{
            int time = Integer.valueOf(key);
            Thread.sleep(time * 1000);
        }
        return "get";
    }

api에서 파라미터로 넘어오는 값만큼 sleep을 줬다.

 

 

CircuitBreakerRegistry 등록

@Bean
public ReactiveResilience4JCircuitBreakerFactory defaultCustomizer(CircuitBreakerRegistry registry) {
Seq<CircuitBreaker> circuits = registry.getAllCircuitBreakers();
if(null != circuits) {
circuits.forEach(c -> c.getEventPublisher().onStateTransition(new ResilienceStateTransitionEventHandler()));
}

ReactiveResilience4JCircuitBreakerFactory factory = new ReactiveResilience4JCircuitBreakerFactory();
factory.configureCircuitBreakerRegistry(registry);
return factory;
}

 

Resilience4j 이벤트 handler 

public class ResilienceStateTransitionEventHandler implements EventConsumer<CircuitBreakerOnStateTransitionEvent> {

    @Override
        public void consumeEvent(CircuitBreakerOnStateTransitionEvent event) {
            CircuitBreaker.StateTransition stateTransition = event.getStateTransition();
            System.out.println("################consumeEvent called################");
            System.out.println(event.toString());
            if(CircuitBreaker.StateTransition.CLOSED_TO_OPEN == stateTransition){
                System.out.println("CLOSED_TO_OPEN");
            }else if(CircuitBreaker.StateTransition.OPEN_TO_HALF_OPEN == stateTransition){
                System.out.println("OPEN_TO_HALF_OPEN");
            }else if(CircuitBreaker.StateTransition.OPEN_TO_CLOSED == stateTransition){
                System.out.println("OPEN_TO_CLOSED");
            }
    }
}

서킷이 오픈되고 닫힐 때 이벤트를 캐치할 수 있다.

 

 

FallbackController

 @GetMapping("/fallback")
 public String fallback(ServerWebExchange ex) {
   Throwable t = ex.getAttribute(ServerWebExchangeUtils.CIRCUITBREAKER_EXECUTION_EXCEPTION_ATTR);
   System.out.println(t.toString());
   System.out.println("Fallback!!!!:" + t.toString());
   return "Recovered with fallback: ";
 }

API가 실패할 경우 호출되는 fallback 이다.

 

Jmeter로 호출테스트

java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 1000ms in 'circuitBreaker' (and no fallback has been configured)
Fallback!!!!:java.util.concurrent.TimeoutException: Did not observe any item or terminal signal within 1000ms in 'circuitBreaker' (and no fallback has been configured)
2021-04-09 18:11:56.646 DEBUG 17016 --- [     parallel-6] o.s.w.r.r.m.a.ResponseBodyResultHandler  : Using 'text/plain;charset=UTF-8' given [*/*] and supported [text/plain;charset=UTF-8, text/event-stream, text/plain;charset=UTF-8, */*]
2021-04-09 18:11:56.647 DEBUG 17016 --- [     parallel-6] o.s.w.r.r.m.a.ResponseBodyResultHandler  : [c42b8c8f-257] 0..1 [java.lang.String]
2021-04-09 18:11:56.650 DEBUG 17016 --- [     parallel-6] o.s.core.codec.CharSequenceEncoder       : [c42b8c8f-257] Writing "Recovered with fallback: "
2021-04-09 18:11:56.661 DEBUG 17016 --- [ctor-http-nio-7] o.s.w.s.adapter.HttpWebHandlerAdapter    : [c42b8c8f-257] Completed 200 OK
2021-04-09 18:11:56.664 DEBUG 17016 --- [ctor-http-nio-7] o.s.w.s.adapter.HttpWebHandlerAdapter    : [c42b8c8f-258] HTTP GET "/scgapi/getmethod?key=6"
################consumeEvent called################
2021-04-09T18:11:57.743+09:00[Asia/Tokyo]: CircuitBreaker 'myCircuitBreaker' changed state from CLOSED to OPEN
CLOSED_TO_OPEN

api 에러가 발생하면 Fallback을 호출하고 

서킷이 오픈되면 이벤트를 잡을 수 있다.

 

서킷 state 체크는 /actuator/health를 통해 가능하다.

 

{
  "status":"UP",
    "components":{
      "circuitBreakers":{
      "status":"UNKNOWN",
      "details":{
        "myCircuitBreaker":{
        "status":"CIRCUIT_OPEN",
        "details":{
          "failureRate":"-1.0%",
          "failureRateThreshold":"10.0%",
          "slowCallRate":"-1.0%",
          "slowCallRateThreshold":"1.0%",
          "bufferedCalls":0,
          "slowCalls":0,
          "slowFailedCalls":0,
          "failedCalls":0,
          "notPermittedCalls":32,
          "state":"OPEN"
      }
      }
    }
  },
  "discoveryComposite":{
    "description":"Discovery Client not initialized",
    "status":"UNKNOWN",
    "components":{
      "discoveryClient":{
        "description":"Discovery Client not initialized",
        "status":"UNKNOWN"
  }
  }
  },
  "diskSpace":{
  "status":"UP",
  "details":{
  "total":245107195904,
  "free":159210278912,
  "threshold":10485760,
  "exists":true
  }
  },
  "ping":{
  "status":"UP"
  },
  "reactiveDiscoveryClients":{
    "description":"Discovery Client not initialized",
    "status":"UNKNOWN",
    "components":{
      "Simple Reactive Discovery Client":{
        "description":"Discovery Client not initialized",
        "status":"UNKNOWN"
      }
    }
  },
  "refreshScope":{
    "status":"UP"
    }
  }
}