I am trying to pass 70k messages through rule engine one by one. I want to bring down the processing time for this. The newKieSession() instantiation as per below implementation takes around 0.8 seconds in each loop which is too much. Although it evaluates messages accurately.
DroolsBeanFactory
public KieSession getKieSession() {
getKieRepository();
KieFileSystem kieFileSystem = kieServices.newKieFileSystem();
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule1.drl"));
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule2.drl"));
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule3.drl"));
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule4.drl"));
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule5.drl"));
kieFileSystem.write(ResourceFactory.newClassPathResource("com/example/demo/Rule6.drl"));
KieBuilder kb = kieServices.newKieBuilder(kieFileSystem);
kb.buildAll();
KieModule kieModule = kb.getKieModule();
KieContainer kieContainer = kieServices.newKieContainer(kieModule.getReleaseId());
return kContainer.newKieSession();
}
When I tried to replace newKieSession creation with KieBase's method the instantiation time is significantly lesser but it does not evaluate rules correctly... I think it stops evaluating messages correctly after 36-37 loops.
kmodule.xml
<?xml version="1.0" encoding="UTF-8"?>
<kmodule xmlns="http://www.drools.org/xsd/kmodule">
<kbase name="rules-kb" packages="com.example.demo">
</kbase>
Implementation file
#Autowired
KieBase kBase
for(String message: messages) {
//KieSession kieSession = new DroolsBeanFactory.getKieSession(); // takes long time, evaluates correctly
KieSession kieSession = kBase.newKieSession(); // quick instantiation, wrong evaluation
kieSession.insert(...);
kieSession.fireAllRules();
KieSession.dispose();
}
Why are they behaving differently ? How can we bring down instantiation time for kiesession and also evaluate messages correctly ?
Related
I am working with resilience4j and spring boot,
I need to accomplish the below scenario,
When I have a failure in the originalMethod
After 5 attempts route to the fallback method
After a specific time like 5 minutes return back to the originalMethod
I tried with retry as below but does not fit the problem ,
#Retry(name = "retryService", fallbackMethod = "fallback")
public String originalMethod(String data) throws InterruptedException {
//..... call external service
}
public String fallback(String data, Throwable t) {
logger.error("Inside retryfallback, cause – {}", t.toString());
return "Inside retryfallback method. Some error occurred ";
}
Added properties
resilience4j.retry:
instances:
retryService:
maxRetryAttempts: 5
waitDuration: 50000
I think you can use a circuit breaker for sometime when a failure limit reached to achieve the behavior you want.
By adding #CircuitBreaker(...) annotation and specifying the failureRateThreshold, waitDurationInOpenState and the other needed config properties for that instance.
I am using Spring integration sftp to put files on a remote server and below is configuration.
<spring-integration.version>5.2.5.RELEASE</spring-integration.version>
I have configurated #MessagingGateway.
#MessagingGateway
public interface SftpMessagingGateway {
#Gateway(requestChannel = "sftpOutputChannel")
void sendToFTP(Message<?> message);
}
I have configured the MessageHandler as below,
#Bean
#ServiceActivator(inputChannel = "sftpOutputChannel" )
public MessageHandler genericOutboundhandler() {
SftpMessageHandler handler = new SftpMessageHandler(outboundTemplate(), FileExistsMode.APPEND);
handler.setRemoteDirectoryExpressionString("headers['remote_directory']");
handler.setFileNameGenerator((Message<?> message) -> ((String) message.getHeaders().get(Constant.FILE_NAME_KEY)));
handler.setUseTemporaryFileName(false);
return handler;
}
I have configured SftpRemoteFileTemplate as below
private SftpRemoteFileTemplate outboundTemplate;
public SftpRemoteFileTemplate outboundTemplate(){
if (outboundTemplate == null) {
outboundTemplate = new SftpRemoteFileTemplate(sftpSessionFactory());
}
return outboundTemplate;
}
This is the configuration for SessionFactory
public SessionFactory<LsEntry> sftpSessionFactory() {
DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory();
factory.setHost(host);
factory.setPort(port);
factory.setUser(username);
factory.setPassword(password);
factory.setAllowUnknownKeys(true);
factory.setKnownHosts(host);
factory.setSessionConfig(configPropertiesOutbound());
CachingSessionFactory<LsEntry> csf = new CachingSessionFactory<LsEntry>(factory);
csf.setSessionWaitTimeout(1000);
csf.setPoolSize(10);
csf.setTestSession(true);
return csf;
}
I have configured all this in one of the service.
Now the problem is,
Sometimes the entire operation takes more than 15 min~ specially if the service is ideal for few hours and I am not sure what is causing this issue.
It looks like it is spending time on getting the active session from CachedSessionFactory the after operations are pretty fast below is the snap from one of the tool where I have managed to capture the time.
It usually takes few miliseconds to transfer files.
I have recently made below changes but before that as well I was getting the same issue,
I have set isShareSession to false earlier it was DefaultSftpSessionFactory factory = new DefaultSftpSessionFactory(true);
There was no pool size I have set it to 10
I think I have configured something incorrectly and that's why I end up piling connection ? Or there is something else ?
Observation :
The time taking to complete the operation is somewhat similar all the time when issue occurs i.e 938000 milliseconds +
If I restart the application daily it works perfectly fine.
I would like to use Resilience4j to deal with fault tolerance, I am using CircuitBreaker and TimerLimit.
I would like to separate business logic of fault tolerance behavior, to not "dirty" my business code.
So, I am thinking to use Command pattern to execute my methods that will be treat, like Hystrix do with HystrixCommand.
Example:
public class MyCommand {
private static final CircuitBreaker circuitBreaker;
private Long param1, param2;
private MyService myService;
private static final TimeLimiter timeLimiter;
static {
long ttl = 50000;
TimeLimiterConfig configTimerLimit
= TimeLimiterConfig.custom().timeoutDuration(Duration.ofMillis(ttl)).build();
timeLimiter = TimeLimiter.of(configTimerLimit);
// I got the configuration from a class that I created.
circuitBreaker = CircuitBreaker.of("my", CircuitBreakerConfigOptions.defaultForExternalService());
}
public MyCommand(Long param1, Long param2, MyService myService) {
this.param1 = param1;
this.param2 = param2;
this.myService = myService;
}
public String run() {
Callable<String> stringCallable = TimeLimiter.decorateFutureSupplier(timeLimiter,
() -> CompletableFuture.supplyAsync(() -> myService.hello(param1, param2)));
Callable<String> callable = CircuitBreaker.decorateCallable(circuitBreaker, stringCallable);
return Try.of(callable::call).recover(t -> fallback(t)).get();
}
protected String fallback(Throwable throwable) {
Callable<String> stringCallable = TimeLimiter.decorateFutureSupplier(timeLimiter,
() -> CompletableFuture.supplyAsync(() -> myService.otherHello(param1, param2)));
return Try.of(stringCallable::call).getOrElse("Fallback");
}
}
Calling in my Controller:
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
return new MyCommand(1L, 2L, new MyService()).run();
}
My doubts:
1 - In this case, circuitBreaker really needs to be static, because I understood that the same object needs to be shared between the same method that you want to threat, I am right?
2 - How I have many instances of this application, circuitBreaker works individually for each instance? I am right?
What I understand from your question is - You need a Resilience4j's circuit breaker which should be sperate i.e. not messed with your business logic.
So I would suggest put circuit breaker protection around run() method. below code will elaborate -
Your Controller -
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
return new MyCommand().run();
}
Now Write your run() method of MyCommand class with #CircuitBreaker
public class MyCommand {
#CircuitBreaker(name = "RUN_METHOD_PROTECTION") // <---- here is our circuit breaker annotation code top of below your business code... and that’s it.
Your_Response run(Your_Request){
// Your business logic written here...
}
Further add circuit breaker configuration in your YAML property file as below (Count based I have used instead of time-based) –
resilience4j.circuitbreaker:
backends:
RUN_METHOD_PROTECTION:
registerHealthIndicator: true
slidingWindowSize: 100 # start rate calc after 100 calls
minimumNumberOfCalls: 100 # minimum calls before the CircuitBreaker can calculate the error rate.
permittedNumberOfCallsInHalfOpenState: 10 # number of permitted calls when the CircuitBreaker is half open
waitDurationInOpenState: 10s # time that the CircuitBreaker should wait before transitioning from open to half-open
failureRateThreshold: 50 # failure rate threshold in percentage
slowCallRateThreshold: 100 # consider all transactions under interceptor for slow call rate
slowCallDurationThreshold: 2s # if a call is taking more than 2s then increase the error rate
recordExceptions: # increment error rate if following exception occurs
- org.springframework.web.client.HttpServerErrorException
- java.io.IOException
- org.springframework.web.client.ResourceAccessException
Now if you are unable to use annotation #CircuitBreaker in your project then you can also do things in functional way i.e.
Let say we you have defined a bean in your configuration,
#Bean
public CircuitBreaker MyCircuitBreaker(){
CircuitBreakerConfig config = CircuitBreakerConfig.custom()
.slidingWindow(100,100, CircuitBreakerConfig.SlidingWindowType.COUNT_BASED)
.failureRateThreshold(50)
.build();
CircuitBreakerRegistry registry = CircuitBreakerRegistry.of(config);
CircuitBreaker circuitBreaker = registry.circuitBreaker("your_run_method_CircuitBreker"); // here you are registering your circuit breaker with a unique tag. And in future you refer this tag you get a same circuit breaker.
return circuitBreaker;
}
Now Your controller code would be below -
private CircuitBreaker circuitBreaker; // Assume you have injected the value from CircuitBreaker bean
#ApiOperation(value = "Only to test")
#GetMapping(value = "/execute", produces = MediaType.APPLICATION_JSON)
public String execute() {
Function<Your_Request, Your_Response> decorated = CircuitBreaker
.decorateFunction(circuitBreaker, new MyCommand().run());
return decorated.apply();
}
This way too you have never interfered with your business logic.
Since it seems that you are using Spring Boot, you could use the resilience4j-spring-boot-2 starter module which also has support for annotations.
https://resilience4j.readme.io/docs/getting-started-3
I am trying to initialize spring context in my Spark application. I want the context in my slave nodes as well as I want to re-use the beans. Here is the code for the same:-
shipperRD2.foreach(shipper->{
AmazonS3 amazonS3Client = AmazonS3ClientBuilder.standard().build();
FileSystemXmlApplicationContext context2 = new FileSystemXmlApplicationContext("https://s3.console.aws.amazon.com/s3/object/spring-configuration/app-context.xml");
PersistenceWrapper persistenceWrapper = context.getBean(PersistenceWrapper.class);
});
However, this is leading to context refresh every time a new task runs on the slave node. Is there any way to avoid this behavior. basically, just initialize the context on the first task run, and re-use that context in the subsequent tasks.
As mentioned by Jacek, I tried the singleton pattern and it worked.
public class SpringInit {
private static FileSystemXmlApplicationContext context = new FileSystemXmlApplicationContext(fileName);
private SpringInit(){
}
public static FileSystemXmlApplicationContext getInstance(){
return context;
}
}
From the spark,
shipperRD2.foreach(shipper->{
FileSystemXmlApplicationContext context = SpringInit.getInstance();
PersistenceWrapper persistenceWrapper = context.getBean(PersistenceWrapper.class);
});
We are experiencing a problem in production where consumers are having zero utilization and the queues keep growing and performance degrades.
Each of the consumers is a container which contains a single instance of a non thread-safe listener bean.
Each listener needs to write to its own set of files. In order to avoid thread contention I would like only one thread to write to its own set of files.
Each listener is only instantiated once by using #Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
I'm using a configuration similar to the one in this question
Each container is also configured with a retry advice which has the following code:
public class RetryMessageAdvice extends StatelessRetryOperationsInterceptorFactoryBean {
private static final int DEFAULT_RETRY_COUNT = 5;
private static final int DEFAULT_BACKOFF_MS = 250;
private int retryCount;
private int backOffPeriodInMS;
public RetryMessageAdvice() {
this.retryCount = DEFAULT_RETRY_COUNT;
this.backOffPeriodInMS = DEFAULT_BACKOFF_MS;
initializeRetryPolicy();
}
public RetryMessageAdvice(int retryCount, int backoff) {
this.retryCount = retryCount;
this.backOffPeriodInMS = backoff;
initializeRetryPolicy();
}
public void initializeRetryPolicy() {
SimpleRetryPolicy retryPolicy = new SimpleRetryPolicy();
retryPolicy.setMaxAttempts(this.retryCount);
FixedBackOffPolicy backOffPolicy = new FixedBackOffPolicy();
backOffPolicy.setBackOffPeriod(backOffPeriodInMS);
RetryTemplate retryTemplate = new RetryTemplate();
retryTemplate.setRetryPolicy(retryPolicy);
retryTemplate.setBackOffPolicy(backOffPolicy);
this.setRetryOperations(retryTemplate);
this.setMessageRecoverer(new RetryMessageRecoverer());
}
public int getRetryCount() {
return retryCount;
}
public void setRetryCount(int retryCount) {
this.retryCount = retryCount;
}
}
The consumer looks something like this:
#Component("exportListenerImpl")
#Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public class ExportListenerImpl extends ExportListenerBase {
private static final Logger LOG = LoggerFactory.getLogger(ExportListenerImpl.class);
private final ExportMapper exportMapper;
private final ExportFormatter exportFormatter;
#Autowired
public ExportListenerImpl(#Qualifier("exportFormatter") ExportFormatter exportFormatter,
#Qualifier("exportMapper") ExportedMapper exportedMapper,
#Value("${export.root.dir}") String exportDirectory) {
super(exportDirectory);
this.exportedFormatter = exportFormatter;
this.exportedMapper = exportedMapper;
}
#Override
public void handle(AnalyticsEvent analyticsEvent) throws Exception {
ExportedEvent exportedEvent = exportMapper.mapPlace(analyticsEvent);
File csvFile = getCsvFile(exportedEvent);
String csvRow = exportFormatter.writeAsString(exportedEvent);
writeCsvRow(csvRow, csvFile);
}
}
Other things to note
Export mapper and export formatter are thread-safe but not using #Scope(value = ConfigurableBeanFactory.SCOPE_PROTOTYPE)
The method writeCsvRow is synchronized.
There is a high number of errors which cause the exportMapper to throw an exception and trigger the retry advice
The incoming message rage is 120/s
The ratio between the incoming and deliver rate is usually 5:1
My theories on what is wrong are
The high number of errors is causing a large number of retries and
degrading performance. I would be better off putting the bad message
in an error queue.
Somehow the synchronized method in writeCsvRow is
causing problems with some higher level thread managed by
spring-amqp.
My question is, which theory is right? Is the impact of the retry advice the problem?
If those beans are also not thread-safe, they must also be prototype scope.
Since there's only one thread, synchronizing that method is not necessary but it shouldn't hurt.
If the errors are irrecoverable, you should configure the retry policy to not retry those exceptions.
.
With those retry settings, you will suspend the container thread for 250ms each time you get an error. So, yes; it will hurt performance.
Shouldn't be a significant overhead.