KieBuilder too much time to load kieFileSystem and build - performance

Question

I have a Drools based application, which has 15-20 files overall. Each file has rules for a particular context. I load the file in session based on the context I want to invoke. Recently the number of executions increased a lot which was followed by latency degradation. I found that building KieBuilder is taking a long time, which grows proportionally to the number of requests on the application. Based on some research, it looks like all calls stop at the building step and continue when building is complete for all. I am not sure why this is happening since the calls are disconnected in the sense that all need different files to be loaded.
I found some references to KieBuilder taking too much time but couldn't find any solution.
Drools version being used: 7.4.1-Final
My code:
private StatelessKieSession loadSession(final String namespace, final String path) {
final KieServices kieServices = KieServices.Factory.get();
final KieResources kieResources = kieServices.getResources();
final KieFileSystem kieFileSystem = kieServices.newKieFileSystem();
final InputStream ruleFile = ruleStore.getRulesForRuleId(namespace, path);
kieFileSystem.write(kieResources.newInputStreamResource(ruleFile)
.setSourcePath(DRL_PATH)
.setResourceType(ResourceType.DRL));
log.info("Creating KieBuilder for {}/{}", namespace, path);
final KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem).buildAll();
if (kieBuilder.getResults().hasMessages(Message.Level.ERROR)) {
throw new InvalidRulesException("Failed to load rules from S3 in "
+ "Drools engine with error: " + kieBuilder.getResults().toString());
}
log.info("Created KieBuilder for {}/{}", namespace, path);
final KieContainer kContainer = kieServices.newKieContainer(
kieServices.getRepository().getDefaultReleaseId());
return kContainer.newStatelessKieSession();
}
Line taking too long:
final KieBuilder kieBuilder = kieServices.newKieBuilder(kieFileSystem).buildAll();
Logs showing application is just waiting for 20 secs for KieBuilder :
(http-bio-0.0.0.0-8080-exec-95) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/CafeteriaTimings.drl
12 Jan 2021 10:36:10,904 (http-bio-0.0.0.0-8080-exec-92) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MandatoryOffDays.drl
12 Jan 2021 10:36:10,911 (http-bio-0.0.0.0-8080-exec-82) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/ConsecutiveOffDaysPerWeek.drl
12 Jan 2021 10:36:10,912 (http-bio-0.0.0.0-8080-exec-93) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxWorkingHoursPerSiteWeek.drl
12 Jan 2021 10:36:10,916 (http-bio-0.0.0.0-8080-exec-53) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxTimeBetweenStartAndLunch.drl
12 Jan 2021 10:36:10,916 (http-bio-0.0.0.0-8080-exec-79) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxWorkingHoursPerDay.drl
12 Jan 2021 10:36:10,922 (http-bio-0.0.0.0-8080-exec-97) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/BreakFrequency.drl
12 Jan 2021 10:36:10,924 (http-bio-0.0.0.0-8080-exec-86) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxTimeBetweenStartAndBreak.drl
12 Jan 2021 10:36:11,004 (http-bio-0.0.0.0-8080-exec-90) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/CanLunchExistOutsideCafeteriaTime.drl
12 Jan 2021 10:36:11,004 (http-bio-0.0.0.0-8080-exec-87) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/LunchDuration.drl
12 Jan 2021 10:36:11,004 (http-bio-0.0.0.0-8080-exec-94) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxConsecutiveDays.drl
12 Jan 2021 10:36:11,005 (http-bio-0.0.0.0-8080-exec-83) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/LunchFrequency.drl
12 Jan 2021 10:36:11,010 (http-bio-0.0.0.0-8080-exec-85) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MinOffDaysPerWeek.drl
12 Jan 2021 10:36:11,015 (http-bio-0.0.0.0-8080-exec-88) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MaxWorkingHoursPerWeek.drl
12 Jan 2021 10:36:11,017 (http-bio-0.0.0.0-8080-exec-84) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MinWorkHoursBeforeLunch.drl
12 Jan 2021 10:36:11,018 (http-bio-0.0.0.0-8080-exec-96) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/BreakDuration.drl
12 Jan 2021 10:36:11,022 (http-bio-0.0.0.0-8080-exec-89) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/WorkIntervalsList.drl
12 Jan 2021 10:36:11,032 (http-bio-0.0.0.0-8080-exec-99) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:11,037 (http-bio-0.0.0.0-8080-exec-80) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MinTimeBetweenStartAndBreak.drl
12 Jan 2021 10:36:11,104 (http-bio-0.0.0.0-8080-exec-78) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/MinConsecutiveOffWorkHours.drl
12 Jan 2021 10:36:11,108 (http-bio-0.0.0.0-8080-exec-81) com.amazon.wforuleengine.drools.RulesSessionFactory: Creating KieBuilder for scheduling/amazon/cs/DayValidationType.drl
12 Jan 2021 10:36:11,203 (http-bio-0.0.0.0-8080-exec-100) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:11,322 (http-bio-0.0.0.0-8080-exec-2) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:14,324 (http-bio-0.0.0.0-8080-exec-1) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:15,224 (http-bio-0.0.0.0-8080-exec-3) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:16,207 (http-bio-0.0.0.0-8080-exec-4) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:16,404 (http-bio-0.0.0.0-8080-exec-6) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:16,503 (http-bio-0.0.0.0-8080-exec-5) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:17,003 (http-bio-0.0.0.0-8080-exec-7) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:17,203 (http-bio-0.0.0.0-8080-exec-8) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:18,513 (http-bio-0.0.0.0-8080-exec-9) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:19,122 (http-bio-0.0.0.0-8080-exec-10) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:19,803 (http-bio-0.0.0.0-8080-exec-11) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:19,903 (http-bio-0.0.0.0-8080-exec-12) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:20,114 (http-bio-0.0.0.0-8080-exec-13) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:20,213 (http-bio-0.0.0.0-8080-exec-14) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:20,407 (http-bio-0.0.0.0-8080-exec-15) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:20,703 (http-bio-0.0.0.0-8080-exec-16) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:21,303 (http-bio-0.0.0.0-8080-exec-17) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:21,503 (http-bio-0.0.0.0-8080-exec-18) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:21,603 (http-bio-0.0.0.0-8080-exec-19) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:21,603 (http-bio-0.0.0.0-8080-exec-20) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:23,711 (MetricAggregationClientScheduled-1) com.amazon.aws.cloudwatch.extension.client.impl.MetricAggregationCloudWatchClient: flush is invoked with sync false
12 Jan 2021 10:36:24,308 (MetricAggregationClientPublisher-1) com.amazon.aws.cloudwatch.extension.client.impl.MetricAggregationCloudWatchClient: Stats after submit are {}
12 Jan 2021 10:36:24,905 (http-bio-0.0.0.0-8080-exec-21) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:25,703 (http-bio-0.0.0.0-8080-exec-22) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:26,518 (http-bio-0.0.0.0-8080-exec-23) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:26,603 (http-bio-0.0.0.0-8080-exec-24) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:26,903 (http-bio-0.0.0.0-8080-exec-25) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:27,218 (http-bio-0.0.0.0-8080-exec-26) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:27,403 (http-bio-0.0.0.0-8080-exec-27) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:28,803 (http-bio-0.0.0.0-8080-exec-28) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:29,503 (http-bio-0.0.0.0-8080-exec-29) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:29,903 (http-bio-0.0.0.0-8080-exec-30) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:30,129 (http-bio-0.0.0.0-8080-exec-31) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:30,803 (http-bio-0.0.0.0-8080-exec-32) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:31,819 (http-bio-0.0.0.0-8080-exec-34) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:31,902 (http-bio-0.0.0.0-8080-exec-33) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:32,103 (http-bio-0.0.0.0-8080-exec-35) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:32,203 (http-bio-0.0.0.0-8080-exec-36) org.jboss.resteasy.core.SynchronousDispatcher: PathInfo: /ping
12 Jan 2021 10:36:32,219 (http-bio-0.0.0.0-8080-exec-90) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl54:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:32,322 (http-bio-0.0.0.0-8080-exec-90) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/CanLunchExistOutsideCafeteriaTime.drl
12 Jan 2021 10:36:32,613 (http-bio-0.0.0.0-8080-exec-82) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl80:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:32,613 (http-bio-0.0.0.0-8080-exec-82) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/ConsecutiveOffDaysPerWeek.drl
12 Jan 2021 10:36:33,913 (http-bio-0.0.0.0-8080-exec-79) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl76:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:33,913 (http-bio-0.0.0.0-8080-exec-79) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/MaxWorkingHoursPerDay.drl
12 Jan 2021 10:36:34,205 (http-bio-0.0.0.0-8080-exec-92) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl0:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:34,205 (http-bio-0.0.0.0-8080-exec-92) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/MandatoryOffDays.drl
12 Jan 2021 10:36:34,210 (http-bio-0.0.0.0-8080-exec-95) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl92:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:34,210 (http-bio-0.0.0.0-8080-exec-95) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/CafeteriaTimings.drl
12 Jan 2021 10:36:34,608 (http-bio-0.0.0.0-8080-exec-81) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl31:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:34,609 (http-bio-0.0.0.0-8080-exec-81) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/DayValidationType.drl
12 Jan 2021 10:36:34,615 (http-bio-0.0.0.0-8080-exec-85) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl8:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:34,615 (http-bio-0.0.0.0-8080-exec-85) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/MinOffDaysPerWeek.drl
12 Jan 2021 10:36:34,716 (http-bio-0.0.0.0-8080-exec-87) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl79:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:34,716 (http-bio-0.0.0.0-8080-exec-87) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/LunchDuration.drl
12 Jan 2021 10:36:35,218 (http-bio-0.0.0.0-8080-exec-94) org.drools.compiler.kie.builder.impl.KieRepositoryImpl: KieModule was added: MemoryKieModule[releaseId=org.default:drl94:1.0.0-SNAPSHOT]
12 Jan 2021 10:36:35,218 (http-bio-0.0.0.0-8080-exec-94) com.amazon.wforuleengine.drools.RulesSessionFactory: Created KieBuilder for scheduling/amazon/cs/MaxConsecutiveDays.drl
I am looking to understand why the time taken for KieBuilder is proportional to the number of calls, and for a way to optimize on the time taken.
Thanks!