I am using existing applicationContext.xml that is being used for our existing web application to create a new console based java process.
This process or accounting server is to process tasks of different companies (lets say 2 different companies at a time). For this ThreadPoolTaskExecutor gives a guarantee that no more than the defined limit threads will be executed.
In below snippet when I try to get the "taskExecutor" from application context it is throwing exception that no such bean exists
is it the correct way to register a component ? as am not getting the thread pool bean from application context, is it properly registered with application context ?
public static void main(String[] args) {
LOGGER.info("Starting Accounting Server ...");
LOGGER.info("initializing applicationContext ... from " + APPLICATION_CONTEXT_XML_FULL_PATH);
try (AbstractApplicationContext applicationContext = new FileSystemXmlApplicationContext("file:" + APPLICATION_CONTEXT_XML_FULL_PATH)) {
LOGGER.info("... applicationContext initialized");
ThreadPoolTaskExecutor threadPool = new ThreadPoolTaskExecutor();
threadPool.setCorePoolSize(MAX_RUNNING_THREADS);
threadPool.setMaxPoolSize(MAX_RUNNING_THREADS);
threadPool.setThreadNamePrefix("T--ACCOUNTING-THREAD--");
threadPool.initialize();
Object obj = applicationContext.getBeanFactory().initializeBean(threadPool, "taskExecutor");
// obj is returned
obj = applicationContext.getBean("taskExecutor"); // <--------- throws exception with no such bean
AccountingQueueTransacitonService accountingQueueTransacitonService = applicationContext.getBean(AccountingQueueTransacitonService.class);
/*
while (hastasks) {
// TODO:
// get tasks
// add to thread and push to thread pool
// get some sleep
// update hastasks flag
}
*/
} catch (Exception e) {
LOGGER.error(e, e);
} finally {
LOGGER.info("closed application context ...");
}
LOGGER.info("Ending Accounting Server gracefully ...");
}
Related
In a Spring Boot application, I'm writing files to S3 using list.stream().parallel().forEach. When trying to get the resource using resourceLoader.getResource(filePath), it throws the exception 'com.amazonaws.services.s3.AmazonS3 referenced from a method is not visible from class loader'. I've noticed that the main thread successfully writes the files, but the child thread throws an exception.
getAccountCommonList()
.stream().parallel()
// .stream()
.forEach(fileNumber -> {
String filePath =
"s3://xxxxx.net/Debug_"
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyyMMdd_HHmm"))
+ "_"
+ fileNumber;
WritableResource resource = (WritableResource) resourceLoader.getResource(filePath);
try (OutputStream outputStream = resource.getOutputStream()) {
write(outputStream);
} catch (IOException e) {
System.console().printf(e.getMessage(), e);
throw new RuntimeException(e);
}
});
If I don't use parallel(), the code executes successfully.
I'd like to know how to proceed if I need to use parallel().
Could it be because the child thread cannot access the applicationContext?
Is it possible that the applicationContext is bound to the main thread through ThreadLocal?
I have a spring boot application which I am trying to deploy on AWS lambda .
I added StreamLambdaHandler as the handler class
public class StreamLambdaHandler implements RequestStreamHandler {
private static SpringBootLambdaContainerHandler<AwsProxyRequest, AwsProxyResponse> handler;
static {
try {
//handler = SpringBootLambdaContainerHandler.getAwsProxyHandler(SituationalFlexibilityApp.class);
// For applications that take longer than 10 seconds to start, use the async builder:
handler = new SpringBootProxyHandlerBuilder<AwsProxyRequest>()
.defaultProxy()
.asyncInit()
.springBootApplication(SituationalFlexibilityApp.class)
.buildAndInitialize();
// we use the onStartup method of the handler to register our custom filter
handler.onStartup(servletContext -> {
FilterRegistration.Dynamic registration = servletContext.addFilter("CognitoIdentityFilter",CognitoIdentityFilter.class);
registration.addMappingForUrlPatterns(EnumSet.of(DispatcherType.REQUEST), true, "/*");
});
} catch (ContainerInitializationException e) {
// if we fail here. We re-throw the exception to force another cold start
e.printStackTrace();
throw new RuntimeException("Could not initialize Spring Boot application", e);
}
}
public StreamLambdaHandler() {
Timer.enable();
}
/*
* public StreamLambdaHandler() throws ContainerInitializationException {
*
* handler = new SpringBootProxyHandlerBuilder() .defaultProxy() .asyncInit()
* .springBootApplication(SlowApplication.class) .buildAndInitialize(); }
*/
#Override
public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException {
handler.proxyStream(input, output, context);
}
When I test it on AWS lambda I get below exception
com.amazonaws.serverless.exceptions.ContainerInitializationException: Could not initialize framework within the 20000ms timeout
so I updated the lambda configuration for a timeout of 5 mins and added below line in the static block of the StreamLambdaHandler class
LambdaContainerHandler.getContainerConfig().setInitializationTimeout(2000000);
Now, I am seeing below exception
Exception in thread "Thread-0" java.lang.IllegalArgumentException: Could not find timer SPRINGBOOT2_COLD_START
Can someone please point me in the correct direction as I a noob in AWS services and lambda
I am not seeing this error after commenting out the below code in StreamLambdaHandler method
// Timer.enable();
I'm using spring and spring-kafka.
When the application starts, I'm loading compacted topics into memory then when the topics are fully read, the application is started.
In order to do that I created #KafkaListeners with autostartup to false and a SmartLifeCycle bean with AbstractMessageListenerContainer.DEFAULT_PHASE - 1 phase which is doing listener.start() on all those listeners (which read compacted topics) then wait for them to be finished.
This is working great but if I set spring.kafka.listener.missing-topics-fatal = true with a missing topic, there is an error :
org.springframework.context.ApplicationContextException: Failed to start bean 'org.springframework.kafka.config.internalKafkaListenerEndpointRegistry'; nested exception is java.lang.IllegalStateException: Topic(s) [mytopic] is/are not present and missingTopicsFatal is true
It's the expected behavior, but the application is not shutdown, my manually started listeners keeps running and sending errors :
java.lang.IllegalStateException: org.springframework.context.annotation.AnnotationConfigApplicationContext#502e1410 has not been refreshed yet
How can I make the application exit in this case ?
Catch the exception and shut down the JVM:
#SpringBootApplication
public class So60036945Application {
public static void main(String[] args) {
try {
SpringApplication.run(So60036945Application.class, args);
}
catch (ApplicationContextException ace) {
if (ace.getCause() instanceof IllegalStateException) {
ace.printStackTrace();
System.exit(1);
}
}
}
#KafkaListener(id = "so60036945", topics = "so60036945a")
public void listen(String in) {
System.out.println(in);
}
}
But, as I said in Gitter, it's better to auto-start the compacted topic listeners and start the other listeners manually (the other way around to what you are doing now).
I have created spring boot thread pool project which has thread that needs to run 24x7 once spawned but when I need to stop the app in server for some maintenance it should shutdown after completing its current task and not taking up any new task.
My code for the same is:
Config class
#Configuration
public class ThreadConfig {
#Bean
public ThreadPoolTaskExecutor taskExecutor(){
ThreadPoolTaskExecutor executorPool = new ThreadPoolTaskExecutor();
executorPool.setCorePoolSize(10);
executorPool.setMaxPoolSize(20);
executorPool.setQueueCapacity(10);
executorPool.setWaitForTasksToCompleteOnShutdown(true);
executorPool.setAwaitTerminationSeconds(60);
executorPool.initialize();
return executorPool;
}
}
Runnable class
#Component
#Scope("prototype")
public class DataMigration implements Runnable {
String name;
private boolean run=true;
public DataMigration(String name) {
this.name = name;
}
#Override
public void run() {
while(run){
System.out.println(Thread.currentThread().getName()+" Start Thread = "+name);
processCommand();
System.out.println(Thread.currentThread().getName()+" End Thread = "+name);
if(Thread.currentThread().isInterrupted()){
System.out.println("Thread Is Interrupted");
break;
}
}
}
private void processCommand() {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
public void shutdown(){
this.run = false;
}
}
Main class:
#SpringBootApplication
public class DataMigrationPocApplication implements CommandLineRunner{
#Autowired
private ThreadPoolTaskExecutor taskExecutor;
public static void main(String[] args) {
SpringApplication.run(DataMigrationPocApplication.class, args);
}
#Override
public void run(String... arg0) throws Exception {
for(int i = 1; i<=20 ; i++){
taskExecutor.execute(new DataMigration("Task " + i));
}
for (;;) {
int count = taskExecutor.getActiveCount();
System.out.println("Active Threads : " + count);
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
if (count == 0) {
taskExecutor.shutdown();
break;
}
}
System.out.println("Finished all threads");
}
}
I need help to understand if I need to stop my spring boot application it should stop all the 20 threads running which runs (24x7) otherwise after completing there current loop in while loop and exit.
I would propose couple of changes in this code to resolve the problem
1) since in your POC processCommand calls Thread.sleep, when you shutdown the executor and it interrupts workers InterruptedException get called but is almost ignored in your code. After that there is if(Thread.currentThread().isInterrupted()) check which will return false for the reason above. Similar problem is outlined in the post below
how does thread.interrupt() sets the flag?
the following code change should fix the problem:
private void processCommand() {
try {
Thread.sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
shutdown();
}
}
2) Also because of ThreadConfig::taskExecutor executorPool.setWaitForTasksToCompleteOnShutdown(true) Spring will call executor.shutdown instead of executor.shutdownNow. According to javadoc ExecutorService.shutdown
Initiates an orderly shutdown in which previously submitted tasks are
executed, but no new tasks will be accepted.
So I would recommend to set
executorPool.setWaitForTasksToCompleteOnShutdown(false);
Other things to improve in this code: although DataMigration is annotated as a component the instances of this class are creared not by Spring. You should try using factory method similar to ThreadConfig::taskExecutor in order to make Spring initiate instances of DataMigration for example to inject other bean into DataMigration instances.
In order to shutdown executor when running jar file on linux environment you can for example add actuator module and enable shutdown endpoint:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
in application.properties:
endpoints.shutdown.enabled=true
It will enable JMX shutdown endpoint and you can call shutdown on it.
If you want current job cycle of the task to be finished you should set
executorPool.setWaitForTasksToCompleteOnShutdown(true);
In order to connect to your jvm process on linux env remotely you have to specify an RMI Registry port.
Here is a detailed article:
How to access Spring-boot JMX remotely
If you just need to connect to JMX from local env you can run jsoncole or command-line tools : Calling JMX MBean method from a shell script
Here is an example uf using one of these tools - jmxterm
$>run -d org.springframework.boot: -b org.springframework.boot:name=shutdownEndpoint,type=Endpoint shutdown
#calling operation shutdown of mbean org.springframework.boot:name=shutdownEndpoint,type=Endpoint with params []
#operation returns:
{
message = Shutting down, bye...;
}
I have a requirement where I do some operation on GUI and once I save the data in DB I need to send a http request to a webservice. But the response to GUI should not wait for result from webservice request.
For this I am using #Async , annotation of Spring.
here is my structure
MyConroller.java
calls a method
goSaveAndCreate
(not Async) in
ServiceA.java
ServiceA has a ServiceB bean injected in it. A method ,
#Async
create()
in ServiceB is annotated with Async.
Now ServiceA.goSaveAndCreate calls a method in itself , save and calls ServiceB.create() (which is Async).
I can see in logs the a new thread is created which is executing create method. But all of a sudden logs after a particular point stop and that thread seems to have got killed or comlpeted.
#Service("MarginCalculationService")
public class ServiceA implements ServiceAI {
private static final String APPROVED = "APPROVED";
public static final String SUCCESS = "SUCCESS";
....
#Autowired
ServiceB serviceB;
public List<VV> goSaveAndCreate(String[] ids,List<XX> calList) throws Exception, DataAccessException {
try {
Pair<List<VG>,List<UU>> resultList = save(ids);
vvList = resultList.getFirst();
/*
* HKAPIL - Send Message to webService callingserviceB
*/
if(resultList.getSecond() != null){
serviceB.create(resultList.getSecond());
}
} catch (DataAccessException e) {
e.printStackTrace();
logger.error("Data Access Exception thrown during - " , e);
throw e;
} catch (Exception e){
e.printStackTrace();
logger.error("Exception thrown during " , e);
throw e;
}
System.out.println("Exiting the method in MCSERVICE");
return vvList;
}
private save(){
...
...
}
}
Second service
#Service("wireInstructionMessageService")
public class ServiceB implements ServiceBI {
#Async
#Override
public void create(List<Ralc> calcList) {
String threadName = Thread.currentThread().getName();
logger.info("Inside a different thread [" + threadName + " ] to send message ." );
..
...
otherMethod(Obj);
}
private otherMethod(Obj obj){
...
...
..
//tills this point logs are getting printed nothing after this
..
...
}
}
applciationContext.xml entry
<!-- Defines a ThreadPoolTaskExecutor instance with configurable pool size, queue-capacity, keep-alive,
and rejection-policy values. The id becomes the default thread name prefix -->
<task:executor id="MyMessageExecutor"
pool-size="5-25"
queue-capacity="100"/>
<task:annotation-driven executor="MyMessageExecutor"/>
Now I have two question
1) is there a way I can add some logs in some method which tell taht the new thread from MyExecutor is getting killed or MyExecutor is getting closed (the way we have in normal Java ExecutorSerrvice)
2) Am I using the Asyn in wrong way? Is it possible that as soon as method returns from Controller or ServiceA , ServiceB instance also is getting cleaned?
Thanks
HKapil