In a Spring Boot application, I'm writing files to S3 using list.stream().parallel().forEach. When trying to get the resource using resourceLoader.getResource(filePath), it throws the exception 'com.amazonaws.services.s3.AmazonS3 referenced from a method is not visible from class loader'. I've noticed that the main thread successfully writes the files, but the child thread throws an exception.
getAccountCommonList()
.stream().parallel()
// .stream()
.forEach(fileNumber -> {
String filePath =
"s3://xxxxx.net/Debug_"
+ LocalDateTime.now().format(DateTimeFormatter.ofPattern("yyyyMMdd_HHmm"))
+ "_"
+ fileNumber;
WritableResource resource = (WritableResource) resourceLoader.getResource(filePath);
try (OutputStream outputStream = resource.getOutputStream()) {
write(outputStream);
} catch (IOException e) {
System.console().printf(e.getMessage(), e);
throw new RuntimeException(e);
}
});
If I don't use parallel(), the code executes successfully.
I'd like to know how to proceed if I need to use parallel().
Could it be because the child thread cannot access the applicationContext?
Is it possible that the applicationContext is bound to the main thread through ThreadLocal?
Related
I am using existing applicationContext.xml that is being used for our existing web application to create a new console based java process.
This process or accounting server is to process tasks of different companies (lets say 2 different companies at a time). For this ThreadPoolTaskExecutor gives a guarantee that no more than the defined limit threads will be executed.
In below snippet when I try to get the "taskExecutor" from application context it is throwing exception that no such bean exists
is it the correct way to register a component ? as am not getting the thread pool bean from application context, is it properly registered with application context ?
public static void main(String[] args) {
LOGGER.info("Starting Accounting Server ...");
LOGGER.info("initializing applicationContext ... from " + APPLICATION_CONTEXT_XML_FULL_PATH);
try (AbstractApplicationContext applicationContext = new FileSystemXmlApplicationContext("file:" + APPLICATION_CONTEXT_XML_FULL_PATH)) {
LOGGER.info("... applicationContext initialized");
ThreadPoolTaskExecutor threadPool = new ThreadPoolTaskExecutor();
threadPool.setCorePoolSize(MAX_RUNNING_THREADS);
threadPool.setMaxPoolSize(MAX_RUNNING_THREADS);
threadPool.setThreadNamePrefix("T--ACCOUNTING-THREAD--");
threadPool.initialize();
Object obj = applicationContext.getBeanFactory().initializeBean(threadPool, "taskExecutor");
// obj is returned
obj = applicationContext.getBean("taskExecutor"); // <--------- throws exception with no such bean
AccountingQueueTransacitonService accountingQueueTransacitonService = applicationContext.getBean(AccountingQueueTransacitonService.class);
/*
while (hastasks) {
// TODO:
// get tasks
// add to thread and push to thread pool
// get some sleep
// update hastasks flag
}
*/
} catch (Exception e) {
LOGGER.error(e, e);
} finally {
LOGGER.info("closed application context ...");
}
LOGGER.info("Ending Accounting Server gracefully ...");
}
I'll generate for each line from an input file a Foo object and save in the next step all the objects in the database (Spring Data JPA).
//read file into stream
try (Stream<String> stream = Files.lines((path), Charset.forName("ISO-8859-1"))) {
Stream<Foo> rStream = stream.map(line -> new Foo(line));
rStream.forEach(line -> fooRepository.save(line));
} catch (IOException e) {
e.printStackTrace();
}
I received an Unhandled exception: java.lang.Exception, because the constructor of the Foo class can propagate an exception from the MyParser class:
public Foo(String row) throws Exception {
String split[] = StringUtils.split(row, "Ú");
field = MyParser.getInt(split[0]);
}
Is it possible to use the Java stream API anyway? Maybe with only one awesome stream? Somethings like this:
stream.map(Foo::new).forEach(foo -> fooRepository.save(foo));
I use Java 8 with Spring Boot 1.5.8.
If you can edit the class, why not simply throw a RuntimeException instead of throwing an Exception. If the API that you use still throws Exception you can just wrap it into:
catch(Exception e){
throw new RuntimeException(e);
}
Then you can simplify it to :
stream.map(Foo::new).forEach(fooRepository::save)
I am using a state machine builder to build state machine in my app.
Also the application has Action classes which implements org.springframework.statemachine.action.Action.
These Action classes are for executing entry actions for each stages.
If any exception is thrown from these Action classes, ie from execute(StateContext paramStateContext) method, I wanted to catch that exception and send an event(Terminated) and drive the state machine to End state, after updating the db with error details.
I tried to use state machine listener by overriding stateMachineError(StateMachine stateMachine, Exception e) method. But unfortunately this is not working.
Any other spring state machine component to catch exceptions, before I go to wrap the entire code in Action classes with try catch, and inside catch block sending the Terminated event so that state machine would navigate End state.
Here is the builder Iam using.
Builder<String, String> builder = StateMachineBuilder
.<String, String> builder();
builder.configureConfiguration()
.withConfiguration()
.autoStartup(false)
.listener(listener())
.beanFactory(
this.applicationContext.getAutowireCapableBeanFactory());
private StateMachineListener<String, String> listener() {
return new StateMachineListenerAdapter<String, String>() {
#Override
public void stateChanged(
org.springframework.statemachine.state.State<String, String> from,
org.springframework.statemachine.state.State<String, String> to) {
LOGGER.debug("State change to " + to.getId());
}
#Override
public void stateMachineError(
StateMachine<String, String> stateMachine, Exception e) {
e.printStackTrace();
LOGGER.debug("Ah... I am not getting executed when exception occurs from entry actions");
LOGGER.debug("Error occured from " + stateMachine.getState()
+ "and the error is" + e.toString());
}
};
}
Iam using 1.1.0.RELEASE version of spring-statemachine-core
You're correct, neither stateMachineError or a method annotated with #onStateMachineError are executed on an error. This is addressed in version 1.2, which is at milestone 1 right now. They introduced errorAction which is executed with the state machine context:
User can always catch exceptions manually but with actions defined for transitions it is possible to define error action which is called if exception is raised. Exception is then available from a StateContext passed to that action.
All that's needed is to specify an error action along with the desired action when defining a transition in the state machine configuration class. From the example in the documentation:
public void configure(StateMachineTransitionConfigurer<States, Events> transitions)
throws Exception {
transitions
.withExternal()
.source(States.S1)
.target(States.S2)
.event(Events.E1)
.action(action(), errorAction());
}
A detailed discussion of this can be found in Spring Statemachine issue #240.
I want to use Java ExecutorService in a Spring Boot application.
Multiple module of the application is called as #Scheduled.
This module calls a Process to get a result from an external application using Runtime.exec. The process is supposed to get a result which is processed by java
This method can be called as part of a Scheduled thread as well as part of a request and response which can be called 24/7 . Multiple instances of the method can be running at a time.
Is it optimum to use ExecutorService defined as a local variable like this or some other method is recommended.
The requirement is that Java should not be infinitely waiting to get a result. It should be timed out.
Here is the method called
public String getExternalInformation(String applicationPath, String command, int noOfTries)
{
ExecutorService executor = Executors.newSingleThreadExecutor();
Future<String> future;
boolean taskExecuted=false;
int trialNumber=0;
String response = "";
while((!taskExecuted)&&(trialNumber<noOfTries))
{
trialNumber++;
log.info("Starting Trial: "+trialNumber);
future= executor.submit(new TestProcessTask(applicationPath,command));
try {
System.out.println("Started Information Fetching ");
response=future.get(3, TimeUnit.MINUTES);
taskExecuted =true;
} catch (TimeoutException e) {
future.cancel(true);
System.out.println("Timed out!");
} catch (InterruptedException e) {
// TODO Auto-generated catch block
e.printStackTrace();
} catch (ExecutionException e) {
e.printStackTrace();
}
}
executor.shutdownNow();
return response;
}
The call() method of the TestProcessTask will call the Runtime.exec and parse the returning OutputStream.
I implemented Async in Spring as suggested by M. Deinum
Using the same Future methods and got it working.,
I have a requirement where I do some operation on GUI and once I save the data in DB I need to send a http request to a webservice. But the response to GUI should not wait for result from webservice request.
For this I am using #Async , annotation of Spring.
here is my structure
MyConroller.java
calls a method
goSaveAndCreate
(not Async) in
ServiceA.java
ServiceA has a ServiceB bean injected in it. A method ,
#Async
create()
in ServiceB is annotated with Async.
Now ServiceA.goSaveAndCreate calls a method in itself , save and calls ServiceB.create() (which is Async).
I can see in logs the a new thread is created which is executing create method. But all of a sudden logs after a particular point stop and that thread seems to have got killed or comlpeted.
#Service("MarginCalculationService")
public class ServiceA implements ServiceAI {
private static final String APPROVED = "APPROVED";
public static final String SUCCESS = "SUCCESS";
....
#Autowired
ServiceB serviceB;
public List<VV> goSaveAndCreate(String[] ids,List<XX> calList) throws Exception, DataAccessException {
try {
Pair<List<VG>,List<UU>> resultList = save(ids);
vvList = resultList.getFirst();
/*
* HKAPIL - Send Message to webService callingserviceB
*/
if(resultList.getSecond() != null){
serviceB.create(resultList.getSecond());
}
} catch (DataAccessException e) {
e.printStackTrace();
logger.error("Data Access Exception thrown during - " , e);
throw e;
} catch (Exception e){
e.printStackTrace();
logger.error("Exception thrown during " , e);
throw e;
}
System.out.println("Exiting the method in MCSERVICE");
return vvList;
}
private save(){
...
...
}
}
Second service
#Service("wireInstructionMessageService")
public class ServiceB implements ServiceBI {
#Async
#Override
public void create(List<Ralc> calcList) {
String threadName = Thread.currentThread().getName();
logger.info("Inside a different thread [" + threadName + " ] to send message ." );
..
...
otherMethod(Obj);
}
private otherMethod(Obj obj){
...
...
..
//tills this point logs are getting printed nothing after this
..
...
}
}
applciationContext.xml entry
<!-- Defines a ThreadPoolTaskExecutor instance with configurable pool size, queue-capacity, keep-alive,
and rejection-policy values. The id becomes the default thread name prefix -->
<task:executor id="MyMessageExecutor"
pool-size="5-25"
queue-capacity="100"/>
<task:annotation-driven executor="MyMessageExecutor"/>
Now I have two question
1) is there a way I can add some logs in some method which tell taht the new thread from MyExecutor is getting killed or MyExecutor is getting closed (the way we have in normal Java ExecutorSerrvice)
2) Am I using the Asyn in wrong way? Is it possible that as soon as method returns from Controller or ServiceA , ServiceB instance also is getting cleaned?
Thanks
HKapil