I'm making use of TaskExecutor to run some tasks in background threads. Following is my configuration:-
#Configuration
public class TaskExecutorConfig
{
#Bean
public TaskExecutor threadPoolTaskExecutor()
{
ThreadPoolTaskExecutor threadPoolTaskExecutor = new ThreadPoolTaskExecutor();
threadPoolTaskExecutor.setCorePoolSize(4);
threadPoolTaskExecutor.setMaxPoolSize(100);
threadPoolTaskExecutor.setWaitForTasksToCompleteOnShutdown(false);
threadPoolTaskExecutor.setThreadNamePrefix("TaskExecutor");
threadPoolTaskExecutor.initialize();
return threadPoolTaskExecutor;
}
}
There can be many tasks to run in background. I have declared a ThreadComponent implementing Runnable class. In this ThreadComponent, I have declared methods which get executed based on condition. Following is the component and what it somewhat looks like:-
public class ThreadComponent implements Runnable
{
....
....
....
....
public ThreadComponent()
{
//................//
//Initialization code for variables and some autowired components//
//................//
}
#Override
public void run()
{
if (condition)
doThis();
else if (condition)
doThat();
else if (condition)
doThen();
else if (condition)
doWhere();
else if (condition)
doHow();
else if (condition)
doWhen();
else if (condition)
doNothing();
}
public void doThis() {
......
.....,
}
public void doThat() {
......
.....,
}
public void doThen() {
......
.....,
}
public void doWhere() {
......
.....,
}
public void doHow() {
......
.....,
}
public void doWhen() {
......
.....,
}
public void doNothing() {
......
.....,
}
}
I call this component using TaskExecutor for tasks. In ThreadComponent, I have also used #Autowired components which I initialize in the constructor using ApplicationContext. But this is making my processing slow. I need help so how can I optimize my code to make it better.
Update
Please find the class variables and constructor of the component:
private DTO dto1;
private DTO2 dto2;
private Map<String, Object> dtoVal;
private String schemaName;
private int checkCondition;
private AppUser user;
private Details oldApprovalDetails;
private Details newApprovalDetails;
private ThreadDTO threadDTO;
#Autowired
OcrService ocrService;
#Autowired
NotificationService notificationService;
#Autowired
CreateRecordsService createRecordService;
#Autowired
DtoService dtoService;
#Autowired
DtoCalculations dtoCalculations;
#Autowired
MailService mailService;
#Autowired
TransDtoRepository transDtoRepository;
#Autowired
TransDtoApproverDetailsRepository approverDetails;
public ThreadComponent(ThreadDTO threadDTO,
List<Details> approvalDetails, ApplicationContext ctx)
{
this.dto1 = threadDTO.getDto1();
this.dto2 = threadDTO.getDto2();
this.dtoVal = threadDTO.getDto1().getConfig();
this.schemaName = threadDTO.getDto1().getSchemaName();
this.checkCondition = threadDTO.getCheckCondition();
this.user = threadDTO.getUser();
this.oldApprovalDetails = approvalDetails.get(0);
this.newApprovalDetails = approvalDetails.get(1);
this.threadDTO = threadDTO;
this.ocrService = ctx.getBean(OcrService.class);
this.notificationService = ctx.getBean(NotificationService.class);
this.createRecordService = ctx.getBean(CreateRecordsService.class);
this.dtoService = ctx.getBean(DtoService.class);
this.dtoCalculations = ctx.getBean(DtoCalculations.class);
this.mailService = ctx.getBean(MailService.class);
this.transDtoRepository = ctx.getBean(TransDtoRepository.class);
this.approverDetails = ctx.getBean(TransDtoApproverDetailsRepository.class);
}
Constructor contains all the variables declared in class scope.
I implemented a work around and that helped me solve my issue. I implemented spring boot #Async annotation service and added all my backend code that doesn't require to interrupt the user in that service. This service then makes call to the TaskExecutor for multiple threads. In short, I implemented both #Async and TaskExecutor framework for one complex functionality and that made my code faster (now it takes 300 milliseconds to 500 milliseconds to give me response).
Hope this helps anyone who is in need.
Related
I have spring component class annotated with #Component and in it I have field ConcurrentHashMap map, which is init in constructor of component and used in spring stream listener:
#Component
public class FooService {
private ConcurrentHashMap<Long, String> fooMap;
public FooService () {
fooMap = new ConcurrentHashMap<>();
}
#StreamListener(value = Sink.INPUT)
private void handler(Foo foo) {
fooMap.put(foo.id, foo.body);
}
}
Listener handle messages sent by rest controller. Can you tell me why I always got there fooMap.put(...) NullPointerException because fooMap is null and not initialzied.
EDIT:
After #OlegZhurakousky answer I find out problem is with async method. When I add #Async on some method and add #EnableAsync I can't anymore use private modificator for my #StreamListener method. Do you have idea why and how to fix it?
https://github.com/schwantner92/spring-cloud-stream-issue
Thanks.
Could you try using #PostConstruct instead of constructor?
#PostConstruct
public void init(){
this.fooMap = new ConcurrentHashMap<>();
}
#Denis Stephanov
When I say bare minimum, here is what I mean. So try this as a start, you'll see that the map is not null and start evolving your app from there.
#SpringBootApplication
#EnableBinding(Processor.class)
public class DemoApplication {
private final Map<String, String> map;
public static void main(String[] args) {
SpringApplication.run(DemoRabbit174Application.class, args);
}
public DemoApplication() {
this.map = new HashMap<>();
}
#StreamListener(Processor.INPUT)
public void sink(String string) {
System.out.println(string);
}
}
With Spring everything has to be injected.
You need to declare a #Bean for the ConcurrentHashMap, that will be injected in you Component. So create a Configuration class like:
#Configuration
public class FooMapConfiguration {
#Bean("myFooMap")
public ConcurrentHashMap<Long, String> myFooMap() {
return new ConcurrentHashMap<>();
}
}
Then modify your Component:
#Component
public class FooService {
#Autowired
#Qualifier("myFooMap")
private ConcurrentHashMap<Long, String> fooMap;
public FooService () {
}
#StreamListener(value = Sink.INPUT)
private void handler(Foo foo) {
fooMap.put(foo.id, foo.body); // <= No more NPE here
}
}
I'm new to Zuul J-unit testing. I have a couple of filters which is ChangeRequestEntityFilter and SessionFilter, Where I pasted my filtercode below. Can someone tell me how to write a Junit for the filter. I've searched and trying to use MockWire for the unit testing(Also I pasted my empty methods with basic annotations and WireMock port). I need at-least one proper example how this J-unit for Zuul works. I've referred the http://wiremock.org/docs/getting-started/ doc. Where I got what to do, but not how to do.
public class ChangeRequestEntityFilter extends ZuulFilter {
#Autowired
private UtilityHelperBean utilityHelperBean;
#Override
public boolean shouldFilter() {
// //avoid http GET request since it does'nt have any request body
return utilityHelperBean.isValidContentBody();
}
#Override
public int filterOrder() {
//given priority
}
#Override
public String filterType() {
// Pre
}
#Override
public Object run() {
RequestContext context = getCurrentContext();
try {
/** get values profile details from session */
Map<String, Object> profileMap = utilityHelperBean.getValuesFromSession(context,
CommonConstant.PROFILE.value());
if (profileMap != null) {
/** get new attributes need to add to the actual origin microservice request payload */
Map<String, Object> profileAttributeMap = utilityHelperBean.getProfileForRequest(context, profileMap);
/** add the new attributes in to the current request payload */
context.setRequest(new CustomHttpServletRequestWrapper(context.getRequest(), profileAttributeMap));
}
} catch (Exception ex) {
ReflectionUtils.rethrowRuntimeException(new IllegalStateException("ChangeRequestEntityFilter : ", ex));
}
return null;
}
}
I know ,I'm asking more. But give me any simple working complete example, I'm fine with it.
My current code with basic annotations and WireMock port.
#RunWith(SpringRunner.class)
#SpringBootTest
#DirtiesContext
#EnableZuulProxy
public class ChangeRequestEntityFilterTest {
#Rule
public WireMockRule wireMockRule = new WireMockRule(8080);
#Mock
ChangeRequestEntityFilter requestEntityFilter;
int port = wireMockRule.port();
#Test
public void changeRequestTest() {
}
}
Have you tried #MockBean?
https://docs.spring.io/spring-boot/docs/current/api/org/springframework/boot/test/mock/mockito/MockBean.html
"When #MockBean is used on a field, as well as being registered in the application context, the mock will also be injected into the field. Typical usage might be:"
#RunWith(SpringRunner.class)
public class ExampleTests {
#MockBean
private ExampleService service;
#Autowired
private UserOfService userOfService;
#Test
public void testUserOfService() {
given(this.service.greet()).willReturn("Hello");
String actual = this.userOfService.makeUse();
assertEquals("Was: Hello", actual);
}
#Configuration
#Import(UserOfService.class) // A #Component injected with ExampleService
static class Config {
}
}
Here there is another approach:
private ZuulPostFilter zuulPostFilter;
#Mock
private anotherService anotherService;
#Mock
private HttpServletRequest request;
#Before
public void before() {
MockitoAnnotations.initMocks(this);
MonitoringHelper.initMocks();
zuulPostFilter = new ZuulPostFilter(anotherService);
doNothing().when(anotherService).saveInformation(null, false);
}
#Test
public void postFilterTest() {
log.info("postFilterTest");
RequestContext context = new RequestContext();
context.setResponseDataStream(new ByteArrayInputStream("Test Stream".getBytes()));
context.setResponseGZipped(false);
RequestContext.testSetCurrentContext(context);
when(request.getScheme()).thenReturn("HTTP");
RequestContext.getCurrentContext().setRequest(request);
ZuulFilterResult result = zuulPostFilter.runFilter();
assertEquals(ExecutionStatus.SUCCESS, result.getStatus());
assertEquals("post", zuulPostFilter.filterType());
assertEquals(10, zuulPostFilter.filterOrder());
}
In this case you can test the filter and mock the services inside it without having to autowire it, the problem with the #autowired is that if you have services inside the filter, then it is going to be an integration test that is going to be more difficult to implement.
I have moved a synchronous process to asynchronous, and now I have some troubles to maintain integration tests. It seems related to the fact that when you create a new thread inside a #Transactional method, then call a new #Transactional, Spring create a new transaction.
During integration tests the problem occurs with #Transactional tests. It seems that the thread transaction is rollbacked before the test finishes because of TransactionalTestExecutionListener in test configuration.
I'have tried many things like
- autowiring EntityManager and manually flushing after thread was finished
- using #Rollback instead of #Transactional in test methods
- managing transactions with TestTransaction
- using #Rollback and TestTransaction together
Here is the simplified source code :
public interface MyService{
public void doThing(someArgs...);
public void updateThings(someArgs...);
}
#Service
public class MyServiceImpl implements MyService{
#Autowired
private AsynchronousFutureHandlerService futureService;
#Autowired
#Qualifier("myExecutorService")
private ScheduledExecutorService myExecutorService;
#Transactional
#Override
public void doThing(someArgs...){
doThingAsync(someArgs...);
}
private void doThingAsync(someArgs...){
AsynchronousHandler runnable = applicationContext.getBean(
AsynchronousHandler.class, someArgs...);
//as we are executing some treatment in a new Thread, a new transaction is automatically created
Future<?> future = myExecutorService.submit(runnable);
//keep track of thread execution
futureService.addFutures(future);
}
#Override
#Transactional
public void updateThings(someArgs...){
//do udpate stuff
}
}
/**
* very basic solution to improve later to consult thread state
*/
#Service
public class AsynchronousFutureHandlerService {
//TODO : pass to ThreadSafe collection
private List<Future<?>> futures = new ArrayList<>();
public void addTheoreticalApplicationFuture(Future<?> future){
futures.add(future);
this.deleteJobsDone();
}
public boolean isThreadStillRunning(){
boolean stillRunning = false;
for(Future<?> f : futures){
if(!f.isDone()){
stillRunning = true;
break;
}
}
return stillRunning;
}
public void deleteJobsDone(){
this.futures.removeIf(f -> f.isDone());
}
}
#Component
#Scope("prototype")
public class AsynchronousHandler implements Runnable {
#Autowired
private MyService myService;
#Override
public void run() {
myService.updateThings(...); //updates data in DB
...
}
}
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration(classes = TestConfiguration.class)
#TestExecutionListeners({ DependencyInjectionTestExecutionListener.class, DataSetTestExecutionListener.class,
TransactionalTestExecutionListener.class })
#DataSet(dbType = DBType.H2, locations = { "classpath:dataset.xml" })
public class MyServiceTest{
#Autowired
private MyService myService;
#Autowired
private AsynchronousFutureHandlerService futureService;
#Test
#Transactional
public void test_doThings(){
myService.doThings(someArgs...);
waitUpdateFinish();
Assert.assertEquals(...); //fails here because Thread transaction has been rollbacked
}
private void waitUpdateFinish() throws InterruptedException{
while(futureService.isThreadStillRunning()){
Thread.sleep(500);
}
}
}
I was trying to use spring stater-cache in spring boot 1.3.5, everything works fine except pre load cache in #Configuration class.
Failed tests:
CacheTest.testCacheFromConfig: expected:<n[eal]> but was:<n[ot cached]>
Please take a look at the code as below, if you met this before, please share it with me :)
#Component
public class CacheObject{
#CachePut(value = "nameCache", key = "#userId")
public String setName(long userId, String name) {
return name;
}
#Cacheable(value = "nameCache", key = "#userId")
public String getName(long userId) {
return "not cached";
}
}
#Component
public class CacheReference {
#Autowired
private CacheObject cacheObject;
public String getNameOut(long userId){
return cacheObject.getName(userId);
}
}
#Configuration
public class SystemConfig {
#Autowired
private CacheObject cacheObject;
#PostConstruct
public void init(){
System.out.println("------------------");
System.out.println("-- PRE LOAD CACHE BUT DIDN'T GET CACHED");
System.out.println("------------------");
cacheObject.setName(2, "neal");
cacheObject.setName(3, "dora");
}
}
#RunWith(SpringJUnit4ClassRunner.class)
#SpringApplicationConfiguration(classes = BootElastic.class)
#WebAppConfiguration
public class CacheTest {
#Autowired
private CacheObject cacheObject;
#Autowired
private CacheReference cacheReference;
#Test
public void testCache(){
String name = "this is neal for cache test";
long userId = 1;
cacheObject.setName(userId, name);
// cacheObject.setName(2, "neal"); // this will make test success
String nameFromCache = cacheReference.getNameOut(userId);
System.out.println("1" + nameFromCache);
Assert.assertEquals(nameFromCache, name);
}
#Test
public void testCacheFromConfig(){
String nameFromCache = cacheReference.getNameOut(2);
System.out.println("4" + nameFromCache);
Assert.assertEquals(nameFromCache, "neal");
}
}
#PostConstruct methods are called right after all postProcessBeforeInitialization() BeanPostProcessor methods invoked, and right before postProcessAfterInitialization() invoked. So it is called before there is any proxy around bean, including one, putting values to cache.
The same reason why you can't use #Transactional or #Async methods in #PostConstruct.
You may call it from some #EventListener on ContextRefreshedEvent to get it working
I would like to start an asynchronous task from within controller like in following code sniplet from Spring docs.
import org.springframework.core.task.TaskExecutor;
public class TaskExecutorExample {
private class MessagePrinterTask implements Runnable {
private int cn;
public MessagePrinterTask() {
}
public void run() {
//dummy code
for (int i = 0; i < 10; i++) {
cn = i;
}
}
}
private TaskExecutor taskExecutor;
public TaskExecutorExample(TaskExecutor taskExecutor) {
this.taskExecutor = taskExecutor;
}
public void printMessages() {
taskExecutor.execute(new MessagePrinterTask());
}
}
afterwards in annother request (in the case that task is running) I need to check the progress of the task. Basicaly get the value of cn.
What would be the best aproach in Spring MVC a how to avoid syncronisation issues.
Thanks
Pepa Procházka
Have you looked at the #Async annotation in the Spring reference doc?
First, create a bean for your asynchronous task:
#Service
public class AsyncServiceBean implements ServiceBean {
private AtomicInteger cn;
#Async
public void doSomething() {
// triggers the async task, which updates the cn status accordingly
}
public Integer getCn() {
return cn.get();
}
}
Next, call it from the controller:
#Controller
public class YourController {
private final ServiceBean bean;
#Autowired
YourController(ServiceBean bean) {
this.bean = bean;
}
#RequestMapping(value = "/trigger")
void triggerAsyncJob() {
bean.doSomething();
}
#RequestMapping(value = "/status")
#ResponseBody
Map<String, Integer> fetchStatus() {
return Collections.singletonMap("cn", bean.getCn());
}
}
Remember to configure an executor accordingly, e.g.
<task:annotation-driven executor="myExecutor"/>
<task:executor id="myExecutor" pool-size="5"/>
One solution could be: in your async thread, write to a DB, and have your checking code check the DB table for progress. You get the additional benefit of persisting performance data for later evaluation.
Also, just use the #Async annotation to kick off the asynchronous thread - makes life easier and is a Spring Way To Do It.
Check this github source, it gives pretty simple way of catching status of the background job using #Async of Spring mvc.
https://github.com/frenos/spring-mvc-async-progress/tree/master/src/main/java/de/codepotion/examples/asyncExample
Ignoring synchronization issues you could do something like this:
private class MessagePrinterTask implements Runnable {
private int cn;
public int getCN() {
return cn;
}
...
}
public class TaskExecutorExample {
MessagePrinterTask printerTask;
public void printMessages() {
printerTask = new MessagePrinterTask();
taskExecutor.execute(printerTask);
}
...
}