Spring dynamic async - spring

Question

Trying to have a dynamic number of async processes running in Spring 2.0.5. I do have it working on a fixed number but need the flexibility.
the current methods are:
in class AsyncFactoryService:
public void doAsyncBetter(long rowsPerBucket) throws InterruptedException, SQLException, ExecutionException {
List<Bucket> buckets = sourceData01Service.bucketsByRowsPerBucket(rowsPerBucket);
AsyncDataLoadService[] asyncDataLoadServiceArray = new AsyncDataLoadService[buckets.size()];
for(int i = 0; i < buckets.size(); i++) {
asyncDataLoadServiceArray[i] = new AsyncDataLoadService();
asyncDataLoadServiceArray[i].loadSourceData01(buckets.get(i));
}
}
in class AsyncDataLoadService
#Autowired SourceData01Service sourceData01Service;
#Async
public Future<String> loadSourceData01(Bucket bucket) throws InterruptedException, SQLException {
LOGGER.info(String.format("(loadSourceData01) [Thread id => %d, bucket => %s]",Thread.currentThread().getId(), bucket.toString()));
List<SourceData01> sourceData01s = null;
sourceData01s = sourceData01Service.rowsByBucket(bucket);
Thread.sleep(5000);
String info = String.format("finish sourceData01s.size() => %d, Thread id => %d", sourceData01s.size(), Thread.currentThread().getId());
return new AsyncResult<>(info);
}
this fails with
Caused by: java.lang.NullPointerException
at demo.service.AsyncDataLoadService.loadSourceData01(AsyncDataLoadService.java:35) ~[classes/:?]
at demo.service.AsyncFactoryService.doAsyncBetter(AsyncFactoryService.java:43) ~[classes/:?]
at demo.RunApp.run(RunApp.java:39) ~[classes/:?]
As it seems in the service the sourceData01Service is null.
When I try with a fixed number like
public void doAsync(long rowsPerBucket) throws InterruptedException, SQLException, ExecutionException {
List<Bucket> buckets = sourceData01Service.bucketsByRowsPerBucket(rowsPerBucket);
Future<String> process0 = asyncDataLoadService.loadSourceData01(buckets.get(0));
Future<String> process1 = asyncDataLoadService.loadSourceData01(buckets.get(1));
Future<String> process2 = asyncDataLoadService.loadSourceData01(buckets.get(2));
Future<String> process3 = asyncDataLoadService.loadSourceData01(buckets.get(3));
Future<String> process4 = asyncDataLoadService.loadSourceData01(buckets.get(4));
while(!(process0.isDone()) && !(process2.isDone()) && !(process3.isDone()) && !(process4.isDone())) {
Thread.sleep(2000);
}
LOGGER.info("Process 0 => " + process0.get());
LOGGER.info("Process 1 => " + process1.get());
LOGGER.info("Process 2 => " + process2.get());
LOGGER.info("Process 3 => " + process3.get());
LOGGER.info("Process 4 => " + process4.get());
}
it works as expected but in my case the number of buckets is dependent on the number of rows in the table so I do not know how many buckets there will be. Any ideas how to do this?

Answer 1

The service (sourceData01Service) injection in AsyncDataLoadService didn't happen as the application is creating the instance of AsyncDataLoadService (in the factory class) instead of spring creating it.
We can modify the implementation to let spring create the required instance.
#Component
class AsyncDataLoadService {
#Autowired SourceData01Service sourceData01Service;
#Async
public Future<String> loadSourceData01(Bucket bucket) throws InterruptedException, SQLException {
LOGGER.info(String.format("(loadSourceData01) [Thread id => %d, bucket => %s]",Thread.currentThread().getId(), bucket.toString()));
List<SourceData01> sourceData01s = null;
sourceData01s = sourceData01Service.rowsByBucket(bucket);
Thread.sleep(5000);
String info = String.format("finish sourceData01s.size() => %d, Thread id => %d", sourceData01s.size(), Thread.currentThread().getId());
return new AsyncResult<>(info);
}
}
We can then change the factory class implementation to autowire AsyncDataLoadService and submit the job to be executed (in this case loadSourceData01).
Since asyncDataLoadService.loadSourceData01 is annotated with #Async, every call to that method will be executed in a separate thread. By default spring use SimpleAsyncTaskExecutor that fires up new thread for every invocation. Configure thread pool implementation if required.
#Component
class AsyncFactoryService {
#Autowired
AsyncDataLoadService asyncDataLoadService;
public void doAsyncBetter(long rowsPerBucket) throws InterruptedException, SQLException, ExecutionException {
List<Bucket> buckets = sourceData01Service.bucketsByRowsPerBucket(rowsPerBucket);
for(int i = 0; i < buckets.size(); i++) {
asyncDataLoadService.loadSourceData01(buckets.get(i));
}
}
}