How to avoid manual sleep in unit test.
Suppose In below code the Process and notify takes around 5 seconds for processing. So in order to complete the processing, i have added sleep of 5 seconds.
public class ClassToTest {
public ProcessService processService;
public NotificationService notificationService;
public ClassToTest(ProcessService pService ,NotificationService nService ) {
this.notificationService=nService;
this.processService = pService;
}
public CompletableFuture<Void> testMethod()
{
return CompletableFuture.supplyAsync(processService::process)
.thenAccept(notificationService::notify);
}
}
is there any better way to handle this ?
#Test
public void comletableFutureThenAccept() {
CompletableFuture<Void> thenAccept =
sleep(6);
assertTrue(thenAccept.isDone());
verify(mocknotificationService, times(1)).notify(Mockito.anystring());
}
Normally, you want to test whether an underlying operation completes with the intended result, has the intended side effect, or at least completes without throwing an exception. This can be achieved as easy as
#Test
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.join();
/* check for class under test to have the desired state */
}
join() will wait for the completion and return the result (which you can ignore in case of Void), throwing an exception if the future completed exceptionally.
If completing withing a certain time is actually part of the test, simply use
#Test(timeout = 5000)
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.join();
/* check for class under test to have the desired state */
}
In the unlikely case that you truly want to test for completion within the specified time only, i.e. do not care whether the operation threw an exception, you can use
#Test(timeout = 5000)
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.exceptionally(t -> null).join();
}
This substitutes an exceptional completion with a null result, hence, join() won’t throw an exception. So only the timeout remains.
Java 9 allows another alternative, not using JUnit’s timeout.
#Test()
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod().orTimeout(5, TimeUnit.SECONDS);
future.join();
/* check for class under test to have the desired state */
}
This has the advantage of not failing if the operation completes in time but the subsequent verification takes longer.
Related
I'm using jobrunr 5.1.4 in my spring boot application. I have a simple service declaring a recurring job which allows for some retries. A single failing job run is not that relevant for me. Instead, I'm interested in getting notified after all jobs, i.e. the initial job including all the retries, have failed.
I thought JobRunr's JobServerFilter would be a good idea. But the onProcessed() method never gets triggered in case of an exception only in case of a successful job run. And the ApplyStateFilter gets triggered on every state change. Far too often for my requirement. Leaving me clueless, if a change to a FAILED state was the last in a series of jobs belonging together (initial job + allowed retried jobs).
A simple example would look like this:
#Service
public class JobScheduler {
#Job(name = "My Recurring Job", retries = 2, jobFilters = ExceptionFilter.class)
#Recurring(id = "my-recurring-job", cron = "*/10 * * * *")
public void recurringJob() {
throw new RuntimeException("foo");
}
}
A basic implementation of my JobFilter looks like this:
#Component
public class ExceptionFilter implements JobServerFilter, ApplyStateFilter {
#Override
public void onProcessing(Job job) {
log.info("onProcessing: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onProcessed(Job job) {
log.info("onProcessed: {}", job.getJobName());
log.info(job.getJobState().getName().name());
}
#Override
public void onStateApplied(Job job, JobState jobState1, JobState jobState2) {
log.info("onStateApplied: {}", job.getJobName());
log.info("jobState1: {}", jobState1.getName().name());
log.info("jobState2: {}", jobState2.getName().name());
}
}
Is this use case even possible with JobRunr? Or does anyone have an idea how to solve this issue in a different way?
Thank you very much in advance for you support.
I think you're on the right track with onStateApplied from ApplyStateFilter.
You can use the following approach:
#Override
public void onStateApplied(Job job, JobState oldState, JobState newState) {
if (isFailed(newState) && maxAmountOfRetriesReached(job)) {
// your logic here
}
}
OnProcessed is not triggered as your job was not processed (due to the failure).
I have next implementation of RecursiveAction, single purpose of this class - is to print from 0 to 9, but from different threads, if possible:
public class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
}
}
And I thought that invoking awaitQuiescence will make current thread to wait until all tasks (submitted and forked) will be completed:
public class Main {
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
}
}
But I don't always get correct result, instead of printing 10 times, prints from 0 to 10 times.
But if I add helpQuiesce to my implementation of RecursiveAction:
public class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
RecursiveAction.helpQuiesce();//here
}
}
Everything works fine.
I want to know for what actually awaitQuiescence waiting?
You get an idea of what happens when you change the System.out.println(num); to System.out.println(num + " " + Thread.currentThread());
This may print something like:
0 Thread[ForkJoinPool-1-worker-3,5,main]
1 Thread[main,5,main]
tasks
2 Thread[ForkJoinPool.commonPool-worker-3,5,main]
When awaitQuiescence detects that there are pending tasks, it helps out by stealing one and executing it directly. Its documentation says:
If called by a ForkJoinTask operating in this pool, equivalent in effect to ForkJoinTask.helpQuiesce(). Otherwise, waits and/or attempts to assist performing tasks until this pool isQuiescent() or the indicated timeout elapses.
Emphasis added by me
This happens here, as we can see, a task prints “main” as its executing thread. Then, the behavior of fork() is specified as:
Arranges to asynchronously execute this task in the pool the current task is running in, if applicable, or using the ForkJoinPool.commonPool() if not inForkJoinPool().
Since the main thread is not a worker thread of a ForkJoinPool, the fork() will submit the new task to the commonPool(). From that point on, the fork() invoked from a common pool’s worker thread will submit the next task to the common pool too. But awaitQuiescence invoked on the custom pool doesn’t wait for the completion of the common pool’s tasks and the JVM terminates too early.
If you’re going to say that this is a flawed API design, I wouldn’t object.
The solution is not to use awaitQuiescence for anything but the common pool¹. Normally, a RecursiveAction that splits off sub tasks should wait for their completion. Then, you can wait for the root task’s completion to wait for the completion of all associated tasks.
The second half of this answer contains an example of such a RecursiveAction implementation.
¹ awaitQuiescence is useful when you don’t have hands on the actual futures, like with a parallel stream that submits to the common pool.
Everything works fine.
No it does not, you got lucky that it worked when you inserted:
RecursiveAction.helpQuiesce();
To explain this let's slightly change your example a bit:
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
}
}
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(2));
}
If you run this, you will notice that you get the result you expect to get. And there are two main reasons for this. First, fork method will execute the task in the common pool as the other answer already explained. And second, is that threads in the common pool are daemon threads. JVM is not waiting for them to finish before exiting, it exists early. So if that is the case, you might ask why it works. It does because of this line:
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(2));
which makes the main thread (which is a non daemon thread) sleep for two seconds, giving enough time for the ForkJoinPool to execute your task.
Now let's change the code closer to your example:
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
}
specifically, you use: forkJoinPool.awaitQuiescence(...), which is documented as:
Otherwise, waits and/or attempts to assist performing tasks...
It does not say that it will necessarily wait, it says it will "wait and/or attempt ...", in this case it is more or, than and. As such, it will attempt to help, but still it will not wait for all the tasks to finish. Is this weird or even stupid?
When you insert RecursiveAction.helpQuiesce(); you are eventually calling the same awaitQuiescence (with different arguments) under the hood - so essentially nothing changes; the fundamental problem is still there:
static ForkJoinPool forkJoinPool = new ForkJoinPool();
static AtomicInteger res = new AtomicInteger(0);
public static void main(String[] args) {
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
System.out.println(res.get());
}
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10_000) {
res.incrementAndGet();
System.out.println(num + " thread : " + Thread.currentThread().getName());
new MyRecursiveAction(num + 1).fork();
}
RecursiveAction.helpQuiesce();
}
}
When I run this, it never printed 10000, showing that the insertions of that line changes nothing.
The usual default way to handle such things is to fork then join. And one more join in the caller, on the ForkJoinTask that you get back when calling submit. Something like:
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool(2);
ForkJoinTask<Void> task = forkJoinPool.submit(new MyRecursiveAction(0));
task.join();
}
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
MyRecursiveAction ac = new MyRecursiveAction(num + 1);
ac.fork();
ac.join();
}
}
}
I wish to batch and process items as they come along so i created a UnicastProcessor and subscribed to it like this
UnicastProcessor<String> processor = UnicastProcessor.create()
processor
.bufferTimeout(10, Duration.ofMillis(500))
.subscribe(new Subscriber<List<String>>() {
#Override
public void onSubscribe(Subscription subscription) {
System.out.println("OnSubscribe");
}
#Override
public void onNext(List<String> strings) {
System.out.println("OnNext");
}
#Override
public void onError(Throwable throwable) {
System.out.println("OnError");
}
#Override
public void onComplete() {
System.out.println("OnComplete");
}
});
And then for testing purposes i created a new thread and started adding items in a loop
new Thread(() -> {
int limit = 100
i = 0
while(i < limit) {
++i
processor.sink().next("Hello $i")
}
System.out.println("Published all")
}).start()
After running this (and letting the main thread sleep for 5 seconds) i can see that all item have been published, but the subscriber does not trigger on any of the events so i can't process any of the published items.
What am I doing wrong here?
Reactive Streams specification is the answer!
The total number of onNext´s signalled by a Publisher to a Subscriber
MUST be less than or equal to the total number of elements requested
by that Subscriber´s Subscription at all times. [Rule 1.1]
In your example, you just simply provide a subscriber who does nothing in any sense. In turn, Reactive Streams specification, directly says that nothing will happen (there will be no onNext invocation) if you have not called Subscription#request method
A Subscriber MUST signal demand via Subscription.request(long n) to
receive onNext signals. [Rule 2.1]
Thus, to fix your problem, one of the possible solutions is changing the code in the following way:
UnicastProcessor<String> processor = UnicastProcessor.create()
processor
.bufferTimeout(10, Duration.ofMillis(500))
.subscribe(new Subscriber<List<String>>() {
#Override
public void onSubscribe(Subscription subscription) {
System.out.println("OnSubscribe");
subscription.request(Long.MAX_VALUE);
}
#Override
public void onNext(List<String> strings) {
System.out.println("OnNext");
}
#Override
public void onError(Throwable throwable) {
System.out.println("OnError");
}
#Override
public void onComplete() {
System.out.println("OnComplete");
}
});
Note, in this example demand in size Long.MAX_VALUE means an unbounded demand so that all messages will be directly pushed to the given Subscriber [Rule 3.17]
Use UnicatProcessor correctly
On the one hand, your example will work correctly with mentioned fixes. However, on the other hand, each invocation of FluxProcessor#sink() (yeah sink is FluxProcessor's method) will lead to a redundant calling of UnicastProcessor's onSubscribe method, which under the hood cause a few atomic reads and writes which might be avoided if create FluxSink once and safely use it as many tame as needed. For example:
UnicastProcessor<String> processor = UnicastProcessor.create()
FluxSink<String> sink = processor.serialize().sink();
...
new Thread(() -> {
int limit = 100
i = 0
while(i < limit) {
++i
sink.next("Hello $i")
}
System.out.println("Published all")
}).start()
Note, in this example, I executed an additional method serialize which provide thread-safe sink and ensure that the calling of FluxSink#next concurrently will not cause a violation of the ReactiveStreams spec.
I have a simple scenario in which am trying to verify some behavior when a method is called (i.e. that a certain method was called with given parameter, a function pointer in this scenario). Below are my classes:
#SpringBootApplication
public class Application {
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(Application.class, args);
AppBootStrapper bootStrapper = context.getBean(AppBootStrapper.class);
bootStrapper.start();
}
}
#Component
public class AppBootStrapper {
private NetworkScanner networkScanner;
private PacketConsumer packetConsumer;
public AppBootStrapper(NetworkScanner networkScanner, PacketConsumer packetConsumer) {
this.networkScanner = networkScanner;
this.packetConsumer = packetConsumer;
}
public void start() {
networkScanner.addConsumer(packetConsumer::consumePacket);
networkScanner.startScan();
}
}
#Component
public class NetworkScanner {
private List<Consumer<String>> consumers = new ArrayList<>();
public void startScan(){
Executors.newSingleThreadExecutor().submit(() -> {
while(true) {
// do some scanning and get/parse packets
consumers.forEach(consumer -> consumer.accept("Package Data"));
}
});
}
public void addConsumer(Consumer<String> consumer) {
this.consumers.add(consumer);
}
}
#Component
public class PacketConsumer {
public void consumePacket(String packet) {
System.out.println("Packet received: " + packet);
}
}
#RunWith(JUnit4.class)
public class AppBootStrapperTest {
#Test
public void start() throws Exception {
NetworkScanner networkScanner = mock(NetworkScanner.class);
PacketConsumer packetConsumer = mock(PacketConsumer.class);
AppBootStrapper appBootStrapper = new AppBootStrapper(networkScanner, packetConsumer);
appBootStrapper.start();
verify(networkScanner).addConsumer(packetConsumer::consumePacket);
verify(networkScanner, times(1)).startScan();
}
}
I want to verify that bootStrapper did in fact do proper setup by registering the packet consumer(there might be other consumers registered later on, but this one is mandatory) and then called startScan. I get the following error message when I execute the test case:
Argument(s) are different! Wanted:
networkScanner bean.addConsumer(
com.spring.starter.AppBootStrapperTest$$Lambda$8/438123546#282308c3
);
-> at com.spring.starter.AppBootStrapperTest.start(AppBootStrapperTest.java:24)
Actual invocation has different arguments:
networkScanner bean.addConsumer(
com.spring.starter.AppBootStrapper$$Lambda$7/920446957#5dda14d0
);
-> at com.spring.starter.AppBootStrapper.start(AppBootStrapper.java:12)
From the exception, clearly the function pointers aren't the same.
Am I approaching this the right way? Is there something basic I am missing? I played around and had a consumer injected into PacketConsumer just to see if it made a different and that was OK, but I know that's certainly not the right way to go.
Any help, perspectives on this would be greatly appreciated.
Java doesn't have any concept of "function pointers"; when you see:
networkScanner.addConsumer(packetConsumer::consumePacket);
What Java actually compiles is (the equivalent of):
networkScanner.addConsumer(new Consumer<String>() {
#Override void accept(String packet) {
packetConsumer.consumePacket(packet);
}
});
This anonymous inner class happens to be called AppBootStrapper$$Lambda$7. Because it doesn't (and shouldn't) define an equals method, it will never be equal to the anonymous inner class that the compiler generates in your test, which happens to be called AppBootStrapperTest$$Lambda$8. This is regardless of the fact that the method bodies are the same, and are built in the same way from the same method reference.
If you generate the Consumer explicitly in your test and save it as a static final Consumer<String> field, then you can pass that reference in the test and compare it; at that point, reference equality should hold. This should work with a lambda expression or method reference just fine.
A more apt test would probably verify(packetConsumer, atLeastOnce()).consumePacket(...), as the contents of the lambda are an implementation detail and you're really more concerned about how your component collaborates with other components. The abstraction here should be at the consumePacket level, not at the addConsumer level.
See the comments and answer on this SO question.
I have one class that extends DeferredResults and extends Runnable as shown below
public class EventDeferredObject<T> extends DeferredResult<Boolean> implements Runnable {
private Long customerId;
private String email;
#Override
public void run() {
RestTemplate restTemplate=new RestTemplate();
EmailMessageDTO emailMessageDTO=new EmailMessageDTO("dineshshe#gmail.com", "Hi There");
Boolean result=restTemplate.postForObject("http://localhost:9080/asycn/sendEmail", emailMessageDTO, Boolean.class);
this.setResult(result);
}
//Constructor and getter and setters
}
Now I have controller that return the object of the above class,whenever new request comes to controller we check if that request is present in HashMap(That stores unprocessed request at that instance).If not present then we are creating object of EventDeferredObject class can store that in HashMap and call start() method on it.If this type request is already present then we will return that from HashMap.On completion on request we will delete that request from HashMap.
#RequestMapping(value="/sendVerificationDetails")
public class SendVerificationDetailsController {
private ConcurrentMap<String , EventDeferredObject<Boolean>> requestMap=new ConcurrentHashMap<String , EventDeferredObject<Boolean>>();
#RequestMapping(value="/sendEmail",method=RequestMethod.POST)
public EventDeferredObject<Boolean> sendEmail(#RequestBody EmailDTO emailDTO)
{
EventDeferredObject<Boolean> eventDeferredObject = null;
System.out.println("Size:"+requestMap.size());
if(!requestMap.containsKey(emailDTO.getEmail()))
{
eventDeferredObject=new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail());
requestMap.put(emailDTO.getEmail(), eventDeferredObject);
Thread t1=new Thread(eventDeferredObject);
t1.start();
}
else
{
eventDeferredObject=requestMap.get(emailDTO.getEmail());
}
eventDeferredObject.onCompletion(new Runnable() {
#Override
public void run() {
if(requestMap.containsKey(emailDTO.getEmail()))
{
requestMap.remove(emailDTO.getEmail());
}
}
});
return eventDeferredObject;
}
}
Now this code works fine if there no identical request comes to that stored in HashMap. If we give number of different request at same time code works fine.
Well, I do not know if I understood correctly, but I think you might have race conditions in the code, for example here:
if(!requestMap.containsKey(emailDTO.getEmail()))
{
eventDeferredObject=new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail());
requestMap.put(emailDTO.getEmail(), eventDeferredObject);
Thread t1=new Thread(eventDeferredObject);
t1.start();
}
else
{
eventDeferredObject=requestMap.get(emailDTO.getEmail());
}
think of a scenario in which you have two requests with the same key emailDTO.getEmail().
Request 1 checks if there is a key in the map, does not find it and puts it inside.
Request 2 comes some time later, checks if there is a key in the map, finds it, and
goes to fetch it; however just before that, the thread started by request 1 finishes and another thread, started by onComplete event, removes the key from the map. At this point,
requestMap.get(emailDTO.getEmail())
will return null, and as a result you will have a NullPointerException.
Now, this does look like a rare scenario, so I do not know if this is the problem you see.
I would try to modify the code as follows (I did not run it myself, so I might have errors):
public class EventDeferredObject<T> extends DeferredResult<Boolean> implements Runnable {
private Long customerId;
private String email;
private ConcurrentMap ourConcurrentMap;
#Override
public void run() {
...
this.setResult(result);
ourConcurrentMap.remove(this.email);
}
//Constructor and getter and setters
}
so the DeferredResult implementation has the responsibility to remove itself from the concurrent map. Moreover I do not use the onComplete to set a callback thread, as it seems to me an unnecessary complication. To avoid the race conditions I talked about before, one needs to combine somehow the verification of the presence of an entry with its fetching into one atomic operation; this is done by the putIfAbsent method of ConcurrentMap. Therefore I change the controller into
#RequestMapping(value="/sendVerificationDetails")
public class SendVerificationDetailsController {
private ConcurrentMap<String , EventDeferredObject<Boolean>> requestMap=new ConcurrentHashMap<String , EventDeferredObject<Boolean>>();
#RequestMapping(value="/sendEmail",method=RequestMethod.POST)
public EventDeferredObject<Boolean> sendEmail(#RequestBody EmailDTO emailDTO)
{
EventDeferredObject<Boolean> eventDeferredObject = new EventDeferredObject<Boolean>(emailDTO.getCustomerId(), emailDTO.getEmail(), requestMap);
EventDeferredObject<Boolean> oldEventDeferredObject = requestMap.putIfAbsent(emailDTO.getEmail(), eventDeferredObject );
if(oldEventDeferredObject == null)
{
//if no value was present before
Thread t1=new Thread(eventDeferredObject);
t1.start();
return eventDeferredObject;
}
else
{
return oldEventDeferredObject;
}
}
}
if this does not solve the problem you have, I hope that at least it might give some idea.