How can I write a script or otherwise simulate about 100
users connection to my own ftp server?
You can prepare a simple Java code.
First, you have to decide how these requests arrive to your server. I.e., completely random, one per minute, following a normal distribution or more likely an exponential distribution.
Then, you have to use a thread that has:
A method to make an ftp connection (e.g. ftpCall())
A method to get the x milliseconds to the next FTP call (e.g. getTimeToNext())
After an FTP call, the method has to stay in sleep for x milliseconds before to make the next call. Here is the outline of the code in Java
public class FTPTest{
class MyFTPThread{
private int numberOfCall=100;
private void ftpCall() {
//DO CONNECTION
}
private long void getTimeToNext() {
//RETURN A RANDOM TIME OR A FIXED VALUE
}
public void run(){
int counter = 0;
while(++counter <= numberOfCall){
ftpCall();
this.sleep(getTimeToNext());
}
}
}
public static void main(String [] args){
MyFTPThread t = new MyFTPThread();
t.start();
}
}
Related
I have next implementation of RecursiveAction, single purpose of this class - is to print from 0 to 9, but from different threads, if possible:
public class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
}
}
And I thought that invoking awaitQuiescence will make current thread to wait until all tasks (submitted and forked) will be completed:
public class Main {
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
}
}
But I don't always get correct result, instead of printing 10 times, prints from 0 to 10 times.
But if I add helpQuiesce to my implementation of RecursiveAction:
public class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
RecursiveAction.helpQuiesce();//here
}
}
Everything works fine.
I want to know for what actually awaitQuiescence waiting?
You get an idea of what happens when you change the System.out.println(num); to System.out.println(num + " " + Thread.currentThread());
This may print something like:
0 Thread[ForkJoinPool-1-worker-3,5,main]
1 Thread[main,5,main]
tasks
2 Thread[ForkJoinPool.commonPool-worker-3,5,main]
When awaitQuiescence detects that there are pending tasks, it helps out by stealing one and executing it directly. Its documentation says:
If called by a ForkJoinTask operating in this pool, equivalent in effect to ForkJoinTask.helpQuiesce(). Otherwise, waits and/or attempts to assist performing tasks until this pool isQuiescent() or the indicated timeout elapses.
Emphasis added by me
This happens here, as we can see, a task prints “main” as its executing thread. Then, the behavior of fork() is specified as:
Arranges to asynchronously execute this task in the pool the current task is running in, if applicable, or using the ForkJoinPool.commonPool() if not inForkJoinPool().
Since the main thread is not a worker thread of a ForkJoinPool, the fork() will submit the new task to the commonPool(). From that point on, the fork() invoked from a common pool’s worker thread will submit the next task to the common pool too. But awaitQuiescence invoked on the custom pool doesn’t wait for the completion of the common pool’s tasks and the JVM terminates too early.
If you’re going to say that this is a flawed API design, I wouldn’t object.
The solution is not to use awaitQuiescence for anything but the common pool¹. Normally, a RecursiveAction that splits off sub tasks should wait for their completion. Then, you can wait for the root task’s completion to wait for the completion of all associated tasks.
The second half of this answer contains an example of such a RecursiveAction implementation.
¹ awaitQuiescence is useful when you don’t have hands on the actual futures, like with a parallel stream that submits to the common pool.
Everything works fine.
No it does not, you got lucky that it worked when you inserted:
RecursiveAction.helpQuiesce();
To explain this let's slightly change your example a bit:
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
new MyRecursiveAction(num + 1).fork();
}
}
}
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(2));
}
If you run this, you will notice that you get the result you expect to get. And there are two main reasons for this. First, fork method will execute the task in the common pool as the other answer already explained. And second, is that threads in the common pool are daemon threads. JVM is not waiting for them to finish before exiting, it exists early. So if that is the case, you might ask why it works. It does because of this line:
LockSupport.parkNanos(TimeUnit.SECONDS.toNanos(2));
which makes the main thread (which is a non daemon thread) sleep for two seconds, giving enough time for the ForkJoinPool to execute your task.
Now let's change the code closer to your example:
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool();
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
}
specifically, you use: forkJoinPool.awaitQuiescence(...), which is documented as:
Otherwise, waits and/or attempts to assist performing tasks...
It does not say that it will necessarily wait, it says it will "wait and/or attempt ...", in this case it is more or, than and. As such, it will attempt to help, but still it will not wait for all the tasks to finish. Is this weird or even stupid?
When you insert RecursiveAction.helpQuiesce(); you are eventually calling the same awaitQuiescence (with different arguments) under the hood - so essentially nothing changes; the fundamental problem is still there:
static ForkJoinPool forkJoinPool = new ForkJoinPool();
static AtomicInteger res = new AtomicInteger(0);
public static void main(String[] args) {
forkJoinPool.execute(new MyRecursiveAction(0));
System.out.println(forkJoinPool.awaitQuiescence(5, TimeUnit.SECONDS) ? "tasks" : "time");
System.out.println(res.get());
}
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10_000) {
res.incrementAndGet();
System.out.println(num + " thread : " + Thread.currentThread().getName());
new MyRecursiveAction(num + 1).fork();
}
RecursiveAction.helpQuiesce();
}
}
When I run this, it never printed 10000, showing that the insertions of that line changes nothing.
The usual default way to handle such things is to fork then join. And one more join in the caller, on the ForkJoinTask that you get back when calling submit. Something like:
public static void main(String[] args) {
ForkJoinPool forkJoinPool = new ForkJoinPool(2);
ForkJoinTask<Void> task = forkJoinPool.submit(new MyRecursiveAction(0));
task.join();
}
static class MyRecursiveAction extends RecursiveAction {
private final int num;
public MyRecursiveAction(int num) {
this.num = num;
}
#Override
protected void compute() {
if (num < 10) {
System.out.println(num);
MyRecursiveAction ac = new MyRecursiveAction(num + 1);
ac.fork();
ac.join();
}
}
}
I have a program to get the values from a Bar code scanner(using jssc library) and it returns value as expected using event listener but I need to access that value from another class.
I tried to instantiate BarcodeScanner class to main method (ProcessTicket class) and called scannerHandler method and also called the getter method of model class to retrieve value but the getter method runs before the scan is completed. Could you please help me to achieve this?
public class BarcodeScanner {
public static Object SerialPortReader;
static SerialPort serialPort;
public void scannerHandler() {
serialPort = new SerialPort("COM4");
try{
serialPort.openPort();//Open serial port
//Set params. Also set params by this string:
serialPort.setParams(9600, 8, 1, 0);
serialPort.setParams(9600, 8, 1, 0);
serialPort.writeString(new String(new byte[]{0x02})); //triggers barcode scanner
serialPort.addEventListener(new SerialPortReader());//Add SerialPortEventListenerS
} catch (SerialPortException ex) {
System.out.println(ex);
}
}
public static class SerialPortReader implements SerialPortEventListener {
String str;
String value;
public void serialEvent(SerialPortEvent event) {
if (event.isRXCHAR() && event.getEventValue() > 0) {//If data is available
//Check bytes count in the input buffer
try {
byte[] bytesCont = serialPort.readBytes(14);
str = new String(bytesCont);
ModelClass modelClass = new ModelClass();
modelClass.setBarcodeValue(str);
} catch (SerialPortException e) {
e.printStackTrace();
}
}
}
My ProcessTicket.java Class
public class ProcessTicket {
public static void main(String[] args) throws SQLException, SerialPortException {
BarcodeScanner bSC = new BarcodeScanner();
bSC.scannerHandler();
BarcodeScanner.SerialPortReader portReader = new BarcodeScanner.SerialPortReader();
ModelClass modelClass = new ModelClass();
String value = modelClass.getBarcodeValue();
System.out.println(value);
}
}
The main problem here is that you're treating an inherently asynchronous operation (reading from an external sensor in the real world) as if it's synchronous.
I simulated that external sensor stuff to make a standalone app that tests your business logic:
HowToRetrieveTheValueFromTheEventListenerFromAnotherClassInJava.java
package com.stackoverflow;
/**
* https://stackoverflow.com/questions/57452205/how-to-retrieve-the-value-from-the-event-listener-from-another-class-in-java
*/
public class HowToRetrieveTheValueFromTheEventListenerFromAnotherClassInJava {
public static void main(String[] args) {
BarcodeScanner barcodeScanner = new BarcodeScanner((String barcode) -> {
System.out.println("Barcode scanned: " + barcode);
});
barcodeScanner.startScan();
MockUser.startScanningStuffLol();
}
}
That call to MockUser.startScanningStuffLol() is only necessary because I'm testing this just in code, without using a real barcode scanner. Please don't focus on it. I'll post its implementation if you ask, but otherwise I'm assuming that your OS/Java/hardware are working the way they were designed to work, and you can just test this with those tools instead of my MockUser software simulation.
Here are the rest of the classes that you need to implement this:
BarcodeScannedCallback.java
package com.stackoverflow;
public interface BarcodeScannedCallback {
void callback(String barcode);
}
Since we're dealing with an asynchronous operation, we can't just start it and then check for a return value, like we would with a synchronous operation. Instead, we need to pass in a function that will be called once the operation is complete, and just wait for it to finish. BarcodeScannedCallback is the signature of that function (in other words, a description of how that function needs to be structured). It takes one string parameter, and returns nothing.
The implementation of BarcodeScannedCallback is this function that I've already mentioned above, which I'm passing into the BarcodeScanner constructor:
(String barcode) -> {
System.out.println("Barcode scanned: " + barcode);
}
As you can see, this function takes one string parameter, and returns nothing. So, it's an implementation of the BarcodeScannedCallback interface.
Now for the last class: the one that bridges our main method and the serial port, using the above interface.
BarcodeScanner.java
package com.stackoverflow;
public class BarcodeScanner implements SerialPortEventListener {
private SerialPort serialPort;
private final BarcodeScannedCallback callback;
public void startScan() {
try {
serialPort = new SerialPort("COM4");
serialPort.openPort();
serialPort.addEventListener(this);
// Also you can set params by this string: serialPort.setParams(9600, 8, 1, 0);
serialPort.setParams(9600, 8, 1, 0);
// Triggers barcode scanner.
serialPort.writeString(new String(new byte[]{0x02}));
} catch (SerialPortException ex) {
System.out.println(ex);
}
}
#Override
public void serialEvent(SerialPortEvent event) {
boolean isDataAvailable = event.isRXCHAR() && event.getEventValue() > 0;
if (isDataAvailable) {
try {
byte[] bytesCont = serialPort.readBytes(14);
String barcode = new String(bytesCont);
callback.callback(barcode);
} catch (SerialPortException ex) {
System.out.println(ex);
}
}
}
public BarcodeScanner(BarcodeScannedCallback callback) {
this.callback = callback;
}
}
So here's the full lifecycle of these events:
You create a BarcodeScanner.
You tell the BarcodeScanner, via the implementation of BarcodeScannedCallback that you pass into its constructor, what code to run once it receives a barcode over the serial port.
You call startScan on the BarcodeScanner, which opens the serial port and starts waiting for the user to scan a barcode.
The user scans a barcode. This data is transmitted over the serial port. The operating system's implementation of SerialPort calls BarcodeScanner.serialEvent.
Your implementation of serialEvent does its validations, pulls the data from the serial port and converts it from bytes to a string, and calls the BarcodeScannedCallback function that was passed in at the beginning.
When I run this (with my MockUser class setting up a background thread that "scans" a barcode every 3 seconds), I get this output:
Barcode scanned: 420L0L
Barcode scanned: 007
Barcode scanned: 12345
In your case, you should be able to scan 3 barcodes with your real-world barcode scanner, and get the same results.
Note that you may need to do something to keep the main method's thread from ending prematurely, depending on the context that you're running this in.
If you're running it in an Android app or a web server, those frameworks keep their main thread running indefinitely, until you kill the app/server.
But if you're running it as a custom command-line app (which it seems like you're doing, based on the existence of a main method), you will need to do something to keep it alive until you decide to kill it. The simplest way is to put an infinite loop like while (true); on the last line of your main method.
How to avoid manual sleep in unit test.
Suppose In below code the Process and notify takes around 5 seconds for processing. So in order to complete the processing, i have added sleep of 5 seconds.
public class ClassToTest {
public ProcessService processService;
public NotificationService notificationService;
public ClassToTest(ProcessService pService ,NotificationService nService ) {
this.notificationService=nService;
this.processService = pService;
}
public CompletableFuture<Void> testMethod()
{
return CompletableFuture.supplyAsync(processService::process)
.thenAccept(notificationService::notify);
}
}
is there any better way to handle this ?
#Test
public void comletableFutureThenAccept() {
CompletableFuture<Void> thenAccept =
sleep(6);
assertTrue(thenAccept.isDone());
verify(mocknotificationService, times(1)).notify(Mockito.anystring());
}
Normally, you want to test whether an underlying operation completes with the intended result, has the intended side effect, or at least completes without throwing an exception. This can be achieved as easy as
#Test
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.join();
/* check for class under test to have the desired state */
}
join() will wait for the completion and return the result (which you can ignore in case of Void), throwing an exception if the future completed exceptionally.
If completing withing a certain time is actually part of the test, simply use
#Test(timeout = 5000)
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.join();
/* check for class under test to have the desired state */
}
In the unlikely case that you truly want to test for completion within the specified time only, i.e. do not care whether the operation threw an exception, you can use
#Test(timeout = 5000)
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod();
future.exceptionally(t -> null).join();
}
This substitutes an exceptional completion with a null result, hence, join() won’t throw an exception. So only the timeout remains.
Java 9 allows another alternative, not using JUnit’s timeout.
#Test()
public void comletableFutureThenAccept() {
CompletableFuture<Void> future = someMethod().orTimeout(5, TimeUnit.SECONDS);
future.join();
/* check for class under test to have the desired state */
}
This has the advantage of not failing if the operation completes in time but the subsequent verification takes longer.
The code below is a Web API that prints on behalf of a SPA. For brevity I've omitted using statements and the actual printing logic. That stuff all works fine. The point of interest is refactoring of the printing logic onto a background thread, with the web api method enqueuing a job. I did this because print jobs sent in quick succession were interfering with each other with only the last job printing.
It solves the problem of serialising print jobs but raises the question of how to detect shutdown and signal the loop to terminate.
namespace WebPrint.Controllers
{
public class LabelController : ApiController
{
static readonly ConcurrentQueue<PrintJob> queue = new ConcurrentQueue<PrintJob>();
static bool running = true;
static LabelController()
{
ThreadPool.QueueUserWorkItem((state) => {
while (running)
{
Thread.Sleep(30);
if (queue.TryDequeue(out PrintJob job))
{
this.Print(job);
}
}
});
}
public void Post([FromBody]PrintJob job)
{
queue.Enqueue(job);
}
}
public class PrintJob
{
public string url { get; set; }
public string html { get; set; }
public string printer { get; set; }
}
}
Given the way I acquire a thread to servicing the print queue, it is almost certainly marked as a background thread and should terminate when the app pool tries to exit, but I am not certain of this, and so I ask you, dear readers, for your collective notion of best practice in such a scenario.
Well, I did ask for best practice.
Nevertheless, I don't have long-running background tasks, I have short-running tasks. They arrive asynchronously on different threads, but must be executed serially and on a single thread because the WinForms printing methods are designed for STA threading.
Matt Lethargic's point about possible job loss is certainly a consideration, but for this case it doesn't matter. Jobs are never queued for more than a few seconds and loss would merely prompt operator retry.
For that matter, using a message queue doesn't solve the problem of "what if someone shuts it down while it's being used" it merely moves it to another piece of software. A lot of message queues aren't persistent, and you wouldn't believe the number of times I've seen someone use MSMQ to solve this problem and then fail to configure it for persistence.
This has been very interesting.
http://thecodelesscode.com/case/156
I would look at your architecture at a higher level, doing 'long running tasks' such as printing should probably live outside of you webapi process entirely.
If this we myself I would:
Create a windows service (or what have you) that has all the printing logic in it, the job of the controller is then to just talk to the service either by http or some kind of queue MSMQ, RabbitMQ, ServiceBus etc.
If via http then the service should internally queue up the print jobs and return 200/201 to the controller as soon as possible (before printing happens) so that the controller can return to the client efficiently and release it's resources.
If via a queuing technology then the controller should place a message on the queue and again return 200/201 as quick as possible, the service can then read the messages at it's own rate and print one at a time.
Doing it this way removes overhead from your api and also the possibility of losing print jobs in the case of a failure in the webapi (if the api crashes any background threads may/will be effected). Also what if you do a deployment at the point of someone printing, there's a high chance the print job will fail.
My 2 cents worth
I believe that the desired behavior is not something that should be done within a Controller.
public interface IPrintAgent {
void Enqueue(PrintJob job);
void Cancel();
}
The above abstraction can be implemented and injected into the controller using the frameworks IDependencyResolver
public class LabelController : ApiController {
private IPrintAgent agent;
public LabelController(IPrintAgent agent) {
this.agent = agent;
}
[HttpPost]
public IHttpActionResult Post([FromBody]PrintJob job) {
if (ModelState.IsValid) {
agent.Enqueue(job);
return Ok();
}
return BadRequest(ModelState);
}
}
The sole job of the controller in the above scenario is to queue the job.
Now with that aspect out of the way I will focus on the main part of the question.
As already mentioned by others, there are many ways to achieve the desired behavior
A simple in memory implementation can look like
public class DefaultPrintAgent : IPrintAgent {
static readonly ConcurrentQueue<PrintJob> queue = new ConcurrentQueue<PrintJob>();
static object syncLock = new Object();
static bool idle = true;
static CancellationTokenSource cts = new CancellationTokenSource();
static DefaultPrintAgent() {
checkQueue += OnCheckQueue;
}
private static event EventHandler checkQueue = delegate { };
private static async void OnCheckQueue(object sender, EventArgs args) {
cts = new CancellationTokenSource();
PrintJob job = null;
while (!queue.IsEmpty && queue.TryDequeue(out job)) {
await Print(job);
if (cts.IsCancellationRequested) {
break;
}
}
idle = true;
}
public void Enqueue(PrintJob job) {
queue.Enqueue(job);
if (idle) {
lock (syncLock) {
if (idle) {
idle = false;
checkQueue(this, EventArgs.Empty);
}
}
}
}
public void Cancel() {
if (!cts.IsCancellationRequested)
cts.Cancel();
}
static Task Print(PrintJob job) {
//...print job
}
}
which takes advantage of async event handlers to process the queue in sequence as jobs are added.
The Cancel is provided so that the process can be short circuited as needed.
Like in Application_End event as suggested by another user
var agent = new DefaultPrintAgent();
agent.Cancel();
or manually by exposing an endpoint if so desired.
I wish to batch and process items as they come along so i created a UnicastProcessor and subscribed to it like this
UnicastProcessor<String> processor = UnicastProcessor.create()
processor
.bufferTimeout(10, Duration.ofMillis(500))
.subscribe(new Subscriber<List<String>>() {
#Override
public void onSubscribe(Subscription subscription) {
System.out.println("OnSubscribe");
}
#Override
public void onNext(List<String> strings) {
System.out.println("OnNext");
}
#Override
public void onError(Throwable throwable) {
System.out.println("OnError");
}
#Override
public void onComplete() {
System.out.println("OnComplete");
}
});
And then for testing purposes i created a new thread and started adding items in a loop
new Thread(() -> {
int limit = 100
i = 0
while(i < limit) {
++i
processor.sink().next("Hello $i")
}
System.out.println("Published all")
}).start()
After running this (and letting the main thread sleep for 5 seconds) i can see that all item have been published, but the subscriber does not trigger on any of the events so i can't process any of the published items.
What am I doing wrong here?
Reactive Streams specification is the answer!
The total number of onNext´s signalled by a Publisher to a Subscriber
MUST be less than or equal to the total number of elements requested
by that Subscriber´s Subscription at all times. [Rule 1.1]
In your example, you just simply provide a subscriber who does nothing in any sense. In turn, Reactive Streams specification, directly says that nothing will happen (there will be no onNext invocation) if you have not called Subscription#request method
A Subscriber MUST signal demand via Subscription.request(long n) to
receive onNext signals. [Rule 2.1]
Thus, to fix your problem, one of the possible solutions is changing the code in the following way:
UnicastProcessor<String> processor = UnicastProcessor.create()
processor
.bufferTimeout(10, Duration.ofMillis(500))
.subscribe(new Subscriber<List<String>>() {
#Override
public void onSubscribe(Subscription subscription) {
System.out.println("OnSubscribe");
subscription.request(Long.MAX_VALUE);
}
#Override
public void onNext(List<String> strings) {
System.out.println("OnNext");
}
#Override
public void onError(Throwable throwable) {
System.out.println("OnError");
}
#Override
public void onComplete() {
System.out.println("OnComplete");
}
});
Note, in this example demand in size Long.MAX_VALUE means an unbounded demand so that all messages will be directly pushed to the given Subscriber [Rule 3.17]
Use UnicatProcessor correctly
On the one hand, your example will work correctly with mentioned fixes. However, on the other hand, each invocation of FluxProcessor#sink() (yeah sink is FluxProcessor's method) will lead to a redundant calling of UnicastProcessor's onSubscribe method, which under the hood cause a few atomic reads and writes which might be avoided if create FluxSink once and safely use it as many tame as needed. For example:
UnicastProcessor<String> processor = UnicastProcessor.create()
FluxSink<String> sink = processor.serialize().sink();
...
new Thread(() -> {
int limit = 100
i = 0
while(i < limit) {
++i
sink.next("Hello $i")
}
System.out.println("Published all")
}).start()
Note, in this example, I executed an additional method serialize which provide thread-safe sink and ensure that the calling of FluxSink#next concurrently will not cause a violation of the ReactiveStreams spec.