I'm trying to capture output written from each task as it is executed. The code below works as expected when running Gradle with --max-workers 1, but when multiple tasks are running in parallel this code below picks up output written from other tasks running simultaneously.
The API documentation states the following about the "getLogging" method on Task. From what it says I judge that it should support capturing output from single tasks regardless of any other tasks running at the same time.
getLogging()
Returns the LoggingManager which can be used to control the logging level and standard output/error capture for this task. https://docs.gradle.org/current/javadoc/org/gradle/api/Task.html
graph.allTasks.forEach { Task task ->
task.ext.capturedOutput = [ ]
def listener = { task.capturedOutput << it } as StandardOutputListener
task.logging.addStandardErrorListener(listener)
task.logging.addStandardOutputListener(listener)
task.doLast {
task.logging.removeStandardOutputListener(listener)
task.logging.removeStandardErrorListener(listener)
}
}
Have I messed up something in the code above or should I report this as a bug?
It looks like every LoggingManager instance shares an OutputLevelRenderer, which is what your listeners eventually get added to. This did make me wonder why you weren't getting duplicate messages because you're attaching the same listeners to the same renderer over and over again. But it seems the magic is in BroadcastDispatch, which keeps the listeners in a map, keyed by the listener object itself. So you can't have duplicate listeners.
Mind you, for that to hold, the hash code of each listener must be the same, which seems surprising. Anyway, perhaps this is working as intended, perhaps it isn't. It's certainly worth an issue to get some clarity on whether Gradle should support listeners per task. Alternatively raise it on the dev mailing list.
Related
I would like to write some customised logs into files with cypress. Such as 'current test case name', 'user's id used for this test', 'fail or pass', etc.
I googled awhile, found cy.writeFile meet my needs. But it seems most people would recommend cy.task for logging.
So is cy.writeFile for logging a good idea? If not, what's the reason?
Thanks!
BTW, here's the codes, very simple:
function logger (log) {
cy.writeFile('logs/combined.log', log + '\n', { flag: 'a+' })
}
module.exports = {
logger
}
The cy.task() command is used generally when you do not want the tests themselves to interrupt the logging or if you want to interact with the node process itself. Whereas the cy.writeFile() has to be called between each test and cannot interact with the node process. You can add things in your plugin files so that continous logs can be produced regardless of the test being ran and concatenate those things into the same file.
// in plugins/index.js file
on('task', {
readJson: (filename) => {
// reads the file relative to current working directory
return fsExtra.readJson(path.join(process.cwd(), filename)
}
})
https://docs.cypress.io/api/commands/task.html#Command
cy.task() provides an escape hatch for running arbitrary Node code, so
you can take actions necessary for your tests outside of the scope of
Cypress. This is great for:
Seeding your test database. Storing state in Node that you want
persisted between spec files. Performing parallel tasks, like making
multiple http requests outside of Cypress. Running an external
process. In the task plugin event, the command will fail if undefined
is returned. This helps catch typos or cases where the task event is
not handled.
If you do not need to return a value, explicitly return null to signal
that the given event has been handled.
When I write tests that involve subscribing to events on the Eventstream or watching actors and listning for "Terminated", the tests work fine running them 1 by 1 but when I run the whole testsuite those tests fail.
Tests also works if each of those tests are in a separate test class with Xunit.
How come?
A repo with those kind of tests: https://github.com/Lejdholt/AkkaTestError
Took a look at your repository. I can reproduce the problems you are describing.
It feels like a bug in the TestKit, some timing issue somewhere. But its hard to pin down.
Also, not all unit test frameworks are created equally. The testkit uses its own TaskDispatcher to enable the testing of what are normally inherently asynchronous processed operations.
This sometimes causes some conflicts with the testframework being used. Is also coincidentally why akka.net also moved to XUnit for their own CI process.
I have managed to fix your problem, by not using the TestProbe. Although im not sure if the problem lies with the TestProbe per say, or the fact that your where using an global reference (your 'process' variable).
I suspect that the testframework, while running tests in parrallel, might be causing some wierd things to happen with your testprobe reference.
Example of how i changed one of your tests:
[Test]
public void GivenAnyTime_WhenProcessTerminates_ShouldLogStartRemovingProcess()
{
IProcessFactory factory = Substitute.For<IProcessFactory>();
var testactor = Sys.ActorOf<FakeActor>("test2");
processId = Guid.NewGuid();
factory.Create(Arg.Any<IActorRefFactory>(), Arg.Any<SupervisorStrategy>()).Returns(testactor);
manager = Sys.ActorOf(Props.Create(() => new Manager(factory)));
manager.Tell(new StartProcessCommand(processId));
EventFilter.Info("Removing process.")
.ExpectOne(() => Sys.Stop(testactor));
}
It should be fairly self explanatory on how you should change your other test.
The FakeActor is nothing more then an empty ReceiveActor implementation.
We have in our build a lifecycle task which is supposed to run everything to do a quick check of the code. It looks like this:
task minimalBuild(dependsOn: [jar, testJar, javadoc])
Someone said it should also run checks, which seems reasonable, except that if I depend on check itself, check depends on test, so all the tests will be run and it's no longer a quick build.
I could manually list the individual checks of course, and that is my best idea right now, but every time someone installs a new checking tool, they will have to update this list, and they might not know they have to.
Is there a way to do this programmatically at least? Can I make minimalBuild depend on "everything check depends on except for test"?
You can onfigure it via taskGraph, like so:
gradle.taskGraph.whenReady { graph ->
if (graph.hasTask(minimalBuild)) {
tasks.withType(Test){
enabled = false
}
}
}
This configuration should be added to root of the script. It waits untill the execution graph is ready (it contains all the task, which should be really executed), after that it checks, whether minimalBuild task will be executed and if yes, then it disables all the tasks wit Test type.
I'm using Sidekiq to perform some heavy processing in the background. I looked online but couldn't find the answers to the following questions. I am using:
Class.delay.use_method(listing_id)
And then, inside the class, I have a
self.use_method(listing_id)
listing = Listing.find_by_id listing_id
UserMailer.send_mail(listing)
Class.call_example_function()
Two questions:
How do I make this function idempotent for the UserMailer sendmail? In other words, in case the delayed method runs twice, how do I make sure that it only sends the mail once? Would wrapping it in something like this work?
mail_sent = false
if !mail_sent
UserMailer.send_mail(listing)
mail_sent = true
end
I'm guessing not since the function is tried again and then mail_sent is set to false for the second run through. So how do I make it so that UserMailer is only run once.
Are functions called within the delayed async method also asynchronous? In other words, is Class.call_example_function() executed asynchronously (not part of the response / request cycle?) If not, should I use Class.delay.call_example_function()
Overall, just getting familiar with Sidekiq so any thoughts would be appreciated.
Thanks
I'm coming into this late, but having been around the loop and had this StackOverflow entry appearing prominently via Google, it needs clarification.
The issue of idempotency and the issue of unique jobs are not the same thing. The 'unique' gems look at the parameters of job at the point it is about to be processed. If they find that there was another job with the same parameters which had been submitted within some expiry time window then the job is not actually processed.
The gems are literally what they say they are; they consider whether an enqueued job is unique or not within a certain time window. They do not interfere with the retry mechanism. In the case of the O.P.'s question, the e-mail would still get sent twice if Class.call_example_function() threw an error thus causing a job retry, but the previous line of code had successfully sent the e-mail.
Aside: The sidekiq-unique-jobs gem mentioned in another answer has not been updated for Sidekiq 3 at the time of writing. An alternative is sidekiq-middleware which does much the same thing, but has been updated.
https://github.com/krasnoukhov/sidekiq-middleware
https://github.com/mhenrixon/sidekiq-unique-jobs (as previously mentioned)
There are numerous possible solutions to the O.P.'s email problem and the correct one is something that only the O.P. can assess in the context of their application and execution environment. One would be: If the e-mail is only going to be sent once ("Congratulations, you've signed up!") then a simple flag on the User model wrapped in a transaction should do the trick. Assuming a class User accessible as an association through the Listing via listing.user, and adding in a boolean flag mail_sent to the User model (with migration), then:
listing = Listing.find_by_id(listing_id)
unless listing.user.mail_sent?
User.transaction do
listing.user.mail_sent = true
listing.user.save!
UserMailer.send_mail(listing)
end
end
Class.call_example_function()
...so that if the user mailer throws an exception, the transaction is rolled back and the change to the user's flag setting is undone. If the "call_example_function" code throws an exception, then the job fails and will be retried later, but the user's "e-mail sent" flag was successfully saved on the first try so the e-mail won't be resent.
Regarding idempotency, you can use https://github.com/mhenrixon/sidekiq-unique-jobs gem:
All that is required is that you specifically set the sidekiq option
for unique to true like below:
sidekiq_options unique: true
For jobs scheduled in the future it is possible to set for how long
the job should be unique. The job will be unique for the number of
seconds configured or until the job has been completed.
*If you want the unique job to stick around even after it has been successfully processed then just set the unique_unlock_order to
anything except :before_yield or :after_yield (unique_unlock_order =
:never)
I'm not sure I understand the second part of the question - when you delay a method call, the whole method call is deferred to the sidekiq process. If by 'response / request cycle' you mean that you are running a web server, and you call delay from there, so all the calls within the use_method are called from the sidekiq process, and hence outside of that cycle. They are called synchronously relative to each other though...
I want to start two Async Tasks but the second will not start until the first has completed.
From what I've googled, people usually suggest this approach:
new MyAsyncTask().execute(params);
new MyAsyncTask().execute(params);
However, I need to instantiate them separately and also keep the handles of the task's (to pass messages for example). Therefore, I SORT OF do this:
onStart()
{
taskA = new MyAsyncTask(paramsA);
taskB = new MyAsyncTask(paramsB);
}
onButtonPress()
{
taskA.execute();
taskB.execute();
}
Edit:
I've noticed that taskB does not actually start executing until taskA completes (which runs a tcp/ip server so it takes a long time). I cannot figure out why. Any thoughts or comments ?
The short answer is that, depending on your version of Android, all AsyncTask subclasses may be using the same thread, so you can only do one at a time. There are two ways around this:
Use Runnable instead of AsyncTask
Replace one call to execute with executeOnExecutor(Executor.THREAD_POOL_EXECUTOR, params)
Clearly, try #2 first - it's less of a code change. But if that doesn't work pretty quickly, I'd switch to #1. In that case, you don't have to worry about how Android might change in the future.
If you want more details about the threading model for AsyncTask, have a look at the Android doc entry.