I am trying to execute a task (Line 3) asynchronously in Quarkus using the worker thread from Infrastructure.getDefaultExecutor(). My task internally invokes a method annotated as #Transactional.
override fun triggerAsyncJob(): Boolean {
// trigger async operation
Uni.createFrom().voidItem()
.emitOn(Infrastructure.getDefaultExecutor()). // -- Line 1
// .emitOn(Executors.newSingleThreadExecutor()) // -- Line 2
.subscribe()
.with { eventDataSource.extractAndPublishEventData() } // Line 3 - some task to execute asynchronously
return true
}
The problem is when I run below method with Line 1, it gives "io.quarkus.runtime.BlockingOperationNotAllowedException: Cannot start a JTA transaction from the IO thread." But when I comment Line 1 and execute method with Line 2, it works perfectly fine.
I want to understand this behaviour. Does Infrastructure.getDefaultExecutor() returns an IO thread to execute the task?
Related
I am trying to use UltimateThreadGroup with the JMeter Java API, I am creating the UltimateThreadGroup object as in the below code[2] and add it to the Hashtree.
Finally handover it to the JMeter engine to execute.
But It gives the following NullPointerException in the middle of execution. As I debugged the code, the Issue seems coming from JMeterThread class following method.
public JMeterThread(HashTree test, JMeterThreadMonitor monitor, ListenerNotifier note, Boolean isSameUserOnNextIteration)
But issue throws on different code lines from execution to execution. So it's difficult figure out what causing the NullPointer.
Does anybody have an idea on what's going on here? Appreciate your answers.
[1]
2020-11-03 13:08:20 DEBUG TestCompiler:273 - adding controller: kg.apc.jmeter.threads.UltimateThreadGroup#30b2b76f to sampler config
2020-11-03 13:08:22 ERROR JMeterThread:319 - Test failed!
java.lang.NullPointerException
at org.apache.jmeter.threads.AbstractThreadGroup.addTestElement(AbstractThreadGroup.java:122)
at org.apache.jmeter.threads.AbstractThreadGroup.addTestElementOnce(AbstractThreadGroup.java:131)
at org.apache.jmeter.threads.TestCompiler.subtractNode(TestCompiler.java:151)
at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:997)
at org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:994)
at org.apache.jorphan.collections.HashTree.traverse(HashTree.java:976)
at org.apache.jmeter.threads.JMeterThread.initRun(JMeterThread.java:704)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
at java.lang.Thread.run(Thread.java:748)
[2]
UltimateThreadGroup ultimateThreadGroup = new UltimateThreadGroup();
ultimateThreadGroup.setName(threadGroupName);
ultimateThreadGroup.setProperty(AbstractThreadGroup.ON_SAMPLE_ERROR, AbstractThreadGroup.ON_SAMPLE_ERROR_CONTINUE);
PowerTableModel dataModel = new PowerTableModel(UltimateThreadGroupGui.columnIdentifiers, UltimateThreadGroupGui.columnClasses);
dataModel.addRow(new Integer[]{2, 4, 10, 60, 10});
dataModel.addRow(new Integer[]{3, 4, 10, 120, 10});
CollectionProperty prop = JMeterPluginsUtils.tableModelRowsToCollectionProperty(dataModel, UltimateThreadGroup.DATA_PROPERTY);
ultimateThreadGroup.setData(prop);
ultimateThreadGroup.setEnabled(setEnabled);
ultimateThreadGroup.setProperty(TestElement.TEST_CLASS, UltimateThreadGroup.class.getName());
ultimateThreadGroup.setProperty(TestElement.GUI_CLASS, UltimateThreadGroupGui.class.getName());
First you need to create a Loop Controller instance like:
LoopController loopController = new LoopController();
loopController.setLoops(1);
loopController.setFirst(true);
loopController.setProperty(TestElement.TEST_CLASS, LoopController.class.getName());
loopController.setProperty(TestElement.GUI_CLASS, LoopControlPanel.class.getName());
loopController.initialize();
Second you need to add the Loop Controller from the step 1 to your Ultimate Thread Group:
ultimateThreadGroup.setSamplerController(loopController);
More information:
Five Ways To Launch a JMeter Test without Using the JMeter GUI
jmeter-from-code
Going forward just compare the .jmx which you generate programatically with the one which is created by JMeter GUI, you will be able to see what fields, properties, etc. are missing
I have created a complex pipeline. In each stage I have called a job. I want to see the console output for each job in a stage in Jenkins. How to get it?
The object returned from a build step can be used to query the log like this:
pipeline {
agent any
stages {
stage('test') {
steps {
echo 'Building anotherJob and getting the log'
script {
def bRun = build 'anotherJob'
echo 'last 100 lines of BuildB'
for(String line : bRun.getRawBuild().getLog(100)){
echo line
}
}
}
}
}
}
The object returned from the build step is a RunWrapper class object. The getRawBuild() call is returning a Run object - there may be other options than reading the log line-by-line from the looks of this class. For this to work you need to either disable the pipeline sandbox or get script approvals for these methods:
method hudson.model.Run getLog int
method org.jenkinsci.plugins.workflow.support.steps.build.RunWrapper getRawBuild
If you are doing this for many builds, it would be worth putting some code in a pipeline shared library to do what you need or define a function in the pipeline.
I came across the Timeout module in Ruby, and wanted to test it out. I looked at their official source code at http://ruby-doc.org/stdlib-2.1.1/libdoc/timeout/rdoc/Timeout.html
Here is the code I had
require 'timeout'
require 'benchmark'
numbers = [*1..80]
Timeout::timeout(5) { numbers.combination(5).count }
=> 24040016
I did some benchmarking tests, and got the following.
10.828000 0.063000 10.891000 11.001676
According to the documentation, this method is supposed to return an exception if the block is not executed within 5 seconds. If it is executed within the time frame, it will return the result of the code block
For what it's worth, I've tried timeout with 1 second, instead of 5 seconds, and I still get returned the result of the code block.
Here is the official documentation
timeout(sec, klass=nil)
Performs an operation in a block, raising an error if it takes longer than sec seconds to complete.
sec: Number of seconds to wait for the block to terminate. Any number may be used,
including Floats to specify fractional seconds. A value of 0 or nil will execute the
block without any timeout.
klass: Exception Class to raise if the block fails to terminate in sec seconds. Omitting
will use the default, Timeout::Error
I am mystified as to why this doesn't work.
The problem is the way MRI (Matz's Ruby Implementation) thread scheduling works. MRI uses a GIL (Global Interpreter Lock), which in practice means only one thread is truly running at a time.
There are some exception, but for the majority of the time there is only one thread executing Ruby code at any one time.
Normally you do not notice this, even during heavy computations that consume 100% CPU, because the MRI keeps time-slicing the threads at regular intervals so that each thread gets a turn to run.
However there's one exception where time-slicing isn't active and that's when a Ruby thread is executing native C-code instead of Ruby code.
Now it so happens that Array#combination is implemented in pure C:
[1] pry(main)> show-source Array#combination
From: array.c (C Method):
static VALUE
rb_ary_combination(VALUE ary, VALUE num)
{
...
}
When we combine this knowledge with how Timeout.timeout is implemented we can start to get a clue of what is happening:
[7] pry(main)> show-source Timeout#timeout
From: /opt/ruby21/lib/ruby/2.1.0/timeout.rb # line 75:
75: def timeout(sec, klass = nil) #:yield: +sec+
76: return yield(sec) if sec == nil or sec.zero?
77: message = "execution expired"
78: e = Error
79: bl = proc do |exception|
80: begin
81: x = Thread.current
82: y = Thread.start {
83: begin
84: sleep sec
85: rescue => e
86: x.raise e
87: else
88: x.raise exception, message
89: end
90: }
91: return yield(sec)
92: ensure
93: if y
94: y.kill
95: y.join # make sure y is dead.
96: end
97: end
98: end
99: ...
1xx: end
Your code running Array.combination most likely actually starts executing even BEFORE the timeout thread runs sleep sec on line 84. Your code is launched on line 91 through yield(sec).
This means the order of execution actually becomes:
1: [thread 1] numbers.combination(5).count
# ...some time passes while the combinations are calculated ...
2: [thread 2] sleep 5 # <- The timeout thread starts running sleep
3: [thread 1] y.kill # <- The timeout thread is instantly killed
# and never times out.
In order to make sure the timeout thread starts first you can try this, which will most likely trigger the timeout exception this time:
Timeout::timeout(5) { Thread.pass; numbers.combination(5).count }
This is because by running Thread.pass you allow the MRI scheduler to start and run the code on line 82 before the native combination C-code executes. However even in this case the exception won't be triggered until combination exits because of the GIL.
There is no way around this unfortunately. You would have to use something like JRuby instead, which has real concurrent threads. Or you could run the combination calculation in a Process instead of a thread.
In my Scalding map reduce code, I want to log out certain steps that are happening so that I can debug the map-reduce jobs if something goes wrong.
How can I add logging to my scalding job?
E.g.
import com.twitter.scalding._
class WordCountJob(args: Args) extends Job(args) {
//LOG: Starting job at time blah..
TextLine( args("input") )
.read
.flatMap('line -> 'word) {
line: String =>
line.trim.toLowerCase.split("\\W+")
}
.groupBy('word) { group => group.size('count) }
}
.write(Tsv(args("output")))
//LOG - ending job at time...
}
Any logging framework will do. You can obviously also use println() - it will appear in your job's stdout log file in the job history of your hadoop cluster (in hdfs mode) or in your console (in local mode).
Also consider defining a trap with the addTrap() method for catching erroneous records.
I have a function in my_sup.erl like this:
init([ems_media_sup]) ->
{ok, {{simple_one_for_one, ?MAX_RESTART, ?MAX_TIME}, [
{ems_media_sup, {ems_media, start_link, []}, temporary, 2000, worker, [ems_media]}]
}};
But there is no function named start_link/1 in ems_media.erl, I want to know why there is no error when run
supervisor:start_link(?MODULE, [ems_media_sup])
So, How to know what happened next after call init([ems_media_sup])
That's because my_sup is of type simple_one_for_one - so it will only start child processes when explicitly asked to do so through supervisor:start_child/2.
If the supervisor had been of any other type (one_for_one, one_for_all or rest_for_one) it would have attempted to start all children in the child specification at startup, but a simple_one_for_one supervisor is for creating large numbers of children that only vary by their argument list, so in that case the child specification in the init function only plays the role of a template.