I have a method where I tried to parallelise the calc using GPARS and calculate an aggregate boolean 'And' result across the calls. This method is wrapped as a #ActiveObject which will deliver the result as a dataflow - the code below has the original approach where I tried to store the aggregate using AtomicBoolean to protect it.
This didn't work (sometimes my tests would pass others they would fail) on the calculated 'end truth'. To fix this I changed from AtomicBoolean to a Agent(boolean) approach and I think it's 'fixed' it - at least my spock tests are continuously succeeding.
Where was my logic flawed trying to use AtomicBoolean to build the final result? It felt like it should work - but doesn't and I don't understand why.
Method below - I've put the original version, and the corrected version below
#ActiveMethod
def evaluateAllAsync () {
AtomicBoolean result = new AtomicBoolean(true)
GParsPool.withPool {
// do as parallel
conditions.eachParallel { condition ->
println "evalAllAsync-parallel intermediate result start value is ${result.get()} and condition with expression ${condition.expression} evaluated to ${condition.evaluate()}"
result.getAndSet(result.get() && condition.evaluate())
println "recalc bool value is now ${result.get()}"
}
}
println "evalAllAsync-parallel final result value is ${result.get()}"
result.get()
}
Fixed issue by using Agent form like this
#ActiveMethod
def evaluateAllAsync () {
def result = new Agent (true)
GParsPool.withPool {
// do as parallel
conditions.eachParallel { condition ->
println "evalAllAsync-parallel intermediate result start value is ${result.val} and condition with expression ${condition.expression} evaluated to ${condition.evaluate()}"
result << { def res = it && condition.evaluate(); println "start> $it and finish> $res"; updateValue(res)}
println "recalc bool value is now ${result.val}"
}
}
println "evalAllAsync-parallel final result value is ${result.val}"
result.val
}
I put debug println in here just so I could see what code was doing.
The version with the Agent to protect the bool aggregate value appears to be working.
Why doesn't the Atomic Boolean work?
Well, this is an issue in how you use AtomicBoolean. You always overwrite by force (getAndSet()) the value stored in it and ignore the possibility that other threads could have changed it while the current thread is busy 'evaluating'.
You perhaps wanted to use the compareAndSet() method instead:
def valueToUse = result.get()
result.compareAndSet(valueToUse, valueToUse && condition.evaluate())
Related
I'm trying to implement a "search as you type" pattern in Java.
The goal of the design is that no change gets lost but at the same time, the (time consuming) search operation should be able to abort early and try with the updated pattern.
Here is what I've come up so far (Java 8 pseudocode):
AtomicReference<String> patternRef
AtomicLong modificationCount
ReentrantLock busy;
Consumer<List<ResultType>> resultConsumer;
// This is called in a background thread every time the user presses a key
void search(String pattern) {
// Update the pattern
synchronized {
patternRef.set(pattern)
modificationCount.inc()
}
try {
if (!busy.tryLock()) {
// Another search is already running, let it handle the change
return;
}
// Get local copy of the pattern and modCount
synchronized {
String patternCopy = patternRef.get();
long modCount = modificationCount.get()
}
while (true) {
// Try the search. It will return false when modificationCount changes before the search is finished
boolean success = doSearch(patternCopy, modCount)
if (success) {
// Search completed before modCount was changed again
break
}
// Try again with new pattern+modCount
synchronized {
patternCopy = patternRef.get();
modCount = modificationCount.get()
}
}
} finally {
busy.unlock();
}
}
boolean doSearch(String pattern, long modCount)
... search database ...
if (modCount != modificationCount.get()) {
return false;
}
... prepare results ...
if (modCount != modificationCount.get()) {
return false;
}
resultConsumer.accept(result); // Consumer for the UI code to do something
return modCount == modificationCount.get();
}
Did I miss some important point? A race condition or something similar?
Is there something in Java 8 which would make the code above more simple?
The fundamental problem of this code can be summarized as “trying to achieve atomicity by multiple distinct atomic constructs”. The combination of multiple atomic constructs is not atomic and trying to reestablish atomicity leads to very complicated, usually broken, and inefficient code.
In your case, doSearch’s last check modCount == modificationCount.get() happens while still holding the lock. After that, another thread (or multiple other threads) could update the search string and mod count, followed by finding the lock occupied, hence, concluding that another search is running and will take care.
But that thread doesn’t care after that last modCount == modificationCount.get() check. The caller just does if (success) { break; }, followed by the finally { busy.unlock(); } and returns.
So the answer is, yes, you have potential race conditions.
So, instead of settling on two atomic variables, synchronized blocks, and a ReentrantLock, you should use one atomic construct, e.g. a single atomic variable:
final AtomicReference<String> patternRef = new AtomicReference<>();
Consumer<List<ResultType>> resultConsumer;
// This is called in a background thread every time the user presses a key
void search(String pattern) {
if(patternRef.getAndSet(pattern) != null) return;
// Try the search. doSearch will return false when not completed
while(!doSearch(pattern) || !patternRef.compareAndSet(pattern, null))
pattern = patternRef.get();
}
boolean doSearch(String pattern) {
//... search database ...
if(pattern != (Object)patternRef.get()) {
return false;
}
//... prepare results ...
if(pattern != (Object)patternRef.get()) {
return false;
}
resultConsumer.accept(result); // Consumer for the UI code to do something
return true;
}
Here, a value of null indicates that no search is running, so if a background thread sets this to a non-null value and finds the old value to be null (in an atomic operation), it knows it has to perform the actual search. After the search, it tries to set the reference to null again, using compareAndSet with the pattern used for the search. Thus, it can only succeed if it has not changed again. Otherwise, it will fetch the new value and repeat.
These two atomic updates are already sufficient to ensure that there is only a single search operation at a time while not missing an updated search pattern. The ability of doSearch to return early when it detects a change, is just a nice to have and not required by the caller’s loop.
Note that in this example, the check within doSearch has been reduced to a reference comparison (using a cast to Object to prevent compiler warnings), to demonstrate that it can be as cheap as the int comparison of your original approach. As long as no new string has been set, the reference will be the same.
But, in fact, you could also use a string comparison, i.e. if(!pattern.equals(patternRef.get())) { return false; } without a significant performance degradation. String comparison is not (necessarily) expensive in Java. The first thing, the implementation of String’s equals does, is a reference comparison. So if the string has not changed, it will return true immediately here. Otherwise, it will check the lengths then (unlike C strings, the length is known beforehand) and return false immediately on a mismatch. So in the typical scenario of the user typing another character or pressing backspace, the lengths will differ and the comparison bail out immediately.
I've encountered a wierd behaviour of a groovy script when running the whole suite. I have a script which orders my testcases alphabetically before they run and it seems to be running forever even when the whole test suite finishes.
After I click on it to see details and immediately go back to test suite it shows that it's finished and no longer running.
Is there something wrong with my script please? I don't see any infinite loops or anything like that. Is it just a bug in the ReadyAPI? Thanks for advice.
My sorting script:
ArrayList<String> testCaseList = new ArrayList<String>();
for (testCase in testRunner.testCase.testSuite.getTestCaseList()) {
testCaseList.add(testCase.getName());
}
testCaseList.sort();
int i = 0;
for (tCase in testCaseList) {
def curCase = testRunner.testCase.testSuite.getTestCaseByName(tCase);
curIndex = testRunner.testCase.testSuite.getIndexOfTestCase(curCase);
testRunner.testCase.testSuite.moveTestCase(curIndex,i-curIndex);
i++;
}
Currently, looks you have separate test case for sorting. But actually, that is not a valid test case of yours.
So, the first change to be made is that the script should be moved from test case to Setup Script of test suite.
Here is the Test Suite's Setup Script which does it alphabetical order. Should be paid special attention in case if the test case names have numbers in it, have to do natural order. Otherwise, it should be ok.
Please follow the in-line comments.
//Get the sorted order of the test case which is expected order
def newList = testSuite.testCaseList.name.sort()
log.info "Expected order of test cases: ${newList}"
//Get the current index of the test case
def getTestCaseIndex = { name -> testSuite.getIndexOfTestCase(testSuite.getTestCaseByName(name))}
//Closure definition and this is being called recursively to make the desired order
def rearrange
rearrange = {
def testCaseNames = testSuite.testCaseList.name
if (testCaseNames != newList) {
log.info testCaseNames
newList.eachWithIndex { tc, index ->
def existingIndex = getTestCaseIndex(tc)
if (index != existingIndex) {
testSuite.moveTestCase(index, existingIndex-index)
rearrange()
}
}
} else {
log.info 'All cases sorted'
}
}
//Call the closure
rearrange()
With that Setup Script, when test suite is executed, automatically the test cases are moved alphabetically. Hence, no separate test case required for just ordering.
Now, the suite gets executed with desired test cases and the current issue mentioned in the question should not be there at all.
Let's imagine the next piece of Kotlin code that performs some query to a database by means a JDBC connector:
var results : ResultSet
preparedStatement.clearParameters()
preparedStatement.setInt(1,value1);
preparedStatement.setInt(2,value2)
results = preparedStatement.executeQuery()
while(results.next()) {
// parse results
}
that compiles without problems. However, when I try to add thread safety to the access to the preparedStatement:
var results : ResultSet
synchronized(preparedStatement) {
preparedStatement.clearParameters()
preparedStatement.setInt(1,value1);
preparedStatement.setInt(2,value2)
results = preparedStatement.executeQuery()
}
while(results.next()) {
// parse results
}
... I got a "Variable 'results' must be initialized". It seems the synchronized block acts as a conditional block, but you can be sure that it will be executed once, before the while block.
I have implemented this same block in Java and I don't get the error. Is this a design/implementation error of Kotlin? Or does it have a good reason to behave like that?
synchronized is just an inline function and compiler doesn't know if lambda will be executed once, or even executed at all. Idiomatic way is to return value from lambda and assign it to the local:
val results =
synchronized(preparedStatement) {
preparedStatement.clearParameters()
preparedStatement.setInt(1,value1);
preparedStatement.setInt(2,value2)
preparedStatement.executeQuery()
}
I am following these examples of C# code. But I am little confused by the Pseudo Code comments all over the place.
For example:
public void addToHead(Object value)
// pre: value non-null
// post: adds element to head of list
{
SinglyLinkedListElement temp =
new SinglyLinkedListElement(value);
if (tail == null) {
tail = temp;
tail.setNext(tail);
}
else {
temp.setNext(tail.next());
tail.setNext(temp);
}
count++;
}
What does Pre and Post mean here?
I've never seen Post used here. I know what Post means in the context of the Web and HTML etc, but not in pure code.
"Pre" indicates an assumption made at the beginning of execution. In this case, it's indicating that the value passed in is assumed to be not null.
"Post" indicates an assumption made at the end of the execution, i.e. what the routine actually does. In this case, when the routine finishes a new element will have been added to the end of the list. If the routine modifies its parameters or has any other side effects, those modifications should be listed in the "Post" as well.
I am a Java programmer, and I am learning Ruby...
But I don't get where those blocks of codes can give me gain... like what's the purpose of passing a block as an argument ? why not have 2 specialized methods instead that can be reused ?
Why have some code in a block that cannot be reused ?
I would love some code examples...
Thanks for the help !
Consider some of the things you would use anonymous classes for in Java. e.g. often they are used for pluggable behaviour such as event listeners or to parametrize a method that has a general layout.
Imagine we want to write a method that takes a list and returns a new list containing the items from the given list for which a specified condition is true. In Java we would write an interface:
interface Condition {
boolean f(Object o);
}
and then we could write:
public List select(List list, Condition c) {
List result = new ArrayList();
for (Object item : list) {
if (c.f(item)) {
result.add(item);
}
}
return result;
}
and then if we wanted to select the even numbers from a list we could write:
List even = select(mylist, new Condition() {
public boolean f(Object o) {
return ((Integer) o) % 2 == 0;
}
});
To write the equivalent in Ruby it could be:
def select(list)
new_list = []
# note: I'm avoid using 'each' so as to not illustrate blocks
# using a method that needs a block
for item in list
# yield calls the block with the given parameters
new_list << item if yield(item)
end
return new_list
end
and then we could select the even numbers with simply
even = select(list) { |i| i % 2 == 0 }
Of course, this functionality is already built into Ruby so in practice you would just do
even = list.select { |i| i % 2 == 0 }
As another example, consider code to open a file. You could do:
f = open(somefile)
# work with the file
f.close
but you then need to think about putting your close in an ensure block in case an exception occurs whilst working with the file. Instead, you can do
open(somefile) do |f|
# work with the file here
# ruby will close it for us when the block terminates
end
The idea behind blocks is that it is a highly localized code where it is useful to have the definition at the call site. You can use an existing function as a block argument. Just pass it as an additional argument, and prefix it with an &