I'm trying parallelise some bits of a code but I do not understand why the following functions main1() and main2() give different results using Julia's multi-threading:
a = rand(4,4);b = rand(4,4);c = rand(4,4);d = rand(4,4)
function main1(a,b,c,d)
L = zeros(2,2,16)
FF = zeros(2,2,16)
FT = zeros(2,2,16)
F = Array{Float32}(undef,2,2)
# L = Array{Array{Float32, 1}, 4}
for i = 1:4
for j = 1:4
ic = i + j*(i-1)
F[1,1] = a[i,j]
F[1,2] = b[i,j]
F[2,1] = c[i,j]
F[2,2] = d[i,j]
L[:,:,ic] .= F * F'
FF[:,:,ic] .= F
FT[:,:,ic] .= F'
end
end
return L,FF,FT
end
function main2(a,b,c,d)
L = zeros(2,2,16)
FF = zeros(2,2,16)
FT = zeros(2,2,16)
F = Array{Float32}(undef,2,2)
# L = Array{Array{Float32, 1}, 4}
Threads.#threads for i = 1:4
Threads.#threads for j = 1:4
ic = i + j*(i-1)
F[1,1] = a[i,j]
F[1,2] = b[i,j]
F[2,1] = c[i,j]
F[2,2] = d[i,j]
L[:,:,ic] .= F * F'
FF[:,:,ic] .= F
FT[:,:,ic] .= F'
end
end
return L,FF,FT
end
How could the parallelisation of main1() be properly fixed?
You cannot nest #threads loops so normally you should do:
Threads.#threads for u in vec(CartesianIndices((4,4)))
i,j = u.I
# your code goes here
end
However, in your code you get the same ic value for different pair of values of (i,j). In the main1 you are overwriting the same parts of L, FF, FT many times which is an obvious bug. Multi-threading will change the order the data is overwritten so it will yields different results. In conclusion, first fix main1 and than parallelize it.
If every letter in the following represents a name. What is the best way to sort them by how common the ancestors are?
A B C D
E F G H
I J K L
M N C D
O P C D
Q R C D
S T G H
U V G H
W J K L
X J K L
The result should be:
I J K L # Three names is more important that two names
W J K L
X J K L
A B C D # C D is repeated more than G H
M N C D
O P C D
Q R C D
E F G H
S T G H
U V G H
EDIT:
Names might have spaces in them (Double names).
Consider the following example where each letter represents a single word:
A B C D M
E F G H M
I J K L M
M N C D M
O P C D
Q R C D
S T G H
U V G H
W J K L
X J K L
The output should be:
A B C D M
M N C D M
I J K L M
E F G H M
W J K L
X J K L
O P C D
Q R C D
S T G H
U V G H
First count the number of occurrences for each chain. Then rank each name according to that count. Try this:
from collections import defaultdict
words = """A B C D
E F G H
I J K L
M N C D
O P C D
Q R C D
S T G H
U V G H
W J K L
X J K L"""
words = words.split('\n')
# Count ancestors
counters = defaultdict(lambda: defaultdict(lambda: 0))
for word in words:
parts = word.split()
while parts:
counters[len(parts)][tuple(parts)] += 1
parts.pop(0)
# Calculate tuple of ranks, used for sorting
ranks = {}
for word in words:
rank = []
parts = word.split()
while parts:
rank.append(counters[len(parts)][tuple(parts)])
parts.pop(0)
ranks[word] = tuple(rank)
# Sort by ancestor count, longest chain comes first
words.sort(key=lambda word: ranks[word], reverse=True)
print(words)
Here's how you could do it in Java - essentially the same method as #fafl's solution:
static List<Name> sortNames(String[] input)
{
List<Name> names = new ArrayList<>();
for (String name : input)
names.add(new Name(name));
Map<String, Integer> partCount = new HashMap<>();
for (Name name : names)
for (String part : name.parts)
partCount.merge(part, 1, Integer::sum);
for (Name name : names)
for (String part : name.parts)
name.counts.add(partCount.get(part));
Collections.sort(names, new Comparator<Name>()
{
public int compare(Name n1, Name n2)
{
for (int c, i = 0; i < n1.parts.size(); i++)
if ((c = Integer.compare(n2.counts.get(i), n1.counts.get(i))) != 0)
return c;
return 0;
}
});
return names;
}
static class Name
{
List<String> parts = new ArrayList<>();
List<Integer> counts = new ArrayList<>();
Name(String name)
{
List<String> s = Arrays.asList(name.split("\\s+"));
for (int i = 0; i < s.size(); i++)
parts.add(String.join(" ", s.subList(i, s.size())));
}
}
Test:
public static void main(String[] args)
{
String[] input = {
"A B C D",
"W J K L",
"E F G H",
"I J K L",
"M N C D",
"O P C D",
"Q R C D",
"S T G H",
"U V G H",
"X J K L" };
for (Name name : sortNames(input))
System.out.println(name.parts.get(0));
}
Output:
I J K L
W J K L
X J K L
A B C D
M N C D
O P C D
Q R C D
E F G H
S T G H
U V G H
I want to write simple insertion sort function using fold_left but I also want to pass function that will specify order in my sort fun.
What I don't know, is how to pass it to fold_left..
let rec insert f l e =
match l with
| [] -> [e]
| h :: t -> if f e h then h :: insert f t e else e :: l;;
let insertion_sort f l = List.fold_left insert f [] l;;
let less x y = x < y;;
let result = insertion_sort less [2 ; 5 ; 1 ; 9 ; 7 ; -2 ; 0 ; 124];;
This what I am talking about but fold_left doesn't accept that solution.
When I make specialization of sort function then it works just fine.
let insertLess = insert less;;
let insertion_sortLess l = List.fold_left insertLess [] l;;
let result = insertion_sortLess [2 ; 5 ; 1 ; 9 ; 7 ; -2 ; 0 ; 124];;
# val result : int list = [124; 9; 7; 5; 2; 1; 0; -2]
List.fold_left insert f ... will apply insert and f as separate arguments to List.fold_left. What you want is List.fold (insert f) ..., which will apply f to insert, and then the result of that to List.fold_left.
Edit: In addition, you don't need to define less. You can pass > as a function directly by surrounding it in parentheses: insertion_sort (<) ...
For example, I have
A = a*c*b + d*c*b + d*c*t
Where b and t are more like variables and a,c,d are more like parameters.
It would be displayed as
a b c + b c d + c d t
what I want is let it be displayed as
a c b + d c b + d c t
Where b and t are in the end.
Dirty trick:
a*c*b + d*c*b + d*c*t /. Thread[# -> (Interpretation[ToString[#], #] & /# #)] &#{b, t}
a c b + c d b + c d t
How do I encode the request in UTF-16? Here's what I have:
# Create Savon client
#client = Savon::Client.new do
wsdl.document = File.expand_path("account_list.wsdl", __FILE__)
end
# Set header encoding
#client.http.headers["Content-Type"] = "text/xml;charset=UTF-16"
# Setup ssl configuration
#client.http.auth.ssl.cert_key_file = "cert_key_file.pem"
#client.http.auth.ssl.cert_file = "cert_file.pem"
#client.http.auth.ssl.ca_cert_file = "ca_cert_file.pem"
#client.http.auth.ssl.verify_mode=:none
# Execute request
response = #client.request :account_list do
soap.body = {
:id => "18615618"
}
end
Here's the begging of what's sent, notice the encoding="UTF-8":
Content-Type: text/xml;charset=UTF-16, SOAPAction: "accountList", Content-Length: 888 <?xml version="1.0" encoding="UTF-8"?><env:Envelope xmlns:xsd="http://www.w3.org/2001/XMLSchema
Here's the error I get:
< s o a p : E n v e l o p e x m l n s : s o a p = " h t t p : / / s c h e m a s . x m l s o a p . o r g / s o a p / e n v e l o p e / " x m l n s : w s>d
< s o a p : B o d y >
< s o a p : F a u l t >
< f a u l t c o d e > s o a p : C l i e n t < / f a u l t c o d e >
< f a u l t s t r i n g > F a i l e d t o p r o c e s s S O A P r e q u e s t . S O A P b o d y n o t i n U T F - 1 6 .
< / f a u l t s t r i n g >
< d e t a i l >
< w s d l _ o p s : e r r o r > F a i l e d t o p r o c e s s S O A P r e q u e s t . S O A P b o d y n o t i n U T F - 1 6 .
< / w s d l _ o p s : e r r o r >
< / d e t a i l >
< / s o a p : F a u l t >
< / s o a p : B o d y >
< / s o a p : E n v e l o p e >
Savon currently only supports changing the XML directive tag via the integrated Builder-method:
response = #client.request(:account_list) do
soap.xml(:xml, :encoding => "UTF-16") { |xml| xml.id("18615618") }
end
You'll miss out a lot of XML-support by using this approach though. No SOAP envelope, no header or body:
<?xml version="1.0" encoding="UTF-16"?><id>18615618</id>
I'll use your ticket to come up with a better solution asap!