I am training in parallel calculations with Erlang. I have following code, that calculates pi number:
start(To)->
Self = self(),
Cores = erlang:system_info(schedulers_online),
{Step, Delta} = {1/To, round(To / Cores)},
Pids = lists:map(fun (El) ->
spawn(fun()->
Self ! {sum, calculate(from(El, Delta), to(El, Delta), Step)}
end)
end, lists:seq(1, Cores)),
lists:sum([receive {sum, S} -> S end || _ <- Pids ]) * Step.
Now, it goes magic to me
lists:sum([receive {sum, S} -> S end || _ <- Pids ])
Please, explain how this magic works?
lists:sum([receive {sum, S} -> S end || _ <- Pids ])
lets go part by part, first the function sum, which just sum all the elements from a list, i.e.:
1> lists:sum([1, 2, 3]).
6
lists:sum(SomeList) * Step. will just do a multiplication between the sum of the element of SomeList and the value in Step.
The interesting part is how the list is built, which is built by the code:
[receive {sum, S} -> S end || _ <- Pids ]
What you have there is a List Comprehension. Pids is an Erlang list which has the Process ID (PID) of each one of the process you created to process the pi number on:
Pids = lists:map(fun (El) ->
spawn(fun()->
Self ! {sum, calculate(from(El, Delta), to(El, Delta), Step)}
end)
end, lists:seq(1, Cores)).
So imagine you have 4 Cores, that function will create 4 process, probably with a list like [<0.36.0>, <0.38.0>, <0.40.0>, <0.42.0>], the main important idea, is that if you have 4 cores, you will be creating 4 processese, if you have 8 cores, 8 processes, and so on.
Each one of the processes, will call the function calculate and will send the result to Self as a message, which you can see here:
Self ! {sum, calculate(from(El, Delta), to(El, Delta), Step)}
So if you have 4 elements, you can try doing something like:
6> [X || X <- [1, 2, 3, 4]].
[1,2,3,4]
And there, you are building a list Compression with each element of the list. Now, lets say you don't care about the Elements of the list you are using to build your new list, you can do something like:
7> [1 || _ <- [1, 2, 3, 4]].
[1,1,1,1]
And you will be using the list compression as a for bucle, you don't really care about the elements of the list you are using to build your new list, which is exactly this case, where you are doing something like:
[1 || _ <- Pids ]
You don't really care what is the content of Pids, you are interested in the amount of elements the list Pids has.
So if the list Pids has 4 elements, you will be executing the receive function four times:
receive
{sum, S} ->
S
end
That receive function, will listen for any Message sent to the process, with the pattern {sum, S} which is the same patter you are sending with {sum, calculate(from(El, Delta), to(El, Delta), Step)}. The return value of the function would be S, which in that case, will be the result of calculate(from(El, Delta), to(El, Delta), Step).
At the end, you will have a list with 4 elements (again, assuming you have 4 cores) with the result calculated by the four processes you started.
Related
This week is my first time doing recursion. One of the problems I was able to solve was Fibonacci's sequence to the nth number; it wasn't hard after messing with it for 5 minutes.
However, I am having trouble understanding why this works with the current return statement.
return array if num == 2
If I push to array, it doesn't work, if I make a new variable sequence and push to that, it returns the correct answer. I am cool with that, but my base case says return array, not sequence. I initially pushed the sequence to the array, the result was not fibs sequence. I only solved the problem when I tried seeing what would happen if I pushed to the sequence array.
Instead of just making it work I was hoping someone could explain what was happening under the hood, what the stacks might be and how the problem works.
I understand recursion to an extent and somehow intuitively can make it work by assuming things, but I feel funny not actually knowing all the whys behind it.
def fib_seq(num)
return [0] if num == 1
return [] if num == 0
array = [0, 1]
return array if num <= 2
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
end
The code can be simplified a bit by removing the temporary array variable. It's a distraction. It also only applies when num == 2; num < 2 will be handled by the other base cases. num < 0 is illegal and should be handled by an error check.
I've also added in an explicit return. Explicit returns make it very obvious what's being returned and that helps understand recursion. In this case it's seq. ("Explicit returns are evil!" all the Ruby style people cry. Tough cookies. Good style isn't an absolute.)
def fib_seq(num)
# Error check
if num < 0 then
raise ArgumentError, "The number must be a positive integer"
end
# Terminating base cases
return [] if num == 0
return [0] if num == 1
return [0,1] if num == 2
# Recursion
seq = fib_seq(num - 1)
# The recursive function
seq << seq[-2] + seq[-1]
return seq
end
Now it's a bit clearer that return [0,1] if num == 2 is one of three base cases for the recursion. These are the terminating conditions which stops the recursion. But processing doesn't end there. The result isn't [0,1] because after that first return the stack has to unwind.
Let's walk through fib_seq(4).
fib_seq(4) calls fib_seq(3)
fib_seq(3) calls fib_seq(2)
fib_seq(2) returns `[0,1]`
We've reached the base case, now we need to unwind that stack of calls.
The call to fib_seq(3) picks up where it left off. seq returned from fib_seq(2) is [0,1]. It adds seq[-2] + seq[-1] onto the end and returns [0,1,1].
fib_seq(4) picks up where it left off. seq returned from fib_seq(3) is [0,1,1]. It adds seq[-2] + seq[-1] to the end and returns [0,1,1,2].
The stack is unwound, so we get back [0,1,1,2].
As you can see, the actual calculation happens backwards. f(n) = f(n-1) + f(n-2) and f(2) = [0,1]. It recurses down to f(2), the base case, then unwinds back up doing f(3) using the result of f(2), and f(4) using the result of f(3) and so on.
Recursive functions need to have an exit condition to prevent them from running forever. The main part of your recursive method is the following:
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
In Ruby, the last expression of a method is considered to be the return value of that method, so the lines above are equivalent to:
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
return seq
Let's run down what would happen if the method only contained these two lines, with num = 4:
call fib_seq(4)
call fib_seq(3)
call fib_seq(2)
call fib_seq(1)
call fib_seq(0)
call fib_seq(-1)
...
Obviously this results in an infinite loop, since we have no exit condition. We always call fib_seq again on the first line, so the code has no chance of ever reaching the return statement at the end. To fix the problem, let's add in these two lines at the beginning:
array = [0, 1]
return array if num <= 2
These can be simplified down to just:
return [0, 1] if num <= 2
Now let's see what happens when we call the method with num = 4:
call fib_seq(4)
4 > 2, exit condition not triggered, calling fib_seq(n - 1)
call fib_seq(3)
3 > 2, exit condition not triggered, calling fib_seq(n - 1)
call fib_seq(2)
2 == 2, exit condition triggered, returning [0, 1]!
fib_seq(2) returned with seq = [0, 1]
add 0 + 1 together, push new value to seq
seq is now [0, 1, 1]
return seq
fib_seq(3) returned with seq = [0, 1, 1]
add 1 + 1 together, push new value to seq
seq is now [0, 1, 1, 2]
return seq
FINAL RESULT: [0, 1, 1, 2]
So it looks like this method is working for values of num that are >= 2:
def fib_seq(num)
return [0, 1] if num <= 2
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
end
There is one bug left: num = 0 and num = 1 both return [0, 1]. Let's fix that:
def fib_seq(num)
return [] if num == 0
return [0] if num == 1
return [0, 1] if num == 2
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
end
Clean it up a little:
def fib_seq(num)
return [0, 1].first(num) if num <= 2
seq = fib_seq(num - 1)
seq << seq[-2] + seq[-1]
end
I always find it confusing when people mix imperative style mutations with functional style recursion – if you're going to do all reassignment and manual array seeking, why bother with using recursion as the looping mechanism? just use a loop.
That's not to say this program can't be expressed in a more functional way, tho. Here, we separate concerns of computing fibonacci numbers and generating a sequence – the result is an extremely easy-to-understand program
def fib n
def aux m, a, b
m == 0 ? a : aux(m - 1, b, a + b)
end
aux n, 0, 1
end
def fib_seq n
(0..n).map &method(:fib)
end
fib_seq 10
#=> [0, 1, 1, 2, 3, 5, 8, 13, 21, 34, 55]
And another way that's a bit more efficient for generating the sequence specifically – Below, I define an axuiliary function aux that utilizes 4 state variables to generate the sequence in a relatively straightforward way.
Note the difference with the input 10 - this one is closer to your proposed function where 0 returns [] despite the 0th fibonacci number is actually 0
def fib_seq n
def aux acc, m, a, b
m == 0 ? acc << a : aux(acc << a, m - 1, b, a + b)
end
case n
when 0; []
when 1; [0]
when 2; [0,1]
else; aux [0,1], n - 3, 1, 2
end
end
fib_seq 10
# => [0, 1, 1, 2, 3, 5, 8, 13, 21, 34]
I was doing polindrom_products exersice from Exercism and found intresting thing.
First version:
#doc """
Generates all palindrome products from an optionally given min factor (or 1) to a given max factor.
"""
#spec generate(non_neg_integer, non_neg_integer) :: map()
def generate(max_factor, min_factor \\ 1) do
palindromes = for a <- min_factor..max_factor,
b <- a..max_factor,
prod = a*b,
palindrome?(prod),
do: {prod, [a, b]}
Enum.reduce(palindromes, %{}, fn {prod, pair}, acc ->
Map.update(acc, prod, [pair], &(&1 ++ [pair]))
end)
end
def palindrome?(n) do
s = Integer.to_string(n)
s == String.reverse(s)
end
Second version:
#doc """
Generates all palindrome products from an optionally given min factor (or 1) to a given max factor.
"""
#spec generate(non_neg_integer, non_neg_integer) :: map()
def generate(max_factor, min_factor \\ 1) do
palindromes = for a <- min_factor..max_factor,
b <- a..max_factor,
prod = a*b,
"#{prod}" == String.reverse("#{prod}"),
do: {prod, [a, b]}
Enum.reduce(palindromes, %{}, fn {prod, pair}, acc ->
Map.update(acc, prod, [pair], &(&1 ++ [pair]))
end)
end
The first version faster than second about 6 times. Replace "#{prod}" == String.reverse("#{prod}") on Integer.to_string(prod) == String.reverse(Integer.to_string(prod)) give a gain.
For example, use this test:
test "smallest palindromes from triple digit factors" do
palindromes = Palindromes.generate(999, 100)
assert palindromes |> Dict.keys |> Enum.sort |> hd == 10201
assert palindromes[10201] == [[101, 101]]
end
First version executed in 0.7s, second version executed in 4s. I am using Elixir v1.1.1 and running code from Sublime Text on local machine.
What is the reason for that?
First I am always wary of timing result reported for Elixir programs. Unless you are benchmarking from within the code, you're including the startup time for the BEAM VM and for such small codes that can skew the results significantly.
However, the crux comes down to why is
"#{prod}" == String.reverse("#{prod}")
much slower than
s = Integer.to_string(n)
s == String.reverse(s)
Lets pull the first line apart
"#{prod}"
-> This is actually 2 functions, the inner function uses the to_string protocol to convert prod to a string and then do string interpolation. Since it uses a protocol, rather than the direct Integer.to_string
function it will likely be slower until you consolidate protocols.
You do this twice in the same line, so you're already up to 4 function calls. Add the String.reverse and == and that is 6 function calls.
The second version properly caches the result and only does 3 function
calls.
If you want to investigate these kinds of micro benchmarks further I highly recommend the benchfella library.
https://github.com/alco/benchfella
Let's say we have an array of age groups and an array of the number of people in each age group
For example:
Ages = ("1-13", "14-20", "21-30", "31-40", "41-50", "51+")
People = (1, 10, 21, 3, 2, 1)
I want to have an algorithm that combines these age groups with the following logic if there are fewer than 5 people in each group. The algorithm that I have so far does the following:
Start from the last element (e.g., "51+") can you combine it with the next group? (here "41-50") if yes add the numbers 1+2 and combine their labels. So we get the following
Ages = ("1-13", "14-20", "21-30", "31-40", "41+")
People = (1, 10, 21, 3, 3)
Take the last one again (here is "41+"). Can you combine it with the next group (31-40)? the answer is yes so we get:
Ages = ("1-13", "14-20", "21-30", "31+")
People = (1, 10, 21, 6)
since the group 31+ now has 6 members we cannot collapse it into the next group.
we cannot collapse "21-30" into the next one "14-20" either
"14-20" also has 10 people (>5) so we don't do anything on this either
for the first one ("1-13") since we have only one person and it is the last group we combine it with the next group "14-20" and get the following
Ages = ("1-20", "21-30", "31+")
People = (11, 21, 6)
I have an implementation of this algorithm that uses many flags to keep track of whether or not any data is changed and it makes a number of passes on the two arrays to finish this task.
My question is if you know any efficient way of doing the same thing? any data structure that can help? any algorithm that can help me do the same thing without doing too much bookkeeping would be great.
Update:
A radical example would be (5,1,5)
in the first pass it becomes (5,6) [collapsing the one on the right into the one in the middle]
then we have (5,6). We cannot touch 6 since it is larger than our threshold:5. so we go to the next one (which is element on the very left 5) since it is less than or equal to 5 and since it is the last one on the left we group it with the one on its right. so we finally get (11)
Here is an OCaml solution of a left-to-right merge algorithm:
let close_group acc cur_count cur_names =
(List.rev cur_names, cur_count) :: acc
let merge_small_groups mini l =
let acc, cur_count, cur_names =
List.fold_left (
fun (acc, cur_count, cur_names) (name, count) ->
if cur_count <= mini || count <= mini then
(acc, cur_count + count, name :: cur_names)
else
(close_group acc cur_count cur_names, count, [name])
) ([], 0, []) l
in
List.rev (close_group acc cur_count cur_names)
let input = [
"1-13", 1;
"14-20", 10;
"21-30", 21;
"31-40", 3;
"41-50", 2;
"51+", 1
]
let output = merge_small_groups 5 input
(* output = [(["1-13"; "14-20"], 11); (["21-30"; "31-40"; "41-50"; "51+"], 27)] *)
As you can see, the result of merging from left to right may not be what you want.
Depending on the goal, it may make more sense to merge the pair of consecutive elements whose sum is smallest and iterate until all counts are above the minimum of 5.
Here is my scala approach.
We start with two lists:
val people = List (1, 10, 21, 3, 2, 1)
val ages = List ("1-13", "14-20", "21-30", "31-40", "41-50", "51+")
and combine them to a kind of mapping:
val agegroup = ages.zip (people)
define a method to merge two Strings, describing an (open ended) interval. The first parameter is, if any, the one with the + in "51+".
/**
combine age-strings
a+ b-c => b+
a-b c-d => c-b
*/
def merge (xs: String, ys: String) = {
val xab = xs.split ("[+-]")
val yab = ys.split ("-")
if (xs.contains ("+")) yab(0) + "+" else
yab (0) + "-" + xab (1)
}
Here is the real work:
/**
reverse the list, combine groups < threshold.
*/
def remap (map: List [(String, Int)], threshold : Int) = {
def remap (mappings: List [(String, Int)]) : List [(String, Int)] = mappings match {
case Nil => Nil
case x :: Nil => x :: Nil
case x :: y :: xs => if (x._2 > threshold) x :: remap (y :: xs) else
remap ((merge (x._1, y._1), x._2 + y._2) :: xs) }
val nearly = (remap (map.reverse)).reverse
// check for first element
if (! nearly.isEmpty && nearly.length > 1 && nearly (0)._2 < threshold) {
val a = nearly (0)
val b = nearly (1)
val rest = nearly.tail.tail
(merge (b._1, a._1), a._2 + b._2) :: rest
} else nearly
}
and invocation
println (remap (agegroup, 5))
with result:
scala> println (remap (agegroup, 5))
List((1-20,11), (21-30,21), (31+,6))
The result is a list of pairs, age-group and membercount.
I guess the main part is easy to understand: There are 3 basic cases: an empty list, which can't be grouped, a list of one group, which is the solution itself, and more than one element.
If the first element (I reverse the list in the beginning, to start with the end) is bigger than 5 (6, whatever), yield it, and procede with the rest - if not, combine it with the second, and take this combined element and call it with the rest in a recursive way.
If 2 elements get combined, the merge-method for the strings is called.
The map is remapped, after reverting it, and the result reverted again. Now the first element has to be inspected and eventually combined.
We're done.
I think a good data structure would be a linked list of pairs, where each pair contains the age span and the count. Using that, you can easily walk the list, and join two pairs in O(1).
I have the following list:
["A";"AA";"ABC";"BCD";"B";"C"]
I am randomly extracting an element from the list. But the element I extract should be of size 3 only not lesser than 3.
I am trying to do this as follows:
let randomnum = (Random.int(List.length (list)));;
let rec code c =
if (String.length c) = 3 then c
else (code ((List.nth (list) (randomnum)))) ;;
print_string (code ( (List.nth (list) (randomnum)))) ;;
This works fine if randomly a string of length 3 is picked out from the list.
But the program does not terminate if a string of length < 3 is picked up.
I am trying to do a recursive call so that new code keeps getting picked up till we get one of length = 3.
I am unable to figure out why this is does not terminate. Nothing gets output by the print statement.
What you probably want to write is
let rec code list =
let n = Random.int (List.length list) in
let s = List.nth list in
if String.length s < 3 then code list else s
Note that, depending on the size of the list and the number of strings of size greater than 3, you might want to work directly on a list with only strings greater than 3:
let code list =
let list = List.filter (fun s -> String.length s >= 3) list in
match list with
| [] -> raise Not_found
| _ -> List.nth list (Random.int (List.length list))
This second function is better, as it always terminate, especially when there are no strings greater than 3.
You only pick a random number once. Say you pick 5. You just keep recursing with 5 over and over and over. You need to get a new random number.
For your code to terminate, it would be better to first filter the list for suitable elements, then take your random number:
let code list =
let suitables = List.filter (fun x -> String.length x = 3) list in
match List.length suitables with
| 0 -> raise Not_found (* no suitable elements at all! *)
| len -> List.nth suitables (Random.int len)
Otherwise your code would take very long to terminate on a large list of elements with size <> 3; or worse on a list with no element of size 3, it would not terminate at all!
I need to make a random list of permutations. The elements can be anything but assume that they are the integers 0 through x-1. I want to make y lists, each containing z elements. The rules are that no list may contain the same element twice and that over all the lists, the number of times each elements is used is the same (or as close as possible). For instance, if my elements are 0,1,2,3, y is 6, and z is 2, then one possible solution is:
0,3
1,2
3,0
2,1
0,1
2,3
Each row has only unique elements and no element has been used more than 3 times. If y were 7, then 2 elements would be used 4 times, the rest 3.
This could be improved, but it seems to do the job (Python):
import math, random
def get_pool(items, y, z):
slots = y*z
use_each_times = slots/len(items)
exceptions = slots - use_each_times*len(items)
if (use_each_times > y or
exceptions > 0 and use_each_times+1 > y):
raise Exception("Impossible.")
pool = {}
for n in items:
pool[n] = use_each_times
for n in random.sample(items, exceptions):
pool[n] += 1
return pool
def rebalance(ret, pool, z):
max_item = None
max_times = None
for item, times in pool.items():
if times > max_times:
max_item = item
max_times = times
next, times = max_item, max_times
candidates = []
for i in range(len(ret)):
item = ret[i]
if next not in item:
candidates.append( (item, i) )
swap, swap_index = random.choice(candidates)
swapi = []
for i in range(len(swap)):
if swap[i] not in pool:
swapi.append( (swap[i], i) )
which, i = random.choice(swapi)
pool[next] -= 1
pool[swap[i]] = 1
swap[i] = next
ret[swap_index] = swap
def plist(items, y, z):
pool = get_pool(items, y, z)
ret = []
while len(pool.keys()) > 0:
while len(pool.keys()) < z:
rebalance(ret, pool, z)
selections = random.sample(pool.keys(), z)
for i in selections:
pool[i] -= 1
if pool[i] == 0:
del pool[i]
ret.append( selections )
return ret
print plist([0,1,2,3], 6, 2)
Ok, one way to approximate that:
1 - shuffle your list
2 - take the y first elements to form the next row
4 - repeat (2) as long as you have numbers in the list
5 - if you don't have enough numbers to finish the list, reshuffle the original list and take the missing elements, making sure you don't retake numbers.
6 - Start over at step (2) as long as you need rows
I think this should be as random as you can make it and will for sure follow your criteria. Plus, you have very little tests for duplicate elements.
First, you can always randomly sort the list in the end, so let's not worry about making "random permutations" (hard); and just worry about 1) making permutations (easy) and 2) randomizing them (easy).
If you want "truly" random groups, you have to accept that randomization by nature doesn't really allow for the constraint of "even distribution" of results -- you may get that or you may get a run of similar-looking ones. If you really want even distribution, first make the sets evenly distributed, and then randomize them as a group.
Do you have to use each element in the set x evenly? It's not clear from the rules that I couldn't just make the following interpretation:
Note the following: "over all the lists, the number of times each elements is used is the same (or as close as possible)"
Based on this criteria, and the rule that z < x*, I postulate that you can simply enumerate all the items over all the lists. So you automatically make y list of the items enumerated to position z. Your example doesn't fulfill the rule above as closely as my version will. Using your example of x={0,1,2,3} y=6 and z=2, I get:
0,1 0,1 0,1 0,1 0,1 0,1
Now I didn't use 2 or 3, but you didn't say I had to use them all. If I had to use them all and I don't care to be able to prove that I am "as close as possible" to even usage, I would just enumerate across all the items through the lists, like this:
0,1 2,3 0,1 2,3 0,1 2,3
Finally, suppose I really do have to use all the elements. To calculate how many times each element can repeat, I just take (y*z)/(count of x). That way, I don't have to sit and worry about how to divide up the items in the list. If there is a remainder, or the result is less than 1, then I know that I will not get an exact number of repeats, so in those cases, it doesn't much matter to try to waste computational energy to make it perfect. I contend that the fastest result is still to just enumerate as above, and use the calculation here to show why either a perfect result was or wasn't achieved. A fancy algorithm to extract from this calculation how many positions will be duplicates could be achieved, but "it's too long to fit here in the margin".
*Each list has the same z number of elements, so it will be impossible to make lists where z is greater than x and still fulfill the rule that no list may contain the same element twice. Therefore, this rule demands that z cannot be greater than x.
Based on new details in the comments, the solution may simply be an implementation of a standard random permutation generation algorithm. There is a lengthy discussion of random permutation generation algorithms here:
http://www.techuser.net/randpermgen.html
(From Google search: random permutation generation)
This works in Ruby:
# list is the elements to be permuted
# y is the number of results desired
# z is the number of elements per result
# equalizer keeps track of who got used how many times
def constrained_permutations list, y, z
list.uniq! # Never trust the user. We want no repetitions.
equalizer = {}
list.each { |element| equalizer[element] = 0 }
results = []
# Do this until we get as many results as desired
while results.size < y
pool = []
puts pool
least_used = equalizer.each_value.min
# Find how used the least used element was
while pool.size < z
# Do this until we have enough elements in this resultset
element = nil
while element.nil?
# If we run out of "least used elements", then we need to increment
# our definition of "least used" by 1 and keep going.
element = list.shuffle.find do |x|
!pool.include?(x) && equalizer[x] == least_used
end
least_used += 1 if element.nil?
end
equalizer[element] += 1
# This element has now been used one more time.
pool << element
end
results << pool
end
return results
end
Sample usage:
constrained_permutations [0,1,2,3,4,5,6], 6, 2
=> [[4, 0], [1, 3], [2, 5], [6, 0], [2, 5], [3, 6]]
constrained_permutations [0,1,2,3,4,5,6], 6, 2
=> [[4, 5], [6, 3], [0, 2], [1, 6], [5, 4], [3, 0]]
enter code here
http://en.wikipedia.org/wiki/Fisher-Yates_shuffle