Slow array access in ruby? - ruby

I would like to present you this small (in this reduced form senseless) snippet of ruby code, which runs awfully slow:
MAX = 28123
M = 7000
abundant = Array.new(M) {|k| k}
checked = Array.new(MAX) {|k| false}
for a in 0..M-1 do
for b in a..M-1 do
checked[abundant[a] + abundant[b]] = true if abundant[a] + abundant[b] < MAX
end
end
It takes about 10 seconds to execute, whereas the equivalent C++ code runs in about 0.2 secs:
int main()
{
int abundant[M];
bool checked[MAX];
for (int n = 0; n < M; n++)
abundant[n] = n;
for (int n = 0; n < MAX; n++)
checked[n] = false;
for (int a = 0; a < M; a++)
for (int b = a; b < M; b++)
if (abundant[a] + abundant[b] < MAX)
checked[abundant[a] + abundant[b]] = true;
}
What am I doing wrong in the ruby implementation? I'm new to ruby - do I use any statement which is known to be thaaat slow?

Ruby is definitely a lot slower than C++, so there is not a lot you could do to make your code faster.
I believe the following code has the same behaviour and is a bit faster (±25%):
MAX = 28123
M = 7000
checked = Array.new(MAX) {|k| false}
(0..M - 1).each do |a|
(a..M - 1).each do |b|
checked[a + b] = true if a + b < MAX
end
end
Using #each makes no difference, but making fewer array access does. I believe one of the reasons C++ is so much faster is because it makes no boundary check for array accesses while Ruby has to do it.
Could you change the C++ version to use std::vector and .at() to access the array and then compare with the Ruby version?

Related

why this while loop performs worse than the other very similar while loop?

I am trying to write a variation of insertion sort. In my algorithm, the swapping of values doesn't happen when finding the correct place for item in hand. Instead, it uses a lookup table (an array containing "links" to smaller values in the main array at corresponding positions) to find the correct position of the item. When we are done with all n elements in the main array, we haven't actually changed any of the elements in the main array itself, but an array named smaller will contain the links to immediate smaller values at positions i, i+1, ... n in correspondence to every element i, i+1, ... n in the main array. Finally, we iterate through the array smaller, starting from the index where the largest value in the main array existed, and populate another empty array in backward direction to finally get the sorted sequence.
Somewhat hacky/verbose implementation of the algorithm just described:
public static int [] sort (int[] a) {
int length = a.length;
int sorted [] = new int [length];
int smaller [] = new int [length];
//debug helpers
long e = 0, t = 0;
int large = 0;
smaller[large] = -1;
here:
for (int i = 1; i < length; i++) {
if (a[i] > a[large]) {
smaller[i] = large;
large = i;
continue;
}
int prevLarge = large;
int temp = prevLarge;
long st = System.currentTimeMillis();
while (prevLarge > -1 && a[prevLarge] >= a[i]) {
e++;
if (smaller[prevLarge] == -1) {
smaller[i] = -1;
smaller[prevLarge] = i;
continue here;
}
temp = prevLarge;
prevLarge = smaller[prevLarge];
}
long et = System.currentTimeMillis();
t += (et - st);
smaller[i] = prevLarge;
smaller[temp] = i;
}
for (int i = length - 1; i >= 0; i--) {
sorted[i] = a[large];
large = smaller[large];
}
App.print("DevSort while loop execution: " + (e));
App.print("DevSort while loop time: " + (t));
return sorted;
}
The variables e and t contain the number of times the inner while loop is executed and total time taken to execute the while loop e times, respectively.
Here is a modified version of insertion sort:
public static int [] sort (int a[]) {
int n = a.length;
//debug helpers
long e = 0, t = 0;
for (int j = 1; j < n; j++) {
int key = a[j];
int i = j - 1;
long st = System.currentTimeMillis();
while ( (i > -1) && (a[i] >= key)) {
e++;
// simply crap
if (1 == 1) {
int x = 0;
int y = 1;
int z = 2;
}
a[i + 1] = a[i];
i--;
}
long et = System.currentTimeMillis();
t += (et - st);
a[i+1] = key;
}
App.print("InsertSort while loop execution: " + (e));
App.print("InsertSort while loop time: " + (t));
return a;
}
if block inside the while loop is introduced just to match the number of statements inside the while loop of my "hacky" algorithm. Note that two variables e and t are introduced also in the modified insertion sort.
The thing that's confusing is that even though the while loop of insertion sort runs exactly equal number of times the while loop inside my "hacky" algorithm, t for insertion sort is significantly smaller than t for my algorithm.
For a particular run, if n = 10,000:
Total time taken by insertion sort's while loop: 20ms
Total time taken by my algorithm's while loop: 98ms
if n = 100,000;
Total time taken by insertion sort's while loop: 1100ms
Total time taken by my algorithm's while loop: 25251ms
In fact, because the condition 1 == 1 is always true, insertion sort's if block inside the while loop must execute more often than the one inside while loop of my algorithm. Can someone explain what's going on?
Two arrays containing same elements in the same order are being sorted using each algorithm.

Algorithm for sorting an array using another array named gaps[]

This is a piece of code from a program. This code tends to sort the array
horses whose size is n. How does the array gap help in sorting the array horses?
int gaps[]={701,301,132,57,23,10,4,1};
for (k = 0; k < 8; k++)
for (i = gaps[k]; i < n; ++i)
{
temp = horses[i];
for (j = i; j >= gaps[k] && horses[j-gaps[k]] > temp; j -= gaps[k])
horses[j] = horses[j-gaps[k]];
horses[j] = temp;
}
gaps[] is an experimentally derived sequence for shell sort.
Take a look at the last entry in the wiki table for shell sort:
https://en.wikipedia.org/wiki/Shellsort#Gap_sequences
Wiki reference for this sequence:
https://oeis.org/A102549

Performance penalty using anonymous function in Julia

I have noticed that there is a performance penalty associated with using anonymous functions in Julia. To illustrate I have two implementations of quicksort (taken from the micro performance benchmarks in the Julia distribution). The first sorts in ascending order
function qsort!(a,lo,hi)
i, j = lo, hi
while i < hi
pivot = a[(lo+hi)>>>1]
while i <= j
while a[i] < pivot; i += 1; end
while pivot < a[j]; j -= 1; end
if i <= j
a[i], a[j] = a[j], a[i]
i, j = i+1, j-1
end
end
if lo < j; qsort!(a,lo,j); end
lo, j = i, hi
end
return a
end
The second takes an additional parameter: an anonymous function that can be used to specify ascending or descending sort, or comparison for more exotic types
function qsort_generic!(a,lo,hi,op=(x,y)->x<y)
i, j = lo, hi
while i < hi
pivot = a[(lo+hi)>>>1]
while i <= j
while op(a[i], pivot); i += 1; end
while op(pivot, a[j]); j -= 1; end
if i <= j
a[i], a[j] = a[j], a[i]
i, j = i+1, j-1
end
end
if lo < j; qsort_generic!(a,lo,j,op); end
lo, j = i, hi
end
return a
end
There is a significant performance penalty when sorting Arrays of Int64, with the default version an order of magnitude faster. Here are times for sorting arrays of length N in seconds:
N qsort_generic qsort
2048 0.00125 0.00018
4096 0.00278 0.00029
8192 0.00615 0.00061
16384 0.01184 0.00119
32768 0.04482 0.00247
65536 0.07773 0.00490
The question is: Is this due to limitations in the compiler that will be ironed out in time, or is there an idiomatic way to pass functors/anonymous functions that should be used in cases like this?
update From the answers it looks like this is something that will be fixed up in the compiler.
In the mean time, there were two suggested work arounds. Both approaches are fairly straightforward, though they do start to feel like the sort of jiggery-pokery that you have to use in C++ (though not on the same scale of awkward).
The first is the FastAnon package suggested by #Toivo Henningsson. I didn't try this approach, but it looks good.
I tried out the second method suggested by #simonstar, which gave me equivalent performance to the non-generic qsort! implementation:
abstract OrderingOp
immutable AscendingOp<:OrderingOp end
immutable DescendingOp<:OrderingOp end
evaluate(::AscendingOp, x, y) = x<y
evaluate(::DescendingOp, x, y) = x>y
function qsort_generic!(a,lo,hi,op=AscendingOp())
i, j = lo, hi
while i < hi
pivot = a[(lo+hi)>>>1]
while i <= j
while evaluate(op, a[i], pivot); i += 1; end
while evaluate(op, pivot, a[j]); j -= 1; end
if i <= j
a[i], a[j] = a[j], a[i]
i, j = i+1, j-1
end
end
if lo < j; qsort_generic!(a,lo,j,op); end
lo, j = i, hi
end
return a
end
Thanks everyone for the help.
It's a problem and will be fixed with an upcoming type system overhaul.
Update: This has now been fixed in the 0.5 version of Julia.
As others have noted, the code you've written is idiomatic Julia and will someday be fast, but the compiler isn't quite there yet. Besides using FastAnonymous, another option is to pass types instead of anonymous functions. For this pattern, you define an immutable with no fields and a method (let's call it evaluate) that accepts an instance of the type and some arguments. Your sorting function would then accept an op object instead of a function and call evaluate(op, x, y) instead of op(x, y). Because functions are specialized on their input types, there is no runtime overhead to the abstraction. This is the basis for reductions and specification of sort order in the standard library, as well as NumericExtensions.
For example:
immutable AscendingSort; end
evaluate(::AscendingSort, x, y) = x < y
function qsort_generic!(a,lo,hi,op=AscendingSort())
i, j = lo, hi
while i < hi
pivot = a[(lo+hi)>>>1]
while i <= j
while evaluate(op, a[i], pivot); i += 1; end
while evaluate(op, pivot, a[j]); j -= 1; end
if i <= j
a[i], a[j] = a[j], a[i]
i, j = i+1, j-1
end
end
if lo < j; qsort_generic!(a,lo,j,op); end
lo, j = i, hi
end
return a
end
Yes, it's due to limitations in the compiler, and there are plans to fix it, see e.g. this issue. In the meantime, the FastAnonymous package might provide a workaround.
The way that you have done it looks pretty idiomatic, there's unfortunately no magic trick that you are missing (except for possibly the FastAnonymous package).

how to skip a few iterations in a loop in Ruby?

Suppose I have the C code below
for(i = 0; i < 10; i++){
printf("Hello");
if(i == 5){
a[3] = a[2] * 2;
if(a[3] == b)
i = a[3]; //Skip to index = a[3]; depends on runtime value
}
}
How to convert to Ruby? I know we can skip one iteration using next, but I have to skip a few iterations depending on conditional value and I don't know how many iterations to skip before runtime?
Here is the code I am actually working on (as mentioned by Coreyward):
I am looking for "straight line" in the array that the values differs less than 0.1(less than 0.1 will considered as a "straight line"). The range has to be longer than 50 to be considered as a long "line". After I find the line range [a,b], i wanna skip the iterations to upper limit b so it would not start again from a+1, and it will start to find new "straight line" from b+1
for(i=0; i<arr.Length; i++){
if(arr[i] - arr[i + 50] < 0.1){
m = i; //m is the starting point
for(j=i; j<arr.Length; j++){ //this loop makes sure all values differs less than 0.1
if(arr[i] - arr[j] < 0.1){
n = j;
}else{
break;
}
}
if(n - m > 50){ //Found a line with range greater than 50, and store the starting point to line array
line[i] = m
}
i = n //Start new search from n
}
}
Your case isn't easily covered by typical ruby iterators, but ruby also has ordinary while loops which can completely cover c-for. the following is equivalent to your c for loop above.
i = 0;
while i < 10
puts "Hello"
if i == 5
a[3] = a[2] * 2
i = a[3] if a[3] == b
end
# in your code above, the for increment i++ will execute after assigning new i,
# though the comment "Skip to index = a[3]" indicates that may not be your intent
i += 1
end
Another way is using the enumerator class:
iter = (1..10).to_enum
while true
value = iter.next
puts "value is #{value.inspect}"
if value == 5
3.times {value = iter.next}
end
end
gives
value is 1
value is 2
value is 3
value is 4
value is 5
value is 9
value is 10
StopIteration: iteration reached at end
from (irb):15:in `next'
from (irb):15
from C:/Ruby19/bin/irb:12:in `<main>'

How to get all the combination in order

For example
Input
2,1,3
Output
1,1,1
1,1,2
1,1,3
2,1,1
2,1,2
2,1,3
If I correctly understand the question then this should work (code is in Haskell, will produce the results in a different order than the example)
combinations [] = []
combinations [x]
|x > 0 = [x]:combinations [(x-1)]
|otherwise = []
combinations (x:xs)
|x > 0 = (map (\c -> x:c) (combinations xs)) ++ combinations((x-1):xs)
|otherwise = []
Or this to get it in the same order as you gave (also just a nicer solution)
combinations' [x] = [[c]|c<-[1..x]]
combinations' (x:xs) = [c:d|c<-[1..x],d<-combinations' xs]
It will take me a bit to produce an answer in an "imperative" language (C, Java, etc). This is the kind of thing where functional languages shine.
Okay, so in Java.
Disclaimer: this code is more or less just a direct translation of the Haskell. It isn't clean, or the best way of doing things. I have not tested it, or really given it enough thought to make sure it is correct
public List<List<Integer>> combinations(List<Integer> workwith){
List<List<Integer>> d = new LinkedList<LinkedList<Integer>>();
if(workwith.size() == 1){
int max = workwith.get(0);
for(int i = 1; i=<max;i++){
List<Integer> toAdd = new LinkedList<Integer>();
toAdd.add(i);
d.add(toAdd);
}
return d
}
Integer max = workwith.remove(0);
List<List<Integer>> back = combinations(workwith);
for(int i = 1, i<=max;i++)
for(List<Integer> b: back){
List<Integer> toAdd = new LinkedList<Integer>();
toAdd.add(i);
toAdd.addAll(b);
d.add(toAdd)
}
}
return d;
}
a[] - is an input vector
int prod = 1;
for (int i = 0; i < a.size(); i++) prod *= a[i]; // find the count of lines in output
for (int i = 0; i < prod; i++){
vector<int> b; // vector of the current output
for (int j = a.size()-1; j >= 0; j--){ // for each output calculate its values
b[j] = i % a[j]; // each value will be between 0 and a[j]
i /= a[j];
}
for (int j = 0; j < a.size(); j++) // output it
cout << b[j] + 1 << " ";
cout << endl;
}
Not the most efficient way to do it, but here's a C implementation:
/* Assumes output is allocated with enough room for 'len' ints. */
/* Generates the 'num'-th combination in 'output'. */
void get_comb_number(int num, int len, int *input, int *output) {
int i;
for (i=num-i-1; i >= 0; --i) {
output[i] = (num % input[i]) + 1;
num /= input[i];
}
}
Then you can just loop from 0 to the product of the input (for the example above it would be 2*1*3 = 6), calling get_comb_number for each and print out each combination. The code is slightly inefficient because it has to call a function for each combination and has to do all the mods and divisions for each combination, but IMO the simplicity of the code makes up for it if you don't need the efficiency. Note that the combination number will overflow somewhat quickly, but assuming 32-bit ints, you'll be spending several minutes just generating all the combinations at that point and much much longer trying to print them all.

Resources