I've just started programming in Ruby. This is my code:
numbers = Array (1..100)
start = Time.now
sum = numbers.inject(0) { |sum, num| sum + num }
puts "sum equals: #{sum}"
time_taken = (Time.now - start) * 1000 * 1000
puts "time taken to execute #{time_taken}"
Why does this code return a different execution time every time I run it?
I mean first time it was:
time taken to execute this: 20.593
second:
time taken to execute this: 27.184
third:
time taken to execute this: 22.516
Why does this code return a different execution time every time I run it?
Because your computer doesn't perform the exact same physical task every time it runs.
Maybe the data happened to be stored in slightly more/less efficient location on the RAM? Maybe your computer was too hot/cold, so performed differently. Maybe you also have 20 StackOverflow tabs open, which was using up system resources :D
Plus, in order to get a more accurate benchmark of how quickly some code runs, you need to decrease the error margin by performing the same operation lots of times.
I would recommend trying the benchmark-ips library if you wish to get some more reliable measurements of your code's performance.
Related
I am currently writing a numerical solver in Julia. I don't think the math behind it matters too much. It all boils down to the fact, that a specific operation is executed several times and uses a large percentage (~80%) of running time.
I tried to reduce it as much as possible and present you this piece of code, which can be saved as dummy.jl and then executed via include("dummy.jl") followed by dummy(10) (for compilation) and then dummy(1000).
function dummy(N::Int64)
A = rand(N,N)
#time timethis(A)
end
function timethis(A::Array{Float64,2})
dummyvariable = 0.0
for k=1:100 # just repeat a few times
for i=2:size(A)[1]-1
for j=2:size(A)[2]-1
dummyvariable += slopefit(A[i-1,j],A[i,j],A[i+1,j],2.0)
dummyvariable += slopefit(A[i,j-1],A[i,j],A[i,j+1],2.0)
end
end
end
println(dummyvariable)
end
#inline function minmod(x::Float64, y::Float64)
return sign(x) * max(0.0, min(abs(x),y*sign(x) ) );
end
#inline function slopefit(left::Float64,center::Float64,right::Float64,theta::Float64)
# arg=ccall((:minmod,"libminmod"),Float64,(Float64,Float64),0.5*(right-left),theta*(center-left));
# result=ccall((:minmod,"libminmod"),Float64,(Float64,Float64),theta*(right-center),arg);
# return result
tmp = minmod(0.5*(right-left),theta*(center-left));
return minmod(theta*(right-center),tmp);
#return 1.0
end
Here, timethis shall imitate the part of the code where I spend a lot of time. I notice, that slopefitis extremely expensive to execute.
For example, dummy(1000) takes roughly 4 seconds on my machine. If instead, slopefit would just always return 1 and not compute anything, the time goes down to one tenth of the overall time.
Now, obviously there is no free lunch.
I am aware, that this is simply a costly operation. But I would still try to optimize it as much as possible, given that a lot of time is spend in something that looks like one could optimize it easily as it is just a few lines of code.
So far, I tried to implement minmod and slopefit as C-functions and call them, however that just increased computing time (maybe I did it wrong).
So my question is, what possibilities do I have to optimize the call of slopefit?
Note, that in the actual code, the arguments of slopefit are not the ones mentioned here, but depend on conditional statements which makes everything hard to vectorize (if that would bring any performance gain I am not sure).
There are two levels of optimization I can think of.
First: the following implementation of minmod will be faster as it avoids branching (I understand this is the functionality you want):
#inline minmod(x::Float64, y::Float64) = ifelse(x<0, clamp(y, x, 0.0), clamp(y, 0.0, x))
Second: you can use #inbounds to speed up loop a bit:
#inbounds for i=2:size(A)[1]-1
I have large loop that contains abs([imaginary]).
It took me a lot of time to complete the program.
I tried multiple other ways to compute abs() such as when C is imaginary, (real(C)^2+imag(C)^2).^.5.
This result same as abs(C).
When I used tic,toc, (real(C)^2+imag(C)^2).^.5 was slightly faster. So I substituted and ran it again.
However profile shows that when I had abs() was much faster.
How can this happen and how can I make abs(C) process faster?
i take it form your comment that you are using large loops, matlab is not that efficent with those, example:
test = randn(10000000,2);
testC = complex(test(:,1),test(:,2));
%%vector
tic
foo = abs(testC);
toc
%%loop
bar = zeros(size(foo));
tic
for i=1:10000000
bar(i) = abs(testC(i));
end
toc
gives you something like
Elapsed time is 0.106635 seconds.
Elapsed time is 0.928885 seconds.
Thats why i would recommend to calculate abs() outside the loop. If replacing the loop in total is no option you can do so only in Parts. For exsample you could use your loop until you got all your complex numbers, end the loop, calc abs() then start a new loop with those results. Also if each iteration of your loop is independat of other iteration results you might want to look in parfor as an replacement for for-loops
Searching for an idea how to avoid using loop in my Matlab code, I found following comments under one question on SE:
The statement "for loops are slow in Matlab" is no longer generally true since Matlab...euhm, R2008a?
and
Have you tried to benchmark a for loop vs what you already have? sometimes it is faster than vectorized code...
So I would like to ask, is there commonly used way to test the speed of a process in Matlab? Can user see somewhere how much time the process takes or the only way is to extend the processes for several minutes in order to compare the times between each other?
The best tool for testing the performance of MATLAB code is Steve Eddins' timeit function, available here from the MATLAB Central File Exchange.
It handles many subtle issues related to benchmarking MATLAB code for you, such as:
ensuring that JIT compilation is used by wrapping the benchmarked code in a function
warming up the code
running the code several times and averaging
Update: As of release R2013b, timeit is part of core MATLAB.
Update: As of release R2016a, MATLAB also includes a performance testing framework that handles the above issues for you in a similar way to timeit.
You can use the profiler to assess how much time your functions, and the blocks of code within them, are taking.
>> profile on; % Starts the profiler
>> myfunctiontorun( ); % This can be a function, script or block of code
>> profile viewer; % Opens the viewer showing you how much time everything took
Viewer also clears the current profile data for next time.
Bear in mind, profile does tend to slow execution a bit, but I believe it does so in a uniform way across everything.
Obviously if your function is very quick, you might find you don't get reliable results so if you can run it many times or extend the computation that would improve matters.
If it's really simple stuff you're testing, you can also just time it using tic and toc:
>> tic; % Start the timer
>> myfunctionname( );
>> toc; % End the timer and display elapsed time
Also if you want multiple timers, you can assign them to variables:
>> mytimer = tic;
>> myfunctionname( );
>> toc(mytimer);
Finally, if you want to store the elapsed time instead of display it:
>> myresult = toc;
I think that I am right to state that many of us time Matlab by wrapping the block of code we're interested in between tic and toc. Furthermore, we take care to ensure that the total time is of the order of 10s of seconds (rather than 1s of seconds or 100s of seconds) and repeat it 3 - 5 times and take some measure of central tendency (such as the mean) and draw our conclusions from that.
If the piece of code takes less than, say 10s, then repeat it as many times as necessary to bring it into the range, being careful to avoid any impact of one iteration on the next. And if the code naturally takes 100s of seconds or longer, either spend longer on the testing or try it with artificially small input data to run more quickly.
In my experience it's not necessary to run programs for minutes to get data on average run time with acceptably low variance. If I run a program 5 times and one (or two) of the results is wildly different from the mean I'll re-run it.
Of course, if the code has any features which make its run time non-deterministic then it's a different matter.
I might have misunderstood something.. but I expected the running of a code in the editor and in the command window to be the same..
I have the following code in the editor
display(' ');
display('script');
fac = #(n) prod(1:n);
n=20;
display(['- use of anonymous function: fac = #(n) prod(1:n); with n = ' num2str(n)]);
tic; fac(n); toc;
display(['- use of build in MatLab function: factorial(); with n = ' num2str(n)]);
tic; factorial(n); toc;
before I run (is that called compiled or execute?) I quickly type in the same command in the command window. Which gives me this:
So all of a sudden the factorial function in the editor got a boost when it comes to performance.. What just happened?
First, the calculation you are trying to measure is too quick to get anything like an accurate reading. It is way below the measurement noise you get when using tic/toc because of other processes and activity on the system. To get more accurate measures repeat the calculation many times... (I would repeat the calculation so it takes 10-20 seconds).
Second, there is indeed a difference between code typed interactively and in a script. I think the JIT only runs for code run from a script or function m-file, where and not for stuff running in the interpreter (but I am struggling to find a reference for this now - I will add if I find it).
you mean the differences in running time between running in command window and running from script? I think they're caused by everything else also running on your computer, matlab's own memory management and a few other things...
it is explained in greater detail here: http://www.mathworks.com/matlabcentral/newsreader/view_thread/296850
Another method of measuring the time spent on cpu is the function cputime. It has however lower precision. So expand your test case in that it takes a longer time (n=2e7 eg factorial is capped at 171) to see results with cputime.
Conclusion: one swallow does not a summer make
Extend your testing with more cases.
n=200;
tic;
for ii=1:1e4
factorial(n);
end;
toc
I have some code that I ported from Matlab to R. I want to compare their performance.
However, I encountered a problem: Using system.time() in R, but I get different results for the same code. Is this supposed to happen? How do I compare it?
You'll get different results if you time your self running a 100m sprint too! The computer has lots of things going on that will slightly vary the time it takes to run your code.
The solution is to run the code many times. The R package benchmark is what you're looking for.
As #Justin said, the times will always vary. Especially the first time couple of times, since the garbage collection system hasn't adjusted itself to your specific use. It might be a good idea to restart R before measuring (and close other programs, ensure the system isn't scanning for viruses at this time etc)...
Note that if the measured time is small (fractions of a second), the relative error will be rather large, so try to adjust the problem so it takes at least a second.
The packages benchmark or rbenchmark can help.
...but I typically just do a for-loop around the problem and adjust it until it takes a second or so - and then I run it several times too.
Here's an example:
f <- function(x, y) {
sum <- 1
for (i in seq_along(x)) sum <- x[[i]] + y[[i]] * sum
sum
}
n <- 10000
x <- 1:n + 0.5
y <- -1:-n + 0.5
system.time(f(x,y)) # 0.02-0.03 secs
system.time(for(i in 1:100) f(x,y)) # 1.56-1.59 secs
...so calling it 100 times reduced the relative error a lot.