curious question on timing. When measuring wall clock time with any language such as python time.time() does the time include the CPU/System time.clock() time in it as well?
In Python, time.time() gives you the elapsed time (also known as wall time).That includes CPU time inasmuch as that's a subset of wall time but you cannot extract CPU time from time.time() itself.
For example, if your process runs for ten seconds but uses the CPU for only five of those seconds, the former includes the latter.
Related
How to convert the time difference given by the GPU timer while rendering into the equivalent CPU timing?
Let's say,
glGetQueryObjectuiv(query, GL_QUERY_RESULT, &elapsed_time) - will return the elapsed time for that query and I presume this elapsed time will correspond to GPU frequency.
How to get the corresponding CPU time which is equivalent to the GPU elapsed time?
It's a timer query - it returns a time in nanoseconds. Time doesn't change with frequency ...
Assume that a query is decomposed into a serial part and a parallel part. The serial part occupies 20% of the entire elapsed time, whereas the rest can be done in parallel.
Given that the one-processor elapsed time is 1 hour, what is the speed up if 10 processors are used? (For simplicity, you may assume that during the parallel processing of the parallel part the task is equally divided among all participating processors).
Even with n processors the serial part will take up the same share (0.2) of computation time as in the one-processor elapsed time (OPELT = 1 hour) scenario. The other 80% can be done in parallel, thus are divided by the number of processors available.
0.2*OPELT + (0.8*OPELT)/n
Speed up S(n) is then the ratio between the one-processor elapsed time and the n-processor elapsed time.
In detail to be found here.
The cputime (); returns the CPU time used by an Octave session. Here, a way is stated to measure the processing time of a code.
t = cputime;
...%your code
...%computing
...%when you are done
printf('Total cpu time: %f seconds\n', cputime-t);
In octave documentation we can use the etime(); function file, it returns the difference (in seconds) between two timestamps from the clock, like this:
t0 = clock ();
# many computations later...
elapsed_time = etime (clock (), t0);
What's the difference between the above two ways? The etime uses the hour of clock of my system? And what's a good way to collect the processing time of an code?
Your first code snippet with cputime measures the time your code was execuded on a CPU. The second snippet measures the wall clock time (btw: use tic and toc to measure this). The wall clock includes time where the CPU was executing some other threads or waiting for IO.
It is up to you what you want to measure. I normally use wall clock with tic/toc if I want to benchmark my code.
On single threaded apps the wall clock time is greater than or equal the cputime (because it includes the time the CPU was also processing some other threads). On multithreaded apps with multiple cores cputime can be greater than the wall clock time.
In trying to choose which indexing method to recommend, I tried to measeure the performance. However, the measurements confused me a lot. I ran this multiple times in different orders, but the measurements remain consistent.
Here is how I measured the performance:
for N = [10000 15000 100000 150000]
x = round(rand(N,1)*5)-2;
idx1 = x~=0;
idx2 = abs(x)>0;
tic
for t = 1:5000
idx1 = x~=0;
end
toc
tic
for t = 1:5000
idx2 = abs(x)>0;
end
toc
end
And this is the result:
Elapsed time is 0.203504 seconds.
Elapsed time is 0.230439 seconds.
Elapsed time is 0.319840 seconds.
Elapsed time is 0.352562 seconds.
Elapsed time is 2.118108 seconds. % This is the strange part
Elapsed time is 0.434818 seconds.
Elapsed time is 0.508882 seconds.
Elapsed time is 0.550144 seconds.
I checked and for values around 100000 this also happens, even at 50000 the strange measurements occur.
So my question is: Does anyone else experience this for a certain range, and what causes this? (Is it a bug?)
I think this is something to do with JIT (results below are using 2011b). Depending on system, version of Matlab, the size of variables, and exactly what is in the loop(s), it is not always faster to use JIT. This is related to the "warm-up" effect, where sometimes if you run an m-file more than once in a session it gets quicker after the first run, as the accelerator only has to compile some parts of the code once.
JIT on (feature accel on)
Elapsed time is 0.176765 seconds.
Elapsed time is 0.185301 seconds.
Elapsed time is 0.252631 seconds.
Elapsed time is 0.284415 seconds.
Elapsed time is 1.782446 seconds.
Elapsed time is 0.693508 seconds.
Elapsed time is 0.855005 seconds.
Elapsed time is 1.004955 seconds.
JIT off (feature accel off)
Elapsed time is 0.143924 seconds.
Elapsed time is 0.184360 seconds.
Elapsed time is 0.206405 seconds.
Elapsed time is 0.306424 seconds.
Elapsed time is 1.416654 seconds.
Elapsed time is 2.718846 seconds.
Elapsed time is 2.110420 seconds.
Elapsed time is 4.027782 seconds.
ETA, kinda interesting to see what happens if you use integers instead of doubles:
JIT on, same code but converted x using int8
Elapsed time is 0.202201 seconds.
Elapsed time is 0.192103 seconds.
Elapsed time is 0.294974 seconds.
Elapsed time is 0.296191 seconds.
Elapsed time is 2.001245 seconds.
Elapsed time is 2.038713 seconds.
Elapsed time is 0.870500 seconds.
Elapsed time is 0.898301 seconds.
JIT off, using int8
Elapsed time is 0.198611 seconds.
Elapsed time is 0.187589 seconds.
Elapsed time is 0.282775 seconds.
Elapsed time is 0.282938 seconds.
Elapsed time is 1.837561 seconds.
Elapsed time is 1.846766 seconds.
Elapsed time is 2.746034 seconds.
Elapsed time is 2.760067 seconds.
This may due to some automatic optimization matlab uses for its Basic Linear Algebra Subroutine.
Just like yours, my configuration (OSX 10.8.4, R2012a with default settings) takes longer to compute idx1 = x~=0 for x (10e5 elements) than x (11e5 elements). See the left panel of the figure where the processing time (y-axis) is measured for different vector size (x-axis). You will see a lower proceesing time for N>103000. In this panel, I also displayed the number of cores that were active during the calculation. You will see that there is no drop for the one-core configuration. It means that matlab do not optimize the execution of ~= when 1 core is active (no parallelization possible). Matlab enables some optimization routines when two conditions are met: multiple cores and a vector of sufficient size.
The right panel displays the results when feature('accel','on'/off') is set to off (doc). Here, only one core is active (1-core and 4-core are identical) and therefore no optimization is possible.
Finally, the function I used for activating/deactivating the cores is maxNumCompThreads. According to Loren Shure, maxNumCompThreads controls both the JIT and BLAS. Since feature('JIT','on'/'off') did not play a role in the performance, BLAS is the last option remaining.
I will leave the final sentence to Loren: "The main message here is that you should not generally need to use this function [maxNumCompThreads] at all! Why? Because we'd like to make MATLAB do the best job possible for you."
accel = {'on';'off'};
figure('Color','w');
N = 100000:1000:105000;
for ind_accel = 2:-1:1
eval(['feature(''accel'',''' accel{ind_accel} ''')']);
tElapsed = zeros(4,length(N));
for ind_core = 1:4
maxNumCompThreads(ind_core);
n_core = maxNumCompThreads;
for ii = 1:length(N)
fprintf('core asked: %d(true:%d) - N:%d\n',ind_core,n_core, ii);
x = round(rand(N(ii),1)*5)-2;
idx1 = x~=0;
tStart = tic;
for t = 1:5000
idx1 = x~=0;
end
tElapsed(ind_core,ii) = toc(tStart);
end
end
h2 = subplot(1,2,ind_accel);
plot(N, tElapsed,'-o','MarkerSize',10);
legend({('1':'4')'});
xlabel('Vector size','FontSize',14);
ylabel('Processing time','FontSize',14);
set(gca,'FontSize',14,'YLim',[0.2 0.7]);
title(['accel ' accel{ind_accel}]);
end
There are only two hard things in Computer Science: cache invalidation
and naming things.
-- Phil Karlton
My app is reporting CPU time, and people reasonably want to know how much time this is out of, so they can compute % CPU utilized. My question is, what's the name for the wall clock time times the number of CPUs?
If you add up the total user, system, idle, etc. time for a system, you get the total wall clock time, times the number of CPUs. What's a good name for that? According to Wikipedia, CPU time is:
CPU time (or CPU usage, process time) is the amount of time for which
a central processing unit (CPU) was used for processing instructions
of a computer program, as opposed to, for example, waiting for
input/output (I/O) operations.
"total time" suggests just wall clock time, and doesn't connote that over a 10 second span, a four-cpu system would have 40 seconds of "total time."
Total Wall Clock Time of all CPUs
Naming things is hard, why waste a good 'un once you've found it ?
Aggregate time: 15 of 40 seconds.