I have two lists:
ActiveExchange = []
ActiveStock = []
My code is very sequential in nature and I'm trying to optimize it:
for exchange in ActiveExchange:
for stock in Activestock:
...
...
...
However, it seems to be highly inefficient due to the for-loop. What can i do to make it run more efficiently?
Does it just seem unefficient or did you run it and it took too long? If you just think its unefficient - test it. And don't uptimize a not existing problem. You can use your time more efficiently.
Anything else will be case-dependent. Like:
if you are searching for one or tow items in ActiveStack for every elemant in ActiveExchange,make the ActiveStack a hash.
If you have if statements, and the statement depends on one variable only, make it the outter one. that way you opyimize brach prediction. This may be done automaticlly by the compiler.
If you want to manipulate/read all pairs - you cannot optimize it.
C/C++ and Python are have row-major arrays. If you are stepping through an array, list, vector, etc, be sure to do this:
for i in range(25):
for j in range(25):
A[j][i] *= 1
rather than:
for i in range(25):
for j in range(25):
A[i][j] *= 1
That can make a difference of 3-6 times in performance.
Related
The problem here is: I have to add two numbers and then do 3 operations with that sum. So either i add them both and put their value inside one variable and compute operations or i re-add like(a+b<c) while doing any operation. So which way is more memory efficient and fast?
val sum = k+d
if(sum<=b && sum>spend){
spend = sum
}
or,
if(k+d<=b && k+d>spend){
spend = k+d
}
It depends on the context. If k and d happen to be compile-time constants then the compiler can just replace k+d with the sum so it won't matter how many times you write k+d. Also if the second form would turn out to be faster and the sum variable would not escape the function (not returned or used as parameter to other functions) then the compiler could replace sum with k+d and again it would make no difference. Compiler optimization is usually pretty good so I don't think you should worry about it.
Since the first is clearer you should go for that, but you can always benchmark (make sure not to use compile-time constants then, as the sums will get optimized away) and tune if that piece of could would turn out to be a bottleneck.
I need to run many many tests of the form a<0 where a is a vector (a relatively short one). I am currently doing it with
all(v<0)
Is there a faster way?
Not sure which one will be faster (that may depend on the machine and Matlab version), but here are some alternatives to all(v<0):
~any(v>0)
nnz(v>=0)==0 %// Or ~nnz(v>=0)
sum(v>=0)==0 %// Or ~sum(v>=0)
isempty(find(v>0, 1)) %// Or isempty(find(v>0))
I think the issue is that the conditional is executed on all elements of the array first, then the condition is tested... That is, for the test "any(v<0)", matlab does the following I believe:
Step 1: compute v<0 for every element of v
Step 2: search through the results of step 1 for a true value
So even if the first element of v is less than zero, the conditional was first computed for all elements, hence wasting a lot of time. I think this is also true for any of the alternative solutions offered above.
I don't know of a faster way to do it easily, but wish I did. In some cases, breaking the array v up into smaller chunks and testing incrementally could speed things up, particularly if the condition is common. For example:
function result = anyLessThanZero(v);
w = v(:);
result = true;
for i=1:numel(w)
if ( w(i) < 0 )
return;
end
end
result = false;
end
but that can be very inefficient if the condition is rare. (If you were to really do this, there is probably a better way than I illustrate above to handle any condition, not just <0, but I show it this way to make it clear).
I am having trouble constructing my own nested selection statements (ifs) and repetition statements (for loops, whiles and do-whiles). I can understand what most simple repetition and selection statements are doing and although it takes me a bit longer to process what the nested statements are doing I can still get the general gist of the code (keeping count of the control variables and such). However, the real problem comes down to the construction of these statements, I just can't for the life of me construct my own statements that properly aligns with the pseudo-code.
I'm quite new to programming in general so I don't know if this is an experience thing or I just genuinely lack a very logical mind. It is VERY demoralising when it takes me a about an hour to complete 1 question in a book when I feel like it should just take a fraction of the time.
Can you people give me some pointers on how I can develop proper nested selection and repetition statements?
First of all, you need to understand this:
An if statement defines behavior for when **a decision is made**.
A loop (for, while, do-while) signifies **repetitive (iterative) work being done** (such as counting things).
Once you understand these, the next step, when confronted with a problem, is to try to break that problem up in small, manageable components:
i.e. decisions, that provide you with the possible code paths down the way,
and various work you need to do, much of which will end up being repetitive,
hence the need for one or more loops.
For instance, say we have this simple problem:
Given a positive number N, if N is even, count (display numbers) from
0(zero) to N in steps of 2, if N is odd, count from 0 to N in steps of
1.
First, let's break up this problem.
Step 1: Get the value of N. For this we don't need any decision, simply get it using the preferred method (from file, read console, etc.)
Step 2: Make a decision: is N odd or even?
Step 3: According to the decision made in Step 2, do work (count) - we will iterate from 0 to N, in steps of 1 or 2, depending on N's parity, and display the number at each step.
Now, we code:
//read N
int N;
cin<<N;
//make decision, get the 'step' value
int step=0;
if (N % 2 == 0) step = 2;
else step = 1;
//do the work
for (int i=0; i<=N; i=i+step)
{
cout >> i >> endl;
}
These principles, in my opinion, apply to all programming problems, although of course, usually it is not so simple to discern between concepts.
Actually, the complicated phase is usually the problem break-up. That is where you actually think.
Coding is just translating your thinking so the computer can understand you.
If I want to speed up the following code, How can I do that?
pcg <- foreach(boot.iter=1:boot.rep） %dopar% {
d.boot<-d[in.sample[[boot.iter]],]
*here in.sample[[boot.iter]] randomly generates 1000 row numbers.
I planned to split the overall tasks and send the seperated trials to each core. for example,
sub_task<-foreach(i=1:cores.use)%dopar%{
for (j in 1:trialsPerCore){
d.boot<-d[in.sample[[structure[i,j]]],]}}
*structure is a matrix which contains from 1 to boot.rep
But this one would not work, seems like we cannot use "for" loop inside the foreach? Also, the d.boot only keeps the last iteration of each core.
I tried to search online, I found the following code works,
sub_task<foreach(i=1:cores.use)%:%
foreach(j=1:trialsPerCore)%dopar%{
d.boot<-d[in.sample[[structure[i,j]]],]｝
But I think it is similar to my original function, and I do not think there is a great enhancement.
Do you guys have any suggestions?
Unless I'm missing something, it doesn't look like you're doing much if any computation in your foreach loop. You appear to be simply creating a list of matrices from d. That wouldn't benefit from parallel computing unless you can perform an operation on those matrices in your loop, and ideally return a relatively small result from that operation.
Although "chunking" often helps to execute parallel loops more efficiently, I don't think it's going to help here. The communication may be a little more efficient, but you're still just doing a lot of communication and essentially no computation.
Note that your attempt at chunking doesn't work because the for loop in the foreach loop is repeatedly assigning a matrix to the same variable. Then, the for loop itself returns a NULL as the body of the foreach loop, so that sub_task is a list of NULL's. An lapply would work much better in this context.
It will help a little to compute the values in the in.sample list in the foreach loop. That will decrease the amount of data that is auto-exported to each of the workers at the cost of a bit more computation on the workers, which is generally what you want to do in parallel loops. At the very least, you could iterate over in.sample directly:
pcg <- foreach(i=in.sample) %dopar% d[i,]
In this form, it's all the more obvious that there isn't enough computation to warrant parallel computing. If there isn't any real computation to perform, you're better off using lapply:
pcg <- lapply(in.sample, function(i) d[i,])
what is considered best-practice in this case?
for (i=0; i<array.length(); ++i)
or
for (i=array.length()-1; i>=0; --i)
assuming i don't want to iterate from a certain direction, but rather over the bare length of the array. also, i don't plan to alter the array's size in the loop body.
so, will the array.length() become constant during compilation? if not, then the second approach should be the one to go for..
I would do the first method, as that is much more readable, and I can look and see you are iterating over the loop. The second one took me a second :(.
array.length will remain constant so long as you arent modifying the array.
In most cases I would expect array.length() to be implemented in such a way that it is O(1), so it would not really impact on the loop's performance. If you are in doubt, or want to make sure it is a constant, just do so explicitly:
// JavaScript
var l = a.length;
for (var i=0; i<l; i++) {
// do something
}
I consider the reversed notation a "clever hack" that falls into the premature optimization category. It's harder to read, more error-prone and does not really provide a benefit over the alternative I suggest.
But since implementations of compilers/interpreters are vastly different and you do not say what language you refer to, it is hard to make an absolute statement about this. I would say unless this is in an absolutely time-critical section of code or otherwise measurably contributing to code running time, and your benchmark tests show that doing it differently provides a real benefit, I would stick to the code that's easier to understand and maintain.
Version 2 is broken and would iterate from one past end of array to 1. (Now corrected)
Stick with version 1. It's well recognised and doesn't leave the reader doing a double-take.
Version 1 is far more widely used and simpler to understand. Version 2 may occasionally be very slightly faster if the compiler doesn't optimize array.length() into a constant, but...insert your own premature optimization comment here
EDIT: as to whether array.length() will be optimized out, it will depend on the language. If the language uses arrays as "normal" objects or arrays can be dynamically sized, it will be just a method call and the compiler can't assume will return a consistent return value. But for languages in which arrays are a special case or object (or the compiler's just really smart...) the speed difference will probably be eliminated.
for (i=0; i<array.length(); ++i) is better for me, but for (i=array.length()-1; i>=0; --i) is fastest, because common processors fastest checking condition comparing to 0 to condition comparing to two varbiales.
array.lenght() is constant if you dont adding/erasing elements from this array under iterations.
Version 1 is quickest. for (i=0; i
For a for loop in my own code, though it's probably overkill, I got into the habit early on of writing my loops such that the length gets calculated exactly once, but the loop proceeds in the natural way:
int len = array.length();
for (int i=0; i<len; ++i) {
doSomething(array[i]);
}
These days, though, I prefer using "for-each" facilities where they're available and convenient; they make loops easier to read and foolproof. In C++ that would be something like:
std::for_each(array.begin(), array.end(), &doSomething);