Given a is much larger than b, would
for (i = 0; i < a; i++)
for (k = 0 k < b; k++)
be faster than
for (i = 0; i < b; i++)
for (k = 0 k < a; k++)
It feels to me the former would be faster but I cannot seem to get my head around this.
Well it really depends on what your doing. It's hard to do runtime analysis without knowing what's being done. That being said, if your using this code to traverse through a large array, its more important to go through each column in each row rather than visa-versa.
[0][1][2]
[3][4][5]
[6][7][8]
is really [0][1][2][3][4][5][6][7][8] in memory.
Your computer's cache provides a greater advantage when memory access is close together, and going sequentially though memory rather than skipping through rows provide much more locality.
Starting a loop takes effort; there's the loop variable itself plus al the variables declared within the loop, which are all allocated memory and pushed onto the stack.
This means the fewer times you enter a loop the better, so loop over the smaller range in the outer loop.
Related
So I've just started learning algorithms and data structures, and I've read about Big O and how it portrays complexity of algorithms based on how the number of operations required scales
But what actually counts as an operation? In this bubble sort, does each iteration of the for loop count as an operation, or only when an if statement is triggered, or all of them?
And since there are so many different algorithms of all kinds, how do you immediately identify what would count as an "operation" happening in the algorithm's code?
function bubbleSort(array) {
for (let i = 0; i < array.length; i++) {
for (let j = 0; j < array.length; j++) {
if (array[j + 1] < array[j]) {
let tmp = array[j]
array[j] = array[j+1]
array[j+1] = tmp
}
}
}
return array
}
You can count anything as an operation that will execute within a constant amount of time, independent of input. In other words, operations that have a constant time complexity.
If we assume your input consists of fixed-size integers (like 32-bit, 64 bit), then all of the following can be considered such elementary operations:
i++
j < array.length
array[j + 1] < array[j]
let tmp = array[j]
...
But that also means you can take several of such operations together and still consider them an elementary operation. So this is also an elementary operation:
if (array[j + 1] < array[j]) {
let tmp = array[j]
array[j] = array[j+1]
array[j+1] = tmp
}
So, don't concentrate on breaking down operations into smaller operations, and those again into even smaller operations, when you are already certain that the larger operation is O(1).
Usually, everything that happens is a single operation. This is one of the reason we don't actually count the exact number of them, but instead use asymptotic notations (big O and big Theta).
However, sometimes you are interested about one kind of operation only. A common example is algorithms that use IO. Since IO is significantly more time consuming than anything happening on the CPU, you often just "count" the number of IO operations instead. In these cases, you often actually care about exact number of times an IO occurs, and can't use only asymptotic notations.
When you have nested for-loops where the amount of loops for the nested one changes each time, what is the easiest approach to analyze the total runtime? It's hard for me to conceptualize how to factor in the changing max value since I've only ever analyzed nested loops where the max was out of N, which led to a pretty simple O(n^2) runtime. Should I make a summation and use that?
For Example:
int val = 1;
for (int i = 0; i < n; i++) {
for (int j = 0; j < val; j++) {
val++;
}
}
My intuition tells me this is 2^n, but I have no practical way of really proving that
In general, to find the time complexity of loops, you need to find how many times they execute, as a function of the input. Sometimes it is straightforward, sometimes it is not. You may end up with complex mathematical expressions, and in some cases you may not be able to decide at all.
As for your example, your outer loop would clearly run exactly n times. Your inner loop, however, checks its loop condition j < val, which the first time is true because j = 0 and val = 1. Then, it increments val by 1 on each iteration so it will always be true that j < val. Therefore we notice that it is an infinite loop, and your program thus runs in O(∞).
(As a side note, in practice, depending on the language of implementation, eventually val may overflow and become smaller than j, which will cause the loop to finish. In this case, it only depends on the integer size you are using.)
So, what is the difference between given below two function in terms of performance and what is the time complexity of both function. It is doing exactly same task with two loop and single loop.
With TWO Loop.
RecipeModel returnRecipe(String? suggestion) {
for (int i = 0; i < _appData.recipeCategories!.length; i++) {
for (int j = 0; j < _appData.recipeCategories![i].recipes!.length; j++) {
if (_appData.recipeCategories![i].recipes![j].recipeName! ==
suggestion) {
return _appData.recipeCategories![i].recipes![j];
}
}
}
return recipe;
}
With Single loop
RecipeModel returnRecipe(String? suggestion) {
int recCategoriesLen = _appData.recipeCategories!.length;
int i = 0
for (int j = 0; j < _appData.recipeCategories![i].recipes!.length;) {
if (_appData.recipeCategories![i].recipes![j].recipeName! ==
suggestion) {
return _appData.recipeCategories![i].recipes![j];
}
j++
if (_appData.recipeCategories![i].recipes!.length == j && i < recCategoriesLen - 1) {
i++
j = 0
}
}
return recipe;
}
It's common for people, when first learning about big-O notation, to assume that big-O notation is calculated by looking at how many loops there are and then multiplying those loops together in some way. While that's often the case, the real guiding principle behind big-O analysis is to think through, conceptually, what it is that the loops are actually doing.
In your case, you have a 2D array of items indexed by i and j. The first version of the code explicitly enumerates all pairs of possible i's and j's, with the outer loop visiting all choices of i and the inner loop visiting all choices of j. The total work done is then proportional to the number of items visited.
The second loop does essentially the same thing, but less explicitly. Specifically, it still generates all possible combinations of i and j, except that there's just a single loop driving the changes to both i and j. Because you're still iterating over all choices, the amount of work done is likely to be pretty similar to what you started with. The actual performance will depend on what optimizations the compiler/interpreter does for you.
To actually reduce the amount of work you're doing, you'll need to find a fundamentally different strategy. Since you're asking the question "across all combinations of i and j, does this item exist?," you might want to store an auxiliary hash table (dictionary) that stores each recipe keyed by its name. That way, you don't need to loop over everything and can instead just do a dictionary lookup, which is much faster.
Does this function have good locality with respect to array a? Justify your answer by calculating the average miss and hit rates if the array size is 10 times larger than the cache.
int sum_array_cols(int a[M][N])
{
int i, j, sum = 0;
for (j = 0; j < N; j++)
for (i = 0; i < M; i++)
sum += a[i][j];
return sum;
}
Probably not.
In the inner loop you increment i what results in accessing memory locations separated by N*sizeof(int) bytes. The ratio of cache misses will depend on constant N and size of cache. The size (understood as sizeof a) will probably have no impact on cache effectiveness.
Moreover, the CPUs often do speculative prefetching of memory by tracing memory access pattern of your program. Therefore there might be little cache misses even though a small protion of cache is actually used. The exact answer must involve benchmarking on specific architecture.
To make thing more 'interesting', nowadays compiler will reorder the loop and improve locality automatically. Therefore I think that there is not enough data to provide a reliable answer to your question.
Would variables declared inside of a for loop that loops N times make the space complexity O(N) even though those variables fall out of scope each time the loop repeats?
for(var i = 0; i < N; i++){
var num = i + 5;
}
Would variables declared inside an O(N) for loop make the space complexity O(N)
No, since variables go out of scope at the end of every iteration, thus they are destroyed.
As a result the space complexity remains constant, i.e. O(1).
1 (fixed-size) variable that you change n times (which could include unallocating and reallocating it) is still just 1 variable, thus O(1) space.
But this may possibly be somewhat language-dependent - if some language (or compiler) decides to keep all of those earlier declarations of the variable in memory, that's going to be O(n), not O(1).
Consider, for example, two ways of doing this in C++:
for (int i = 0; i < N; i++)
int num = i + 5;
for (int i = 0; i < N; i++)
int* num = new int(i + 5);
In the former case, the variable can be reused and it will be O(1).
When you use new, that memory will not be automatically freed, so every iteration in the latter case will assign more memory instead of reusing the old (technically the pointer will get reused, but what it pointed to will remain), thus it will use O(n) space. Doing this is a terrible idea and will be a memory leak, but it's certainly possible.
(I'm not too sure what the C++ standard says about what compilers are or are not required to do in each case, this is mostly just meant to show that this type of in-loop assignment is not necessarily always O(1)).
No, it remains O(1) as explained below:
for(var i = 0; i < N; i++){
var num = i + 5; //allocate space for var `num`
} // release space acquired by `num`