What is the space complexity of the following algorithm?
Given an array of n 32-bit signed integers, where each value is positive and less than two to the power of 30, negate all values in the array, then negate all values in the array a second time.
The question arose for me out of a discussion in the comment section here: Rearrange an array so that arr[i] becomes arr[arr[i]] with O(1) extra space
I am specifically interested in different opinions and definitions. I think subtle distinctions and definitions may be missing sometimes in some stackoverflow discussions on this subject.
Space complexity usually refers to the added space requirements for an algorithm, over and above the original data set itself. In this case, the original data set is the n 32-bit signed integers, so you're only concerned with extra storage above that.
In that case, that extra storage is is basically nothing, which translates to constant O(1) space complexity.
If you were required to create a separate array (negated, then negated again), it would be O(n) since the space required is in proportion to the original data set.
But, since you're doing the negations in-place, that's not the case.
You are confusing two different, though related, things: computer-theoretic space complexity of an algorithm, and practical memory requirements of a program.
Algorithms in computer science are normally not formulated in terms of integers of certain predefined size which is imposed by currently predominant computer architectures. If anything, they are parameterized by integer size. So "given an array of n 32-bit signed integers" should be replaced with "given an array of n k-bit signed integers".
Now if all integers of the input array are actually m<k bit wide, and all integers of the output array are also known to be m<k bit wide, and nothing else outside your algorithm imposes k bit wide integers, then sneaking k in the problem description is just cheating in order to make your complexity look better then it actually is. Likewise, saying "signed" if both input and output data is supposed to be positive is cheating.
Real-life programs don't have complexity, they have memory requirements. It is perfectly fine to say that your program does not use any extra memory if it only temporarily uses otherwise unused sign bits of your array elements. Just don't act surprised when one fine day you discover you have too large an array and you must pack it, so that it no longer has any unused bits. That is, you are reusing your algorithm in a different program with a different data representation, one that does not have any spare bits. Then you are forced to recall that the added space complexity of your algorithm is actually O(n).
Since you're interested in space complexity, the only relevant part of the question is:
"an array of n 32-bit signed integers"
From the above, the answer is pretty straightforward - O(n)
This whole blurb:
negate all values in the array, then negate all values in the array a
second time
only affects the time complexity, which seems like a poorly crafted distraction in a homework assignment.
Related
More specifically, is there an algorithm that can generate, deterministically, provided a seed, n integers from 0-(n-1), with no duplicates or missing numbers, in linear or sub-linear time and constant space?
All the answers i've found or seen online require linear space, as they need to store information about every digit in the sequence before they can give the first number at all. This becomes unreasonable memory usage in the millions/trillions of possible numbers, which is useful for random id generation. Is there an algorithm, say an iterative formula, which nicely spits out one number after another, without having to know any information about all the numbers before it or after it? Or am I living in a pipe dream right now?
I am currently reading the Introduction to Algorithms book and I have a question in regard to analyzing an algorithm:
The computational cost for merge sort is c lg n according to the book and it says that
We restrict c to be a constant so that the word size does not grow arbirarily (If the word size could grow arbitrarily, we could store huge amounts of data in one word and operate on it all in constant time)
I do not understand the meaning of "constant" here. Could anyone explain clearly what this means?
Computational complexity in the study of algorithms deals with finding function(s) which provide upper and lower bounds for how much time (or space) the algorithm requires. Recall basic algebra in high school where you learned about the general point-slope formula for a line? That formula, y = mx + b, provided two parameters, m (slope), and b (y intercept), which described a line completely. Those constants (m,b) described where the line lay, and a larger slope meant that the line was steeper.
Algorithmic complexity is just a way to describe the upper (and possibly lower) bounds for how long an algorithm takes to run (and/or how much space is required). With big-O (and big-Theta) notation, you are finding a function which provides upper (and lower) bounds for the algorithm costs. The constants are just shifting the curve, not changing the shape of the curve.
We restrict c to be a constant so that the word size does not grow arbirarily (If the word size could grow arbitrarily, we could store huge amounts of data in one word and operate on it all in constant time)
On a physical computer, there is some maximum size to a machine word. On a 32-bit system, that would be 32 bits, and on a 64-bit system, it's probably 64 bits. Operations on machine words are (usually) assumed to take time O(1) even though they operate on lots of bits at the same time. For example, if you use a bitwise OR or bitwise AND on a machine word, you can think of it as performing 32 or 64 parallel OR or AND operations in a single unit of time.
When trying to build a theoretical model for a computing system, it's necessary to assume an upper bound on the maximum size of a machine word. If you don't do this, then you could claim that you could perform operations like "compute the OR of n values in time O(1)" or "add together two arbitrary-precision numbers in time O(1)," operations that you can't actually do on a real computer. Therefore, there's usually an assumption that the machine word has some maximum size so that if you do want to compute the OR of n values, you can still do so, but you can't do it instantaneously by packing all the values into one machine word and performing a single assembly instruction to get the result.
Hope this helps!
This is a general question, which could be applicable to any given language like C,C++,Java etc.
I figured any way you implement it, you can't get more efficient than using 2 loops, which gives an efficiency of n^2.
for(i=0;i<n;i++)
for(j=0;j<n;j++)
a[i][j]=1;
I was asked this at an interview recently, and couldn't think of anything more efficient. All I got from the interviewer was that I could use recursion or convert the 2D array to a linked list to make it more efficient than n^2. Anyone know if this is possible, and if yes, how? At least theoretically, if not practically.
edit: The actual question gives me the coordinates of two cells, and I have to fill the paths taken by all possible shortest routes with 1.
eg, if i have a 5x5 matrix, and my two coordinates are (2,0) and (3,3), I'd have to fill:
(2,0)(2,1)(2,2)(2,3)
(3,0)(3,1)(3,2)(3,3)
while leaving the rest of the cells as they were.
It depends on what you mean. If the question is about plain arrays, meaning a sequence of contiguos memory locations and for initialization you mean putting a value in every memory location of this "matrix" then the answer is no, better than O(n*m) is not possible and we can prove it:
Let us assume that algorithm fill(A[n][m], init_val) is correct(i.e. fills all the memory locations of A) has complexity g(n,m) which is less than O(n*m)(meaning g(n,m) is not part of Ω(n*m)), then for big enough n and m we will have that g(n,m) < n*m = number of memory locations. Since filling a memory location requires one operation the algorithm fill can fill at most g(n,m) locations[actually half because it must also do at least an operation to "select" a different memory location, except if the hardware provides a combined operation] which is strictly less than n*m, which imply that the algorithm fill is not correct.
The same applies if filling k memory locations takes constant time, you simply have to choose bigger n and m values.
As other already suggested you can use other data-structures to avoid the O(n^2) initialization time. amit suggestion uses some kind of lazy-evaluation, which allows you to not initialize the array at all but do it only when you access the elements.
Note that this removes the Ω(n^2) cost at the beginning, but requires more complex operations to access the array's elements and also requires more memory.
It is not clear what your interviewer meant: converting an array into a linked-list requires Ω(L) time(where L is the length of the array), so simply converting the whole matrix into a linked-list would require Ω(n^2) time plus the real initialization. Using recursion does not help at all,
you simply end up in recurrences such as T(n) = 2T(n/2) + O(1) which would again result in no benefit for the asymptotic complexity.
As a general rule all algorithms have to scan at least all of their input, except it they have some form of knowledge beforehand(e.g. elements are sorted). In your case the space to scan is Θ(n^2) and thus every algorithm that wants to fill it must be at least Ω(n^2). Anything with less than this complexity either make some assumption(e.g. the memory contains the initializer value by default -> O(1)), or solves a different problem(e.g. use lazy arrays, or other data structures).
You can initialize an array in O(1), but it consumes triple the amount of space, and extra "work" for each element access in the matrix.
Since in practice, a matrix is a 1D array in memory, the same principles still hold.
The page describes how it can be done in details.
When you fill a 2d-array with same element, if you really will fill every element at least n^2 operations should be made.(given 2-d array is n*n).
The only way to decrease complexity is use a parallel programming approach.For example, given n processor, first input is is assigned the first row of the array.This is n operations. Then each processor Pi assigns array[i] of row k to array[i] of row k+1 for k=0 to n-1. This will be again O(n) since we have n processor working parallel.
If you really want to implement this approach you can look for free parallel programming environments like OpenMPI and mpich
I had this maybe stupid thought
since we have linear time sorting algorithms for constrained categroies like integers using counting sort, radix sort.
as in computer word, all categories of number types are finally encoded in byte sequences (which are to some extent similar with integers etc... ). is it able to state that we can do linear time sorting for all these numbers using those linear time sorting algorithms ?
Sure, although details vary from type to type. One simple example is IEEE-754 floating point values (both 32-bit and 64-bit), which can almost be sorted as though they were integers. (More specifically, they can be sorted as though they were sign-magnitude integers.) So a radix-sort would work fine.
For character strings, a not-uncommon technique when you have too many of them to fit in memory is to "bin" them by prefix, which is a variety of radix-sort.
For short bit-field values (like integers or, as above, floating point numbers), a left-to-right bit-at-a-time radix sort is really just a variant of quicksort, since it is basically just a way to find a plausible pivot. Unlike quicksort, it guarantees a finite recursion depth (32 in the case of 32-bit values). On the other hand, quicksort usually has a much smaller recursion depth, since log2 of the dataset size is usually a lot less than 32.
The main advantage of quicksort is that you can write the algorithm (STL style) without knowing anything about the datatype being sorted at all, other than how to call a function to compare two values. The same cannot be said of radix-sort; it's a lot harder to make a generic version.
Edited to add one important point:
It's very common to overemphasize the difference between O(n) and O(n log n). For very large n, they are different. But for most real-world non-Google-sized problems, log n is a small integer. It wouldn't make sense to use an O(n) algorithm which takes 100n seconds when there is an O(n log n) algorithm which takes 2n log2 n seconds, unless log n were greater than 50, which is to say that n were greater than 1,125,899,906,842,624.
No you cannot. if you have a piece of data represented by the bytes below:
11001100 00110011
(204) (51)
If you were to sort these using something like radix sort you would get:
00110011 11001100
(51) (204)
The only problem with this is that this is no longer the piece of data you wrote to the disk, it is a completely different piece of data that may not even mean anything at all(garbage).
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
What data-structure should I use to create my own “BigInteger” class?
Out of pure interest, I am trying to design a type that can hold an arbitrarily large integer. I want to support four basic operations [+, -, *, /] and optimise for speed of those operations.
I was thinking about some sort of a doubly-linked list and a bit flag to indicate positive or negative value. But I am not really sure how to add, for example, to large numbers of different sizes. Shall I walk to the last element of both numbers and then return back (using the second reverse pointer to the previous element).
123456789 //one large number
+ 123 //another large number with different size
Providing I can have an arbitrarily large memory, what is the best data structure for this task?
I would appreciate a small hint and any comments on worst-case complexity of the arithmetic operations. Thanks!
Usually one would go for an array/vector in this case, perhaps little-endian (lowest-significant word first). If you implement in-place operations, use a constant factor when growing the array, then amortized complexity for the reallocation remains O(1).
All operations should be doable in O(n) run time where n is the size of the input. EDIT: No, of course, multiplication and division will need more, this answer says it's at least O(N log N).
Just out of curiosity: Why are you reimplementing the wheel? Find here an implementation in Java. C# has one with .NET 4.0, too. While it might be a good exercise to implement this yourself (I remember myself doing it once), if you just need the functionality then it's there in many computing environments already.