What is the main difference between divide and conquer and dynamic programming? If we take an example merge sort is basically solved by divide and conquer which uses recursion . Dynamic programming is also based on recursion than why not Merge sort considered to be an example of dynamic programming?
The two are similar in that they both break up the problem into small problems and solve those. However, in divide and conquer, the subproblems are independent, while in dynamic programming, the subproblems are dependent. Both requiring recombining the subproblems in some way, but the distinction comes from whether or not the subproblems relate to other subproblems (of the same "level")
D&C example: Mergesort
In Mergesort, you break the sorting into a lot of little "sub-sorts", that is instead of sorting 100 items, you sort 50, then 25, etc. However, after breaking the original into (for example) 4 "sub-sorts", it doesn't matter which you do first; order is irrelevant because they are independent. All that matter is that they eventually get done. As such, each time, you get an entirely independent problem with its own right answer.
DP example: Recursive Fibonacci
Though there are sub-problems, each is directly built on top of the other. If you want the 10th digit, you have to the solve the problems building up to that (1+2, 2+3, etc) in a specific order. As such, they are not independent.
D&C is used when sub-problems are independent. Dynamic programming needed when a recursive function repeats same recursive calls.
Take fibonacci recurrence: f(n)=f(n-1)+f(n-2)
For example:
f(8) = f(7) + f(6)
= ( f(6) + f(5) ) + f(6)
As you can see f(6) will be calculated twice. From the recurrence relation, obviously there are too many repeating values. It's better to memorize these values rather than calculating over and over again. Most important thing in dp is memorizing these calculated values. If you look at dp problems generally an array or a matrix is used for preventing repetitive calculations.
Comparing to dp, d&c generally divides problem into independent sub-problems and memorizing any value is not necessary.
So I would say that D&C is a bigger concept and DP is special kind of D&C. Specifically, when you found that your subproblems need to share some calculations of same smaller subproblem, you may not want them to calculate the same things again and again, you cache the intermediate results to speed up time, that comes the DP. So, essentially, I would way, DP is a fast version of D&C.
Related
Let’s use as an example the problem LeetCode 322. Coin Change
I know it is best solved by using Dynamic Programming, but I want to focus on my Brute Force solution:
class Solution:
def coinChange(self, coins: List[int], amount: int) -> int:
curr_min = float('inf')
def helper(amount):
nonlocal curr_min
if amount < 0:
return float('inf')
if amount == 0:
return 0
for coin in coins:
curr_min = min(curr_min, helper(amount-coin) + 1)
return curr_min
ans = helper(amount)
return -1 if ans == float('inf') else ans
The Recursion Tree looks like: Recursion Tree
I can say it is Divide and Conquer: We are dividing the problem into smaller sub-problems, solving individually and using those individual results to construct the result for the original problem.
I can also say it is Backtracking: we are enumerating all combinations of coin frequencies which satisfy the constraints.
I know both are implemented via Recursion, but I would like to know which paradigm my Brute Force solution belongs to: Divide and Conquer or Backtracking.
A complication in categorizing your algorithm is that there aren’t clear, well-defined boundaries between different classes of algorithms and different people might have slightly different definitions in mind.
For example, generally speaking, divide-and-conquer algorithms involve breaking the problem apart into non-overlapping subproblems. (See, for example, mergesort, quicksort, binary search, closest pair of points, etc.) In that sense, your algorithm doesn’t nicely map onto the divide-and-conquer paradigm, since the subproblems you’re considering involve some degree of overlap in the subproblems they solve. (Then again, not all divide-and-conquer algorithms have this property. See, for example, stoogesort.)
Similarly, backtracking algorithms usually, but not always, work by committing to a decision, recursively searching to see whether a solution exists given that decision, then unwinding the choice if it turns out not to lead to a solution. Your algorithm doesn’t have this property, since it explores all options and then takes the best. (When I teach intro programming, I usually classify algorithms this way. But my colleagues sometimes describe what you’re doing as backtracking!)
I would classify your algorithm as belonging to a different family of exhaustive search. The algorithm you’ve proposed essentially works by enumerating all possible ways of making change, then returning the one that uses the fewest coins. Exhaustive search algorithms are ones that work by trying all possible options and returning the best, and I think that’s the best way of classifying your strategy.
To me this doesn't fit with either paradigm.
Backtracking to me is associated with reaching a point where the candidate cannot be further developed, but here we develop it to it's end, infinity, and we don't throw it away, we use it in comparisons.
Divide and conquer I associate with a division into a relatively small number of candidate groups (the classic example is two, like binary search). To call each path in a recursion a group for the sake of Divide and Conquer would lose the latter's meaning.
The most practical answer is it doesn't matter.
Safest answer recursion. My best interpretation is that its backtracking.
I think the options here are recursion, backtracking, divide-and-conquer, and dynamic programming.
Recursion being the most general and encapsulating of backtracking, D&C, and DP. If indeed it has backtracking and D&C algorithms then recursion would be the best answer as it contains both.
In Skiena's ADM (Section 5.3.1), it says:
A typical divide-and-conquer algorithm breaks a given problem into a smaller pieces, each of which is of size n/b.
By this interpretation is doesn't meet the as we divide our solution by coins and each coin amount being a different size.
In Erickson's Algorithms (section 1.6), it says:
divide and conquer:
Divide the given instance of the problem into several independent smaller instances of exactly the same problem.
So in this case, according to the recursion tree, are not always independent (they overlap).
Which leaves backtracking. Erickson defines the 'recursive strategy' as:
A backtracking algorithm tries to construct a solution to a computational problem incrementally, one small piece at a time.
Which seems general enough to fit all DP problems under it. The provided code can be said it backtracks when a solution path fails.
Additionally, according to Wikipedia:
It is often the most convenient technique for parsing, for the knapsack problem and other combinatorial optimization problems.
Coin Change being an Unbounded Knapsack type problem, then it fits into the description of backtracking.
I am thinking about QuickSort specifically: each subproblem is about half the size of the main problem -- is it that the subproblems, including the overhead of dividing the main problem and then recombining the results of the solved subproblems, tend to be solved in less than half the time of the main problem?
I understand that solving the subproblems is parallel is also a way that of speeding up the algorithm but most of the discussions of QuickSort don't mention parallelism.
Not really. If you analyze QuickSort you'll find its efficiency is actually a function of the quality of the pivot, and there are always cases which either by bad luck or deliberate maliciousness will run as O(N^2).
But in general, the complexity of a D&C algorithm is due to a combination of the cost of solving the subproblems and of merging the solutions. Coming back to your question, if the cost of merging the solutions is of the order of solving the problem with the naive algorithm, your D&C algorithm will never outperform the naive one.
You can (almost) always work out the complexity of some D&C algorithm using the Master theorem:
https://en.wikipedia.org/wiki/Master_theorem_(analysis_of_algorithms)
D&C is just a meta-strategy to design algorithms that may or may not better solve a certain problem; there is no guarantee of "success" i.e. lower complexity, much less lowest one.
QuickSort, for instance, runs in randomized loglinear time and has an acceptable average performance for sorting generic arrays in standard hardware, but is surpassed by other algorithms for specific cases (such as integer arrays) or hardware architectures (such as massively parallel computers).
Quicksort isn't a good example of divide and conquer, since its primary function of ordering data (< pivot, == pivot, > pivot) is performed before dividing. Once the base case of a partition size == 1 is reached, then all of the elements in that part of the call chain have been ordered.
A typical top down merge sort is a better example, since it does nothing other than push indexes (or pointers) onto the stack via recursive calls, and no merging takes place until two instances of sub-array size of 1 occur. Then the merging is done, following the call chain up and down. However, bottom up merge sort skips all of the recursive generation of indexes, and starts by treating an array of n elements as n sub-arrays of size 1, and starts merging immediately. Top down is mostly used for educational purposed, while most libraries use some hybrid of bottom up merge sort and insertion sort.
This is a paragraph of the book: Introduction to Algorithms, 3rd Edition. p.336
"These two approaches yield algorithms with the same asymptotic running time,
expect in unusual circumstances where the top-down approach does not actually
recurse to examine all possible subproblems. The bottom-up approach often has
much better constant factors, since it has less overhead for procedure calls."
The Context : two approaches are first top-down + memoization(DP) and second
bottom-up method.
I got a question for you one more. Does 'overhead' of function call mean every function call needs time? Even if we solve all subproblems, top-down takes more time because of the 'overhead'?
A bottom-up approach to dynamic programming means solving all the small problems first, and then using them to find answers to the next smallest, and so on. So, for instance, if the solution to a problem of length n depends only on answers to problems of length n-1, you might start by putting in all the solutions for length 0, then you'd iteratively fill in solutions to length 1, 2, 3, and so on, each time using the answers you'd already calculated at the previous level. It is efficient in that it means you don't end up solving a sub-problem twice.
A top-down with memoization approach would look at it the other way. If you want the solution to a problem of length 10, then you do so recursively. You notice that it relies on (say) three problems of length 9, so you recursively solve them, and then you know the answer of length 10. But whenever you solve a sub-problem, you remember the answer, and whenever you need the answer to a sub-problem, you look first to see whether you've already solved it, and if you have, you return the cached answer.
The bottom-up approach is good in that it can be written iteratively (using for loops) rather than recursively, which means you don't run out of stack space on large problems, and loops are also faster. Its disadvantage is that you solve all the sub-problems, and you might not need them all to be solved in order to solve the large problem you want the answer to.
The top-down approach is slower if you need all the sub-problems solved anyway, because of the recursion overhead. But it is faster if the problem you're solving only needs a smallish subset of the sub-problems to be solved, because it only solves the ones that it needs.
It is essentially the same as the difference between eager evaluation (bottom up) and lazy evaluation (top down).
What is the difference between Divide and Conquer Algorithms and Dynamic Programming Algorithms? How are the two terms different? I do not understand the difference between them.
Please take a simple example to explain any difference between the two and on what ground they seem to be similar.
Divide and Conquer
Divide and Conquer works by dividing the problem into sub-problems, conquer each sub-problem recursively and combine these solutions.
Dynamic Programming
Dynamic Programming is a technique for solving problems with overlapping subproblems. Each sub-problem is solved only once and the result of each sub-problem is stored in a table ( generally implemented as an array or a hash table) for future references. These sub-solutions may be used to obtain the original solution and the technique of storing the sub-problem solutions is known as memoization.
You may think of DP = recursion + re-use
A classic example to understand the difference would be to see both these approaches towards obtaining the nth fibonacci number. Check this material from MIT.
Divide and Conquer approach
Dynamic Programming Approach
Dynamic Programming and Divide-and-Conquer Similarities
As I see it for now I can say that dynamic programming is an extension of divide and conquer paradigm.
I would not treat them as something completely different. Because they both work by recursively breaking down a problem into two or more sub-problems of the same or related type, until these become simple enough to be solved directly. The solutions to the sub-problems are then combined to give a solution to the original problem.
So why do we still have different paradigm names then and why I called dynamic programming an extension. It is because dynamic programming approach may be applied to the problem only if the problem has certain restrictions or prerequisites. And after that dynamic programming extends divide and conquer approach with memoization or tabulation technique.
Let’s go step by step…
Dynamic Programming Prerequisites/Restrictions
As we’ve just discovered there are two key attributes that divide and conquer problem must have in order for dynamic programming to be applicable:
Optimal substructure — optimal solution can be constructed from optimal solutions of its subproblems
Overlapping sub-problems — problem can be broken down into subproblems which are reused several times or a recursive algorithm for the problem solves the same subproblem over and over rather than always generating new subproblems
Once these two conditions are met we can say that this divide and conquer problem may be solved using dynamic programming approach.
Dynamic Programming Extension for Divide and Conquer
Dynamic programming approach extends divide and conquer approach with two techniques (memoization and tabulation) that both have a purpose of storing and re-using sub-problems solutions that may drastically improve performance. For example naive recursive implementation of Fibonacci function has time complexity of O(2^n) where DP solution doing the same with only O(n) time.
Memoization (top-down cache filling) refers to the technique of caching and reusing previously computed results. The memoized fib function would thus look like this:
memFib(n) {
if (mem[n] is undefined)
if (n < 2) result = n
else result = memFib(n-2) + memFib(n-1)
mem[n] = result
return mem[n]
}
Tabulation (bottom-up cache filling) is similar but focuses on filling the entries of the cache. Computing the values in the cache is easiest done iteratively. The tabulation version of fib would look like this:
tabFib(n) {
mem[0] = 0
mem[1] = 1
for i = 2...n
mem[i] = mem[i-2] + mem[i-1]
return mem[n]
}
You may read more about memoization and tabulation comparison here.
The main idea you should grasp here is that because our divide and conquer problem has overlapping sub-problems the caching of sub-problem solutions becomes possible and thus memoization/tabulation step up onto the scene.
So What the Difference Between DP and DC After All
Since we’re now familiar with DP prerequisites and its methodologies we’re ready to put all that was mentioned above into one picture.
If you want to see code examples you may take a look at more detailed explanation here where you'll find two algorithm examples: Binary Search and Minimum Edit Distance (Levenshtein Distance) that are illustrating the difference between DP and DC.
The other difference between divide and conquer and dynamic programming could be:
Divide and conquer:
Does more work on the sub-problems and hence has more time consumption.
In divide and conquer the sub-problems are independent of each other.
Dynamic programming:
Solves the sub-problems only once and then stores it in the table.
In dynamic programming the sub-problem are not independent.
sometimes when programming recursivly, you call the function with the same parameters multiple times which is unnecassary.
The famous example Fibonacci numbers:
index: 1,2,3,4,5,6...
Fibonacci number: 1,1,2,3,5,8...
function F(n) {
if (n < 3)
return 1
else
return F(n-1) + F(n-2)
}
Let's run F(5):
F(5) = F(4) + F(3)
= {F(3)+F(2)} + {F(2)+F(1)}
= {[F(2)+F(1)]+1} + {1+1}
= 1+1+1+1+1
So we have called :
1 times F(4)
2 times F(3)
3 times F(2)
2 times F(1)
Dynamic Programming approach: if you call a function with the same parameter more than once, save the result into a variable to directly access it on next time. The iterative way:
if (n==1 || n==2)
return 1
else
f1=1, f2=1
for i=3 to n
f = f1 + f2
f1 = f2
f2 = f
Let's call F(5) again:
fibo1 = 1
fibo2 = 1
fibo3 = (fibo1 + fibo2) = 1 + 1 = 2
fibo4 = (fibo2 + fibo3) = 1 + 2 = 3
fibo5 = (fibo3 + fibo4) = 2 + 3 = 5
As you can see, whenever you need the multiple call you just access the corresponding variable to get the value instead of recalculating it.
By the way, dynamic programming doesn't mean to convert a recursive code into an iterative code. You can also save the subresults into a variable if you want a recursive code. In this case the technique is called memoization. For our example it looks like this:
// declare and initialize a dictionary
var dict = new Dictionary<int,int>();
for i=1 to n
dict[i] = -1
function F(n) {
if (n < 3)
return 1
else
{
if (dict[n] == -1)
dict[n] = F(n-1) + F(n-2)
return dict[n]
}
}
So the relationship to the Divide and Conquer is that D&D algorithms rely on recursion. And some versions of them has this "multiple function call with the same parameter issue." Search for "matrix chain multiplication" and "longest common subsequence" for such examples where DP is needed to improve the T(n) of D&D algo.
I assume you have already read Wikipedia and other academic resources on this, so I won't recycle any of that information. I must also caveat that I am not a computer science expert by any means, but I'll share my two cents on my understanding of these topics...
Dynamic Programming
Breaks the problem down into discrete subproblems. The recursive algorithm for the Fibonacci sequence is an example of Dynamic Programming, because it solves for fib(n) by first solving for fib(n-1). In order to solve the original problem, it solves a different problem.
Divide and Conquer
These algorithms typically solve similar pieces of the problem, and then put them together at the end. Mergesort is a classic example of divide and conquer. The main difference between this example and the Fibonacci example is that in a mergesort, the division can (theoretically) be arbitrary, and no matter how you slice it up, you are still merging and sorting. The same amount of work has to be done to mergesort the array, no matter how you divide it up. Solving for fib(52) requires more steps than solving for fib(2).
I think of Divide & Conquer as an recursive approach and Dynamic Programming as table filling.
For example, Merge Sort is a Divide & Conquer algorithm, as in each step, you split the array into two halves, recursively call Merge Sort upon the two halves and then merge them.
Knapsack is a Dynamic Programming algorithm as you are filling a table representing optimal solutions to subproblems of the overall knapsack. Each entry in the table corresponds to the maximum value you can carry in a bag of weight w given items 1-j.
Divide and Conquer involves three steps at each level of recursion:
Divide the problem into subproblems.
Conquer the subproblems by solving them recursively.
Combine the solution for subproblems into the solution for original problem.
It is a top-down approach.
It does more work on subproblems and hence has more time
consumption.
eg. n-th term of Fibonacci series can be computed in O(2^n) time complexity.
Dynamic Programming involves the following four steps:
1. Characterise the structure of optimal solutions.
2. Recursively define the values of optimal solutions.
3. Compute the value of optimal solutions.
4. Construct an Optimal Solution from computed information.
It is a Bottom-up approach.
Less time consumption than divide and conquer since we make use of the values computed earlier, rather than computing again.
eg. n-th term of Fibonacci series can be computed in O(n) time complexity.
For easier understanding, lets see divide and conquer as a brute force solution and its optimisation as dynamic programming.
N.B. divide and conquer algorithms with overlapping subproblems can only be optimised with dp.
Divide and Conquer
They broke into non-overlapping sub-problems
Example: factorial numbers i.e. fact(n) = n*fact(n-1)
fact(5) = 5* fact(4) = 5 * (4 * fact(3))= 5 * 4 * (3 *fact(2))= 5 * 4 * 3 * 2 * (fact(1))
As we can see above, no fact(x) is repeated so factorial has non overlapping problems.
Dynamic Programming
They Broke into overlapping sub-problems
Example: Fibonacci numbers i.e. fib(n) = fib(n-1) + fib(n-2)
fib(5) = fib(4) + fib(3) = (fib(3)+fib(2)) + (fib(2)+fib(1))
As we can see above, fib(4) and fib(3) both use fib(2). similarly so many fib(x) gets repeated. that's why Fibonacci has overlapping sub-problems.
As a result of the repetition of sub-problem in DP, we can keep such results in a table and save computation effort. this is called as memoization
Divide and Conquer
In this problem is solved in following three steps:
1. Divide - Dividing into number of sub-problems
2. Conquer - Conquering by solving sub-problems recursively
3. Combine - Combining sub-problem solutions to get original problem's solution
Recursive approach
Top Down technique
Example: Merge Sort
Dynamic Programming
In this the problem is solved in following steps:
1. Defining structure of optimal solution
2. Defines value of optimal solutions repeatedly.
3. Obtaining values of optimal solution in bottom-up fashion
4. Getting final optimal solution from obtained values
Non-Recursive
Bottom Up Technique
Example: Strassen's Matrix Multiplication
Divide and Conquer:
This paradigm involves three stages:
Divide the problem into smaller sub-problems
Conquer, i.e., solve these smaller sub-problems
Combine these sub-problems' solutions to get the final answer.
Dynamic Programming:
DP is an optimization of recursive solutions. The primary difference it makes is that it stores the solution to sub-problems, which can later be accessed during the process of finding solutions of the remaining sub-problems. This is done so that we don't have to calculate the solution to a sub-problem every time, rather we can simply look it up the computer memory to retrieve its value, given that it has been solved earlier. We can simply add this as our base case in recursion. For example, we are solving a problem through recursion, we can store the solutions to sub-problems in an array and access them by adding the relevant code in one of our base cases in the recursive method.
There are two ways in which DP is done:
Consider a problem: To find factorial of x.
Tabulation: We use the bottom up approach, that is we go from the smallest numbers all the way upto x, to find the solution.
Pseudo Code:
1. int array
2. for int=1, i<=x, i++
3. array[i] = array[i-1]*i
Memoization: We use the top down approach, that is we take the problem and then break it down into smaller parts and solve them, to get the final solution
Pseudo Code:
fac():
1. int array
2. if(x==0): return 1
3. if(array[x]!=null): return array[x]
4. return array[x] = x*fac(x-1)
I was asked if a Binary Search is a divide and conquer algorithm at an exam. My answer was yes, because you divided the problem into smaller subproblems, until you reached your result.
But the examinators asked where the conquer part in it was, which I was unable to answer. They also disapproved that it actually was a divide and conquer algorithm.
But everywhere I go on the web, it says that it is, so I would like to know why, and where the conquer part of it is?
The book:
Data Structures and Algorithm Analysis in Java (2nd Edition), by Mark Allen Weiss
Says that a D&C algorithm should have two disjoint recursive calls, just like QuickSort does.
Binary Search does not have this, even though it can be implemented recursively.
I think it is not divide and conquer, see first paragraph in http://en.wikipedia.org/wiki/Divide_and_conquer_algorithm
recursively breaking down a problem into two or more sub-problems
which are then combined to give a solution
In binary search there is still only one problem which does just reducing data by half every step, so no conquer (merging) phase of the results is needed.
It isn't.
To complement #Kenci's post, DnC algorithms have a few general/common properties; they:
divide the original problem instance into a set of smaller sub-instances of itself;
independently solve each sub-instance;
combine smaller/independent sub-instance solutions to build a single solution for the larger/original instance
The problem with Binary Search is that it does not really even generate a set of independent sub-instances to be solved, as per step 1; it only simplifies the original problem by permanently discarding sections it's not interested in. In other words, it only reduces the problem's size and that's as far as it ever goes.
A DnC algorithm is supposed to not only identify/solve the smaller sub-instances of the original problem independently of each other, but also use that set of partial independent solutions to "build up" a single solution for the larger problem instance as a whole.
The book Fundamentals of Algorithmics, G. Brassard, P. Bratley says the following (bold my emphasis, italics in original):
It is probably the simplest application of divide-and-conquer, so simple in fact that strictly speaking this is an application of simplification rather than divide-and-conquer: the solution to any sufficiently large instance is reduced to that of a single smaller one, in this case of half size.
Section 7.3 Binary Search on p.226.
In a divide and conquer strategy :
1.Problem is divided into parts;
2.Each of these parts is attacked/solved independently, by applying the algorithm at hand (mostly recursion is used for this purpose) ;
3.And then the solutions of each partition/division and combined/merged together to arrive at the final solution to the problem as a whole (this comes under conquer)
Example, Quick sort, merge sort.
Basically, the binary search algorithm just divides its work space(input (ordered) array of size n) into half in each iteration. Therefore it is definitely deploying the divide strategy and as a result, the time complexity reduces down to O(lg n).So,this covers up the "divide" part of it.
As can be noticed, the final solution is obtained from the last comparison made, that is, when we are left with only one element for comparison.
Binary search does not merge or combine solution.
In short, binary search divides the size of the problem (on which it has to work) into halves but doesn't find the solution in bits and pieces and hence no need of merging the solution occurs!
I know it's a bit too lengthy but i hope it helps :)
Also you can get some idea from : https://www.khanacademy.org/computing/computer-science/algorithms/binary-search/a/running-time-of-binary-search
Also i realised just now that this question was posted long back!
My bad!
Apparently some people consider binary search a divide-and-conquer algorithm, and some are not. I quickly googled three references (all seem related to academia) that call it a D&C algorithm:
http://www.cs.berkeley.edu/~vazirani/algorithms/chap2.pdf
http://homepages.ius.edu/rwisman/C455/html/notes/Chapter2/DivConq.htm
http://www.csc.liv.ac.uk/~ped/teachadmin/algor/d_and_c.html
I think it's common agreement that a D&C algorithm should have at least the first two phases of these three:
divide, i.e. decide how the whole problem is separated into sub-problems;
conquer, i.e. solve each of the sub-problems independently;
[optionally] combine, i.e. merge the results of independent computations together.
The second phase - conquer - should recursively apply the same technique to solve the subproblem by dividing into even smaller sub-sub-problems, and etc. In practice, however, often some threshold is used to limit the recursive approach, as for small size problems a different approach might be faster. For example, quick sort implementations often use e.g. bubble sort when the size of an array portion to sort becomes small.
The third phase might be a no-op, and in my opinion it does not disqualify an algorithm as D&C. A common example is recursive decomposition of a for-loop with all iterations working purely with independent data items (i.e. no reduction of any form). It might look useless at glance, but in fact it's very powerful way to e.g. execute the loop in parallel, and utilized by such frameworks as Cilk and Intel's TBB.
Returning to the original question: let's consider some code that implements the algorithm (I use C++; sorry if this is not the language you are comfortable with):
int search( int value, int* a, int begin, int end ) {
// end is one past the last element, i.e. [begin, end) is a half-open interval.
if (begin < end)
{
int m = (begin+end)/2;
if (value==a[m])
return m;
else if (value<a[m])
return search(value, a, begin, m);
else
return search(value, a, m+1, end);
}
else // begin>=end, i.e. no valid array to search
return -1;
}
Here the divide part is int m = (begin+end)/2; and all the rest is the conquer part. The algorithm is explicitly written in a recursive D&C form, even though only one of the branches is taken. However, it can also be written in a loop form:
int search( int value, int* a, int size ) {
int begin=0, end=size;
while( begin<end ) {
int m = (begin+end)/2;
if (value==a[m])
return m;
else if (value<a[m])
end = m;
else
begin = m+1;
}
return -1;
}
I think it's quite a common way to implement binary search with a loop; I deliberately used the same variable names as in the recursive example, so that commonality is easier to see. Therefore we might say that, again, calculating the midpoint is the divide part, and the rest of the loop body is the conquer part.
But of course if your examiners think differently, it might be hard to convince them it's D&C.
Update: just had a thought that if I were to develop a generic skeleton implementation of a D&C algorithm, I would certainly use binary search as one of API suitability tests to check whether the API is sufficiently powerful while also concise. Of course it does not prove anything :)
The Merge Sort and Quick Sort algorithms use the divide and conquer technique (because there are 2 sub-problems) and Binary Search comes under decrease and conquer (because there is 1 sub-problem).
Therefore, Binary Search actually uses the decrease and conquer technique and not the divide and conquer technique.
Source: https://www.geeksforgeeks.org/decrease-and-conquer/
Binary search is tricky to describe with divide-and-conquer because the conquering step is not explicit. The result of the algorithm is the index of the needle in the haystack, and a pure D&C implementation would return the index of the needle in the smallest haystack (0 in the one-element list) and then recursively add the offsets in the larger haystacks that were divided in the divison step.
Pseudocode to explain:
function binary_search has arguments needle and haystack and returns index
if haystack has size 1
return 0
else
divide haystack into upper and lower half
if needle is smaller than smallest element of upper half
return 0 + binary_search needle, lower half
else
return size of lower half + binary_search needle, upper half
The addition (0 + or size of lower half) is the conquer part. Most people skip it by providing indices into a larger list as arguments, and thus it is often not readily available.
The divide part is of course dividing the set into halves.
The conquer part is determining whether and on what position in the processed part there is a searched element.
Dichotomic in computer science refers to choosing between two antithetical choices, between two distinct alternatives. A dichotomy is any splitting of a whole into exactly two non-overlapping parts, meaning it is a procedure in which a whole is divided into two parts. It is a partition of a whole (or a set) into two parts (subsets) that are:
1. Jointly Exhaustive: everything must belong to one part or the other, and
2. Mutually Exclusive: nothing can belong simultaneously to both parts.
Divide and conquer works by recursively breaking down a problem into two or more sub-problems of the same type, until these become simple enough to be solved directly.
So the binary search halves the number of items to check with each iteration and determines if it has a chance of locating the "key" item in that half or moving on to the other half if it is able to determine keys absence. As the algorithm is dichotomic in nature so the binary search will believe that the "key" has to be in one part until it reaches the exit condition where it returns that the key is missing.
Divide and Conquer algorithm is based on 3 step as follows:
Divide
Conquer
Combine
Binary Search problem can be defined as finding x in the sorted array A[n].
According to this information:
Divide: compare x with middle
Conquer: Recurse in one sub array. (Finding x in this array)
Combine: it is not necessary.
A proper divide and conquer algorithm will require both parts to be processed.
Therefore, many people will not call binary-search a divide and conquer algorithm, it does divide the problem, but discards the other half.
But most likely, your examiners just wanted to see how you argue. (Good) exams aren't about the facts, but about how you react when the challenge goes beyond the original material.
So IMHO the proper answer would have been:
Well, technically, it consists only of a divide step, but needs to conquer only half of the original task then, the other half is trivially done already.
BTW: there is a nice variation of QuickSort, called QuickSelect, which actually exploits this difference to obtain an on average O(n) median search algorithm. It's like QuickSort - but descends only into the half it is interested in.
Binary Search is not a divide and conquer approach. It is a decrease and conquer approach.
In divide and conquer approach, each subproblem must contribute to the solution but in binary search, all subdivision does not contribute to the solution. we divide into two parts and discard one part because we know that the solution does not exist in this part and look for the solution only in one part.
The informal definition is more or less: Divide the problem into small problems. Then solve them and put them together (conquer). Solving is in fact deciding where to go next (left, right, element found).
Here a quote from wikipedia:
The name "divide and conquer" is sometimes applied also to algorithms that reduce each problem to only one subproblem, such as the binary search algorithm for finding a record in a sorted list.
This states, it's NOT [update: misread this phrase:)] only one part of divide and conquer.
Update:
This article made it clear for me. I was confused since the definition says you have to solve every sub problem. But you solved the sub problem if you know you don't have to keep on searching..
The Binary Search is a divide and conquer algorithm:
1) In Divide and Conquer algorithms, we try to solve a problem by solving a smaller sub problem (Divide part) and use the solution to build the solution for our bigger problem(Conquer).
2) Here our problem is to find an element in the sorted array. We can solve this by solving a similar sub problem. (We are creating sub problems here based on a decision that the element being searched is smaller or bigger than the middle element). Thus once we know that the element can not exist surely in one half, we solve a similar sub-problem in the the other half.
3) This way we recurse.
4) The conquer part here is just returning the value returned by the sub problem to the top the recursive tree
I think it is Decrease and Conquer.
Here is a quote from wikipedia.
"The name decrease and conquer has been proposed instead for the
single-subproblem class"
http://en.wikipedia.org/wiki/Divide_and_conquer_algorithms#Decrease_and_conquer
According to my understanding, "Conquer" part is at the end when you find the target element of the Binary search. The "Decrease" part is reducing the search space.
Binary Search and Ternary Search Algorithms are based on Decrease and Conquer technique. Because, you do not divide the problem, you actually decrease the problem by dividing by 2(3 in ternary search).
Merge Sort and Quick Sort Algorithms can be given as examples of Divide and Conquer technique. You divide the problem into two subproblems and use the algorithm for these subproblems again to sort an array. But, you discard the half of array in binary search. It means you DECREASE the size of array, not divide.
No, binary search is not divide and conquer. Yes, binary search is decrease and conquer. I believe divide and conquer algorithms have an efficiency of O(n log(n)) while decrease and conquer algorithms have an efficiency of O(log(n)). The difference being whether or not you need to evaluate both parts of the split in data or not.