In 16.1 An activity-selection problem of Introduction to Algorithm, the dynamic programming solution for this problem was given as
c[i, j] = 0 if S(i, j) is empty
c[i, j] = max { c[i, k] + c[k, j] + 1 } if S(i, j) is not empty
where S(i, j) denotes the set of activities that start after activity a(i) finishes and that finish before activity a(j) starts, and c[i, j] denotes the size of an optimal solution for the set S(i, j)
However, I am thinking of another simpler solution
c[i] = max { c[i - 1], c[f(i)] + 1 }
where f(i) gives the activity that is compatible with a(i) and has the max finish time and finishes before a(i) starts.
Will this work? If yes, why the author provides this complex solution. If not, what am I missing?
I think you are missing many details of designing the dp solution.
What is the initial value?
What is the base case?
what happens if there are several activities compatible with a(i) with same finishing time?
When designing a dp solution, one of the properties needed is optimal substructure
The computing order of a particular state (i.e. c[i]) is important, it can only be computed by its subproblems. Your solution does not meet this requirement, as when you computing c[i], you have to computer c[j] first with j = f(i), let's assume j > i (or even j = i+1) , then you have to compute c[i] before computing c[j]! So c[i] depends on c[j] while c[j] depends on c[i] ==> not correct
Another example very similar to this question is Matrix chain mutiplication
You may want to have a look :)
Edit:
After seeing you edit the question, then here's my response:
Assuming you can precompute f(i) in reasonable time (which obviously can), your solution is correct as it IS the greedy solution as other answers told you.
The reason why it works is quite straight forward, literally speaking,
until the i-th activity, you either choose activity i (thats the c[f(i)]+1 part) or not choose it (the c[i-1]) part
You can try to construct a formal proof as well, the correctness of a greedy method can usually be proofed by contradiction (roughly speaking, you can try to see why it is NOT possible to have a larger set other than c[i-1] if you do not choose activity i, similar for the case that you choose activity i)
To answer your question about why writer demonstrate the dp solution, I think it's out of programming context, but my thought is the user is trying to demonstrate two different ways to solve a problem, and furthermore to illustrate an idea here: given a problem which can be solved by greedy method, it can also be solved by dp but IT IS OVERKILLING.
Then the writer try to help the reader to recognize the difference between Greedy and dp as they are quite similar to a new learner. And that's why the writer first give the DP solution to show the pain, then the greedy solution, lastly a paragraph Greedy versus DP in Page 382
So TL;DR: Your solution is correct as it is basically the greedy method to solve the problem, and of course it is much easier than the DP solution given in the book, as this IS the point the book would like to illustrate.
A quote from the book at P.382: ...One might be temped to generate a dp solution to a problem when a greedy solution suffices, or one might mistakenly think that a greedy solution suffices...when a dp solution is required...
Will this work?
Yes, that will work too.
why the author provides this complex solution.
That wasn't their proposed solution, but a part of the analysis of the problem. In the next paragraph after the equation you cited the authors say:
But we would be overlooking another important characteristic of the
activity-selection problem that we can use to great advantage.
The final solution (Greedy-Activity-Selector) is similar to yours, but even simpler.
Their point, as I understand it, is that a DP solution can be built almost mechanically (as described in the chapter 15.3), without considering the specifics of that particular problem, but coming up with a better algorithm requires some insight into the problem beyond the optimal substructure.
Your solution relies on the theorem 16.1, but once the theorem is proven, it doesn't make sense to create another DP algorithm, because you already know enough about the problem to create a simpler greedy algorithm.
Related
I was looking at the balanced partitioning problem here and here (problem 7).
The problem basically asks to partition a given array of numbers into 2 subsets (S1 and S2) such that absolute difference between the sums of numbers is S1 ans S2 |sum(S1) - sum(S2)| needs to be minimum. One thing I didn't understand is why doesn't anyone suggest greedy approach:
def balanced_partition(lst):
idx = 0
S1 = 0
S2 = 0
result_partition=[None]*len(lst)
while idx < len(lst):
new_S1 = S1 + lst[idx]
new_S2 = S2 + lst[idx]
if abs(new_S1 - S2) < abs(new_S2 - S1):
result_partition[idx] = 1
S1 = new_S1
else:
result_partition[idx] = 2
S2 = new_S2
idx += 1
print("final sums s1 = {S1} and s2 = {S2} ".format(S1=S1, S2=S2))
return result_partition
What is wrong with my approach? It seems to pass all the test cases I can come up with.
The simple counterexample is [1,1,1,1,1,1,6]. The greedy approach will spread the ones between the two sets, while the optimal solution is [1,1,1,1,1,1],[6].
There is nothing wrong with your implementation and approach. However if you consider all subsets in this particular problem, you may find a better answer than the greedy output. Even in the wiki page that you shared has some examples.
Probably you already know the difference between those two approaches. Although, greedy algorithm will always give you a pretty good result, so close or maybe equal to the best one, you have to consider all options to be sure. Dynamic programming approach checks all the possible subsets in a way. As it saves results from previously computed sub-problems, it is faster than brute forcing basically.
The question is when to use greedy or dynamic programming approach. I have done some competitive programming and when I see a DP problem (problems like partitioning, subset sum, knapsack and so on), I sometimes come up with a greedy solution immediately because most of the times they are more obvious. People use greedy approach all the time in daily life. Before implementing, I test my algorithm with examples and if I convince myself that this is right approach, I implement it. It is kinda intuitive in some way.
If you find a test case that should have a better answer, most probably it means you have to find a DP solution. If you got WA from judge system, it means you haven't find good test cases but that's okay you don't have to find that exact test case because it won't help you to find a better solution.
The mathematical problem
Let there be 2n persons, and C(i,j) the "cost" of having i and j work together (the function C is quick to compute, in my case it is a given matrix, and is symmetric). The question is to find the arrangement of 2n pairs of persons that minimizes the sum of the costs of each pair.
This should be done in polynomial complexity in n, and implemented relatively easily in the Scilab language (input : cost matrix, output : pairings, for instance a n-by-2 matrix of indexes). I am aware that "relatively easily" is subject to interpretation...
Previous research
This problem is actually solved by the Blossom algorithm. See for instance this paper.
However, this (and its variants) looks like a nightmare to implement. My real problem is for n=20, so although brute force (= trying all possible pairings) is not OK (brute-forcing n=8 took an hour on my computer), pretty much anything better than brute force should do the trick; if I can avoid one week of coding at the cost of one hour of computation I'm in.
I was thinking along the lines of using the Hungarian/Munkres algorithm on a 2n-by-2n array filling the diagonal with +%inf and other elements by the symmetric cost matrix, then somehow selecting from the resulting permutation a relevant pairing, but I fail to find a reliable way to do this. (Note, the Hungarian algorithm is already coded for a separate section, so you may use it without cost to the "easy to implement" requirement.)
I hope that compared to the blossom-algorithm problem, the completeness of the graph allows for some shortcuts... (Edit: see DE's comment below, this is wrong for semi-obvious reasons)
I do not know Scilab I am afraid, but if you are willing to use Python it is very easy as the Networkx library provides support for this function:
import networkx as nx
import networkx.algorithms.matching as matching
def C(i,j):
return i*j
n=40
G=nx.Graph()
for i in range(n):
for j in range(n):
G.add_edge(i,j,weight = -C(i,j))
M = matching.max_weight_matching(G,maxcardinality=True)
for i in M:
print i,'with',M[i]
This code prints out the answer within a second.
The function C defines the cost of pairing i with j. Note that the weights are set to -C(i,j) in order to transform the max_weight_matching into a min_weight_matching algorithm.
The "simple/naive backtracking brute force algorithm", "Straightforward Depth-First Search" for sudoku is commonly known and implemented.
and no different implementation seems to exist.
(when i first wrote this question.. i wanted to mean we could completely standardize it, but the wording is bad..)
This guy has described the algorithm well i think: https://stackoverflow.com/a/2075498/3547717
Edit: So let me have it more specified with pseudo code...
var field[9][9]
set the givens in 'field'
if brute (first empty grid) = true then
output solution
else
output no solution
end if
function brute (cx, cy)
for n = 1 to 9
if (n doesn't present in row cy) and (n doesn't present in column cx) and (n doesn't present in block (cx div 3, cy div 3)) then
let field[cx][cy] = n
if (cx, cy) this is the last empty grid then
return true
elseif brute (next empty grid) = true then
return true
end if
let field[cx][cy] = empty
end if
next n
end function
I want to find the puzzle that requires most time. We may call it "hardest" for this particular "standardized" algorithm, but this one is not like those questions asking for "Hardest sudoku".
In fact, a "hard" puzzle under this definition may turn super easy when simply rotated or flipped.
According to the rule "for each grid try number 1 to 9", it tries from 1 on, so we may somehow let it try more by using proper number, by the way there won't be permutation problem.
The sudoku puzzle must be valid, i.e. it should have exactly 1 solution. Some guy got a puzzle requiring 1439 seconds, but it's not valid because of having no solution.
I define the time required (or say time complexity) equivalent to how many times the recursive function is entered. (in my implementation, it's slightly different from the pseudo code above, because of the last entrance, and ensuring unique solution, etc.)
Is there any good way to construct it, or we have to use approximate ones like heuristic algorithms to find inexact solutions?
I've implemented a backtracking with both naive strategy (that I referred to as "simple" above, it's unique) and Peter Norvig's "Least Candidates First" strategy (my implementation is deterministic, but not unique. As Peter has also mentioned, the order of python dict changes the result a lot, in case of a tie on the number of candidates).
https://github.com/farteryhr/labs/blob/master/sudoku.c
The no-solution one:
.....5.8....6.1.43..........1.5........1.6...3.......553.....61........4.........
takes 60 seconds on my laptop to get the no-solution conclusion, entering the recursion function 2549798781 times (called "cycles" later). With my implementation of LCF, 78308087 cycles in 30 seconds to conclude. It's because finding the grid with least candidates needs more operations, a single cycle of LCF strategy uses about 16x more time.
The topmost one on the Hardest list:
4.....8.5.3..........7......2.....6.....8.4......1.......6.3.7.5..2.....1.4......
takes 3.0s, found the solution at cycle 9727397, and 142738236 cycles for ensuring unique solution. (my LCF: 981/7216 in 0.004s)
Many in the "hard" list are still easy for naive, though a larger portion of them needs 10^7 to 10^9 cycles.
On Wikipedia: Sudoku solving algorithms (Original) it's stated that such puzzles against backtracking algorithm can be constructed, by making as many empty grids at the beginning as possible and the permutation of the top row 987654321.
Well the test..
..............3.85..1.2.......5.7.....4...1...9.......5......73..2.1........4...9
takes 1.4s, 69175317 cycles for finding solution, 69207227 cycles ensuring unique solution. Not as good as the hard one provided by Peter, but OK, and it's almost right after finding the solution, the search ends. That's probably how the first row works by being lexicographically large. (my LCF: 29206/46160 in 0.023s)
Yes these are obvious, I'm just asking for better ways...
There are also other ways of measuring the difficulty of Sudoku (through solving)
Sudoku Analyst will get stuck with the multiple-solution puzzle given by Peter (naive 419195/419256, LCF 2529478/2529482, yes, there are some puzzles that make LCF do worse):
.....6....59.....82....8....45........3........6..3.54...325..6..................
This one is easy for both naive backtracking (10008/76703) and LCF backtracking (313/1144), but also gets Sudoku Analyst stuck.
..53.....8......2..7..1.5..4....53...1..7...6..32...8..6.5....9..4....3......97..
Another update:
The most difficult Sudoku puzzles are quickly solved by a straightforward depth-first search algorithm
Ha, finally someone also looking for it, and a super tough one is given! The following valid puzzle:
9..8...........5............2..1...3.1.....6....4...7.7.86.........3.1..4.....2..
In this paper, the algorithm is named SDFS, Straightforward Depth-First Search. The number of cycles stated by the author is 1553023932/1884424814, and with my implementation, it's 1305263522/1584688020. Yes, there will be some difference on precisely where to pop the counter, but the basic behavior matches. On repl.it 's server, it took 97s to find the answer and 119s to finish the search.
You can easily generate the worst case by recording the time taken / no. of operations taken by your code to solve hard sudoku puzzles. You can either use a random generator that generates valid sudoku puzzles (or) you can take hard sudoku puzzles from the internet and run your code against it to measure the time/number of operations. Once you run your code against 10000 such cases the slowest 5 (and the unsolved ones) would be the worst cases for your solution.
I have a few questions regarding the semantics of terminology used when describing algorithms.
Firstly, what is meant by a 'naive' algorithm? How does this differ from other solutions to a given problem? What other forms can solutions take?
Secondly, I have heard much reference to having a 'closed - form' solution. I have no idea what this means either - but often it appears when trying to solve recurrence relations...
Thanks for your time
A Naive algorithm is usually the most obvious solution when one is asked a problem. It may not be a smart algorithm but will probably get the job done (...eventually.)
Eg. Trying to search for an element in a sorted array.
A Naive algorithm would be to use a Linear Search.
A Not-So Naive Solution would be to use the Binary Search.
A better example, would be in case of substring search Naive Algorithm is far less efficient than Boyer–Moore or Knuth–Morris–Pratt Algorithm
A Closed Form Solution is a simple Solution that works instantly without any loops,functions etc..
Eg:
Iterative Algorithm for sum of integer from 1 to n
s= 0
for i in 1 to n
s = s + i
end for
print s
Closed Form (for the same problem)
s = n * (n + 1 ) /2
Naive algorithm is a very simple algorithm, one with very simple rules. Sometimes the first one that comes to mind. It may be stupid and very slow, it may not even solve the problem. It may sometimes be the best possible. Here's an example of a problem and "naive" algorithms:
Problem: You are in a (2-dimensional) maze. Find your way out. (meaning: to a spot with an "EXIT" sign :)
Naive algorithm 1: Start walking and choose the right one in every intersection you meet (until you find "EXIT").
Naive algorithm 2: Start walking and choose a random one in every intersection you meet (until you find "EXIT").
Algorithm 1 will not even get you out of some mazes!
Algorithm 2 will get you out of all mazes (although this is rather hard to prove).
Closed form means you can give the one expression as solution, that does solve it without recurrence/recursive. Here one should remark, that it is not always possible to find such a closed form.
Naive means just that what it says: A first, stupid solution to the problem, that solves it, but maybe not very time-/space efficient. What one really considers 'naive' depends on the speaker, the context, and the weather of the next day. Often it is used to distinguish a very sophisticated solution (that uses some kind of trick) from the obvious implementation.
This question is an extension to the following one. The difference is that now our function to optimize will have higher order relations between elements:
We have an array of elements a1,a2,...aN from an alphabet E. Assuming |N| >> |E|.
For each symbol of the alphabet we define an unique integer priority = V(sym). Let's define V{i} := V(symbol(ai)) for the simplicity.
The task is to find a priority function V for which:
Count(i)->MIN | V{i} > V{i+1} <= V{i+2}
In other words, I need to find the priorities / permutation of the alphabet for which the number of positions i, satisfying the condition V{i}>V{i+1}<=V{i+2}, is minimum.
Maximum required abstraction (low priority for me). I guess once the solution model for the initial question is extended to cover the first part of this one, extending it farther (see below) will be easier.
Given a matrix of signs B of size MxK (basically B[i,j] is from the set {<,>,<=,>=}), find the priority function V for which:
Sum(for all j in range [1,M]) {Count(i)}->EXTREMUM | V{i} B[j,1] V{i+1} B[j,2] ... B[j,K] V{i+K}
As an example, find the priority function V, for which the number of i, satisfying V{i}<V{i+1}<V{i+2} or V{i}>V{i+1}>V{i+2}, is minimum.
My intuition is that all variations on this problem will prove to be NP-hard. So I'd begin looking for heuristics that produce reasonable answers. This may involve some trial and error.
A simplistic approach is to write down a possible permutation. And then try possible swaps until you've arrived at a local minimum. Try several times, and pick the best answer.
Simulated annealing provides a more sophisticated version of this approach, see http://en.wikipedia.org/wiki/Simulated_annealing for a description. It may take some experimentation to find a set of parameters that seems to converge relatively well.
Another idea is to look for a genetic algorithm. Based on a quick Google search it looks like the standard way to do this is to try to turn an NP-complete problem into a SAT problem, and then use a genetic algorithm on that problem. This approach would require turning this into a SAT problem in some reasonable way. Unfortunately it is not obvious to me how one would go about doing this reduction. Indeed in the first version that you had, your problem was closely connected to a classic NP-hard problem. The fact that it is labeled NP-hard rather than NP-complete is evidence that people haven't found a good way to transform it into a SAT problem. So if it isn't obvious how to turn the simple version into a SAT problem, then you are unlikely to convert the hard problem either.
But you could still try some variation on genetic algorithms. Mutation is pretty simple, just swap some elements around. One way to combine elements would be to take 3 permutations and use quicksort to find the combination as follows: take a random pivot, and then use "majority wins" to bucket elements into bigger and smaller. Sort each half in the same way.
I'm sorry that I can't just give you an approach and say, "This should work." You've got what looks like an open-ended research project, and the best I can do is give you some ideas about things you can try that might work reasonably well.