I'm not a student taking a computational complexity course, just interested in the subject. I came across this section:
Suppose we have a problem that we've proven is hard to solve,
and we have a similar new problem. We might suspect that it is also
hard to solve. We argue by contradiction: suppose the new problem is
easy to solve. Then, if we can show that every instance of the old
problem can be solved easily by transforming it into instances of the
new problem and solving those, we have a contradiction. This
establishes that the new problem is also hard.
Source: Wikipedia
I can't seem to wrap my head around what this means. Can you explain this process in layman's terms (as much as possible) how a proof by contradiction as such would work?
Is the contradiction that we know the old problem is hard but we're able to reduce it to a new problem and solving that means old is at least as easy as new? This whole idea is confusing to me. Can someone explain it to me, who doesn't have a solid background in computational complexity theory?
Say that we know for a fact that task A is a hard task. Now say we are given task B and asked to determine whether it's hard or not. Pretend that we figure out a way to efficiently break down task A into a few parts and each part turns out to be task B. That is, task A can be easily performed by doing a small number of task Bs. Now, can task B be easy? If it were, then I'd be able to do task A easily as well. That would be a contradiction though, since i know task A to be hard. Therefore task B must be hard as well.
It's important to note that all cases of task A must reduce to B in order for this to hold. If just some cases do then B could be easy but still not provide an easy solution to A in general, therfore there isn't necessarily a contradiction. Also, if it takes a lot to reduce A to B (ie the reduction itself is hard) then there isn't necessarily a contradiction if B is easy.
Related
Let's start here:
It is said that all NP problems can be reduced to SAT(boolean satisfiability problem). To be more accurate to Circuit SAT, because all decision problems like NP should end up with answer Yes or No.
But now, if I have a random NP problem, how to build a boolean circuit to test, how to group my input, what kind of gates(AND, NOT, OR etc..) should connect those inputs. So basically, my question how to design boolean Circuit which gives an answer TRUE or FALSE.
Last thing, what that answer means. I understand TRUE as this NP problem can be solved in polynomial time and FALSE cannot, am I correct?
It is huge mess in my mind, don't be really outrageous if I made logical mistakes explaining my question :) I hope you understood it.
Excitingly waiting for answers.
I understand the confusion but your understanding is not quite how it works.
NP-hardness is a qualification of decision problems, that is, a problem with answer is yes or no. If we want to show that a decision problem is NP-hard, we do so by showing that it is as least as hard as a problem of which we know it is NP-hard already, for instance SAT.
How can we show that problem A is at least as hard as problem B? Well, we can phrase that as
if we can solve A, we can also solve B
So, given an instance of problem B, we convert it to an instance of problem A, use our solution to A to solve it, and convert it back to a solution to B. Assuming that both conversations are easy, we know that A cannot be easier than B, since a solution to A is also a solution to B.
Your understanding thus had it backwards. In order to show that some problem is NP-hard, we to show that it is at least as hard as SAT, that is, given an arbitrary instance of SAT, convert it to an instance of your problem, and then solve that problem. If the answer is "yes", then the original SAT problem was satisfiable, otherwise it wasn't.
Now, as I wrote in a comment, there is no standard way to do the conversion. You somehow need to manipulate your problem, such that it "looks like SAT", in order to make the conversion. For some problems that's easier then others, but I'd claim it's the hardest part of the NP-hardness proof.
What people typically do instead is that they look for another problem, which is known to be NP-hard already, but looks a bit more like their own problem. That way, the reduction becomes a bit easier. But still it requires a lot of work and creativity. I recommend you look at some existing proofs to see how others do this.
Maybe this is very obvious, but if we had an algorithm in P (so this algorithm gives a yes/no answer in polynomial time), is there a more efficient way to find the solution beyond just guessing and checking?
So, suppose SAT is in P (I know this is an NP-Complete problem, but this seems like the best example for what I'm trying to ask). This means that there is a polynomial time algorithm that will tell you yes or no depending on whether or not the given input is satisfiable.
It would seem that there should thus be an efficient way to find/extract this satisfying assignment (rather than just know it exists, if there is one). However, I can't think of any efficient way to utilize this poly-time algorithm to find such an assignment.
** side note **
For maximization/minimization (e.g. Knapsack) problems I know that you can use binary search to find your solution, but my question is more pertaining to these non-maximization type problems like SAT
You don't have to guess the entire thing and then test it.
You can get a satisfying valuation (if it exists) like this:
Pick a variable, make it false, remove it from all clauses/remove satisfied clauses. Consult the SAT oracle, which apparently runs in polynomial time today. If it's still satisfiable, fine, keep it. Otherwise it must be true, restore the clauses and clean up the clauses again. There's no backtracking, there's just one call to SAT for every variable. That whole thing is still in polynomial time.
Or if that's what you had in mind, then well, that's it. Does it really matter though? Polynomial time is polynomial time, and this isn't usable in practice anyway so wall-clock time is hardly a concern.
I'm not sure if I'm completely understanding genetic algorithms and how they work, I'm trying to learn via ai4r http://ai4r.rubyforge.org/geneticAlgorithms.html
If in Job Shop Scheduling, which I believe can be solved by GA(?), isn't cost of any single job is based on how it related to it's predecessors? I was thinking I would calculate a cost based on the placement of the chromosome with a dynamic score of how well it is placed rather than a binary value, but I'm not sure this works.
Anybody have any experience with this? or does a GA only work when the difference between any two genomes is static?
I hope I have the right terminology here, as I mentioned, I'm just learning.
-----------------------update-----------------------------------
I think I'm using a bit of the wrong terminology here. I referred to 'fitness' when I think what I actually wanted to use was cost matrix.
The example I'm going from describes this
Each chromosome must represent a posible solution for the problem. This class conatins an array with the list of visited nodes (cities of the tour). The size of the tour is obtained automatically from the traveling costs matrix. You have to assign the costs matrix BEFORE you run the genetic search. The following costs matrix could be used to solve the problem with only 3 cities:
data_set = [ [ 0, 10, 5],
[ 6, 0, 4],
[25, 4, 0]
]
Ai4r::GeneticAlgorithm::Chromosome.set_cost_matrix(data_set)
so in my instance, I'm thinking the 'cost' of each chromosome is dynamic based on it's neighbours.
Since you asked in a comment to make this an answer, I took the liberty of summarizing my earlier responses as well so it's all in one place. The answer to the specific question of "what is a penalty term" is in item #3 below.
The way a standard genetic algorithm works is that each "chromosome" is a complete solution to the problem. In your case, an ordering for the jobs to be submitted. The confusion, I think, centers around the notion that because the individual contributions to fitness made by a particular job in that schedule varies according to the rest of the schedule, you must need something "dynamic". That's not really true. From the point of view of the GA, the only thing that has a fitness is the entire solution. So a dynamic problem is one in which the fitness of a whole schedule can change over time. Going back to the TSP, a dynamic problem would be one in which touring cities in order of A, B, C, D, and then E actually had a different distance each time you tried it. Even though the cost of a tour through B depends on which cities come before and after B in the tour, once you decide that, the costs are static, and because the GA only ever receives costs for entire tours, all it knows is that [A,B,C,D,E] has a constant fitness. No dynamic trickery needed.
Now, your second question was how to handle constraints like, for the TSP example, what if you need to ensure that the salesman gets to Paris by a certain time? Typically, there are three ways to try to handle this.
Never allow a solution to be generated in which he doesn't get there before 2:00. Sometimes this is easy, other times it's very hard. For instance, if the constraint was "he cannot start at city X", it's fairly easy to just not generate solutions that don't start with X. Often though, simply finding valid solutions can be hard, and so this approach doesn't really work.
Allow constraints to be violated, but fix them afterward. In the TSP example, you let crossover and mutation produce any possible tour, but then scan through it to see if he gets to Paris too late. If so, swap the position of Paris with some earlier city in the tour. Again though, sometimes it can be hard to figure out a good way to repair violations.
Penalize the fitness of an infeasible solution. Here, the idea is that even if I can't prevent him from getting to Paris too late and I can't fix it if he does, I can at least make the fitness arbitrarily worse. For TSP, the fitness is the length of the tour. So you might say that if a tour gets him to Paris too late, the fitness is the length of the tour + 100. That let's the solution stay in the population (it might be very good otherwise, so you want it to have a chance to pass on some of its genes), but you make it less likely to be selected, because your selection and replacement methods pick individuals with better fitness values.
For your JSP problem, typically you're looking to minimize the makespan. The same three options are available to you if you do have some constraints. But from what I can tell, you don't really have such constraints. I think you're trying to inject too much knowledge into the process rather than letting the evolutionary algorithm come up with it on its own. That is, you don't necessarily worry about telling the GA that some arrangements of jobs are better than others. You just assign higher fitness to the better ones and let the process converge.
That said, injecting information like this is often a really good thing to do, but you want to have a good understanding of the basic algorithm first. Let's say that we know that for TSP, it's more likely that a good solution will connect cities that are close to one another. The way I would use that information inside a GA would be to generate random solutions non-uniformly (perhaps with a greedy heuristic). I might also replace the standard crossover and mutation algorithms with something customized. Mutation is typically easier to do this with than crossover. To mutate a TSP solution, I might pick two connected cities, break the connection, and then look for a way to reconnect them that was "closer". That is, if a tour is [A,B,C,D,E,F,G,H], I might pick the edge [B,C] at random, and then look for another edge, maybe [F,G], such that when I connected them crossways to get [A,B,G,D,E,F,C,H], the total tour length was lower. I could even extend that mutation beyond one step -- create a loop that keeps trying to break and reconnect edges until it can't find a shorter tour. This leads to what is usually called a hybrid GA because it's a GA hybridized with a local search; sometimes also called a Memetic Algorithm. These sorts of algorithms usually outperform a black-box GA because you're giving the algorithm "hints" to bias it towards trying things you expect to be good.
I think this idea of a memetic algorithm is pretty close to what you were hitting on in your original question of wondering how to deal with the fact that the contribution to fitness from a particular job depends on where the other jobs are in the schedule. The only stumbling block there is that you were a bit unlucky in that the somewhat reasonable idea of thinking of this as "dynamic" leads you a bit astray, as "dynamic" actually means something entirely different here.
So to wrap up, there's nothing "dynamic" about your problem, so the things people do with GAs for dynamic problems will be entirely unhelpful. A standard GA will work with no fancy tricks. However, the idea of using information you have about what schedules work better can be introduced into the genetic operators, and will probably result in a significantly better overall algorithm.
You'd use GA to find say the best order to do a number of jobs in, or those jobs which made say best use of a day's resources. So yes they'd be related to each other.
So your fitness measure would be for seq 1,3,4,5,6,2.
Look at say find shortest path algorithm, starts to make sense then
Sometimes I come across interview questions like this: "Find the common parent of any 2 nodes in a tree". I noticed that they ask LCA questions also at Google, Amazon, etc.
As wikipedia says LCA can be found by intersection of the paths from the given nodes to the root and it takes O(H), where H is the height of the tree. Besides, there are more advanced algorithms that process trees in O(N) and answer LCA queries in O(1).
I wonder what exactly interviewers want to learn about candidates asking this LCA question. The first algorithm of paths intersection seems trivial. Do they expect the candidates to remember pre-processing algorithms ? Do they expect the candidates to invent these algorithms and data structures on the fly ?
They want to see what you're made of. They want to see how you think, how you tackle on problem, and handle stress from deadline. I suppose that if you solve the problem because you already know the solution then they will just pose you another problem. It's not the solution they want, it's your brainz.
With many algorithms questions in an interview, I don't care (or want) you to remember the answer. I want you to be able to derive the answer from the main principles you know.
Realistically: I could give two cares less if you already know the answer to "how to do X", if you are able to quickly construct an answer on the fly. Ultimately: in the real world, I can't necessarily assume you have experience tackling problems of domain X, but if you run across a problem in said domain, I will certainly hope you'll have the analytical ability to figure out a reasonable answer, based on the general knowledge you hold.
A tree is a common data structure it's safe to assume most will know - if you know the answer offhand, you should be able to explain it. If you don't know the answer, but understand the data structure, you should be able to fairy easily derive the answer.
I'm writing a program which solves a 24-puzzle (5x5 grid) using two heuristic. The first uses how many blocks the incorrect place and the second uses the Manhattan distance between the blocks current place and desired place.
I have different functions in the program which use each heuristic with an A* and a greedy search and compares the results (so 4 different parts in total).
I'm curious whether my program is wrong or whether it's a limitation of the puzzle. The puzzle is generated randomly with pieces being moved around a few times and most of the time (~70%) a solution is found with most searches, but sometimes they fail.
I can understand why greedy would fail, as it's not complete, but seeing as A* is complete this leads me to believe that there's an error in my code.
So could someone please tell me whether this is an error in my thinking or a limitation of the puzzle? Sorry if this is badly worded, I'll rephrase if necessary.
Thanks
EDIT:
So I"m fairly sure it's something I'm doing wrong. Here's a step-by-step list of how I'm doing the searches, is anything wrong here?
Create a new list for the fringe, sorted by whichever heuristic is being used
Create a set to store visited nodes
Add the initial state of the puzzle to the fringe
while the fringe isn't empty..
pop the first element from the fringe
if the node has been visited before, skip it
if node is the goal, return it
add the node to our visited set
expand the node and add all descendants back to the fringe
If you mean that sliding puzzle: This is solvable if you exchange two pieces from a working solution - so if you don't find a solution this doesn't tell anything about the correctness of your algorithm.
It's just your seed is flawed.
Edit: If you start with the solution and make (random) legal moves, then a correct algorithm would find a solution (as reversing the order is a solution).
It is not completely clear who invented it, but Sam Loyd popularized the 14-15 puzzle, during the late 19th Century, which is the 4x4 version of your 5x5.
From the Wikipedia article, a parity argument proved that half of the possible configurations are unsolvable. You are probably running into something similar when your search fails.
I'm going to assume your code is correct, and you implemented all the algorithms and heuristics correctly.
This leaves us with the "generated randomly" part of your puzzle initialization. Are you sure you are generating correct states of the puzzle? If you generate an illegal state, obviously there will be no solution.
While the steps you have listed seem a little incomplete, you have listed enough to ensure that your A* will reach a solution if there is one (albeit not optimal as long as you are just simply skipping nodes).
It sounds like either your puzzle generation is flawed or your algorithm isn't implemented correctly. To easily verify your puzzle generation, store the steps used to generate the puzzle, and run it in reverse and check if the result is a solution state before allowing the puzzle to be sent to the search routines. If you ever generate an invalid puzzle, dump the puzzle, and expected steps and see where the problem is. If the puzzle passes and the algorithm fails, you have at least narrowed down where the problem is.
If it turns out to be your algorithm, post a more detailed explanation of the steps you have actually implemented (not just how A* works, we all know that), like for instance when you run the evaluation function, and where you resort the list that acts as your queue. That will make it easier to determine a problem within your implementation.