I'm trying to reconcile two seemingly contradictory ideas:
The unbounded knapsack optimization problem is known to be NP-hard
My colleague and I think we can solve a minor variation on it in polynomial time using A*
Sounds crazy, right? That's what I think!
The variation of the problem is described in terms of a cargo plane that must unload some of its goods in order to reduce its payload to the plane's capacity. So there's a set of items each with a weight and a value, and a target weight which must be unloaded -- optimize the goods to unload so that you have at least W weight removed, and minimize the total value of the goods. Consider the unbounded problem where there are arbitrarily many items available each of N different types.
The proposed solution uses a graph which starts at a node (vertex) representing nothing unloaded. Each unload operation represents an edge, so the graph grows exponentially out from the starting point with every possible combination of goods unloaded. The destination node is a virtual aggregate in that all combinations with total weight >= the target are considered the target node. The total weight unloaded so far gets stored in each node and is used to determine whether the target has been reached or not. The cost of each edge is the value of the item being unloaded. So a shortest-path algorithm such as Dijkstra or A* will find the optimum set of goods.
Dijkstra clearly takes exponential time since it is exploring all possible combinations. But with an admissible heuristic, I think A* should run in polynomial time. And I think the following heuristic should work. For every good, calculate the "specific value" which is the ratio of value to weight. Pick the good with the highest specific value. As a heuristic for a given node, calculate the weight still needed to be unloaded times the maximum specific value. This provides an estimate which is either exactly correct in the case that the target weight can be achieved by an integer number of optimal goods, or in all other cases underestimates the distance (weight) remaining because the actual number of goods would need to be rounded up. So the heuristic is admissible.
I haven't proven runtime complexity in any rigorous way. But the way A* works, it will greedily add items towards the goal, exploring the best options quickly, which intuitively feels like it should run in polynomial time for N. And with a properly admissible heuristic the solution is guaranteed to be optimal.
So what's wrong with this solution? I absolutely do not believe we have found a novel solution to a well-studied problem by applying a well-known algorithm. But this seems like it should work.
This sounds like the standard branch and bound method for knapsack. It's good when there's variety in the ratios but devolves to exponential-time brute force when the ratios are the same.
Related
I am using A* in order to solve the Asymmetric Traveling Salesman problem.
My state representation has 4 variables:
1 - Visited cities (List)
2 - Unvisitied cities (List)
3 - Current City (Integer)
4 - Current Cost (Integer)
However, even tho I find many path-construction algorithms such as Nearest Neighbor, k-opt and so on, I can't find an heuristic suitable for A*, which is, a h(n) function that takes a state as input and returns an integer corresponding to that state's quality.
So my question is, are there such heuristics? Any recommendations?
Thanks in advance
The weight of the minimum spanning tree of the subgraph that contains all unvisited vertices and the current vertex is a lower bound for the cost to finish the current path. It can be used with the A* algorithm as it can't overestimate the remaining distance (otherwise, the weight of the remaining path is smaller than the weight of minimum spanning tree and it spans the given vertices, which is a contradiction).
I've never tried it though so I don't know how well it'll work in practice.
There always are: h(n) = 0 always works. It is useless, turning A* into Dijkstra, but it's definitely admissible.
An other obvious one: let h(n) be the shortest edge from the current city back to the beginning. Still a huge underestimation, but at least it's not necessarily zero. It's obviously valid, the loop has to be closed eventually and (given this partial route) there is no shorter way to do it.
You can be a bit more clever here, for example you could use linear programming (make two variables for each edge, one for each direction, then for every city make a constraint forcing the sum of entering edges to be 1 and a constraint forcing the sum of exiting edges to be one, weights are obviously the distances) to find an underestimation of the length from the current node back to the beginning while touching every city in the set of unvisited cities. Of course if you're doing that, you might as well drop A* and just use the usual integer linear programming tricks. A* doesn't seem like a good fit here (especially in the beginning, the branching factor is too high and the heuristics won't guide it enough yet), but I haven't tried this so who knows.
Also, given the solution from the LP, you can improve it a lot by using some simple tricks (and some advanced tricks that whole books have been written about, but let's not go there, read the books if you want to know). For example, one thing the LP likes to do is form lots of little triangles. This will satisfy the degree constraints everywhere locally and keeps everything nice and short. But it's not a tour, and forcing it be more like a tour will make the heuristic higher=better. To remove the sub-tours, you can detect them in the fractional solution and then force the number of entries to the subgraph to be at least 1 (it may have to become more than 1 at some point, so don't force it to be exactly 1) and force the number of exits to be at least 1, by adding the corresponding constraints and solving again. There are many more tricks, but this should already give a very reasonable heuristic, much closer to the actual cost than using any of the overestimating heuristics and dividing them by their worst case overestimation factor. The problem with those is that usually the heuristic is pretty good, much better than their worst case factor, and then dividing by the worst case factor really kills the quality of the heuristic.
I have a graph that represents a city. I know the location of places of interest (nodes, which have a Importance value), the location of the hotel I'm staying in, how the nodes are connected, the traversal time between them and have acess to latitude and longitude. There are no issues converting from time to distance and vice-versa.
The objective is to tour the city, maximizing the importance per day but limiting one day of travel to 10 hours. A day begins and ends at the hotel. I have a working A* algorithm that chooses the lowest value but with no heuristic yet, which I guess makes it a BB for now. With that in mind:
Since I have access to Lat/Long, my first stab at an heuristic, while
only dealing with times, would be the distance as the crow flies
between a node and the hotel. Would this be an admissible heuristic?
It gives me the shortest possible distance and time, so it wouldn't
overestimate.
Now let's say the Importance of a node is between 1-4. In order to factor it in, one idea could be g(neighbor) = g(current) + (edge_cost / Importance^2). Assuming this would be valid (if not, why?):
But now the heuristic values would be in a different unit. Could a solution to this simply be give the Hotel Importance = 1? If the value is the same, will it still be admissible? EDIT: I think this will end up giving me problems because of the difference in scale.
I still have to restrict the total amount of time. Should each node keep track of the total time spent, in order to compare to the limit, plus the g() and h() values, because of the different units?
And finally:
Since I have to start and end in the same node, what comes to mind is to explore a node and should I find the hotel see if I still have time to explore the neighbors instead of going back. However, if I still have time to expand to one more node, but time runs out and I can't get to the hotel from there, I'm assuming I'll have to backtrack to the parent.
I can't help but see similarities to the knapsack problem. Even though I have to use A*, is there any lesson I can take from it?
Must my heuristic be consistent in this case? If so, why?
By the way, the purpose here is pathfinding first, optimizations second.
This actually looks like a combination of the travelling salesman problem (TSP) and knapsack problem (KP). It's KP in this respect: the knapsack capacity is 10 (for total hours available in a day) and the locations are the items. The item value equals the location value. The item weight is equal to the time it takes to travel to the location (plus the location's portion of the trip back to the hotel). The challenge arises from the fact that an item's weight is unknown until you solve the optimal tour through the selected locations--enter the TSP and Pathfinding.
One approach might be to use a pathfinding algorithm (e.g. A*, Bellman–Ford, or Dijkstra's algorithm) primarily to compute a distance matrix between each node. The distance matrix can then be leveraged while solving the TSP portion of the problem: finding a tour through the locations and using the total time as the weight.
The next step is up to you. If you are looking for an approximate solution, many heuristics exist for both TSP and KP: See Christofides TSP Heuristic, or the Minimum TSP and Maximum Knapsack entries at the Compendium of NP Optimization problems.
If on the other hand you seek an optimal solution, you may be out of luck. Still I recommend you find a copy of Graph Theory. An Algorithmic Approach by Nicos Christofides (ISBN-13: 978-0121743505). It provides heuristics for early backtracking in a Depth-First-Search that expedite the search for optimal solutions to several NP-Complete problems.
Given an undirected graph G = G(V, E), how can I find the size of the largest clique in it in polynomial time? Knowing the number of edges, I could put an upper limit on the maximal clique size with
https://cs.stackexchange.com/questions/11360/size-of-maximum-clique-given-a-fixed-amount-of-edges
, and then I could iterate downwards from that upper limit to 1. Since this upper cap is O(sqrt(|E|)), I think I can check for the maximal clique size in O(sqrt(|E|) * sqrt(|E|) * sqrt(|E|)) time.
Is there a more efficient way to solve this NP-complete problem?
Finding the largest clique in a graph is the clique number of the graph and is also known as the maximum clique problem (MCP). This is one of the most deeply studied problems in the graph domain and is known to be NP-Hard so no polynomial time algorithm is expected to be found to solve it in the general case (there are particular graph configurations which do have polynomial time algorithms). Maximum clique is even hard to approximate (i.e. find a number close to the clique number).
If you are interested in exact MCP algorithms there have been a number of important improvements in the past decade, which have increased performance in around two orders of magnitude. The current leading family of algorithms are branch and bound and use approximate coloring to compute bounds. I name the most important ones and the improvement:
Branching on color (MCQ)
Static initial ordering in every subproblem (MCS and BBMC)
Recoloring: MCS
Use of bit strings to encode the graph and the main operations (BBMC)
Reduction to maximum satisfiability to improve bounds (MaxSAT)
Selective coloring (BBMCL)
and others.
It is actually a very active line of research in the scientific community.
The top algorithms are currently BBMC, MCS and I would say MaxSAT. Of these probably BBMC and its variants (which use a bit string encoding) are the current leading general purpose solvers. The library of bitstrings used for BBMC is publicly available.
Well I was thinking a bit about some dynamic programming approach and maybe I figured something out.
First : find nodes with very low degree (can be done in O(n)). Test them, if they are part of any clique and then remove them. With a little "luck" you can crush graph into few separate components and then solve each one independently (which is much much faster).
(To identify component, O(n) time is required).
Second : For each component, you can find if it makes sense to try to find any clique of given size. How? Lets say, you want to find clique of size 19. Then there has to exist at least 19 nodes with at least 19 degree. Otherwise, such clique cannot exist and you dont have to test it.
I am trying to find a optimal solution for the following problem
The numbers denoted inside each node are represented as (x,y).
The adjacent nodes to a node always have a y value that is (current nodes y value +1).
There is a cost of 1 for a change in the x value when we go from one node to its adjacent
There is no cost for going from node to its adjacent, if there is no change in the value of x.
No 2 nodes with the same y value are considered adjacent.
The optimal solution is the one with the lowest cost, I'm thinking of using A* path finding algorithm for finding an optimal solution.
My question, Is A* a good choice for the this kind of problems, or should i look at any other algorithm, and also i was thinking of using recursive method to calculate the Heuristic cost, but i get the feeling that it is not a good idea.
This is the example of how I'm thinking the heuristic function will be like this
The heuristic weight of a node = Min(heuristic weight of it's child nodes)
The same goes for the child nodes too.
But as far as my knowledge goes, heuristic is meant to be an approximate, so I think I'm going in the wrong direction as far as the heuristic function is concerned
A* guarantees to find the lowest cost path in a graph with nonnegative edge path costs, provided that you use an appropriate heuristic. What makes a heuristic function appropriate?
First, it must be admissible, i. e. it should, for any node, produce either an underestimate or a correct estimate for the cost of the cheapest path from that node to any of goal nodes. This means the heuristic should never overestimate the cost to get from the node to the goal.
Note that if your heuristic computes the estimate cost of 0 for every node, then A* just turns into breadth-first exhaustive search. So h(n)=0 is still an admissible heuristic, only the worst possible one. So, of all admissible heuristics, the tighter one estimates the cost to the goal, the better it is.
Second, it must be cheap to compute. It should be certainly O(1), and should preferably look at the current node alone. Recursively evaluating the cost as you propose will make your search significantly slower, not faster!
The question of A* applicability is thus whether you can come up with a reasonably good heuristic. From your problem description, it is not clear whether you can easily come up with one.
Depending on the problem domain, A* may be very useful if requirements are relaxed. If heuristic becomes inadmissible, then you lose the guarantee of finding the best path. Depending on the degree of overestimation of the distance, hovewer, the solution might still be good enough (for problem specific definition of "good enough"). The advantage is that sometimes you can compute that "good enough" path much faster. In some cases, probabilistic estimate of heuristics works good (it can have additional constraints on it to stay in the admissible range).
So, in general, you have breadth-first search for tractable problems, next faster you have A* for tractable problems with admissible heuristic. If your problem is intractable for breadth-first exhaustive search and does not admit a heuristic, then your only option is to settle for a "good enough" suboptimal solution. Again, A* may still work with inadmissible heuristic here, or you should look at beam search varieties. The difference is that beam searches have a limit on the number of ways the graph is currently being explored, while A* limits them indirectly by choosing some subset of less costly ones. There are practical cases not solvable by A* even with relaxed admissbility, when difference in cost among different search path is slight. Beam search with its hard limit on the number of paths works more efficiently in such problems.
Seems like an overkill to use A* when something like Dijkstra's would work. Dijkstra's will only work on non-negative cost transitions which seems true in this example. If you can have negative cost transitions then Bellman-Ford should also work.
A-star is used to find the shortest path between a startnode and an endnode in a graph. What algorithm is used to solve something were the target state isn't specifically known and we instead only have a criteria for the target state?
For example, can a sudoku puzzle be solved with an Astar-like algorithm? We dont know how the endstate will look like (which number is where) but we do know the rules of sudoku, a criteria for a winning state. Therefore I have a startnode and just a criteria for the endnode, which algorithm to use?
A* requires a graph, a cost function for traversal of that graph, a heuristic as to whether a node in the graph is closer to the goal than another, and a test whether the goal is reached.
Searching a Sudoku solution space doesn't really have a traversal cost to minimize, only a global cost ( the number of unsolved squares ), so all traversals would be equal cost, so A* doesn't really help - any cell you could assign costs one move and moves you one closer to the goal, so A* would be no better than choosing the next step at random.
It might be possible to apply an A* search based on the estimated/measured cost of applying the different techniques at each point, which would then try to find a path through the solution space with the least computational cost. In that case the graph would not just be the solution states of the puzzle, but you'd be choosing between the techniques to apply at that point - you'd know an estimate of the cost of a transition, but not where that transition 'goes', except that if successful, it's one step closer to the goal.
Yes, A* can be used when a specific goal state cannot be identified. (Pete Kirkham's answer implies this, but doesn't emphasise it much.)
When a specific goal state can't be identified, it's sometimes harder to come up with a useful heuristic lower bound on the remaining cost needed to complete a partial solution -- and the efficiency of A* depends on choosing an effective heuristic. But it doesn't mean it can't be applied. Any problem that can be solved on a computer can be solved using a breadth-first search, plus an array of flags indicating whether a state has been seen before; which is the same as A* with a heuristic lower bound that is always zero. (Of course, this is not the most efficient algorithm for solving many problems.)
You dont have to know the exact target endstate. It all comes down to the heuristic function, when it returns 0 you could assume to have found (at least) one of the valid endstates.
So during the a*, instead of checking if current_node == target_node, check if current_node.h() returns 0. If so, it should be infinitely close and/or overlapping the goal/endstate.