I just finished with the first module in the algo specialization course in coursera.
There was an exam question that i could not quite understand. I have passed that exam, so there's no point for me to retake it.
Out of curiosity, I want to learn the principles around this question.
The question was posted as such:
Suppose that a randomized algorithm succeeds (e.g., correctly computes
the minimum cut of a graph) with probability p (with 0 < p < 1). Let ϵ
be a small positive number (less than 1).
How many independent times do you need to run the algorithm to ensure
that, with probability at least 1−ϵ, at least one trial succeeds?
The options given were:
log(1−p)/logϵ
log(p)/logϵ
logϵ/log(p)
logϵ/log(1−p)
I made two attempts and both were wrong. My attempts were:
log(1−p)/logϵ
logϵ/log(1−p)
It's not so much I want to know the right answer. I want to learn the principles behind this question and what it's asking for. So that I know how to answer similar questions in future.
I have posted this on the forum, but nobody answered after a month. So I am trying it out here.
NO need to post the answer directly. If you got me to get to aha moment, i will mark it as correct.
Thanks.
How many independent times do you need to run the algorithm to ensure that, with probability at least 1−ϵ, at least one trial succeeds?
Let's rephrase it a bit:
What is the smallest number of independent trials such that the probability of all of them failing is less than or equal to ϵ?
By the definition of independent events, the probability of all of them occurring is the product of their individual probabilities. Since the probability of one trial failing is (1-p), the probability of n trials failing is (1-p)^n.
This gives us an inequality for n:
(1-p)^n <= ϵ
Related
I am looking for a faster way to solve this problem:
Let's suppose we have n boxes and n marbles (each of them has a different kind). Every box can contain only some kinds of marbles (It is shown it the example below), and only one marble fits inside one box. Please read the edits. The whole algorithm has been described in the post linked below but it was not precisely described, so I am asking for a reexplenation.
The question is: In how many ways can I put marbles inside the boxes in polynomial time?
Example:
n=3
Marbles: 2,5,3
Restrictions of the i-th box (i-th box can only contain those marbles): {5,2},{3,5,2},{3,2}
The answer: 3, because the possible positions of the marbles are: {5,2,3},{5,3,2},{2,5,3}
I have a solution which works in O(2^n), but it is too slow. There are also one limitation about the boxes tolerance, which I don't find very important, but I will write them also. Each box has it's own kind-restrictions, but there is one list of kinds which is accepted by all of them (in the example above this widely accepted kind is 2).
Edit: I have just found this question but I am not sure if it works in my case, and the dynamic solution is not well described. Could somebody clarify this? This question was answered 4 years ago, so I won't ask it there. https://math.stackexchange.com/questions/2145985/how-to-compute-number-of-combinations-with-placement-restrictions?rq=1
Edit#2: I also have to mention that excluding widely-accepted list the maximum size of the acceptance list of a box has 0 1 or 2 elements.
Edit#3: This question refers to my previos question(Allowed permutations of numbers 1 to N), which I found too general. I am attaching this link because there is also one more important information - the distance between boxes in which a marble can be put isn't higher than 2.
As noted in the comments, https://cs.stackexchange.com/questions/19924/counting-and-finding-all-perfect-maximum-matchings-in-general-graphs is this problem, with links to papers on how to tackle it, and counting the number of matchings is #P-complete. I would recommend finding those papers.
As for dynamic programming, simply write a recursive solution and then memoize it. (That's top down, and is almost always the easier approach.) For the stack exchange problem with a fixed (and fairly small) number of boxes, that approach is manageable. Unfortunately in your variation with a large number of boxes, the naive recursive version looks something like this (untested, probably buggy):
def solve (balls, box_rules):
ball_can_go_in = {}
for ball in balls:
ball_can_go_in[ball] = set()
for i in range(len(box_rules)):
for ball in box_rules[i]:
ball_can_go_in[ball].add(i)
def recursive_attempt (n, used_boxes):
if n = len(balls):
return 1
else:
answer = 0
for box in ball_can_go_in[balls[n]]:
if box not in used_boxes:
used_boxes.add(box)
answer += recursive_attempt(n+1, used_boxes)
used_boxes.remove(box)
return answer
return recursive_attempt(0, set())
In order to memoize it you have to construct new sets, maybe use bit strings, BUT you're going to find that you're calling it with subsets of n things. There are an exponential number of them. Unfortunately this will take exponential time AND use exponential memory.
If you replace the memoizing layer with an LRU cache, you can control how much memory it uses and probably still get some win from the memoizing. But ultimately you will still use exponential or worse time.
If you go that route, one practical tip is sort the balls by how many boxes they go in. You want to start with the fewest possible choices. Since this is trying to reduce exponential complexity, it is worth quite a bit of work on this sorting step. So I'd first pick the ball that goes in the fewest boxes. Then I'd next pick the ball that goes in the fewest new boxes, and break ties by fewest overall. The third ball will be fewest new boxes, break ties by fewest boxes not used by the first, break ties by fewest boxes. And so on.
The idea is to generate and discover forced choices and conflicts as early as possible. In fact this is so important that it is worth a search at every step to try to discover and record forced choices and conflicts that are already visible. It feels counterintuitive, but it really does make a difference.
But if you do all of this, the dynamic programming approach that was just fine for 5 boxes will become faster, but you'll still only be able to handle slightly larger problems than a naive solution. So go look at the research for better ideas than this dynamic programming approach.
(Incidentally the inclusion-exclusion approach has a term for every subset, so it also will blow up exponentially.)
I was discussing with a co-worker a problem we were having with a piece of software we deploy, and he mentioned how it was similar to the conceptual problem of booking rooms over a course of time and the algorithm should output the room bookings that requires the least switches (so for example, an optimal solution may be staying in one room for 3 days, and the rest in another room, only requiring two switches).
Is there a name for such a problem in algorithms?
Originally I posted something regarding the minimum set cover problem. Although you can describe your problem as a minimum set cover problem, if we assume "room bookings" are over consecutive days, your problem can be more succinctly described with a different problem.
The interval cover problem1 consists of one big interval (call it (a,b)), and a bunch of subintervals (call them (ai, bi)). Our goal is to cover the one big interval with as few subintervals as possible.
Finding the minimal coverage of an interval with subintervals is a question posted about 5 years ago which asks for an efficient solution, and the accepted answer shows that the greedy solution is optimal. Within the context of room bookings, the "greedy solution" would be basically to start from the beginning of the period and always pick the booking with the latest end date.
The idea of course with this problem is that the each "subinterval" is a booking, so the fewer subintervals we need, the fewer bookings, and hence the fewer "switches" we need.
1 I'm not actually 100% sure that this is the correct name, but if you were to say "interval cover problem", the listener would probably think of the same thing.
I'm trying to find the best algorithmic solution to the following problem. It's a real-world problem, but I'm going to present it in an abstract manner.
There is a community of 1000 people. Each user is provided with a set number of tickets. There are four types of tickets (each one corresponds to a different event). However, some people are willing to make trades (for example, I want one A-ticket and am willing to give up two B-tickets). Moreover, some people have extra tickets that they are willing to give away for nothing (for example, I'll give away two C-tickets to whoever wants them). Assuming that I know what each person is willing to give away / trade, how do I satisfy the most number of people?
I tried Googling, but I don't know how to word this problem to avoid getting results related to algorithmic trading of financial instruments.
Thanks.
Given that it has multiple dimensions, it is likely an NP-complete problem. It has parallels to a multi-dimensional knapsack problem.
Therefore, I recommend trying a backtracking approach.
Start with everybody involved in the trade.
Sort the people who cause the most deficit descending (here you can weight the deficit caused by each ticket type by the shortfall in each ticket).
Then in a backtracking manner, kick the person causing the next highest deficit person out of the trade.
Repeat until you either have no more deficit in any ticket (record as possible answer), or you have kicked everybody out.
When that happens, backtrack 1 step and continue (if you already tried kicking the highest deficit, kick the next highest deficit causing person).
Repeat until end or you run out of time. Get the optimal answer out of the possible answers you found.
If the problem is too hard, it will probably run out of time. Otherwise, this algorithm should give you a reasonable answer (perhaps near to optimal).
How well this method works depends on how generous/greedy the people are, how many people there are, and how fast your computer is.
Look for a bipartite minimum weigth matching problem. The idea is to find the shortest distance from i to j using vertices 1 .. k only.
I've always been writing software to solve business problems. I came across about LIP while I was going through one of the SO posts. I googled it but I am unable to relate how I can use it to solve business problems. Appreciate if some one can help me understand in layman terms.
ILP can be used to solve essentially any problem involving making a bunch of decisions, each of which only has several possible outcomes, all known ahead of time, and in which the overall "quality" of any combination of choices can be described using a function that doesn't depend on "interactions" between choices. To see how it works, it's easiest to restrict further to variables that can only be 0 or 1 (the smallest useful range of integers). Now:
Each decision requiring a yes/no answer becomes a variable
The objective function should describe the thing we want to maximise (or minimise) as a weighted combination of these variables
You need to find a way to express each constraint (combination of choices that cannot be made at the same time) using one or more linear equality or inequality constraints
Example
For example, suppose you have 3 workers, Anne, Bill and Carl, and 3 jobs, Dusting, Typing and Packing. All of the people can do all of the jobs, but they each have different efficiency/ability levels at each job, so we want to find the best task for each of them to do to maximise overall efficiency. We want each person to perform exactly 1 job.
Variables
One way to set this problem up is with 9 variables, one for each combination of worker and job. The variable x_ad will get the value 1 if Anne should Dust in the optimal solution, and 0 otherwise; x_bp will get the value 1 if Bill should Pack in the optimal solution, and 0 otherwise; and so on.
Objective Function
The next thing to do is to formulate an objective function that we want to maximise or minimise. Suppose that based on Anne, Bill and Carl's most recent performance evaluations, we have a table of 9 numbers telling us how many minutes it takes each of them to perform each of the 3 jobs. In this case it makes sense to take the sum of all 9 variables, each multiplied by the time needed for that particular worker to perform that particular job, and to look to minimise this sum -- that is, to minimise the total time taken to get all the work done.
Constraints
The final step is to give constraints that enforce that (a) everyone does exactly 1 job and (b) every job is done by exactly 1 person. (Note that actually these steps can be done in any order.)
To make sure that Anne does exactly 1 job, we can add the constraint that x_ad + x_at + x_ap = 1. Similar constraints can be added for Bill and Carl.
To make sure that exactly 1 person Dusts, we can add the constraint that x_ad + x_bd + x_cd = 1. Similar constraints can be added for Typing and Packing.
Altogether there are 6 constraints. You can now supply this 9-variable, 6-constraint problem to an ILP solver and it will spit back out the values for the variables in one of the optimal solutions -- exactly 3 of them will be 1 and the rest will be 0. The 3 that are 1 tell you which people should be doing which job!
ILP is General
As it happens, this particular problem has a special structure that allows it to be solved more efficiently using a different algorithm. The advantage of using ILP is that variations on the problem can be easily incorporated: for example if there were actually 4 people and only 3 jobs, then we would need to relax the constraints so that each person does at most 1 job, instead of exactly 1 job. This can be expressed simply by changing the equals sign in each of the 1st 3 constraints into a less-than-or-equals sign.
First, read a linear programming example from Wikipedia
Now imagine the farmer producing pigs and chickens, or a factory producing toasters and vacuums - now the outputs (and possibly constraints) are integers, so those pretty graphs are going to go all crookedly step-wise. That's a business application that is easily represented as a linear programming problem.
I've used integer linear programming before to determine how to tile n identically proportioned images to maximize screen space used to display these images, and the formalism can represent covering problems like scheduling, but business applications of integer linear programming seem like the more natural applications of it.
SO user flolo says:
Use cases where I often met it: In digital circuit design you have objects to be placed/mapped onto certain parts of a chip (FPGA-Placing) - this can be done with ILP. Also in HW-SW codesign there often arise the partition problem: Which part of a program should still run on a CPU and which part should be accelerated on hardware. This can be also solved via ILP.
A sample ILP problem will looks something like:
maximize 37∙x1 + 45∙x2
where
x1,x2,... should be positive integers
...but, there is a set of constrains in the form
a1∙x1+b1∙x2 < k1
a2∙x1+b2∙x2 < k2
a3∙x1+b3∙x2 < k3
...
Now, a simpler articulation of Wikipedia's example:
A farmer has L m² land to be planted with either wheat or barley or a combination of the two.
The farmer has F grams of fertilizer, and P grams of insecticide.
Every m² of wheat requires F1 grams of fertilizer, and P1 grams of insecticide
Every m² of barley requires F2 grams of fertilizer, and P2 grams of insecticide
Now,
Let a1 denote the selling price of wheat per 1 m²
Let a2 denote the selling price of barley per 1 m²
Let x1 denote the area of land to be planted with wheat
Let x2 denote the area of land to be planted with barley
x1,x2 are positive integers (Assume we can plant in 1 m² resolution)
So,
the profit is a1∙x1 + a2∙x2 - we want to maximize it
Because the farmer has a limited area of land: x1+x2<=L
Because the farmer has a limited amount of fertilizer: F1∙x1+F2∙x2 < F
Because the farmer has a limited amount of insecticide: P1∙x1+P2∙x2 < P
a1,a2,L,F1,F2,F,P1,P2,P - are all constants (in our example: positive)
We are looking for positive integers x1,x2 that will maximize the expression stated, given the constrains stated.
Hope it's clear...
ILP "by itself" can directly model lots of stuff. If you search for LP examples you will probably find lots of famous textbook cases, such as the diet problem
Given a set of pills, each with a vitamin content and a daily vitamin
quota, find the cheapest cocktail that matches the quota.
Many such problems naturally have instances that require varialbe to be integers (perhaps you can't split pills in half)
The really interesting stuff though is that actually a big deal of combinatorial problems reduce to LP. One of my favourites is the assignment problem
Given a set of N workers, N tasks and an N by N matirx describing how
much each worker charges for the each task, determine what task to
give to each worker in order to minimize cost.
Most solution that naturally come up have exponential complexity but there is a polynomial solution using linear programming.
When it comes to ILP, ILP has the added benefit/difficulty of being NP-complete. This means that it can be used to model a very wide range of problems (boolean satisfiability is also very popular in this regard). Since there are many good and optimized ILP solvers out there it is often viable to translate an NP-complete problem into ILP instead of devising a custom algorithm of your own.
You can apply linear program easily everywhere you want to optimize and the target function is linear. You can make schedules (I mean big, like train companies, who need to optimize the utilization of the vehicles and tracks), productions (optimize win), almost everything. Sometimes it is tricky to formulate your problem as IP and/or sometimes you meet the problem that your solution is, that you have to produce e.g. 0.345 cars for optimum win. That is of course not possible, and so you constraint even more: Your variable for the number of cars must be integer. Even when it now sounds simpler (because you have infinite less choices for your variable), its actually harder. In this moment it gets NP-hard. Which actually means you can solve ANY problem from your computer with ILP, you just have to transform it.
For you I would recommend an intro into reading some basic (I)LP stuff. From my mind I dont know any good online site (but if you goolge you will find some), as book I can recommend Linear Programming from Chvatal. It has very good examples, and describes also real use cases.
The other answers here have excellent examples. Two of the gold standards in business of using integer programming and more generally operations research are
the journal Interfaces published by INFORMS (The Institute for Operations Research and the Management Sciences)
winners of the the Franz Edelman Award for Achievement in Operations Research and the Management Sciences
Interfaces publishes research that uses operations research applied to real-world problems, and the Edelman award is a highly competitive award for business use of operations research techniques.
My best shot so far:
A delivery vehicle needs to make a series of deliveries (d1,d2,...dn), and can do so in any order--in other words, all the possible permutations of the set D = {d1,d2,...dn} are valid solutions--but the particular solution needs to be determined before it leaves the base station at one end of the route (imagine that the packages need to be loaded in the vehicle LIFO, for example).
Further, the cost of the various permutations is not the same. It can be computed as the sum of the squares of distance traveled between di -1 and di, where d0 is taken to be the base station, with the caveat that any segment that involves a change of direction costs 3 times as much (imagine this is going on on a railroad or a pneumatic tube, and backing up disrupts other traffic).
Given the set of deliveries D represented as their distance from the base station (so abs(di-dj) is the distance between two deliveries) and an iterator permutations(D) which will produce each permutation in succession, find a permutation which has a cost less than or equal to that of any other permutation.
Now, a direct implementation from this description might lead to code like this:
function Cost(D) ...
function Best_order(D)
for D1 in permutations(D)
Found = true
for D2 in permutations(D)
Found = false if cost(D2) > cost(D1)
return D1 if Found
Which is O(n*n!^2), e.g. pretty awful--especially compared to the O(n log(n)) someone with insight would find, by simply sorting D.
My question: can you come up with a plausible problem description which would naturally lead the unwary into a worse (or differently awful) implementation of a sorting algorithm?
I assume you're using this question for an interview to see if the applicant can notice a simple solution in a seemingly complex question.
[This assumption is incorrect -- MarkusQ]
You give too much information.
The key to solving this is realizing that the points are in one dimension and that a sort is all that is required. To make this question more difficult hide this fact as much as possible.
The biggest clue is the distance formula. It introduces a penalty for changing directions. The first thing an that comes to my mind is minimizing this penalty. To remove the penalty I have to order them in a certain direction, this ordering is the natural sort order.
I would remove the penalty for changing directions, it's too much of a give away.
Another major clue is the input values to the algorithm: a list of integers. Give them a list of permutations, or even all permutations. That sets them up to thinking that a O(n!) algorithm might actually be expected.
I would phrase it as:
Given a list of all possible
permutations of n delivery locations,
where each permutation of deliveries
(d1, d2, ...,
dn) has a cost defined by:
Return permutation P such that the
cost of P is less than or equal to any
other permutation.
All that really needs to be done is read in the first permutation and sort it.
If they construct a single loop to compare the costs ask them what the big-o runtime of their algorithm is where n is the number of delivery locations (Another trap).
This isn't a direct answer, but I think more clarification is needed.
Is di allowed to be negative? If so, sorting alone is not enough, as far as I can see.
For example:
d0 = 0
deliveries = (-1,1,1,2)
It seems the optimal path in this case would be 1 > 2 > 1 > -1.
Edit: This might not actually be the optimal path, but it illustrates the point.
YOu could rephrase it, having first found the optimal solution, as
"Give me a proof that the following convination is the most optimal for the following set of rules, where optimal means the smallest number results from the sum of all stage costs, taking into account that all stages (A..Z) need to be present once and once only.
Convination:
A->C->D->Y->P->...->N
Stage costs:
A->B = 5,
B->A = 3,
A->C = 2,
C->A = 4,
...
...
...
Y->Z = 7,
Z->Y = 24."
That ought to keep someone busy for a while.
This reminds me of the Knapsack problem, more than the Traveling Salesman. But the Knapsack is also an NP-Hard problem, so you might be able to fool people to think up an over complex solution using dynamic programming if they correlate your problem with the Knapsack. Where the basic problem is:
can a value of at least V be achieved
without exceeding the weight W?
Now the problem is a fairly good solution can be found when V is unique, your distances, as such:
The knapsack problem with each type of
item j having a distinct value per
unit of weight (vj = pj/wj) is
considered one of the easiest
NP-complete problems. Indeed empirical
complexity is of the order of O((log
n)2) and very large problems can be
solved very quickly, e.g. in 2003 the
average time required to solve
instances with n = 10,000 was below 14
milliseconds using commodity personal
computers1.
So you might want to state that several stops/packages might share the same vj, inviting people to think about the really hard solution to:
However in the
degenerate case of multiple items
sharing the same value vj it becomes
much more difficult with the extreme
case where vj = constant being the
subset sum problem with a complexity
of O(2N/2N).
So if you replace the weight per value to distance per value, and state that several distances might actually share the same values, degenerate, some folk might fall in this trap.
Isn't this just the (NP-Hard) Travelling Salesman Problem? It doesn't seem likely that you're going to make it much harder.
Maybe phrasing the problem so that the actual algorithm is unclear - e.g. by describing the paths as single-rail railway lines so the person would have to infer from domain knowledge that backtracking is more costly.
What about describing the question in such a way that someone is tempted to do recursive comparisions - e.g. "can you speed up the algorithm by using the optimum max subset of your best (so far) results"?
BTW, what's the purpose of this - it sounds like the intent is to torture interviewees.
You need to be clearer on whether the delivery truck has to return to base (making it a round trip), or not. If the truck does return, then a simple sort does not produce the shortest route, because the square of the return from the furthest point to base costs so much. Missing some hops on the way 'out' and using them on the way back turns out to be cheaper.
If you trick someone into a bad answer (for example, by not giving them all the information) then is it their foolishness or your deception that has caused it?
How great is the wisdom of the wise, if they heed not their ego's lies?