I know if this were one variable I could easily find the maximum by hand, but I now have two variables to solve for.
I want to maximize f(x) = delay(v_dd, v_t) * energy(v_dd, v_t)
Both delay and energy depend on a I = K_1 * (v_dd - v_t)**2 And v_dd must be between [.3, 1] and v_t = v_dd - .1
What is the general technique to maximize a function of two variables with constraints?
Would the general technique be
def edp(v_dd, v_t):
e_total = v_dd ** 2 + v_dd
return e_total
fmin(-1*edp)
How do I enforce the constraints?
Related
I have a curvefit problem
I have two functions
y = ax+b
y = ax^2+bx-2.3
I have one set of data each for the above functions
I need to find a and b using least square method combining both the functions
I was using fminsearch function to minimize the sum of squares of errors of these two functions.
I am unable to use this method in lsqcurvefit
Kindly help me
Regards
Ram
I think you'll need to worry less about which library routine to use and more about the math. Assuming you mean vertical offset least squares, then you'll want
D = sum_{i=1..m}(y_Li - a x_Li + b)^2 + sum_{i=j..n}(y_Pj - a x_Pj^2 - b x_Pj + 2.3)^2
where there are m points (x_Li, y_Li) on the line and n points (x_Pj, y_Pj) on the parabola. Now find partial derivatives of D with respect to a and b. Setting them to zero provides two linear equations in 2 unknowns, a and b. Solve this linear system.
y = ax+b
y = ax^2+bx-2.3
In order to not confuse y of the first equation with y of the second equation we use distinct notations :
u = ax+b
v = ax^2+bx+c
The method of linear regression combined for the two functions is shown on the joint page :
HINT : If you want to find by yourself the matrixial equation appearing above, follow the Gene's answer.
Imagine I have person 1, 2, 3 and 4, then I have shirt styles A, B, C, D and I want to distribute the shirt styles to the people such that 25% of them get style A, 25% get style B, 25% get style C and 25% get style D but some of the people refuse to wear certain styles, these people are represented by Fs. How can I randomly match all the people with the styles they are willing to wear to get the approximate distribution?
A B C D
1 T F T T
2 T F F F
3 T T T T
4 T T T F
In this case this is easy and 25% is can be fully achieved, just give each person a different style. However, I intend to take this problem beyond this simple situation, my solution has to be generic. The number or styles, the number of people, and the distribution is all variable. Sometimes, the distribution will be impossible to create 100% accurately, approximate/close/best effor is expected. The selection process should be random and attempt to maintain the distribution.
I'm pretty agnostic to the language here, I'm just seeking the algorithm. Though preferably it would be able to be distributed.
Finding a solution when you are hampered by the Fs is https://en.wikipedia.org/wiki/Assignment_problem. One way to select an arbitrary assignment when there are many would be to set random costs where a style is acceptable to a person and then let it find the assignment with lowest possible cost. However it is not obvious that this will fit any natural definition of random. One (very inefficient) natural definition of random would be to select from all possible assignments at random until you get one that is acceptable to everybody. The distribution you get from this might not be the same as the one you would get by setting up random costs and then solving the resulting assignment problem.
You are using the term 'randomly match' which should be used with caution. The correct interpretation, I believe, is a random selection from the set of all valid solutions, so basically if we could enumerate all valid solution - we could trivially solve the problem.
You are looking for a close-enough solution, so we need to better define what a valid solution is. I suggest to define some threshold (say 1% error at most).
In your example, there are 4 groups (those assigned with shirt style A/B/C/D). Therefore, there are 2^4-1 possible person archetypes (love/hate each of A/B/C/D, with the assumption that anyone loves at least one shirt style). Each archetype population has a given population size, and each archetype can be assigned to some of the 4 groups (1 or more).
The goal is to divide the population of each archetype between the 4 groups, such that say, the size of each group is between L and H.
Lets formalize it.
Problem statement:
Denote A(0001),...,A(1111): the population size of each of the 15 archetypes
Denote G1(0001): the size of A(0001) assigned to G1, etc.
Given
L,H: constants
A(0001),...,A(1111): 15 constants
Our goal is to find all integer solutions for
G1(0001),G1(0011),G1(0101),G1(0111),G1(1001),G1(1011),G1(1101),G1(1111),
G2(0010),G2(0011),G2(0110),G2(0111),G2(1010),G2(1011),G2(1110),G2(1111),
G3(0100),G3(0101),G3(0110),G3(0111),G3(1100),G3(1101),G3(1110),G3(1111),
G4(1000),G4(1001),G4(1010),G4(1011),G4(1100),G4(1101),G4(1110),G4(1111)
subject to:
G1(0001) = A(0001)
G2(0010) = A(0010)
G2(0011) + G1(0011) = A(0011)
G3(0100) = A(0100)
G3(0101) + G1(0101) = A(0101)
G3(0110) + G2(0110) = A(0110)
G3(0111) + G2(0101) + G1(0101) = A(0111)
G4(1000) = A(1000)
G4(1001) + G1(1001) = A(1001)
G4(1010) + G2(1010) = A(1010)
G4(1011) + G2(1011) + G1(1011) = A(1011)
G4(1100) + G3(1100) = A(1100)
G4(1101) + G3(1101) + G1(1101) = A(1101)
G4(1110) + G3(1110) + G2(1110) = A(1110)
G4(1111) + G3(1111) + G2(1111) + G1(1111) = A(1111)
L < G1(0001)+G1(0011)+G1(0101)+G1(0111)+G1(1001)+G1(1011)+G1(1101)+G1(1111) <
H
L < G2(0010)+G2(0011)+G2(0110)+G2(0111)+G2(1010)+G2(1011)+G2(1110)+G2(1111) <
H
L < G3(0100)+G3(0101)+G3(0110)+G3(0111)+G3(1100)+G3(1101)+G3(1110)+G3(1111) <
H
L < G4(1000)+G4(1001)+G4(1010)+G4(1011)+G4(1100)+G4(1101)+G4(1110)+G4(1111) <
H
Now we can use an integer programming solver for the job.
I am currently working on some MatLab code to fit experimental data to a sum of exponentials following a method described in this paper.
According to the paper, the data has to follow the following equation (written in pseudo-code):
y = sum(v(i)*exp(-x/tau(i)),i=1..n)
Here tau(i) is a set of n predefined constants. The number of constants also determines the size of the summation, and hence the size of v. For example, we can try to fit a sum of 100 exponentials, each with a different tau(i) to our data. However, due to the nature of the fitting and the exponential sum, we need to add another constraint to the problem, and hence to the cost function of the least-squares method used.
Normally, the cost function of the least-squares method is given by:
(y_data - sum(v(i)*exp(-x/tau(i)),i=1..n)^2
And this has to be minimized. However, to prevent over-fitting that would make the time-constant spectrum extremely noisy, the paper adds the following constraint to the cost function:
|v(i) - v(i+1)|^2
Because of this extra constraint, as far as I know, the regular algorithms, like lsqcurvefit aren't useable any longer, and I have to use fminsearch to search the minimum of my least-squares cost function with a constraint. The function that has to be minimized, according to me, is the following:
(y_data - sum(v(i)*exp(-x/tau(i)),i=1..n)^2 + sum(|v(j) - v(j+1)|^2,j=1..n-1)
My attempt to code this in MatLab is the following. Initially we define the function in a function script, then we use fminsearch to actually minimize the function and get values for v.
function res = funcost( v )
%FUNCOST Definition of the function that has to be minimised
%We define a function yvalues with 2 exponentials with known time-constants
% so we know the result that should be given by minimising.
xvalues = linspace(0,50,10000);
yvalues = 3-2*exp(-xvalues/1)-exp(-xvalues/10);
%Definition of 30 equidistant point in the logarithmic scale
terms = 30;
termsvector = [1:terms];
tau = termsvector;
for i = 1:terms
tau(i) = 10^(-1+3/terms*i);
end
%Definition of the regular function
res_1 = 3;
for i=1:terms
res_1 =res_1+ v(i).*exp(-xvalues./tau(i));
end
res_1 = res_1-yvalues;
%Added constraint
k=1;
res_2=0;
for i=1:terms-1
res_2 = res_2 + (v(i)-v(i+1))^2;
end
res=sum(res_1.*res_1) + k*res_2;
end
fminsearch(#funcost,zeros(30,1),optimset('MaxFunEvals',1000000,'MaxIter',1000000))
However, this code is giving me inaccurate results (no error, just inaccurate results), which leads me to believe I either made a mistake in the coding or in the interpretation of the added constraint for the least-squares method.
I would try to introduce the additional constrain in following way:
res_2 = max((v(1:(end-1))-v(2:end)).^2);
e.g. instead of minimizing an integrated (summed up) error, it does minmax.
You may also make this constrain stiff by
if res_2 > some_number
k = a_very_big_number;
else
k=0; % or k = a_small_number
end;
I am currently doing some calculations with trees. Each node has 5 values I am trying to calculate and a type deciding how these values are calculated. Some calculations can be pretty complicated algorithms. All calculations within a node depend solely on the values of its child nodes, so I am doing calculations from down to top. For each node type, a value depends on different values of the childnodes. I am interested mainly in the 5 values in the root node, which depend on all values in all other nodes ofc. All this is working just fine. A node can only have 1 or 2 childnodes, and the tree usually is no deeper than 5 levels.
For some node-types, there is a tolerance; meaning some values there would not matter, see this picture, I marked those with XX. Sometimes even, some values would be in relation, like C = XX * A. Currently, these values are just set to some default values. Sometimes there would be a complicated relationship even, like multiple possible solutions of an algorithm like Newton's Method, depending on starting values.
Now there is a rating I can apply on the values of the root node. What I would like is to optimize this rating by adjusting the XX-values deep within the tree. The calculations within each node can be a range of many possible formulas and the tolerance can be one of many possible patterns, so I cannot just figure out some formula but I would need some algorithm which is very flexible. I do not know of such an algorithm. Does anyone have an idea?
/Edit: To clarify, it is unclear how many values in the tree will be free. There is not just one XX, but there may be any number of them (I guess max. 10), so my first step would be identifying these values. Also, I will be doing this on many generated trees within a time window, so speed is not unimportant as well. Thanks:)
If you have 3 input values XX, YY, and ZZ, you are searching a 3 dimension space. What you are looking to do is to apply an optimisation algorithm, or Heuristic algorithm. Your choice of algorithm is key, a cost benefit between your time and the computer's time. I am guessing that you just want to do this once.
What ever method you use, you need to understand the problem, which means to understand how your algorithm changes with different input values. Some solutions have a very nice minimum that is easy to find (e.g. using Newton's Method), some don't.
I suggest starting simple. One of the most basic is just to do an iterative search. It's slow, but it works. You need to make sure that your iteration step is not too large, such that you don't miss some sweet spots.
For XX = XXmin to XXmax
For YY = YYmin to YYmax
For ZZ = ZZmin to ZZmax
result = GetRootNodeValue(XX, YY, ZZ)
If result < best_result then
print result, XX, YY, ZZ
best_result = result
End if
End For
End For
End For
Below is another method, it's a stochastic optimisation method (uses random points to converge on the best solution), it's results are reasonable for most conditions. I have used this successfully and it's good at converging to the minimum value. This is good to use if there is no clear global minimum. You will have to configure the parameters for your problem.
' How long to search for, a larger value will result in long search time
max_depth = 20000
' Initial values
x0 = initial XX value
y0 = initial YY value
z0 = initial ZZ value
' These are the delta values, how far should the default values range
dx = 5
dy = 5
dz = 5
' Set this at a large value (assuming the best result is a small number)
best_result = inf
' Loop for a long time
For i = 1 To max_depth
' New random values near the best result
xx = x0 + dx * (Rnd() - 0.5) * (Rnd() - 0.5)
yy = y0 + dy * (Rnd() - 0.5) * (Rnd() - 0.5)
zz = y0 + dy * (Rnd() - 0.5) * (Rnd() - 0.5)
' Do the test
result = GetRootNodeValue(xx, yy, zz)
' We have found the best solution so far
If result < best_result Then
x0 = xx
y0 = yy
z0 = zz
best_result = result
End If
Print progress
Next i
There are many optimisation algorithms to choose from. Above are some very simple ones, but they may not be the best for your problem.
As another answer has pointed out, this looks like a optimization problem. You may consider using a genetic algorithm. Basically, you try to mimic the evolution process by "mating" different individuals (in your case trees) with different traits (in your case the values on the leaves) and make them survive based on an objective function (in your case, what you obtain on the root node). The algorithm can be improved by adding mutations to your populations (as in nature's evolution).
Here is a real-world combinatorial optimization problem.
We are given a large set of value propositions for a certain product. The value propositions are of different types but each type is independent and adds equal benefit to the overall product. In building the product, we can include any non-negative integer number of "units" of each type. However, after adding the first unit of a certain type, the marginal benefit of additional units of that type continually decreases. In fact, the marginal benefit of a new unit is the inverse of the number of units of that type, after adding the new unit. Our product must have a least one unit of some type, and there is a small correction that we must make to the overall value because of this requirement.
Let T[] be an array representing the number of each type in a certain production run of the product. Then the overall value V is given by (pseudo code):
V = 1
For Each t in T
V = V * (t + 1)
Next t
V = V - 1 // correction
On cost side, units of the same type have the same cost. But units of different types each have unique, irrational costs. The number of types is large, but we are given an array of type costs C[] that is sorted from smallest to largest. Let's further assume that the type quantity array T[] is also sorted by cost from smallest to largest. Then the overall cost U is simply the sum of each unit cost:
U = 0
For i = 0, i < NumOfValueTypes
U = U + T[i] * C[i]
Next i
So far so good. So here is the problem: Given product P with value V and cost U, find the product Q with the cost U' and value V', having the minimal U' such that U' > U, V'/U' > V/U.
The problem you've described is nonlinear integer programming problem because it contains a product of integer variables t. Its feasibility set is not closed because of strict inequalities which can be worked around by using non-strict inequalities and adding a small positive number (epsilon) to the right hand sides. Then the problem can be formulated in AMPL as follows:
set Types;
param Costs{Types}; # C
param GivenProductValue; # V
param GivenProductCost; # U
param Epsilon;
var units{Types} integer >= 0; # T
var productCost = sum {t in Types} units[t] * Costs[t];
minimize cost: productCost;
s.t. greaterCost: productCost >= GivenProductCost + Epsilon;
s.t. greaterValuePerCost:
prod {t in Types} (units[t] + 1) - 1 >=
productCost * GivenProductValue / GivenProductCost + Epsilon;
This problem can be solved using a nonlinear integer programming solver such as Couenne.
Honestly I don't think there is an easy way to solve this. The best thing would be to write the system and solve it with a solver ( Excel solver will do the tricks, but you can use Ampl to solve this non lienar program.)
The Program:
Define: U;
V;
C=[c1,...cn];
Variables: T=[t1,t2,...tn];
Objective Function: SUM(ti.ci)
Constraints:
For all i: ti integer
SUM(ti.ci) > U
(PROD(ti+1)-1).U > V.SUM(ti.ci)
It works well with excel, (you just replace >U by >=U+d where d is the significative number of the costs- (i.e if C=[1.1, 1.8, 3.0, 9.3] d =0.1) since excel doesn't allow stric inequalities in the solver.)
I guess with a real solver like Ampl it will work perfectly.
Hope it helps,