Generalized Linear Modelling on R data set Trucks
When I tried to key in the formula, an error pops out saying that my y values should be between 0 and 1. Why does this occur?
This is for R studio. Data set is from R package - Trucks
This is the code I used
print (Data)
lrfit<-glm(Freq ~ +period +collision + parked +light , family =
binomial (link = logit)glm
Error in eval (family$initialize) : y values must be 0<= y <= 1
Related
So I'm trying to make sense of a scenario in my class exercise which is to find the max and min value of a function. I have two vectors, w and v, of weights which are to sum to 1. The vectors are w = [0.6, 0.2, 0.2]^T v = [0.8, -0.2, 0.4]^T
These vectors form a linear combination of weights M = Aw + Bv, and A and B must sum to 1.
The function we are then optimizing is r = [0.1, 0.2, 0.1] • M
The constraints are as follows: 0 ≤ (0.6A + 0.8B) <= 1 , 0 ≤ (0.2A - 0.2B) <= 1 , 0 ≤ (0.2A + 0.4B) <= 1
The answer we should get are A = B = .5 for the minimum value of r which is 0.1. For the maximum we should get A = 2, B = -1 with r = 0.16. But the values I'm getting for the max are A = 3.5714286, B = -1.4285714, and for the min I'm getting A = B = 0.
Below is the code.
import pulp as p
from pulp import *
problem = LpProblem('Car Factory', LpMaximize)
A = LpVariable('Amound of w', cat=LpContinuous)
B = LpVariable('Amount of v', cat=LpContinuous)
#Objective Function
problem += (0.1)*(0.6*A + 0.8*B) + (0.2)*(0.2*A - 0.2*B) + (0.1)*(0.2*A + 0.4*B) , 'Objective Function'
#Constraints
problem += (0.6*A + 0.8*B) <= 1 , 'A'
problem += (0.6*A + 0.8*B) >= 0 , 'AL'
problem += (0.2*A - 0.2*B) <= 1, 'B'
problem += (0.2*A - 0.2*B) >= 0, 'BL'
problem += (0.2*A + 0.4*B) <= 1, 'C'
problem += (0.2*A + 0.4*B) >= 0, 'CL'
problem.solve()
print("Amount of w: ", A.varValue)
print("Amount of v: ", B.varValue)
print("total: ", value(problem.objective))
I'm sure it has to do with the set up which I'm just not seeing. And also is there a more efficient way to put this together?
I think you are missing a constraint, which would explain your deviation from the expected result. Where is your constraint that:
A + B == 1
Also, you are importing pulp twice, which may cause some confusion in the namespace of your code. Do one or the other, not both.
On expressing the problem more efficiently...? Nahh. You could treat your two column vectors as arrays of length 3 and do the math in your objective a bit differently, but it probably isn't worth it and your variables are just scalars, so I'd write it as you did. Now if the vectors were much larger, or if the variables were vectors, sure, I'd do something else.
pulp doesn't naturally handle vectors (like numpy arrays) to my knowledge. If you are going to be doing a lot of optimization in vector-matrix format and you are comfortable with the linear algebra, you might look at cvxpy which handles them naturally. If you're in a class that uses pulp, it's just fine to learn the basics.
I am implementing a Szudik's pairing function in Matlab, where i pair 2 values coming from 2 different matrices X and Y, into a unique value given by the function 'CantorPairing2D(X,Y), After this i reverse the process to check for it's invertibility given by the function 'InverseCantorPairing2( X )'. But I seem to get an unusual problem, when i check this function for small matrices of size say 10*10, it works fine, but the for my code i have to use a 256 *256 matrices A and B, and then the code goes wrong, actually what it gives is a bit strange, because when i invert the process, the values in the matrix A, are same as cvalues of B in some places, for instance A(1,1)=B(1,1), and A(1,2)=B(1,2). Can somebody help.
VRNEW=CantorPairing2D(VRPRO,BLOCK3);
function [ Z ] = CantorPairing2D( X,Y )
[a,~] =(size(X));
Z=zeros(a,a);
for i=1:a
for j=1:a
if( X(i,j)~= (max(X(i,j),Y(i,j))) )
Z(i,j)= X(i,j)+(Y(i,j))^2;
else
Z(i,j)= (X(i,j))^2+X(i,j)+Y(i,j);
end
end
end
Z=Z./1000;
end
function [ A,B ] = InverseCantorPairing2( X )
[a, ~] =(size(X));
Rfinal=X.*1000;
A=zeros(a,a);
B=zeros(a,a);
for i=1:a
for j=1:a
if( ( Rfinal(i,j)- (floor( sqrt(Rfinal(i,j))))^2) < floor(sqrt(Rfinal(i,j))) )
T=floor(sqrt(Rfinal(i,j)));
B(i,j)=T;
A(i,j)=Rfinal(i,j)-T^2;
else
T=floor( (-1+sqrt(1+4*Rfinal(i,j)))/2 );
A(i,j)=T;
B(i,j)=Rfinal(i,j)-T^2-T;
end
end
end
end
Example if A= 45 16 7 17
7 22 11 25
11 12 9 17
2 11 3 5
B= 0 0 0 1
0 0 0 1
1 1 1 1
1 3 0 0
Then after pairing i get
C =2.0700 0.2720 0.0560 0.3070
1.4060 0.5060 0.1320 0.6510
0.1330 0.1570 0.0910 0.3070
0.0070 0.1350 0.0120 0.0300
after the inverse pairing i should get the same A and same B. But for bigger matrices it is giving unusual behaviour, because some elements of A are same as B.
If possible it would help immensely a counter example where your code does fail.
I got to reproduce your code behaviour and I have rewritten your code in a vectorised fashion. You should get the bug, but hopefully it is a first step to uncover the underlying logic and find the bug itself.
I am not familiar with the specific algorithm, but I observe a discrepancy in the CantorPairing definition.
for elements where Y = X your if statement would be false, since X = max(X,X); so for those elements your Z would be X^2+X+Y, but for hypothesis X =Y, therefore your would have:
X^2+X+X = X^2+2*X;
now, if we perturb slightly the equation and suppose Y = X + 10*eps, your if statement would be true (since Y > X) and your Z would be X + Y ^2; since X ~=Y we can approximate to X + X^2
therefore your equation is very temperamental to numerical approximation ( and you definitely have a discontinuity in Z). Again, I am not familiar with the algorithm and it may very well be the behaviour you want, but it is unlikely: so I am pointing this out.
Following is my version of your code, I report it also because I hope it will be pedagogical in getting you acquainted with logical indexing and vectorized code (which is the idiomatic form for MATLAB, let alone much faster than nested for loops).
function [ Z ] = CantorPairing2D( X,Y )
[a,~] =(size(X));
Z=zeros(a,a);
firstConditionIndeces = Y > X; % if Y > X then X is not the max between Y and X
% update elements on which to apply first equation
Z(firstConditionIndeces) = X(firstConditionIndeces) + Y(firstConditionIndeces).^2;
% update elements on the remaining elements
Z(~firstConditionIndeces) = X(~firstConditionIndeces).^2 + X(~firstConditionIndeces) + Y(~firstConditionIndeces) ;
Z=Z./1000;
end
function [ A,B ] = InverseCantorPairing2( X )
[a, ~] =(size(X));
Rfinal=X.*1000;
A=zeros(a,a);
B=zeros(a,a);
T = zeros(a,a) ;
% condition deciding which updates to be applied
indecesToWhichApplyFstFcn = Rfinal- (floor( sqrt(Rfinal )))^2 < floor(sqrt(Rfinal)) ;
% elements on which to apply the first update
T(indecesToWhichApplyFstFcn) = floor(sqrt(Rfinal )) ;
B(indecesToWhichApplyFstFcn) = floor(Rfinal(indecesToWhichApplyFstFcn)) ;
A(indecesToWhichApplyFstFcn) = Rfinal(indecesToWhichApplyFstFcn) - T(indecesToWhichApplyFstFcn).^2;
% updates on which to apply the remaining elements
A(~indecesToWhichApplyFstFcn) = floor( (-1+sqrt(1+4*Rfinal(~indecesToWhichApplyFstFcn )))/2 ) ;
B(~indecesToWhichApplyFstFcn) = Rfinal(~indecesToWhichApplyFstFcn) - T(~indecesToWhichApplyFstFcn).^2 - T(~indecesToWhichApplyFstFcn) ;
end
Let X be a mealy machine. More specifically, I'm representing it as a 3 by 3 by 2 by N array where N is the number of states. View it as a row of N 3 by 3 matrices and another such layer behind it. The layer behind gives the state transitions and the one in front the output values. So say I start from state N and input (x,y), I get output X(x,y,1,N) and then move to state X(x,y,2,N). For a sequence of inputs (x,y)...(x',y') I want to know the output sequence and am currently using the following code. Any speed up suggestions would be very welcome. A related question is does 'isequal' work faster when comparing rows of matrices than when comparing cells entries (related to another 'checking' code not the code below).
function [ T, O ] = mealy( X ,w ,k )
%Given a Mealy machine X, input word w and starting state k, T == 1 means
%this mealy machine does not have sufficient information to give an output
%for input word w. Else T == 0 and O is the output sequence for w.
L = length(w);
S = zeros(1,L); %state sequence
O = cell(1,L); %letter and output sequence
T = 0;
S(1) = k; %start from state k
% build output sequence O from input word w
for i = 2:L
e = w{i-1};
S(i) = X(e(1),e(2),2,S(i-1));
if S(i) < 0, T = 1; return, end
t = X(e(1),e(2),1,S(i-1));
if t < -1.5, T = 1; return, end
O{i-1} = [e(1),e(2),t];
end
e = w{L};
t = X(e(1),e(2),1,S(L));
if t < -1.5, T = 1;return, end
O{L}= [e(1),e(2),t];
end
I have a data set of the form:
[9.1 5.6 7.4] => 8.5, [4.1 4.4 5.2] => 4.9, ... , x => y(x)
So x is a real vector of three elements and y is a scalar function.
I'm assuming a weighted average model of this data:
y(x) = (a * x[0] + b * x[1] + c * x[2]) / (a+b+c) + E(x)
where E is an unknown random error term.
I need an algorithm to find a,b,c, that minimizes total sum square error:
error = sum over all x of { E(x)^2 }
for a given data set.
Assume that the weights are normalized to sum to 1 (which happily is without loss of generality), then we can re-cast the problem with c = 1 - a - b, so we are actually solving for a and b.
With this we can write
error(a,b) = sum over all x { a x[0] + b x[1] + (1 - a - b) x[2] - y(x) }^2
Now it's just a question of taking the partial derivatives d_error/da and d_error/db and setting them to zero to find the minimum.
With some fiddling, you get a system of two equations in a and b.
C(X[0],X[0],X[2]) a + C(X[0],X[1],X[2]) b = C(X[0],Y,X[2])
C(X[1],X[0],X[2]) a + C(X[1],X[1],X[2]) b = C(X[1],Y,X[2])
The meaning of X[i] is the vector of all i'th components from the dataset x values.
The meaning of Y is the vector of all y(x) values.
The coefficient function C has the following meaning:
C(p, q, r) = sum over i { p[i] ( q[i] - r[i] ) }
I'll omit how to solve the 2x2 system unless this is a problem.
If we plug in the two-element data set you gave, we should get precise coefficients because you can always approximate two points perfectly with a line. So for example the first equation coefficients are:
C(X[0],X[0],X[2]) = 9.1(9.1 - 7.4) + 4.1(4.1 - 5.2) = 10.96
C(X[0],X[1],X[2]) = -19.66
C(X[0],Y,X[2]) = 8.78
Similarly for the second equation: 4.68 -13.6 4.84
Solving the 2x2 system produces: a = 0.42515, b = -0.20958. Therefore c = 0.78443.
Note that in this problem a negative coefficient results. There is nothing to guarantee they'll be positive, though "real" data sets may produce this result.
Indeed if you compute weighted averages with these coefficients, they are 8.5 and 4.9.
For fun I also tried this data set:
X[0] X[1] X[2] Y
0.018056028 9.70442075 9.368093544 6.360312244
8.138752835 5.181373099 3.824747424 5.423581239
6.296398214 4.74405298 9.837741509 7.714662742
5.177385358 1.241610571 5.028388255 4.491743107
4.251033792 8.261317658 7.415111851 6.430957844
4.720645386 1.0721718 2.187147908 2.815078796
1.941872069 1.108191586 6.24591771 3.994268819
4.220448549 9.931055481 4.435085917 5.233711923
9.398867623 2.799376317 7.982096264 7.612485261
4.971020963 1.578519218 0.462459906 2.248086465
I generated the Y values with 1/3 x[0] + 1/6 x[1] + 1/2 x[2] + E where E is a random number in [-0.1..+0.1]. If the algorithm is working correctly we'd expect to get roughly a = 1/3 and b = 1/6 from this result. Indeed we get a = .3472 and b = .1845.
OP has now said that his actual data are larger than 3-vectors. This method generalizes without much trouble. If the vectors are of length n, then you get an n-1 x n-1 system to solve.
We have need for a "rating" system in a project we are working on, similar to the one in SO. However, in ours there are multiple entities that need to be "tagged" with a vote up (only up, never down, like an increment). Sometimes we will need to show all of the entities in order of what is rated highest, regardless of entity type, basically mixing the result sets, I guess. What data structures / algorithms do you use to implement this so that is flexible and still scalable?
Since reddit's ranking algorithm rocks, it makes very much sense to have a look at it, if not copy it:
Given the time the entry was posted A and the time of 7:46:43 a.m. December 8, 2005 B we have ts as their difference in seconds:
ts = A - B
and x as the difference between the number of up votes U and the number of down votes D:
x = U - D
Where
y = 1 if x > 0
y = 0 if x = 0
y = -1 if x < 0
and z as the maximal value of the absolute value of x and 1:
z = |x| if |x| >= 1
z = 1 if |x| < 1
we have the rating as a function ƒ(ts, y, z):
ƒ(ts, y, z) = log10 z + (y • ts)/45000