Goal
I want to solve a system of 9 non-linear equation with 9 unknowns with solve Matlab.
All 9 unknowns are coupled as a ration of polynoms (see myfun lower)
Fsolve
x02=[5000,5000,5000,0.4,0.4,0.4,0.4,0.4,0.4];
ctrl2=[9894+1i*0.118,9894+1i*0.118,9894+1i*0.118,0.5,0.5,0.5,0.5,0.5,0.5];
f2 = #(x) myfun(x,Peff);
options2 = optimoptions('fsolve','Algorithm','trust-region-dogleg','Display','iter-detailed'...
,'MaxFunEvals', 100000, 'MaxIter', 100000,'TolX',1e-12,'TolFun',1e-12,...
'Jacobian','on');
[x2,F2,exitflag2,output2] = fsolve(f2,x02,options2);
Function myfun, return the system of equation to feed into fsolve an corresponding Jacobian
function [F,J] = myfun( x, p)
% System of equation
F(1) = -(x(1) - x(1)*x(8)*x(9))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(1,1);
F(2) = -(x(2)*x(4) + x(2)*x(6)*x(9))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(1,2);
F(3) = -(x(3)*x(6) + x(3)*x(4)*x(8))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(1,3);
F(4) = -(x(1)*x(5) + x(1)*x(8)*x(7))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(2,1);
F(5) = -(x(2) - x(2)*x(6)*x(7))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(2,2);
F(6) = -(x(3)*x(8) + x(3)*x(6)*x(5))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(2,3);
F(7) = -(x(1)*x(7) + x(1)*x(5)*x(9))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(3,1);
F(8) = -(x(2)*x(9) + x(2)*x(4)*x(7))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(3,2);
F(9) = -(x(3) - x(3)*x(4)*x(5))/(x(4)*x(5) + x(6)*x(7) + x(8)*x(9) + x(4)*x(8)*x(7) + x(6)*x(5)*x(9) - 1) - p(3,3);
%% Jacobian
I compute the Jacobian myself but will spare you the detail, as it is considerably long
end
Results
Norm of First-order Trust-region
Iteration Func-count f(x) step optimality radius
0 1 2.45042e+19 1.39e+14 1
1 2 2.45042e+19 1 1.39e+14 1
2 3 2.45031e+19 0.25 4.77e+15 0.25
3 4 2.45031e+19 0.625 4.77e+15 0.625
4 5 2.45031e+19 0.15625 4.77e+15 0.156
5 6 2.44992e+19 0.0390625 6.8e+16 0.0391
6 7 2.44992e+19 0.0976562 6.8e+16 0.0977
7 8 2.44992e+19 0.0244141 6.8e+16 0.0244
8 9 2.4495e+19 0.00610352 2.03e+17 0.0061
9 10 2.4495e+19 0.0152588 2.03e+17 0.0153
10 11 2.4486e+19 0.0038147 7.67e+17 0.00381
11 12 2.4486e+19 0.00953674 7.67e+17 0.00954
12 13 2.44592e+19 0.00238419 4.62e+18 0.00238
13 14 2.44592e+19 0.00596046 4.62e+18 0.00596
14 15 2.40048e+19 0.00149012 5.62e+20 0.00149
15 16 2.40048e+19 0.00372529 5.62e+20 0.00373
16 17 2.40048e+19 0.000931323 5.62e+20 0.000931
17 18 2.40048e+19 0.000232831 5.62e+20 0.000233
18 19 2.36832e+19 5.82077e-05 1.52e+21 5.82e-05
19 20 2.36832e+19 0.000145519 1.52e+21 0.000146
20 21 2.3131e+19 3.63798e-05 4.24e+21 3.64e-05
21 22 2.3131e+19 9.09495e-05 4.24e+21 9.09e-05
22 23 2.21355e+19 2.27374e-05 1.26e+22 2.27e-05
23 24 2.21355e+19 5.68434e-05 1.26e+22 5.68e-05
24 25 2.01772e+19 1.42109e-05 4.2e+22 1.42e-05
25 26 2.01772e+19 3.55271e-05 4.2e+22 3.55e-05
26 27 1.5592e+19 8.88178e-06 1.76e+23 8.88e-06
27 28 1.5592e+19 2.22045e-05 1.76e+23 2.22e-05
28 29 1.17854e+18 5.55112e-06 7.24e+23 5.55e-06
29 30 3.43734e+16 1.38778e-05 1.9e+23 1.39e-05
30 31 1.23843e+15 3.46945e-05 4.04e+22 3.47e-05
31 32 1.23843e+15 8.67362e-05 4.04e+22 8.67e-05
32 33 7.49991e+13 2.1684e-05 4.25e+21 2.17e-05
33 34 7.49991e+13 5.42101e-05 4.25e+21 5.42e-05
34 35 3.19073e+13 1.35525e-05 3.46e+21 1.36e-05
35 36 3.19073e+13 3.38813e-05 3.46e+21 3.39e-05
36 37 3.19073e+13 8.47033e-06 3.46e+21 8.47e-06
37 38 3.19073e+13 2.11758e-06 3.46e+21 2.12e-06
38 39 3.19073e+13 5.29396e-07 3.46e+21 5.29e-07
39 40 3.19073e+13 1.32349e-07 3.46e+21 1.32e-07
40 41 3.19073e+13 3.30872e-08 3.46e+21 3.31e-08
41 42 3.19073e+13 8.27181e-09 3.46e+21 8.27e-09
42 43 3.19073e+13 2.06795e-09 3.46e+21 2.07e-09
43 44 3.19073e+13 5.16988e-10 3.46e+21 5.17e-10
44 45 3.16764e+13 1.29247e-10 3e+21 1.29e-10
45 46 3.16764e+13 1.29247e-10 3e+21 1.29e-10
46 47 3.16764e+13 3.23117e-11 3e+21 3.23e-11
47 48 3.16764e+13 8.07794e-12 3e+21 8.08e-12
48 49 3.16764e+13 2.01948e-12 3e+21 2.02e-12
49 50 3.16764e+13 5.04871e-13 3e+21 5.05e-13
50 51 3.16764e+13 1.26218e-13 3e+21 1.26e-13
51 52 3.16764e+13 3.15544e-14 3e+21 3.16e-14
52 53 3.16764e+13 7.88861e-15 3e+21 7.89e-15
53 54 3.16764e+13 1.97215e-15 3e+21 1.97e-15
54 55 3.16764e+13 4.93038e-16 3e+21 4.93e-16
fsolve stopped because the relative norm of the current step, 1.097484e-16, is less than
max(options.TolX^2,eps) = 2.220446e-16. However, the sum of squared function values,
r = 3.167644e+13, exceeds sqrt(options.TolFun) = 1.000000e-06.
Optimization Metric Options
relative norm(step) = 1.10e-16 max(TolX^2,eps) = 2e-16 (selected)
r = 3.17e+13 sqrt(TolFun) = 1.0e-06 (selected)
exitflag2 =
-2
output2 =
iterations: 54
funcCount: 55
algorithm: 'trust-region-dogleg'
firstorderopt: 3.000805388686251e+21
message: 'No solution found.…'
Problem
Initial guess
x02 =
1.0e+03 *
Columns 1 through 5
5.000000000000000 5.000000000000000 5.000000000000000 0.000400000000000 0.000400000000000
Columns 6 through 9
0.000400000000000 0.000400000000000 0.000400000000000 0.000400000000000
Solution given by fsolve
x2 =
1.0e+03 *
Columns 1 through 2
5.000098340971978 - 0.000000066639557i 5.000100855522207 + 0.000000027141142i
Columns 3 through 4
5.000100887684736 + 0.000000021333305i 0.000500827051867 + 0.000000033172152i
Columns 5 through 6
0.000498312570833 - 0.000000060511167i 0.000500859436647 + 0.000000027409553i
Columns 7 through 8
0.000500831092720 + 0.000000033506374i 0.000500831171443 + 0.000000033543065i
Column 9
0.000498337065684 - 0.000000066909835i
What the solution should be (control)
ctrl2 =
1.0e+03 *
Columns 1 through 2
9.894000000000000 + 0.355900000000000i 9.894000000000000 + 0.355900000000000i
Columns 3 through 4
9.894000000000000 + 0.355900000000000i 0.000499999000000 + 0.000000000000000i
Columns 5 through 6
0.000499999000000 + 0.000000000000000i 0.000499999000000 + 0.000000000000000i
Columns 7 through 8
0.000499999000000 + 0.000000000000000i 0.000499999000000 + 0.000000000000000i
Column 9
0.000499999000000 + 0.000000000000000i
Comments
As you can see solve crashes fairly quick without going very far from the initial guess.
I tried changing TolX TolFun Algorithm but still crashes (following advice from Matlab website under what to do when algorithm fails)
The algorithm crashes similarly for other algorithm in fsolve and lsqnonlin (levnberg, trust-region-dogleg, trust-region-reflective)
Question to you:
Can you help figure out what I need to do in order to build an
algorithm that will converge to the control values?
Thank you
What you are experiencing is actually nothing wrong with the solver. More than likely, you simply have many local minimizers in the problem, and the optimization algorithm is getting stuck there. There is really not much you can do about that... Almost all optimization solvers that you get your hand on are based on the principle of gradient descent (more or less), so they will guide you to a minimizer, but not necessarily the global one. What some people do is to generate a large set of random initial conditions, and to run the optimization solver for each initial condition. This isn't guaranteed to mean that one of the solutions you get back is a global minimizer, but at least you should be able to do better than a single initial guess.
If you really NEED to find the global minimizer, then you need to use some type of "Groebner-basis" solution method. These types of solvers are based on "elimination theory" and will yield a univariate polynomiial containing all of the global minimizers. However, these solvers are usually symbolic, extremely memory intensive, and basically aren't guaranteed to finish in a finite amount of time. I would suspect (I don't really know) that these problems are NP hard. In my experience, a seemingly small problem like the one you show can take very long to solve (if at all).
Related
General Problem Description
Hi, it is actually a special assignment problem( check wiki if interested). Suppose I have 10 agents denoted as A1, A2, ... A10 and I need them to work in pairs. While, according to previous experience, I know the working efficiency of each two-agent pair so that I have an efficiency matrix shown as follows whose ( i, j ) element represents the working efficiency of agent pair ( Ai, Aj ). Hence, we know it should be a symmetric matrix, which means E( i, j )=E( j, i ) and E( i, i ) should be 0. Now, I need divide these 10 agents into 5 groups such that the overall (sum) efficiency is maximal.
E =
0 25 28 23 39 77 56 58 85 41
25 0 18 77 32 52 69 59 47 18
28 18 0 20 55 75 63 38 5 56
23 77 20 0 59 76 24 82 68 64
39 32 55 59 0 49 70 28 42 31
77 52 75 76 49 0 33 84 50 29
56 69 63 24 70 33 0 15 49 83
58 59 38 82 28 84 15 0 68 40
85 47 5 68 42 50 49 68 0 56
41 18 56 64 31 29 83 40 56 0
N.B.
From the matrix view of this problem, I need pick 5 elements from above matrix such that none of them share a same index with others. ( if you pick E( 2, 3 ), then you cannot pick any elments with index containing 2 or 3 since A2 and A3 are assigned to work. In other words, you cannot pick any elements from the 2nd, 3rd row and 2nd, 3rd column.)
The title of this problem is an equivalent problem to the special assignment problem mentinoned above.
You may find the Hungarian(munkres) algorithm helpful! Here is the matlab code.
Another view of this problem is to solve a normal assignment problem, but we need to find a solution whose elements are symmetrically distributed about the diagonal.
Directly applying Hungarian(munkres) algorithm to the symmetric efficiency matrix does not always work. Sometimes it will give asymmetric permutations e.g.
E =
0 30 63 32 20 40
30 0 67 84 75 63
63 67 0 37 79 88
32 84 37 0 43 59
20 75 79 43 0 56
40 63 88 59 56 0
The optimal solution is:
assignment =
0 0 0 0 0 1
0 0 0 1 0 0
1 0 0 0 0 0
0 1 0 0 0 0
0 0 1 0 0 0
0 0 0 0 1 0
This can be solved as weighted maximum matching problem, where:
G = (V,E,w)
V = { all numbers }
E = { (v,u) | v,u in V, v!=u }
w(u,v) = -|u-v|
The solution to maximum matching will pair all your vertices (numbers), such that sum of: sum { -|u-v| : u,v paired } is maximum, which means sum { |u-v| : u,v paired is minimum.
I have some problem while trying to calculate time complexity for this code:
function foo (int a):
if a < 1:
return 1
else:
for i = 1 to 4:
foo(a - 3)
for i = 1 to 4:
foo(a / 2)
end function
As far as I can go:
T(n) = 1 if n<1
T(n) = 4T(n-3) + 4T(n/2) if n>=1
= 4(4T(n-6) + 4T((n-3)/2)) + 4(4T(n/2 - 3) + 4T(n/4))
~ 4^2 (T(n-6) + T((n-3)/2) + T(n/2-3) + T(n/4))
Now, it is very complicated, since number of the next T increase by 2^n and also the child is quite complicated.
Is there any other ways to solve this problem?
Let's expand the recursive cost function:
T(n) = 4 [T(n-3) + T(n/2)]
T(n) = 4^2 [T(n-6) + T((n-3)/2) + T((n-6)/2) + T(n/4)]
T(n) = 4^n [T(n-9) + 2*T((n-6)/2) + T((n-9)/2) + T((n-12)/4) + T((n-3)/4) + T((n-6)/4) + T(n/8)]
From the moment the x in T(x) drops below 1, you should replace T(x) with 1. And from that moment on, the T(x) doesn't generate any "children" anymore so to speak.
what does this means? It means that after the k-'th expansion of T(n), the function will look like:
T(n) = 4^k [number of paths with length `k`]
and keep increasing k until all paths have "died". This is definitely the case after n/3 iterations, because that's the longest possible path.
We thus have some kind of graph, for instance for n=9:
9 + 6 + 3 + 0
| | ` 1
| `3 + 0
| ` 1
`4 + 1
` 2 + -1
` 1
so 6 paths. Now the problem is how to count the number of paths. In order to do this we will first represent the main path: n, n-3, n-6, etc. as a horizontal line of nodes, this is definitely the longest path:
n n-3 n-6 n-9 ... 1
Now out of all these nodes, do originate nodes of i -> i/2 (except for one)
n n-3 n-6 n-9 ... 4 1
| | | |
n/2 (n-3)/2 (n-6)/2 (n-9)/2 ... 2
(the second row shows all nodes created by divisions by 2). Now these nodes, generate again offsprong n -> n-3, which is, since it is divided by two n/2 -> (n-6)/2, in other words, there are edges that make jumps of two:
n n-3 n-6 n-9 ... 4 1
| | /-----+-------(n-9)/2 |
n/2 (n-3)/2 (n-6)/2 (n-9)/2 ... 2
\---------->(n-6)/2 \------->...
on other words, except for the first two elements, all other nodes in the second row count for two. If we would represent it as some kind of graph with the nodes labeled by their weight, it would look like:
1 -- 1 -- 1 -- 1 -- 1 -- .. -- .. -- 1
| | | | | | |
1 -- 1 -- 2 -- 2 -- 2 -- .. -- 2
Or if we keep doing this for this process:
1 -- 1 -- 1 -- 1 -- 1 -- .. -- .. -- .. -- .. -- ..-- 1
| | | | | | | | | |
1 -- 1 -- 2 -- 2 -- 2 -- .. -- .. -- .. -- .. -- 2
| | | | | | | |
1 -- 1 -- 2 -- 2 -- 3 -- .. -- .. -- 4
(the third row generates children 4 items further)
Now we need to calculate the sum of the last row. This is at most O(log n).
Which thus results in an upper bound of O(4^(n/3)*log n) maximum. It is definitely possible that the bound is tighter, or 4^(n/3+epsilon), the log doesn't really matter when it comes down to the exponent.
Experiments
One can turn the program into a program that calculates the cost (used Python):
def memodict(f):
""" Memoization decorator for a function taking a single argument """
class memodict(dict):
def __missing__(self, key):
ret = self[key] = f(key)
return ret
return memodict().__getitem__
#memodict
def foo (a):
if a < 1:
return 1
else:
return 1+4*(foo(a-3)+foo(a//2))
for i in range(1000) :
print '{0} {1}'.format(i,foo(i))
mind the 1+ (this is due to the fact that calling a method not at the leaves requires computational cost as well).
It shows the following graph (with the y axis in log space):
If one looks very closely it looks as if log n is a better estimate. Although I don't know if it is safe to say this.
This results in a table (below, calculated it further up to 2'000).
1 9
2 41
3 41
4 201
5 329
6 329
7 969
8 2121
9 2121
10 5193
11 9801
12 9801
13 22089
14 43081
15 43081
16 96841
17 180809
18 180809
19 395849
20 744009
21 744009
22 1622601
23 3015241
24 3015241
25 6529609
26 12149321
27 12149321
28 26290761
29 48769609
30 48769609
31 105335369
32 195465801
33 195465801
34 422064713
35 782586441
36 782586441
37 1688982089
38 3131929161
39 3131929161
40 6758904393
41 12530692681
42 12530692681
43 27038593609
44 50129261129
45 50129261129
46 108166435401
47 200529105481
48 200529105481
49 432677802569
50 802142540361
51 802142540361
52 1730759807561
53 3208618758729
54 3208618758729
55 6923087827529
56 12834580197961
57 12834580197961
58 27692546388553
59 51338515870281
60 51338515870281
61 110770380632649
62 205354484822601
63 205354484822601
64 443082304393801
65 821418721153609
66 821418721153609
67 1772329999438409
68 3285676572873289
69 3285676572873289
70 7089323128099401
71 13142709421838921
72 13142709421838921
73 28357295642743369
74 52570844443284041
75 52570844443284041
76 113429195098690121
77 210283390300852809
78 210283390300852809
79 453716792922477129
80 841133588239028809
81 841133588239028809
82 1814867221812679241
83 3364534403078885961
84 3364534403078885961
85 7259468937373487689
86 13458137720469918281
87 13458137720469918281
88 29037875950010995273
89 53832551082396717641
90 53832551082396717641
91 116151504000561025609
92 215330204762252612169
93 215330204762252612169
94 464606016804360524361
95 861320819851126870601
96 861320819851126870601
97 1858424068019558519369
98 3445283281135218692681
99 3445283281135218692681
100 7433696275286804238921
101 13781133127749444932169
102 13781133127749444932169
103 29734785104355787117129
104 55124532517920818958921
105 55124532517920818958921
106 118939140430257623503433
107 220498130084517750870601
108 220498130084517750870601
109 475756561733864969048649
110 881992520365763354792521
111 881992520365763354792521
112 1903026246986798196986441
113 3527970081514391739961929
114 3527970081514391739961929
115 7612104987998531108737609
116 14111880326168337145401929
117 14111880326168337145401929
118 30448419952199478498431561
119 56447521304878702645088841
120 56447521304878702645088841
121 121793679809003268057207369
122 225790085219957892102885961
123 225790085219957892102885961
124 487174719236834490168119881
125 903160340880652986350834249
126 903160340880652986350834249
127 1948698876948159378611769929
128 3612641363524384274620912201
129 3612641363524384274620912201
130 7794795507795923189331694153
131 14450565454100822773368263241
132 14450565454100822773368263241
133 31179182031186978432211391049
134 57802261816410380413470806601
135 57802261816410380413470806601
136 124716728124761056435137057353
137 231209047265654664360174719561
138 231209047265654664360174719561
139 498866912499057368446839722569
140 924836189062647014733211275849
141 924836189062647014733211275849
142 1995467649996282044625046245961
143 3699344756250640629770532459081
144 3699344756250640629770532459081
145 7981870599985180749337872339529
146 14797379025002675948264700809801
147 14797379025002675948264700809801
148 31927482399940933280729262494281
149 59189516100010914076436576375369
150 59189516100010914076436576375369
151 127709929599763943406294823113289
152 236758064400044110022526700261961
153 236758064400044110022526700261961
154 510839718399056614758740495864393
155 947032257600177281223668004459081
156 947032257600177281223668004459081
157 2043358873596227300168523186868809
158 3788129030400710939761843707744841
159 3788129030400710939761843707744841
160 8173435494384912565208445703590473
161 15152516121602847123581727787094601
162 15152516121602847123581727787094601
163 32693741977539653625368135770477129
164 60610064486411395753795798399095369
165 60610064486411395753795798399095369
166 130774967910158627959610155397452361
167 242440257945645596473320805911925321
168 242440257945645596473320805911925321
169 523099871640634525296578233905353289
170 969761031782582414931158973141652041
171 969761031782582414931158973141652041
172 2092399486562538155018863817501086281
173 3879044127130329713557186774446281289
174 3879044127130329713557186774446281289
175 8369597946250152673908006151884018249
176 15516176508521318970380250897829106249
177 15516176508521318970380250897829106249
178 33478391785000610910962228937122943561
179 62064706034085276096851207920903295561
180 62064706034085276096851207920903295561
181 133913567140002443859179120078078644809
182 248258824136341104852010847685857284681
183 248258824136341104852010847685857284681
184 535654268560009776298037299361325027913
185 993035296545364420269364209792439587401
186 993035296545364420269364209792439587401
187 2142617074240039106053470016494310560329
188 3972141186181457682935880906387200447049
189 3972141186181457682935880906387200447049
190 8570468296960156427659163345381749723721
191 15888564744725830735188806904953309270601
192 15888564744725830735188806904953309270601
193 34281873187840625714081936660931506377289
194 63554258978903322948188923891891471159881
195 63554258978903322948188923891891471159881
196 137127492751362502870108879768266900279881
197 254217035915613291806536828692106759410249
198 254217035915613291806536828692106759410249
199 548509971005450011494216652197608475890249
200 1016868143662453167255882099869574254596681
(Rewrote to give a better answer.)
Here is a simple and rigorous analysis that shows why T(n) ~ 4^{n/3} is a tight estimate.
We have the recurrence
T(n) = 4T(n-3) + 4T(n/2)
To get the tight result, we want to see that T(n/2) is negligible compared to T(n-3). We can do this as follows.
First, T is nonnegative for all n, so in particular T(n/2) >= 0, so for all n we have an inequality,
T(n) >= 4T(n-3)
Now, we want to use that inequality to compare T(n-3) and T(n/2).
By applying that inqeuality n/6 - 1 times, we get that
T(n-3) >= 4^{n/6 - 1} * T(n/2)
(Because, (n/6 - 1) * 3 = n/2 - 3, and n/2 - 3 + n/2 = n - 3).
It implies that T(n/2) is small compared to T(n-3):
T(n/2) <= 4^{-n/6 + 1} * T(n-3)
Now, for any epsilon > 0, there is an n_0 such that for n > n_0, 4^{-n/6 + 1} < epsilon. (Because, the limit of 4^{-n/6 + 1} is zero as n gets large.)
This implies that for any epsilon > 0, there is large enough n so that
4T(n-3) <= T(n) <= (4 + epsilon) T(n-3)
This yields the tight bound T(n) = 4^(n/3 + o(n)).
Getting a sharper estimate
There's some question in the comments about getting rid of the o(n) above, to get an even sharper estimate.
I fear this is basically just going to get pedantic -- usually no one cares about the low order terms, and nailing them down exactly is just some calculus work. But we can do a little more today anyways.
What's the difference
First of all, what is the difference between O(4^{n/3}) and 4^{n/3 + o(n)}? (Alternatively, we could write the latter as (4+o(1))^{n/3}.)
The difference is in how tightly they control the low order terms. O(4^{n/3}) controls them very tightly -- it says you don't exceed the (concrete) value 4^{n/3}) by more than a constant factor.
4^{n/3 + o(n)}, allows that you may exceed 4^{n/3} by more than a constant factor. But that factor is subexponential in n, it's negligible compared to 4^{n/3}.
For example, consider the function f(n) = n * 4^{n/3}. This function is not O(4^{n/3}). Indeed, it exceeds it by a factor n, more than a constant factor.
However, f(n) is in the class 4^{n/3 + o(n)}. Why? Because n = O(4^{epsilon n}) for every epsilon > 0.
When you have an inequality like,
4T(n-3) <= T(n) <= (4 + epsilon) T(n-3)
for every epsilon > 0, you can only deduce from this T(n) = (4 + o(1))^{n/3}.
To get a sharper bound, we need to treat epsilon as a function of n and not as a constant (like I did in the lazier version.)
Proof
Let epsilon(n) = 4^{-n/6 + 1} in what follows. Then we already showed
T(n) <= (4 + epsilon(n)) T(n-3)
and we want to see T = O(4^{n/3}).
This is can be expanded as an iterated product:
T(n) = PI_{i=1}^{n/3} (4 + epsilon(3i))
We can factor each term and pull out a factor of 4 to get
T(n) = 4^{n/3} * PI_{i=1}^{n/3} (1 + epsilon(3i)/4 )
The goal is now to show that
PI_{i=1}^{n/3} (1 + epsilon(3i)/4 ) = O(1)
and then we will be finished.
To do this we take the log, and show that that is O(1).
SUM_{i=1}^{n/3} log(1 + epsilon(3i/4))
We bound that using log(1+x) <= x for x >= 0.
SUM_{i=1}^{n/3} epsilon(3i/4)
Now we use the definition of epsilon. In fact we only need to know epsilon(n) <= C^{-n} for some C > 1. The above becomes
SUM_{i=1}^{n/3} C'^{-i}
for some constant C' > 1. But this is a geometric series, so it is bounded above by the infinite geometric series as
1 / (1 - 1/C') = O(1)
Thus T(n) = O(4^{n/3}).
Since we already had T(n) = Omega(4^{n/3}) we now have it tight up to constants, T(n) = Θ(4^{n/3})
You can decide for yourself if this extra work made things any more clear :p Personally I prefer to leave the o(n)'s in there usually.
IMO, the time complexity is Θ(r^n), where r=³√4.
Indeed, plugging this expression in the recurrence relation,
r^n = 1 + 4 r^n / r³ + 4 r^(n/2) = 1 + r^n + 4 √(r^n),
where the second term dominates asymptotically.
Here is a plot of the exact number of total calls to foo, divided by r^n for easy reading. We assumed the floor [n/2] in f(n/2).
The ratios tend to the repeating sequence 46.6922952502, 63.4656065932
74.1193985991. This seems to confirm Θ(r^n).
Update:
By induction we can show that for n >= 21,
T(n) < B(n) = 75.(s^(2n) - 4.s^n),
with s=³√2.
Indeed, by the recurrence equation and the induction hypothesis,
T(n+3) = 1 + 4.T(n) + 4.T([(n+3)/2])
< 1 + 4.75.(s^(2n) - 4.s^n) + 4.75.(s^(2[(n+3)/2])) - 4.s^[(n+3)/2])
We compare this to the bound B(n+3) to establish
1 + 4.75.(s^(2n) - 4.s^n) + 4.75.(s^(2[(n+3)/2])) - 4.s^[(n+3)/2])
< 75.(s^(2n+6) - 4.s^[(n+3)/2]
We can simplify the terms 4.75.s^(2n) and divide by 300.s^n:
s^(-n)/300 - 4 + s^(-(n+3)%2) - 4.s^([(n+3)/2]-n) < - s^([(n+3)/2]-n)
or
s^(-n)/300 + s^(-(n+3)%2) < 4 + 5.s^([(n+3)/2]-n).
This inequality is true for any n, so that T(n) < B(n) => T(n+3) < B(n+3).
Now for the base case, we use the table of T(n) given by #CommuSoft (and checked independently) and verify numerically
T(21) = 744009 < 75.(s^42 - 4.s^21) = 1190400
T(22) = 1622601 < 75.(s^44 - 4.s^22) = 1902217.444...
T(23) = 3015241 < 75.(s^46 - 4.s^23) = 3035425.772...
...
T(41) = 12530692681 < 75.(s^82 - 4.s^41) = 12678879361
This shows that the induction step can be applied from n=39 onwards ([(39+3)/2]=21).
Then
T(n) = O(75.(s^(2n) - 4.s^n)) = O(r^n).
(Actually, for all n >= 23, 46.r^n < T(n) < 75.r^n and this is very tight; T(n) = Θ(r^n).)
Suppose we have a 2D (5x5) matrix:
test =
39 13 90 5 71
60 78 38 4 11
87 92 46 45 35
40 96 61 17 1
90 50 46 89 63
And a second 2D (5x2) matrix:
tidx =
1 3
2 4
2 3
2 4
4 5
And now we want to use tidx as an idex into test, so that we get the following output:
out =
39 90
78 4
92 46
96 17
89 63
One way to do this is with a for loop...
for i=1:size(test,1)
out(i,:) = test(i,tidx(i,:));
end
Question:
Is there a way to vectorize this so the same output is generated without a for loop?
Here is one way:
test(repmat([1:rows(test)]',1,columns(tidx)) + (tidx-1)*rows(test))
What you describe is an index problem. When you place a matrix all in one dimension, you get
test(:) =
39
60
87
40
90
13
78
92
96
50
90
38
46
61
46
5
4
45
17
89
71
11
35
1
63
This can be indexed using a single number. Here is how you figure out how to transform tidx into the correct format.
First, I use the above reference to figure out the index numbers which are:
outinx =
1 11
7 17
8 13
9 19
20 25
Then I start trying to figure out the pattern. This calculation gives a clue:
(tidx-1)*rows(test) =
0 10
5 15
5 10
5 15
15 20
This will move the index count to the correct column of test. Now I just need the correct row.
outinx-(tidx-1)*rows(test) =
1 1
2 2
3 3
4 4
5 5
This pattern is created by the for loop. I created that matrix with:
[1:rows(test)]' * ones(1,columns(tidx))
*EDIT: This does the same thing with a built in function.
repmat([1:rows(test)]',1,columns(tidx))
I then add the 2 together and use them as the index for test.
Alright. Now this question is pretty hard. I am going to give you an example.
Now the left numbers are my algorithm classification and the right numbers are the original class numbers
177 86
177 86
177 86
177 86
177 86
177 86
177 86
177 86
177 86
177 89
177 89
177 89
177 89
177 89
177 89
177 89
So here my algorithm merged 2 different classes into 1. As you can see it merged class 86 and 89 into one class. So what would be the error at the above example ?
Or here another example
203 7
203 7
203 7
203 7
16 7
203 7
17 7
16 7
203 7
At the above example left numbers are my algorithm classification and the right numbers are original class ids. As can be seen above it miss classified 3 products (i am classifying same commercial products). So at this example what would be the error rate? How would you calculate.
This question is pretty hard and complex. We have finished the classification but we are not able to find correct algorithm for calculating success rate :D
Here's a longish example, a real confuson matrix with 10 input classes "0" - "9"
(handwritten digits),
and 10 output clusters labelled A - J.
Confusion matrix for 5620 optdigits:
True 0 - 9 down, clusters A - J across
-----------------------------------------------------
A B C D E F G H I J
-----------------------------------------------------
0: 2 4 1 546 1
1: 71 249 11 1 6 228 5
2: 13 5 64 1 13 1 460
3: 29 2 507 20 5 9
4: 33 483 4 38 5 3 2
5: 1 1 2 58 3 480 13
6: 2 1 2 294 1 1 257
7: 1 5 1 546 6 7
8: 415 15 2 5 3 12 13 87 2
9: 46 72 2 357 35 1 47 2
----------------------------------------------------
580 383 496 1002 307 670 549 557 810 266 estimates in each cluster
y class sizes: [554 571 557 572 568 558 558 566 554 562]
kmeans cluster sizes: [ 580 383 496 1002 307 670 549 557 810 266]
For example, cluster A has 580 data points, 415 of which are "8"s;
cluster B has 383 data points, 249 of which are "1"s; and so on.
The problem is that the output classes are scrambled, permuted;
they correspond in this order, with counts:
A B C D E F G H I J
8 1 4 3 6 7 0 5 2 6
415 249 483 507 294 546 546 480 460 257
One could say that the "success rate" is
75 % = (415 + 249 + 483 + 507 + 294 + 546 + 546 + 480 + 460 + 257) / 5620
but this throws away useful information —
here, that E and J both say "6", and no cluster says "9".
So, add up the biggest numbers in each column of the confusion matrix
and divide by the total.
But, how to count overlapping / missing clusters,
like the 2 "6"s, no "9"s here ?
I don't know of a commonly agreed-upon way
(doubt that the Hungarian algorithm
is used in practice).
Bottom line: don't throw away information; look at the whole confusion matrix.
NB such a "success rate" will be optimistic for new data !
It's customary to split the data into say 2/3 "training set" and 1/3 "test set",
train e.g. k-means on the 2/3 alone,
then measure confusion / success rate on the test set — generally worse than on the training set alone.
Much more can be said; see e.g.
Cross-validation.
You have to define the error criteria if you want to evaluate the performance of an algorithm, so I'm not sure exactly what you're asking. In some clustering and machine learning algorithms you define the error metric and it minimizes it.
Take a look at this
https://en.wikipedia.org/wiki/Confusion_matrix
to get some ideas
You have to define a error metric to measure yourself. In your case, a simple method should be to find the properties mapping of your product as
p = properties(id)
where id is the product id, and p is likely be a vector with each entry of different properties. Then you can define the error function e (or distance) between two products as
e = d(p1, p2)
Sure, each properties must be evaluated to a number in this function. Then this error function can be used in the classification algorithm and learning.
In your second example, it seems that you treat the pair (203 7) as successful classification, so I think you have already a metric yourself. You may be more specific to get better answer.
Classification Error Rate(CER) is 1 - Purity (http://nlp.stanford.edu/IR-book/html/htmledition/evaluation-of-clustering-1.html)
ClusterPurity <- function(clusters, classes) {
sum(apply(table(classes, clusters), 2, max)) / length(clusters)
}
Code of #john-colby
Or
CER <- function(clusters, classes) {
1- sum(apply(table(classes, clusters), 2, max)) / length(clusters)
}
I have a set of N^2 numbers and N bins. Each bin is supposed to have N numbers from the set assigned to it. The problem I am facing is finding a set of distributions that map the numbers to the bins, satisfying the constraint, that each pair of numbers can share the same bin only once.
A distribution can nicely be represented by an NxN matrix, in which each row represents a bin. Then the problem is finding a set of permutations of the matrix' elements, in which each pair of numbers shares the same row only once. It's irrelevant which row it is, only that two numbers were both assigned to the same one.
Example set of 3 permutations satisfying the constraint for N=8:
0 1 2 3 4 5 6 7
8 9 10 11 12 13 14 15
16 17 18 19 20 21 22 23
24 25 26 27 28 29 30 31
32 33 34 35 36 37 38 39
40 41 42 43 44 45 46 47
48 49 50 51 52 53 54 55
56 57 58 59 60 61 62 63
0 8 16 24 32 40 48 56
1 9 17 25 33 41 49 57
2 10 18 26 34 42 50 58
3 11 19 27 35 43 51 59
4 12 20 28 36 44 52 60
5 13 21 29 37 45 53 61
6 14 22 30 38 46 54 62
7 15 23 31 39 47 55 63
0 9 18 27 36 45 54 63
1 10 19 28 37 46 55 56
2 11 20 29 38 47 48 57
3 12 21 30 39 40 49 58
4 13 22 31 32 41 50 59
5 14 23 24 33 42 51 60
6 15 16 25 34 43 52 61
7 8 17 26 35 44 53 62
A permutation that doesn't belong in the above set:
0 10 20 30 32 42 52 62
1 11 21 31 33 43 53 63
2 12 22 24 34 44 54 56
3 13 23 25 35 45 55 57
4 14 16 26 36 46 48 58
5 15 17 27 37 47 49 59
6 8 18 28 38 40 50 60
7 9 19 29 39 41 51 61
Because of multiple collisions with the second permutation, since, for example they're both pairing the numbers 0 and 32 in one row.
Enumerating three is easy, it consists of 1 arbitrary permutation, its transposition and a matrix where the rows are made of the previous matrix' diagonals.
I can't find a way to produce a set consisting of more though. It seems to be either a very complex problem, or a simple problem with an unobvious solution. Either way I'd be thankful if somebody had any ideas how to solve it in reasonable time for the N=8 case, or identified the proper, academic name of the problem, so I could google for it.
In case you were wondering what is it useful for, I'm looking for a scheduling algorithm for a crossbar switch with 8 buffers, which serves traffic to 64 destinations. This part of the scheduling algorithm is input traffic agnostic, and switches cyclically between a number of hardwired destination-buffer mappings. The goal is to have each pair of destination addresses compete for the same buffer only once in the cycling period, and to maximize that period's length. In other words, so that each pair of addresses was competing for the same buffer as seldom as possible.
EDIT:
Here's some code I have.
CODE
It's greedy, it usually terminates after finding the third permutation. But there should exist a set of at least N permutations satisfying the problem.
The alternative would require that choosing permutation I involved looking for permutations (I+1..N), to check if permutation I is part of the solution consisting of the maximal number of permutations. That'd require enumerating all permutations to check at each step, which is prohibitively expensive.
What you want is a combinatorial block design. Using the nomenclature on the linked page, you want designs of size (n^2, n, 1) for maximum k. This will give you n(n+1) permutations, using your nomenclature. This is the maximum theoretically possible by a counting argument (see the explanation in the article for the derivation of b from v, k, and lambda). Such designs exist for n = p^k for some prime p and integer k, using an affine plane. It is conjectured that the only affine planes that exist are of this size. Therefore, if you can select n, maybe this answer will suffice.
However, if instead of the maximum theoretically possible number of permutations, you just want to find a large number (the most you can for a given n^2), I am not sure what the study of these objects is called.
Make a 64 x 64 x 8 array: bool forbidden[i][j][k] which indicates whether the pair (i,j) has appeared in row k. Each time you use the pair (i, j) in the row k, you will set the associated value in this array to one. Note that you will only use the half of this array for which i < j.
To construct a new permutation, start by trying the member 0, and verify that at least seven of forbidden[0][j][0] that are unset. If there are not seven left, increment and try again. Repeat to fill out the rest of the row. Repeat this whole process to fill the entire NxN permutation.
There are probably optimizations you should be able to come up with as you implement this, but this should do pretty well.
Possibly you could reformulate your problem into graph theory. For example, you start with the complete graph with N×N vertices. At each step, you partition the graph into N N-cliques, and then remove all edges used.
For this N=8 case, K64 has 64×63/2 = 2016 edges, and sixty-four lots of K8 have 1792 edges, so your problem may not be impossible :-)
Right, the greedy style doesn't work because you run out of numbers.
It's easy to see that there can't be more than 63 permutations before you violate the constraint. On the 64th, you'll have to pair at least one of the numbers with another its already been paired with. The pigeonhole principle.
In fact, if you use the table of forbidden pairs I suggested earlier, you find that there are a maximum of only N+1 = 9 permutations possible before you run out. The table has N^2 x (N^2-1)/2 = 2016 non-redundant constraints, and each new permutation will create N x (N choose 2) = 28 new pairings. So all the pairings will be used up after 2016/28 = 9 permutations. It seems like realizing that there are so few permutations is the key to solving the problem.
You can generate a list of N permutations numbered n = 0 ... N-1 as
A_ij = (i * N + j + j * n * N) mod N^2
which generates a new permutation by shifting the columns in each permutation. The top row of the nth permutation are the diagonals of the n-1th permutation. EDIT: Oops... this only appears to work when N is prime.
This misses one last permutation, which you can get by transposing the matrix:
A_ij = j * N + i