I have to solve a system of non-linear equations for a wide range of parameter space. I'm using FindRoot, which is sensitive to initial start point so I have to do it by hand and by trial and error and plotting, rather than putting the equations in a loop or in a table.
So what I want to do is create a database or a Matrix with a fixed number of columns but variable number of rows so I can keep appending it with new results as and when I solve for them.
Right now I've used something like:
{{{xx, yy}} = {x, y} /. FindRoot[{f1(x,y) == 0,f2(x,y)==0}, {x,a},{y,b}],
g(xx,yy)} >>> "Attempt1.txt"
Where I am solving for two variables and then storing the variables and also a function g(xx,yy) of the variables.
This seems to work for me but the result is not a Matrix any more but the data is stored as some text type thing.
Is there anyway I can get this to stay a matrix or a database where I keep adding rows to it each time I solve for FindRoot by hand? Again, I need to do FindRoot by hand because it is sensitive to the start points and I don't know the good start points without first plotting it.
Thanks a lot
Unless I'm not understanding what you're trying to do, this should work
results = {};
results = Append[Flatten[{{xx, yy} = {x, y} /. FindRoot[{f1(x,y) == 0,f2(x,y)==0}, {x,a},{y,b}],g(xx,yy)}],results];
Then every time you're tying to add a line to the matrix results by hand, you would just type
results = Append[Flatten[{{xx, yy} = {x, y} /. FindRoot[{f1(x,y) == 0,f2(x,y)==0}, {x,a},{y,b}],g(xx,yy)}],results];
By the way, to get around the problem of sensitivity to the initial a and b values, you could explore the parameter space in a loop, varying the parameters slowly and using the solution of x and y from the previous loop iteration for your new a and b values each time.
What you want to do can be achieved by using Read instead of Get. While Get reads the complete file in one run, Read can be adjusted to extract a single Expression, Byte, Number and many more. So what you should do is open your file and read expression after expression and pack it inside a list.
PutAppend[{{1, 2}, {3, 4}}, "tmp.mx"]
PutAppend[{{5, 6}, {7, 8}}, "tmp.mx"]
PutAppend[{{9, 23}, {11, 12}}, "tmp.mx"]
PutAppend[{{13, 14}, {15, 16}}, "tmp.mx"]
stream = OpenRead["tmp.mx"];
mat = ArrayPad[
NestWhileList[Read[stream, Expression] &,
stream = OpenRead["tmp.mx"], # =!= EndOfFile &], -1];
Close[stream];
And now you have in mat a list containing all lines. The ArrayPad, which cuts off one element at each end is necessary because the first element contains the output of the OpenRead and the last element contains EndOfFile. If you are not familiar with functional constructs like NestWhileList then you can put it in a loop as you like, since it is really just the iterated calls to Read
stream = OpenRead["tmp.mx"];
mat = {};
AppendTo[mat, Read[stream, Expression]];
AppendTo[mat, Read[stream, Expression]];
AppendTo[mat, Read[stream, Expression]];
AppendTo[mat, Read[stream, Expression]];
Close[stream];
Related
I have not used PackedArray before, but just started looking at using them from reading some discussion on them here today.
What I have is lots of large size 1D and 2D matrices of all reals, and no symbolic (it is a finite difference PDE solver), and so I thought that I should take advantage of using PackedArray.
I have an initialization function where I allocate all the data/grids needed. So I went and used ToPackedArray on them. It seems a bit faster, but I need to do more performance testing to better compare speed before and after and also compare RAM usage.
But while I was looking at this, I noticed that some operations in M automatically return lists in PackedArray already, and some do not.
For example, this does not return packed array
a = Table[RandomReal[], {5}, {5}];
Developer`PackedArrayQ[a]
But this does
a = RandomReal[1, {5, 5}];
Developer`PackedArrayQ[a]
and this does
a = Table[0, {5}, {5}];
b = ListConvolve[ {{0, 1, 0}, {1, 4, 1}, {0, 1, 1}}, a, 1];
Developer`PackedArrayQ[b]
and also matrix multiplication does return result in packed array
a = Table[0, {5}, {5}];
b = a.a;
Developer`PackedArrayQ[b]
But element wise multiplication does not
b = a*a;
Developer`PackedArrayQ[b]
My question : Is there a list somewhere which documents which M commands return PackedArray vs. not? (assuming data meets the requirements, such as Real, not mixed, no symbolic, etc..)
Also, a minor question, do you think it will be better to check first if a list/matrix created is already packed before calling calling ToPackedArray on it? I would think calling ToPackedArray on list already packed will not cost anything, as the call will return right away.
thanks,
update (1)
Just wanted to mention, that just found that PackedArray symbols not allowed in a demo CDF as I got an error uploading one with one. So, had to remove all my packing code out. Since I mainly write demos, now this topic is just of an academic interest for me. But wanted to thank everyone for time and good answers.
There isn't a comprehensive list. To point out a few things:
Basic operations with packed arrays will tend to remain packed:
In[66]:= a = RandomReal[1, {5, 5}];
In[67]:= Developer`PackedArrayQ /# {a, a.a, a*a}
Out[67]= {True, True, True}
Note above that that my version (8.0.4) doesn't unpack for element-wise multiplication.
Whether a Table will result in a packed array depends on the number of elements:
In[71]:= Developer`PackedArrayQ[Table[RandomReal[], {24}, {10}]]
Out[71]= False
In[72]:= Developer`PackedArrayQ[Table[RandomReal[], {24}, {11}]]
Out[72]= True
In[73]:= Developer`PackedArrayQ[Table[RandomReal[], {25}, {10}]]
Out[73]= True
On["Packing"] will turn on messages to let you know when things unpack:
In[77]:= On["Packing"]
In[78]:= a = RandomReal[1, 10];
In[79]:= Developer`PackedArrayQ[a]
Out[79]= True
In[80]:= a[[1]] = 0 (* force unpacking due to type mismatch *)
Developer`FromPackedArray::punpack1: Unpacking array with dimensions {10}. >>
Out[80]= 0
Operations that do per-element inspection will usually unpack the array,
In[81]:= a = RandomReal[1, 10];
In[82]:= Position[a, Max[a]]
Developer`FromPackedArray::unpack: Unpacking array in call to Position. >>
Out[82]= {{4}}
There penalty for calling ToPackedArray on an already packed list is small enough that I wouldn't worry about it too much:
In[90]:= a = RandomReal[1, 10^7];
In[91]:= Timing[Do[Identity[a], {10^5}];]
Out[91]= {0.028089, Null}
In[92]:= Timing[Do[Developer`ToPackedArray[a], {10^5}];]
Out[92]= {0.043788, Null}
The frontend prefers packed to unpacked arrays, which can show up when dealing with Dynamic and Manipulate:
In[97]:= Developer`PackedArrayQ[{1}]
Out[97]= False
In[98]:= Dynamic[Developer`PackedArrayQ[{1}]]
Out[98]= True
When looking into performance, focus on cases where large lists are getting unpacked, rather than the small ones. Unless the small ones are in big loops.
This is just an addendum to Brett's answer:
SystemOptions["CompileOptions"]
will give you the lengths being used for which a function will return a packed array. So if you did need to pack a small list, as an alternative to using Developer`ToPackedArray you could temporarily set a smaller number for one of the compile options. e.g.
SetSystemOptions["CompileOptions" -> {"TableCompileLength" -> 20}]
Note also some difference between functions which to me at least doesn't seem intuitive so I generally have to test these kind of things whenever I use them rather than instinctively knowing what will work best:
f = # + 1 &;
g[x_] := x + 1;
data = RandomReal[1, 10^6];
On["Packing"]
Timing[Developer`PackedArrayQ[f /# data]]
{0.131565, True}
Timing[Developer`PackedArrayQ[g /# data]]
Developer`FromPackedArray::punpack1: Unpacking array with dimensions {1000000}.
{1.95083, False}
Another addition to Brett's answer: If a list is a packed array then a ToPackedArray is very fast since this checked quite early. Also you might find this valuable:
http://library.wolfram.com/infocenter/Articles/3141/
In general for numerics stuff look for talks from Rob Knapp and/or Mark Sofroniou.
When I develop numerics codes, I write the function and then use On["Packing"] to make sure that everything is packed that needs to be packed.
Concerning Mike's answer, the threshold has been introduced since for small stuff there is overhead. Where the threshold is is hardware dependent. It might be an idea to write a function that sets these threshold based on measurements done on the computer.
Is there a way to make MatrixForm display a row vector horizontally on the line and not vertically as it does for column vectors? As this confuses me sometimes. Do you think it will be hard to write wrapper around matrix form to adjust this behavior?
For example, here is a 2 by 2 matrix. The rows display the same as the columns. Both are shown vertical.
Question: Is it possible to make MatrixForm display row vectors laid out horizontally and not vertically?
Sorry if this was asked before, a quick search shows nothing.
thanks
update (1)
fyi, this is in Matlab, it prints rows horizontally and column vertically automatically, I was hoping for something like this. But I'll use suggestion by Heike below for now as it solves this at the cost of little extra typing.
update (2)
Using Hilderic solution is nice also, I always had hard time printing 3D matrix in a way I can read it. Here it is now using the {} trick
For both arrayname[[All,1]] and arrayname[[1,All]], Part delivers a vector, and MatrixForm has no way of determining which "orientation" it has. Accordingly, it always prints vectors as columns.
About the only thing you can do is provide your own output routine for row vectors, e.g., by wrapping it in an enclosing list, converting it back to a (single-row) matrix:
rowVector[a_List] := MatrixForm[{a}]
columnVector = MatrixForm (*for symmetry*)
It's still up to you to remember whether a vector came from a row or a column, though.
Or you could just cook up your own RowForm function, e.g.:
RowForm[(m_)?VectorQ] := Row[{"(",Row[m," "],
")"}, "\[MediumSpace]"];
Then
RowForm[twoRowsMatrix[[All,1]]]
looks kind of o.k.
Alternatively, if you really just care about displaying vectors, you could do:
twoRowsMatrix = {{a11, a12}, {a21, a22}};
TakeColumn[m_?MatrixQ, i_] := (Print[MatrixForm[#]]; #) &#m[[All, i]];
TakeRow[m_?MatrixQ, i_] := (Print[MatrixForm[{#}]]; #) &#m[[i]];
TakeColumn[twoRowsMatrix, 1]
TakeRow[twoRowsMatrix, 1]
If you don't care about the () part, then you can append with ,{}, wrap in curly brackets, and use TableForm or Grid instead:
vec = {x, y, z};
TableForm[{vec, {}}]
Grid[{vec, {}}]
When I get worried about this, I use {{a,b,c}} to specify a row of a,b,c (they can be any kind of list) and Transpose[{{a,b,c}}] to specify a column of a,b,c.
MatrixForm[a = RandomInteger[{0, 6}, {2, 2}]]
MatrixForm[b = RandomInteger[{0, 6}, {2, 2}]]
MatrixForm[c = RandomInteger[{0, 6}, {2, 2}]]
w = {a, b, c};
MatrixForm[w]
w = {{a, b, c}};
MatrixForm[w]
w = Transpose[{{a, b, c}}];
MatrixForm[w]
Graphics#Flatten[Table[
(*colors, dont mind*)
{ColorData["CMYKColors"][(a[[r, t]] - .000007)/(.0003 - 0.000007)],
(*point size, dont mind*)
PointSize[1/Sqrt[r]/10],
(*Coordinates for your points "a" is your data matrix *)
Point[
{(rr =Log[.025 + (.58 - .25)/64 r]) Cos#(tt = t 5 Degree),
rr Sin#tt}]
} &#
(*values for the iteration*)
, {r, 7, 64}, {t, 1, 72}], 1]
(*Rotation, dont mind*)
/. gg : Graphics[___] :> Rotate[gg, Pi/2]
Okay, I'll bite. First, Mathematica allows functions to be applied via one of several forms: standard form - f[x], prefix form - f # x, postfix form - f // x, and infix form - x ~ f ~ y. Belisarius's code uses both standard and prefix form.
So, let's look at the outermost functions first: Graphics # x /. gg : Graphics[___]:> Rotate[gg,Pi/2], where x is everything inside of Flatten. Essentially, what this does is create a Graphics object from x and using a named pattern (gg : Graphics[___]) rotates the resulting Graphics object by 90 degrees.
Now, to create a Graphics object, we need to supply a bunch of primitives and this is in the form of a nested list, where each sublist describes some element. This is done via the Table command which has the form: Table[ expr, iterators ]. Iterators can have several forms, but here they both have the form {var, min, max}, and since they lack a 4th term, they take on each value between min and max in integer steps. So, our iterators are {r, 7, 64} and {t, 1, 72}, and expr is evaluated for each value that they take on. Since, we have two iterators this produces a matrix, which would confuse Graphics, so we using Flatten[ Table[ ... ], 1] we take every element of the matrix and put it into a simple list.
Each element that Table produces is simply: color (ColorData), point size (PointSize), and point location (Point). So, with Flatten, we have created the following:
Graphics[{{color, point size, point}, {color, point size, point}, ... }]
The color generation is taken from the data, and it assumes that the data has been put into a list called a. The individual elements of a are accessed through the Part construct: [[]]. On the surface, the ColorData construct is a little odd, but it can be read as ColorData["CMYKColors"] returns a ColorDataFunction that produces a CMYK color value when a value between 0 and 1 is supplied. That is why the data from a is scaled the way it is.
The point size is generated from the radial coordinate. You'd expect with 1/Sqrt[r] the point size should be getting smaller as r increases, but the Log inverts the scale.
Similarly, the point location is produced from the radial and angular (t) variables, but Point only accepts them in {x,y} form, so he needed to convert them. Two odd constructs occur in the transformation from {r,t} to {x,y}: both rr and tt are Set (=) while calculating x allowing them to be used when calculating y. Also, the term t 5 Degree lets Mathematica know that the angle is in degrees, not radians. Additionally, as written, there is a bug: immediately following the closing }, the terms & and # should not be there.
Does that help?
I'm a newbie in Mathematica and I'm having a major malfunction with adding columns to a data table. I'm running Mathematica 7 in Vista. I have spent a lot of time RFD before asking here.
I have a data table (mydata) with three columns and five rows. I'm trying to add two lists of five elements to the table (effectively adding two columns to the data table).
This works perfectly:
Table[AppendTo[mydata[[i]],myfirstlist[[i]]],{i,4}]
Printing out the table with: mydata // TableForm shows the added column.
However, when I try to add my second list
Table[AppendTo[mydata[[i]],mysecondlist[[i]]],{i,5}]
either Mathematica crashes(!) or I get a slew of Part::partw and Part::spec errors saying Part 5 does not exist.
However, after all the error messages (if Mathematica does not crash), again printing out the data table with: mydata // TableForm shows the data table with five columns just like I intended. All TableForm formatting options on the revised data table work fine.
Could anyone tell me what I'm doing wrong? Thanks in advance!
Let's try to clarify what the double transpose method consists of. I make no claims about the originality of the approach. My focus is on the clarity of exposition.
Let's begin with 5 lists. First we'll place three in a table. Then we'll add the final two.
food = {"bagels", "lox", "cream cheese", "coffee", "blueberries"};
mammals = {"fisher cat", "weasel", "skunk", "raccon", "squirrel"};
painters = {"Picasso", "Rembrandt", "Klee", "Rousseau", "Warhol"};
countries = {"Brazil", "Portugal", "Azores", "Guinea Bissau",
"Cape Verde"};
sports = {"golf", "badminton", "football", "tennis", "rugby"};
The first three lists--food, mammals, painters--become the elements of lists3. They are just lists, but TableForm displays them in a table as rows.
(lists3 = {food, mammals, painters}) // TableForm
mydata will be the name for lists3 transposed. Now the three lists appear as columns. That's what transposition does: columns and rows are switched.
(mydata = Transpose#lists3) // TableForm
This is where the problem actually begins. How can we add two additional columns (that is, the lists for countries and sports)? So let's work with the remaining two lists.
(lists2 = {countries, sports}) // TableForm
So we can Join Transpose[mydata] and lists2....
(lists5 = Join[Transpose[mydata], lists2]) // TableForm
[Alternatively, we might have Joined lists3 and lists2 because the second transposition, the transposition of mydata undoes the earlier transposition.
lists3 is just the transposition of mydata. (and vice-versa).]
In[]:= lists3 === Transpose[mydata]
Out[]:= True
Now we only need to Transpose the result to obtain the desired final table of five lists, each occupying its own column:
Transpose#lists5 // TableForm
I hope that helps shed some light on how to add two columns to a table. I find this way reasonably clear. You may find some other way clearer.
There are several things to cover here. First, the following code does not give me any errors, so there may be something else going on here. Perhaps you should post a full code block that produces the error.
mydata = Array[Subscript[{##}] &, {5, 3}];
myfirstlist = Range[1, 5];
mysecondlist = Range[6, 10];
Table[AppendTo[mydata[[i]], myfirstlist[[i]]], {i, 4}];
mydata // TableForm
Table[AppendTo[mydata[[i]], mysecondlist[[i]]], {i, 5}];
mydata // TableForm
Second, there is no purpose in using Table here, as you are modifying mydata directly. Table will use up memory pointlessly.
Third, there are better ways to accomplish this task.
See How to prepend a column and Inserting into a 2d list
I must retract my definitive statement that there are better ways. After changing Table to Do and running a few quick tests, this appears to be a competitive method for some data.
I am using Mathematica 7, so that does not appear to be the problem.
As mentioned before, there are better alternatives to adding columns to a list, and like Gareth and Mr.Wizard, I do not seem to be able to reproduce the problem on v. 7. But, I want to focus on the error itself, and see if we can correct it that way. When Mathematica produces the message Part::partw it spits out part of the offending list like
Range[1000][[1001]]
Part::partw: Part 1001 of
{1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,
31,32,33,34,35,36,37,38,39,40,41,42,43,44,45,46,47,48,49,50,<<950>>}
does not exist.
So, the question I ask is which list is giving me the problems? My best guess is it is mysecondlist, and I'd check Length # mysecondlist to see if it is actually 5 elements long.
Well, here's my two cents with what I believe is a very fast and IMHO most easily understandable construction.
First, some test arrays:
m = RandomInteger[100, {2000, 10000}];
l1 = RandomInteger[100, 2000];
l2 = RandomInteger[100, 2000];
{r, c} = Dimensions[m];
I increased the test array sizes somewhat to improve accuracy of the following timing measurements.
The method involves the invoking of the powers of Part ([[...]]), All and Span (;;).
Basically, I set up a new working matrix with the future dimensions of the data array after addition of the two columns, then add the original matrix using All and Span and add the additional columns with All only. I then copy back the scrap matrix to our original matrix, as the other methods also return the modified data matrix.
n = ConstantArray[0, {r, c} + {0, 2}];
n[[All, 1 ;; c]] = m;
n[[All, c + 1]] = l1;
n[[All, c + 2]] = l2;
m = n;
As for timing:
Mean[
Table[
First[
AbsoluteTiming[
n2 = ConstantArray[0, {r, c} + {0, 2}];
n2[[All, 1 ;; c]] = m;
n2[[All, c + 1]] = l1;
n2[[All, c + 2]] = l2;
m2 = n2;
]
],
{10}
]
]
0.1056061
(an average of 10 runs)
The other proposed method with Do (Mr.Wizard and the OP):
Mean[
Table[
n1 = m;
First[
AbsoluteTiming[
Do[AppendTo[n1[[i]], l1[[i]]], {i, 2000}];
Do[AppendTo[n1[[i]], l2[[i]]], {i, 2000}];
]
],
{10}
]
]
0.4898280
The result is the same:
In[9]:= n2 == n1
Out[9]= True
So, a conceptually easy and quick (5 times faster!) method.
I tried to reproduce this but failed. I'm running Mma 8 on Windows XP; it doesn't seem like the difference should matter, but who knows? I said, successively,
myData = {{1, 2, 3}, {2, 3, 4}, {8, 9, 10}, {1, 1, 1}, {2, 2, 2}}
myFirstList = {9, 9, 9, 9, 9}
mySecondList = {6, 6, 6, 6, 6}
Table[AppendTo[myData[[i]], myFirstList[[i]]], {i, 4}]
Table[AppendTo[myData[[i]], mySecondList[[i]]], {i, 5}]
myData // TableForm
and got (0) no crash, (1) no errors or warnings, and (2) the output I expected. (Note: I used 4 rather than 5 in the limit of the first set of appends, just like in your question, in case that was somehow provoking trouble.)
The Mma documentation claims that AppendTo[a,b] is always equivalent to a=Append[a,b], which suggests that it isn't modifying the list in-place. But I wonder whether maybe AppendTo sometimes does modify the list when it thinks it's safe to do so; then if it thinks it's safe and it isn't, there could be nasty consequences. Do the weird error messages and crashes still happen if you replace AppendTo with Append + ordinary assignment?
I know that using the Insert menu, you can create a matrix with vertical and horizontal lines, but not a more generic partition, such as dividing a 4x4 matrix into 4 2x2 partitions. Nor, can MatrixForm do any sort of partitioning. So, how would I go about programmatically displaying such a partitioned matrix? I would like to retain the ability of MatrixForm to act only as a wrapper and not affect subsequent evaluations, but it is not strictly necessary. I suspect this would involve using a Grid, but I haven't tried it.
After playing around for far too long trying to make Interpretation drop the displayed form and use the matrix when used in subsequent lines, I gave up and just made a wrapper that acts almost exactly like MatrixForm. This was really quick as it was a simple modification of this question.
Clear[pMatrixForm,pMatrixFormHelper]
pMatrixForm[mat_,col_Integer,row_:{}]:=pMatrixForm[mat,{col},row]
pMatrixForm[mat_,col_,row_Integer]:=pMatrixForm[mat,col,{row}]
pMatrixFormHelper[mat_,col_,row_]:=Interpretation[MatrixForm[
{Grid[mat,Dividers->{Thread[col->True],Thread[row->True]}]}],mat]
pMatrixForm[mat_?MatrixQ,col:{___Integer}:{},row:{___Integer}:{}]:=
(CellPrint[ExpressionCell[pMatrixFormHelper[mat,col,row],
"Output",CellLabel->StringJoin["Out[",ToString[$Line],"]//pMatrixForm="]]];
Unprotect[Out];Out[$Line]=mat;Protect[Out];mat;)
Then the postfix command //pMatrixForm[#, 3, 3]& will give the requested 2x2 partitioning of a 4x4 matrix. It maybe useful to change the defaults of pMatrixForm from no partitions to central partitions. This would not be hard.
So this is what I came up with. For a matrix M:
M = {{a, b, 0, 0}, {c, d, 0, 0}, {0, 0, x, y}, {0, 0, z, w}};
you construct two list of True/False values (with True for places where you want separators) that take two arguments; first the matrix and second a list of positions for separators.
colSep = Fold[ReplacePart[#1, #2 -> True] &,
Table[False, {First#Dimensions##1 + 1}], #2] &;
rowSep = Fold[ReplacePart[#1, #2 -> True] &,
Table[False, {Last#Dimensions##1 + 1}], #2] &;
Now the partitioned view using Grid[] is made with the use of Dividers:
partMatrix = Grid[#1, Dividers -> {colSep[#1, #2], rowSep[#1, #3]}] &;
This takes three arguments; first the matrix, second the list of positions for column dividers, and third the list of values for row dividers.
In order for it to display nicely you just wrap it in brakets and use MatrixForm:
MatrixForm#{partMatrix[M, {3}, {3}]}
Which does the 2by2 partitioning you mentioned.