Related
I'm looking for a fast implementation for the following, I'll call it Position2D for lack of a better term:
Position2D[ matrix, sub_matrix ]
which finds the locations of sub_matrix inside matrix and returns the upper left and lower right row/column of a match.
For example, this:
Position2D[{
{0, 1, 2, 3},
{1, 2, 3, 4},
{2, 3, 4, 5},
{3, 4, 5, 6}
}, {
{2, 3},
{3, 4}
}]
should return this:
{
{{1, 3}, {2, 4}},
{{2, 2}, {3, 3}},
{{3, 1}, {4, 2}}
}
It should be fast enough to work quickly on 3000x2000 matrices with 100x100 sub-matrices. For simplicity, it is enough to only consider integer matrices.
Algorithm
The following code is based on an efficient custom position function to find positions of (possibly overlapping) integer sequences in a large integer list. The main idea is that we can first try to eficiently find the positions where the first row of the sub-matrix is in the Flatten-ed large matrix, and then filter those, extracting full sub-matrices and comparing to the sub-matrix of interest. This will be efficient for most cases except very pathological ones (those, for which this procedure would generate a huge number of potential position candidates, while the true number of entries of the sub-matrix would be much smaller. But such cases seem rather unlikely generally, and then, further improvements to this simple scheme can be made).
For large matrices, the proposed solution will be about 15-25 times faster than the solution of #Szabolcs when a compiled version of sequence positions function is used, and 3-5 times faster for the top-level implementation of sequence positions - finding function. The actual speedup depends on matrix sizes, it is more for larger matrices. The code and benchmarks are below.
Code
A generally efficient function for finding positions of a sub-list (sequence)
These helper functions are due to Norbert Pozar and taken from this Mathgroup thread. They are used to efficiently find starting positions of an integer sequence in a larger list (see the mentioned post for details).
Clear[seqPos];
fdz[v_] := Rest#DeleteDuplicates#Prepend[v, 0];
seqPos[list_List, seq_List] :=
Fold[
fdz[#1 (1 - Unitize[list[[#1]] - #2])] + 1 &,
fdz[Range[Length[list] - Length[seq] + 1] *
(1 - Unitize[list[[;; -Length[seq]]] - seq[[1]]])] + 1,
Rest#seq
] - Length[seq];
Example of use:
In[71]:= seqPos[{1,2,3,2,3,2,3,4},{2,3,2}]
Out[71]= {2,4}
A faster position-finding function for integers
However fast seqPos might be, it is still the major bottleneck in my solution. Here is a compiled-to-C version of this, which gives another 5x performance boost to my code:
seqposC =
Compile[{{list, _Integer, 1}, {seq, _Integer, 1}},
Module[{i = 1, j = 1, res = Table[0, {Length[list]}], ctr = 0},
For[i = 1, i <= Length[list], i++,
If[list[[i]] == seq[[1]],
While[j < Length[seq] && i + j <= Length[list] &&
list[[i + j]] == seq[[j + 1]],
j++
];
If[j == Length[seq], res[[++ctr]] = i];
j = 1;
]
];
Take[res, ctr]
], CompilationTarget -> "C", RuntimeOptions -> "Speed"]
Example of use:
In[72]:= seqposC[{1, 2, 3, 2, 3, 2, 3, 4}, {2, 3, 2}]
Out[72]= {2, 4}
The benchmarks below have been redone with this function (also the code for main function is slightly modified )
Main function
This is the main function. It finds positions of the first row in a matrix, and then filters them, extracting the sub-matrices at these positions and testing against the full sub-matrix of interest:
Clear[Position2D];
Position2D[m_, what_,seqposF_:Automatic] :=
Module[{posFlat, pos2D,sp = If[seqposF === Automatic,seqposC,seqposF]},
With[{dm = Dimensions[m], dwr = Reverse#Dimensions[what]},
posFlat = sp[Flatten#m, First#what];
pos2D =
Pick[Transpose[#], Total[Clip[Reverse#dm - # - dwr + 2, {0, 1}]],2] &#
{Mod[posFlat, #, 1], IntegerPart[posFlat/#] + 1} &#Last[dm];
Transpose[{#, Transpose[Transpose[#] + dwr - 1]}] &#
Select[pos2D,
m[[Last## ;; Last## + Last#dwr - 1,
First## ;; First## + First#dwr - 1]] == what &
]
]
];
For integer lists, the faster compiled subsequence position-finding function seqposC can be used (this is a default). For generic lists, one can supply e.g. seqPos, as a third argument.
How it works
We will use a simple example to dissect the code and explain its inner workings. This defines our test matrix and sub-matrix:
m = {{0, 1, 2, 3}, {1, 2, 3, 4}, {2, 3, 4, 5}};
what = {{2, 3}, {3, 4}};
This computes the dimensions of the above (it is more convenient to work with reversed dimensions for a sub-matrix):
In[78]:=
dm=Dimensions[m]
dwr=Reverse#Dimensions[what]
Out[78]= {3,4}
Out[79]= {2,2}
This finds a list of starting positions of the first row ({2,3} here) in the Flattened main matrix. These positions are at the same time "flat" candidate positions of the top left corner of the sub-matrix:
In[77]:= posFlat = seqPos[Flatten#m, First#what]
Out[77]= {3, 6, 9}
This will reconstruct the 2D "candidate" positions of the top left corner of a sub-matrix in a full matrix, using the dimensions of the main matrix:
In[83]:= posInterm = Transpose#{Mod[posFlat,#,1],IntegerPart[posFlat/#]+1}&#Last[dm]
Out[83]= {{3,1},{2,2},{1,3}}
We can then try using Select to filter them out, extracting the full sub-matrix and comparing to what, but we'll run into a problem here:
In[84]:=
Select[posInterm,
m[[Last##;;Last##+Last#dwr-1,First##;;First##+First#dwr-1]]==what&]
During evaluation of In[84]:= Part::take: Cannot take positions 3 through 4
in {{0,1,2,3},{1,2,3,4},{2,3,4,5}}. >>
Out[84]= {{3,1},{2,2}}
Apart from the error message, the result is correct. The error message itself is due to the fact that for the last position ({1,3}) in the list, the bottom right corner of the sub-matrix will be outside the main matrix. We could of course use Quiet to simply ignore the error messages, but that's a bad style. So, we will first filter those cases out, and this is what the line Pick[Transpose[#], Total[Clip[Reverse#dm - # - dwr + 2, {0, 1}]], 2] &# is for. Specifically, consider
In[90]:=
Reverse#dm - # - dwr + 2 &#{Mod[posFlat, #, 1],IntegerPart[posFlat/#] + 1} &#Last[dm]
Out[90]= {{1,2,3},{2,1,0}}
The coordinates of the top left corners should stay within a difference of dimensions of matrix and a sub-matrix. The above sub-lists were made of x and y coordiantes of top - left corners. I added 2 to make all valid results strictly positive. We have to pick only coordiantes at those positions in Transpose#{Mod[posFlat, #, 1], IntegerPart[posFlat/#] + 1} &#Last[dm] ( which is posInterm), at which both sub-lists above have strictly positive numbers. I used Total[Clip[...,{0,1}]] to recast it into picking only at those positions at which this second list has 2 (Clip converts all positive integers to 1, and Total sums numbers in 2 sublists. The only way to get 2 is when numbers in both sublists are positive).
So, we have:
In[92]:=
pos2D=Pick[Transpose[#],Total[Clip[Reverse#dm-#-dwr+2,{0,1}]],2]&#
{Mod[posFlat,#,1],IntegerPart[posFlat/#]+1}&#Last[dm]
Out[92]= {{3,1},{2,2}}
After the list of 2D positions has been filtered, so that no structurally invalid positions are present, we can use Select to extract the full sub-matrices and test against the sub-matrix of interest:
In[93]:=
finalPos =
Select[pos2D,m[[Last##;;Last##+Last#dwr-1,First##;;First##+First#dwr-1]]==what&]
Out[93]= {{3,1},{2,2}}
In this case, both positions are genuine. The final thing to do is to reconstruct the positions of the bottom - right corners of the submatrix and add them to the top-left corner positions. This is done by this line:
In[94]:= Transpose[{#,Transpose[Transpose[#]+dwr-1]}]&#finalPos
Out[94]= {{{3,1},{4,2}},{{2,2},{3,3}}}
I could have used Map, but for a large list of positions, the above code would be more efficient.
Example and benchmarks
The original example:
In[216]:= Position2D[{{0,1,2,3},{1,2,3,4},{2,3,4,5},{3,4,5,6}},{{2,3},{3,4}}]
Out[216]= {{{3,1},{4,2}},{{2,2},{3,3}},{{1,3},{2,4}}}
Note that my index conventions are reversed w.r.t. #Szabolcs' solution.
Benchmarks for large matrices and sub-matrices
Here is a power test:
nmat = 1000;
(* generate a large random matrix and a sub-matrix *)
largeTestMat = RandomInteger[100, {2000, 3000}];
what = RandomInteger[10, {100, 100}];
(* generate upper left random positions where to insert the submatrix *)
rposx = RandomInteger[{1,Last#Dimensions[largeTestMat] - Last#Dimensions[what] + 1}, nmat];
rposy = RandomInteger[{1,First#Dimensions[largeTestMat] - First#Dimensions[what] + 1},nmat];
(* insert the submatrix nmat times *)
With[{dwr = Reverse#Dimensions[what]},
Do[largeTestMat[[Last#p ;; Last#p + Last#dwr - 1,
First#p ;; First#p + First#dwr - 1]] = what,
{p,Transpose[{rposx, rposy}]}]]
Now, we test:
In[358]:= (ps1 = position2D[largeTestMat,what])//Short//Timing
Out[358]= {1.39,{{{1,2461},{100,2560}},<<151>>,{{1900,42},{1999,141}}}}
In[359]:= (ps2 = Position2D[largeTestMat,what])//Short//Timing
Out[359]= {0.062,{{{2461,1},{2560,100}},<<151>>,{{42,1900},{141,1999}}}}
(the actual number of sub-matrices is smaller than the number we try to generate, since many of them overlap and "destroy" the previously inserted ones - this is so because the sub-matrix size is a sizable fraction of the matrix size in our benchmark).
To compare, we should reverse the x-y indices in one of the solutions (level 3), and sort both lists, since positions may have been obtained in different order:
In[360]:= Sort#ps1===Sort[Reverse[ps2,{3}]]
Out[360]= True
I do not exclude a possibility that further optimizations are possible.
This is my implementation:
position2D[m_, k_] :=
Module[{di, dj, extractSubmatrix, pos},
{di, dj} = Dimensions[k] - 1;
extractSubmatrix[{i_, j_}] := m[[i ;; i + di, j ;; j + dj]];
pos = Position[ListCorrelate[k, m], ListCorrelate[k, k][[1, 1]]];
pos = Select[pos, extractSubmatrix[#] == k &];
{#, # + {di, dj}} & /# pos
]
It uses ListCorrelate to get a list of potential positions, then filters those that actually match. It's probably faster on packed real matrices.
As per Leonid's suggestion here's my solution. I know it isn't very efficient (it's about 600 times slower than Leonid's when I timed it) but it's very short, rememberable, and a nice illustration of a rarely used function, PartitionMap. It's from the Developer package, so it needs a Needs["Developer`"] call first.
Given that, Position2D can be defined as:
Position2D[m_, k_] := Position[PartitionMap[k == # &, m, Dimensions[k], {1, 1}], True]
This only gives the upper-left coordinates. I feel the lower-right coordinates are really redundant, since the dimensions of the sub-matrix are known, but if the need arises one can add those to the output by prepending {#, Dimensions[k] + # - {1, 1}} & /# to the above definition.
How about something like
Position2D[bigMat_?MatrixQ, smallMat_?MatrixQ] :=
Module[{pos, sdim = Dimensions[smallMat] - 1},
pos = Position[bigMat, smallMat[[1, 1]]];
Quiet[Select[pos, (MatchQ[
bigMat[[Sequence##Thread[Span[#, # + sdim]]]], smallMat] &)],
Part::take]]
which will return the top left-hand positions of the submatrices.
Example:
Position2D[{{0, 1, 2, 3}, {1, 2, 3, 4}, {2, 3, 4, 5}, {3, 5, 5, 6}},
{{2, 3}, {3, _}}]
(* Returns: {{1, 3}, {2, 2}, {3, 1}} *)
And to search a 1000x1000 matrix, it takes about 2 seconds on my old machine
SeedRandom[1]
big = RandomInteger[{0, 10}, {1000, 1000}];
Position2D[big, {{1, 1, _}, {1, 1, 1}}] // Timing
(* {1.88012, {{155, 91}, {295, 709}, {685, 661},
{818, 568}, {924, 45}, {981, 613}}} *)
When I type the following
lis = {1, 2};
pos = Position[lis, 1];
result = Extract[lis, pos]
the result is always a list.
{1}
another example
lis = {{1}, {2}};
pos = Position[lis, {1}];
result = Extract[lis, pos]
{{1}}
Mathematica always adds an extra {} in the result. What would be the best way to remove this extra {}, other than applying Flatten[result,1] each time? And is there a case where removing these extra {} can cause a problem?
You probably realise this, but Position and Extract return lists because the requested values may be found in more than one position. So in general, removing the outer brackets doesn't make sense.
If you are sure the result is a singleton list, using Flatten would destroy information if the element is itself a list. For example,
Position[{{1}},1]
gives a list whose sole element is a list. So in this case, using Extract would make more sense.
Even so, there are many situations where Mathematica treats {x} very differently to x, as in
Position[1,1]
Position[{1},1]
which have very different results. So whether you can remove the outer braces from a one-member list depends on what you plan to do with it.
If I understood your question correctly, you are asking why
lis = {{1}, {2}};
pos = Position[lis, {1}];
result = Extract[lis, pos]
returns
{{1}}
rather than
{1}
The answer is, I think, simple: Position[lis,{1}] gives the position at which {1}, not 1 appears in lis; when you then go and look at that position using Extract, you do indeed get {1} which is then wrapped in a list (which is exactly what happened in the first case, when you looked for 1 and obtained {1} as a result; just replace 1 by {1}, because that is now what you are asking for.
To see this more clearly, try
lis = {f[1], f[2]};
pos = Position[lis, f[1]];
result = Extract[lis, pos]
which gives
{f[1]}
The point here is that List in {1} (which is the same as List[1] if you check look at the FullForm) before was just a head, like f here. Should mathematica have remove f here? If not, then why should it have removed the innermost List earlier?
And finally, if you really want to remove the inner {} in your second example, try
lis = {{1}, {2, {1}}};
pos = Position[lis, {1}];
result = Extract[lis, pos, #[[1]] &]
giving
{1, 1}
EDIT: I am becoming puzzled with some of the answers here. If I do
lis = {{1}, {2, {1, 2, {1}}}};
pos = Position[lis, 1];
result = Extract[lis, pos]
then I get
{1, 1, 1}
in result. I only get the extra brackets when I actually obtain the positions of {1} in pos instead of the positions of 1 (and then when I look at those positions, I find {1}). Or am I missing something in your question?
Short answer: You should probably use First#Position[...]
Long answer:
Lets separate the question to 2 parts:
Why do you have the extra {} in the result for Position?
i.e. why:
lis = {1, 2};
Position[lis, 1]
returns {{1}}?
This is in order to work consistently with n-dimensional list, that may have the requested values in more than one position. For example:
lis = {{1, 2, 3}, {1, 5, 6}, {1, 2, 1}};
Position[lis, 1]
returns {{1, 1}, {2, 1}, {3, 1}, {3, 3}}
which is a list of the coordinates the result is found in.
So in your case:
lis = {1, 2};
Position[lis, 1]
return {{1}}, as in: we found your requested value one time, in the coordinate-set {1}.
Now, a lot of times Mathematica assume that there might be a list of solutions (for example, in Solve), but the user know that he expect only one. A suitable code to this in your case will be First#Position[...]. this will return the first (and, assumebly, only) element in the list of positions --
So, if you are sure that the element you are searching for exist only once in the list and want to know where, use this way.
Why do you have the extra {} in the result for Extract?
Extract can work in two different ways.
If I'm doing Extract[{{a, b, c}, {d, e, f}, {g, e, h}}, {1, 2}]
I will get b, so extract with a 1 dimensional list of is just choosing and returning this element. In fact, Extract[lis, {1, 2}] is equal to lis[[1, 2]]
If I'm doing Extract[{{a, b, c}, {d, e, f}, {g, e, h}}, {{1, 2}, {3, 4}}]
I will get {b, h}, so extract with a 2 dimensional list is choosing and returning a list of elements.
In your case(s), you are doing Extract[lis, {{1}}], as in: give me a list containing only the element lis[[1]]. The result is always this element in a list, which is the extra {}
I have the following Nested table
(myinputmatrix = Table[Nest[function, myinputmatrix[[i]][[j]],
myinputmatrix[[i]][[j]][[2]][[2]] +
myinputmatrix[[i]][[j]][[3]][[2]]], {i,
Dimensions[myinputmatrix][[1]]}, {j,
Dimensions[myinputmatrix][[2]]}]) // TableForm
fq[k_?NumericQ] := Count[RandomReal[{0, 1}, k], x_ /; x < .1]
function[x_List] := ReplacePart[
x, {{2, 1} -> x[[2]][[1]] - #1,
{2, 2} -> x[[2]][[2]] + #1,
{3, 1} -> x[[3]][[1]] - #2, {3, 2} ->
x[[3]][[2]] + #2}] &[fq[x[[2]][[1]]], fq[x[[2]][[1]]]];
My problem is that I want to add only the #1 in the bold part above, but not only the new one, I want it to add all #1 for the n times (Nest function times]
If I try the function
function[x_List] := ReplacePart[
x, {{2, 1} -> x[[2]][[1]] - #1, {2, 2} -> #1,
{3, 1} -> x[[3]][[1]] - #2, {3, 2} -> #2}] &[fq[x[[2]][[1]]],
fq[x[[2]][[1]]]];
I am having as a result the last value of fq[k]. I thought of replacing that part in my table with 0 but is not going to work since I am using it in my nested list, also I thought of substricting that part from my initial table but I am not sure which way is the best to do it and if the way I am thinking is the correct one. Can anyone help me?
If I may restate the problem and hopefully clarify the question for myself. At each iteration in the Nest, you want to add not the current (random) output from fq, but the cumulation of the current and all past values of it. But because the random output depends at each iteration on the input matrix, you need to calculate both the random number and the current value of the matrix in the same iteration.
If that hadn't been true you could use Fold.
Restating fq as Sasha suggested EDIT with some type checking to avoid problems with incorrect input:
fq[k_Integer?Positive]:=RandomVariate[BinomialDistribution[k,.1]]
You might want to add some other error checking code. Something like this, depending on your requirements, might do.
fq[0]:= 0;
fq[k_Real?Positive]:=RandomVariate[BinomialDistribution[Round[k],.1]]
You need function to take the random numbers as parameters. EDIT 1 and 2 I have changed the syntax of this function to use the parameters explicitly instead of the original question's anonymous function within a function. This should avoid some syntax errors. Also note that I have used "NumericQ" rather than "Real" as the type for the rv1 and rv2 parameters, because they can be integers at the start of the Nest iteration.
function[x_List, rv1_?NumericQ, rv2_?NumericQ] := ReplacePart[
x, {{2, 1} -> x[[2]][[1]] - rv1, {2, 2} -> rv1,
{3, 1} -> x[[3]][[1]] - rv2, {3, 2} -> rv2}]
And then pass the current random number as a local constant using With to a Nest function that works on a list containing your matrix and the cumulation of the random variates. I have used myoutputmatrix because I really don't like the idea of rewriting existing expressions all the time. That's just me. Now, the one other thing is that you need to set n, the number of iterates. I've set it to 5 but you can make this a parameter in a function if you want (see below).
(myoutputmatrix = Table[ First[Nest[With[{rv=fq[#1[[1]][[2]][[1]] ]},
{function[#1[[1]],rv, rv+#1[[2]] ],rv+#1[[2]] }]&,
{ myinputmatrix[[i]][[j]], 0 }, 5]],
{i, Dimensions[myinputmatrix][[1]]}, {j,
Dimensions[myinputmatrix][[2]]}]) // TableForm
The First is there because in the end you only want the matrix, not the cumulation of the random variates.
outputmatrix[input_List, n_Integer?Positive] /;
Length[Dimensions[input]] == 4 :=
Table[First[
Nest[With[{rv = fq[#1[[1]][[2]][[1]]]}, {function[#1[[1]], rv,
rv + #1[[2]]], rv + #1[[2]]}] &, {input[[i]][[j]], 0}, n]],
{i, Dimensions[input][[1]]}, {j, Dimensions[input][[2]]}]
outputmatrix[myinputmatrix, 10] // TableForm
EDIT I have checked this now and it runs, but note that you can get negative numbers in the output, which is not what you want, I don't think.
Suppose we want to generate a list of primes p for which p + 2 is also prime.
A quick solution is to generate a complete list of the first n primes and use the Select function to return the elements which meet the condition.
Select[Table[Prime[k], {k, n}], PrimeQ[# + 2] &]
However, this is inefficient as it loads a large list into the memory before returning the filtered list. A For loop with Sow/Reap (or l = {}; AppendTo[l, k]) solves the memory issue, but it is far from elegant and is cumbersome to implement a number of times in a Mathematica script.
Reap[
For[k = 1, k <= n, k++,
p = Prime[k];
If[PrimeQ[p + 2], Sow[p]]
]
][[-1, 1]]
An ideal solution would be a built-in function which allows an option similar to this.
Table[Prime[k], {k, n}, AddIf -> PrimeQ[# + 2] &]
I will interpret this more as a question about automation and software engineering rather than about the specific problem at hand, and given a large number of solutions posted already. Reap and Sow are good means (possibly, the best in the symbolic setting) to collect intermediate results. Let us just make it general, to avoid code duplication.
What we need is to write a higher-order function. I will not do anything radically new, but will simply package your solution to make it more generally applicable:
Clear[tableGen];
tableGen[f_, iter : {i_Symbol, __}, addif : Except[_List] : (True &)] :=
Module[{sowTag},
If[# === {}, #, First##] &#
Last#Reap[Do[If[addif[#], Sow[#,sowTag]] &[f[i]], iter],sowTag]];
The advantages of using Do over For are that the loop variable is localized dynamically (so, no global modifications for it outside the scope of Do), and also the iterator syntax of Do is closer to that of Table (Do is also slightly faster).
Now, here is the usage
In[56]:= tableGen[Prime, {i, 10}, PrimeQ[# + 2] &]
Out[56]= {3, 5, 11, 17, 29}
In[57]:= tableGen[Prime, {i, 3, 10}, PrimeQ[# + 1] &]
Out[57]= {}
In[58]:= tableGen[Prime, {i, 10}]
Out[58]= {2, 3, 5, 7, 11, 13, 17, 19, 23, 29}
EDIT
This version is closer to the syntax you mentioned (it takes an expression rather than a function):
ClearAll[tableGenAlt];
SetAttributes[tableGenAlt, HoldAll];
tableGenAlt[expr_, iter_List, addif : Except[_List] : (True &)] :=
Module[{sowTag},
If[# === {}, #, First##] &#
Last#Reap[Do[If[addif[#], Sow[#,sowTag]] &[expr], iter],sowTag]];
It has an added advantage that you may even have iterator symbols defined globally, since they are passed unevaluated and dynamically localized. Examples of use:
In[65]:= tableGenAlt[Prime[i], {i, 10}, PrimeQ[# + 2] &]
Out[65]= {3, 5, 11, 17, 29}
In[68]:= tableGenAlt[Prime[i], {i, 10}]
Out[68]= {2, 3, 5, 7, 11, 13, 17, 19, 23, 29}
Note that since the syntax is different now, we had to use the Hold-attribute to prevent the passed expression expr from premature evaluation.
EDIT 2
Per #Simon's request, here is the generalization for many dimensions:
ClearAll[tableGenAltMD];
SetAttributes[tableGenAltMD, HoldAll];
tableGenAltMD[expr_, iter__List, addif : Except[_List] : (True &)] :=
Module[{indices, indexedRes, sowTag},
SetDelayed ## Prepend[Thread[Map[Take[#, 1] &, List ## Hold ### Hold[iter]],
Hold], indices];
indexedRes =
If[# === {}, #, First##] &#
Last#Reap[Do[If[addif[#], Sow[{#, indices},sowTag]] &[expr], iter],sowTag];
Map[
First,
SplitBy[indexedRes ,
Table[With[{i = i}, Function[Slot[1][[2, i]]]], {i,Length[Hold[iter]] - 1}]],
{-3}]];
It is considerably less trivial, since I had to Sow the indices together with the added values, and then split the resulting flat list according to the indices. Here is an example of use:
{i, j, k} = {1, 2, 3};
tableGenAltMD[i + j + k, {i, 1, 5}, {j, 1, 3}, {k, 1, 2}, # < 7 &]
{{{3, 4}, {4, 5}, {5, 6}}, {{4, 5}, {5, 6}, {6}}, {{5, 6}, {6}}, {{6}}}
I assigned the values to i,j,k iterator variables to illustrate that this function does localize the iterator variables and is insensitive to possible global values for them. To check the result, we may use Table and then delete the elements not satisfying the condition:
In[126]:=
DeleteCases[Table[i + j + k, {i, 1, 5}, {j, 1, 3}, {k, 1, 2}],
x_Integer /; x >= 7, Infinity] //. {} :> Sequence[]
Out[126]= {{{3, 4}, {4, 5}, {5, 6}}, {{4, 5}, {5, 6}, {6}}, {{5, 6}, {6}}, {{6}}}
Note that I did not do extensive checks so the current version may contain bugs and needs some more testing.
EDIT 3 - BUG FIX
Note the important bug-fix: in all functions, I now use Sow with a custom unique tag, and Reap as well. Without this change, the functions would not work properly when expression they evaluate also uses Sow. This is a general situation with Reap-Sow, and resembles that for exceptions (Throw-Catch).
EDIT 4 - SyntaxInformation
Since this is such a potentially useful function, it is nice to make it behave more like a built-in function. First we add syntax highlighting and basic argument checking through
SyntaxInformation[tableGenAltMD] = {"ArgumentsPattern" -> {_, {_, _, _., _.}.., _.},
"LocalVariables" -> {"Table", {2, -2}}};
Then, adding a usage message allows the menu item "Make Template" (Shift+Ctrl+k) to work:
tableGenAltMD::usage = "tableGenAltMD[expr,{i,imax},addif] will generate \
a list of values expr when i runs from 1 to imax, \
only including elements if addif[expr] returns true.
The default of addiff is True&."
A more complete and formatted usage message can be found in this gist.
I think the Reap/Sow approach is likely to be most efficient in terms of memory usage. Some alternatives might be:
DeleteCases[(With[{p=Prime[#]},If[PrimeQ[p+2],p,{}] ] ) & /# Range[K]),_List]
Or (this one might need some sort of DeleteCases to eliminate Null results):
FoldList[[(With[{p=Prime[#2]},If[PrimeQ[p+2],p] ] )& ,1.,Range[2,K] ]
Both hold a big list of integers 1 to K in memory, but the Primes are scoped inside the With[] construct.
Yes, this is another answer. Another alternative that includes the flavour of the Reap/Sow approach and the FoldList approach would be to use Scan.
result = {1};
Scan[With[{p=Prime[#]},If[PrimeQ[p+2],result={result,p}]]&,Range[2,K] ];
Flatten[result]
Again, this involves a long list of integers, but the intermediate Prime results are not stored because they are in the local scope of With. Because p is a constant in the scope of the With function, you can use With rather than Module, and gain a bit of speed.
You can perhaps try something like this:
Clear[f, primesList]
f = With[{p = Prime[#]},Piecewise[{{p, PrimeQ[p + 2]}}, {}] ] &;
primesList[k_] := Union#Flatten#(f /# Range[k]);
If you want both the prime p and the prime p+2, then the solution is
Clear[f, primesList]
f = With[{p = Prime[#]},Piecewise[{{p, PrimeQ[p + 2]}}, {}] ] &;
primesList[k_] :=
Module[{primes = f /# Range[k]},
Union#Flatten#{primes, primes + 2}];
Well, someone has to allocate memory somewhere for the full table size, since it is not known before hand what the final size will be.
In the good old days before functional programming :), this sort of thing was solved by allocating the maximum array size, and then using a separate index to insert to it so no holes are made. Like this
x=Table[0,{100}]; (*allocate maximum possible*)
j=0;
Table[ If[PrimeQ[k+2], x[[++j]]=k],{k,100}];
x[[1;;j]] (*the result is here *)
{1,3,5,9,11,15,17,21,27,29,35,39,41,45,51,57,59,65,69,71,77,81,87,95,99}
Here's another couple of alternatives using NextPrime:
pairs1[pmax_] := Select[Range[pmax], PrimeQ[#] && NextPrime[#] == 2 + # &]
pairs2[pnum_] := Module[{p}, NestList[(p = NextPrime[#];
While[p + 2 != (p = NextPrime[p])];
p - 2) &, 3, pnum]]
and a modification of your Reap/Sow solution that lets you specify the maximum prime:
pairs3[pmax_] := Module[{k,p},
Reap[For[k = 1, (p = Prime[k]) <= pmax, k++,
If[PrimeQ[p + 2], Sow[p]]]][[-1, 1]]]
The above are in order of increasing speed.
In[4]:= pairs2[10000]//Last//Timing
Out[4]= {3.48,1261079}
In[5]:= pairs1[1261079]//Last//Timing
Out[5]= {6.84,1261079}
In[6]:= pairs3[1261079]//Last//Timing
Out[7]= {0.58,1261079}
Does mathematica have something like "select any" that gets any element of a list that satisfies a criterion?
If you just want to return after the first matching element, use the optional third argument to Select, which is the maximum number of results to return. So you can just do
Any[list_List, crit_, default_:"no match"] :=
With[{maybeMatch = Select[list, crit, 1]},
If[maybeMatch =!= {},
First[maybeMatch],
default]
Mathematica lacks a great way to signal failure to find an answer, since it lacks multiple return values, or the equivalent of Haskell's Maybe type. My solution is to have a user-specifiable default value, so you can make sure you pass in something that's easily distinguishable from a valid answer.
Well, the downside of Eric's answer is that it does execute OddQ on all elements of the list. My call is relatively costly, and it feels wrong to compute it too often. Also, the element of randomness is clearly unneeded, the first one is fine with me.
So, how about
SelectAny[list_List, criterion_] :=
Catch[Scan[ If[criterion[#], Throw[#, "result"]] &, list];
Throw["No such element"], "result"]
And then
SelectAny[{1, 2, 3, 4, 5}, OddQ]
returns 1.
I still wish something were built into Mathematica. Using home-brew functions kind of enlarges your program without bringing much direct benefit.
The Select function provides this built-in, via its third argument which indicates the maximum number of items to select:
In[1]:= Select[{1, 2, 3, 4, 5}, OddQ, 1]
Out[1]= {1}
When none match:
In[2]:= Select[{2, 4}, OddQ, 1]
Out[2]= {}
Edit: Oops, missed that nes1983 already stated this.
You can do it relatively easily with Scan and Return
fstp[p_, l_List] := Scan[ p## && Return## &, l ]
So
In[2]:= OddQ ~fstp~ Range[1,5]
Out[2]= 1
In[3]:= EvenQ ~fstp~ Range[1,5]
Out[3]= 2
I really wish Mathematica could have some options to make expressions evaluated lazily. In a lazy language such as Haskell, you can just define it as normal
fstp p = head . filter p
There's "Select", that gets all the elements that satisfy a condition. So
In[43]:= Select[ {1, 2, 3, 4, 5}, OddQ ]
Out[43]= {1, 3, 5}
Or do you mean that you want to randomly select a single matching element? I don't know of anything built-in, but you can define it pretty quickly:
Any[lst_, q_] :=
Select[ lst, q] // (Part[#, 1 + Random[Integer, Length[#] - 1]]) &
Which you could use the same way::
In[51]:= Any[ {1, 2, 3, 4, 5}, OddQ ]
Out[51]= 3