Accidental shadowing and `Removed[symbol]` - wolfram-mathematica

If you evaluate the following code twice, results will be different. Can anyone explain what's going on?
findHull[points_] := Module[{},
Needs["ComputationalGeometry`"];
ConvexHull[points]
];
findHull[RandomReal[1, {10, 2}]];
Remove["Global`ConvexHull"];
findHull[RandomReal[1, {10, 2}]]

The problem is that even though the module is not evaluated untill you call findHull, the symbols are resolved when you define findHull (i.e.: The new downvalue for findHull is stored in terms of symbols, not text).
This means that during the first round, ConvexHull resolves to Global`ConvexHull because the Needs is not evaluated.
During the second round, ComputationalGeometry is on $ContextPath and so ConvexHull resolves as you intended.
If you really cannot bear to load ComputationalGeometry beforehand, just refer to ConvexHull by its full name: ComputationalGeometry`ConvexHull. See also this related answer.
HTH

Not a direct answer to the question, but a bit too large for a comment. As another alternative, a general way to delay the symbol parsing until run-time is to use Symbol["your-symbol-name"]. In your case, you can replace ConvexHull on the r.h.s. of your definition by Symbol["ConvexHull"]:
findHull[points_] :=
Module[{},
Needs["ComputationalGeometry`"];
Symbol["ConvexHull"][points]];
This solution is not very elegant though, since Symbol["ConvexHull"] will be executed every time afresh. This can also be somewhat error-prone, if you do non-trivial manipulations with $ContextPath. Here is a modified version, combined with a generally useful trick with self-redefinition, that I use in similar cases:
Clear[findHull];
findHull[points_] :=
Module[{},
Needs["ComputationalGeometry`"];
With[{ch = Symbol["ConvexHull"]},
findHull[pts_] := ch[pts];
findHull[points]]];
For example,
findHull[RandomReal[1, {10, 2}]]
{4, 10, 9, 1, 6, 2, 5}
What happens is that the first time the function is called, the original definition with Module gets replaced by the inner one, and that happens already after the needed package is loaded and its context placed on the $ContextPath. Here we exploit the fact that Mathematica replaces an old definition with a new one if it can determine that the patterns are the same - as it can in such cases.
Other instances when self-redefinition trick is useful are cases when, for example, a function call results in some expensive computation, which we want to cache, but we are not sure whether or not the function will be called at all. Then, such construct allows to cache the computed (say, symbolically) result automatically upon the first call to the function.

Related

Using a struct to pass parameters by reference

This question is on the subject of passing by reference in M (a related question of mine is here simple question on passing data between functions)
While I was trying to find a way to pass things by reference without using Unevaluted[] or HoldFirst[] , I hit on this method by mistake and it looks really working well for me, even though I do not understand how it works and any hidden risks of using it. So I'd like to show it here to the experts and ask if they think it is safe to use (I have very large demo and needed to package parameters into number of different structs to help manage them, and this is how I found this method while I was trying things).
Here is the method: First we know that one can not write the following:
Remove[p]
foo[p_] := Module[{u},
u = Table[99, {10}];
p = u
];
p = 0;
foo[p];
One way to to update 'p' in the above is to change to call to become
foo[Unevaluated#p];
Or by defining foo[] with HoldFirst.
But here is the way I found, which does the pass by reference, without either of these:
I put all the parameters in a struct (I do this anyway now), and pass the struct, and then one can update the fields of the struct inside foo[] and the updates will be reflected in the way back from the function call:
Remove[parms]
foo[parms_] := Module[{u},
u = Table[99, {10}];
parms["p"] = u
];
parms["p"] = 0;
foo[parms];
Now, parms["p"] contained the new list {99, 99, 99, 99, 99, 99, 99, 99, 99, 99}
So, the parms was overwritten/updated inside foo[] without me having to tell M to pass parms by reference !
I tried this in my program, and I see no strange side effects. The CDF was updated ok with no error. I do not know how this works, may be M treats all the fields inside parms in this example as global?
But either case, I am happy with this so far as it provides me a way to package my many parameters into structs, and at the same time I am able to do the updated inside the functions to the struct.
But my question is: do you see a major problems with this method? How does it work internally? I mean how does M handle this passing without me doing HoldFirst or Unevaluated? I know that I lost the ability now to do parameters checking as before, but I can't have everything I want. As I said before, M needs a real build-in struct as part of the language and integrated into it. But this is for another time to talk about.
Btw, the best struct emulation I've see so far was by this one by Leonid Shifrin posted at the end of this thread here but unfortunately I could not use it in my demo as it uses symbols not allowed in a demo CDF.
thanks
Update:
Btw, this below is what I think how M handles this. This is just a guess of mine: I think param is some kind of a lookup table, and its fields are just used as an index into it (may be a hash table?) Then param["p1"] will contain in it the address (not the value) of a location in the heap of where the actual value of param["p1"] live.
Something like this:
So, when passing param, then inside the function foo[], when typing param["p1"]=u it will cause the current memory pointed to by "p1" to be freed up, and then a new memory allocated from the heap, and in it the value of u is copied to.
And so, on return back, this is why we see the content of param["p1"] has changed. But what actually changed, is the content of the memory being pointed to by the address which param["p1"] represent. (there is a name, which is "p1", and an address field, which points to the content that the name "p1" represents)
But then this means, that the address itself, which "p1" represents has changed, but the lookup name itself (which is "p1") did not change.
So, as the name itself did not change, there was no need to use HoldFirst, even though the data being pointed to by what this name represent has been modified?
Update:
There is one small glitch in this method, but it is not a big deal to work around it: when passing things using this indexed object method, it turns out that one can't modify part of the object. For example, this below does not work:
foo[param_] := Module[{},
param["u"][[3]] = 99 (*trying to update PART of u *)
];
param["u"] = Table[0, {5}];
foo[param];
The above gives the error
Set::setps: "param[u] in the part assignment is not a symbol"
But the workaround is easy. make a local copy of the whole field that one wants to update part of it, then update (part) of the local copy, then write the copy back to the field, like this
foo[param_] := Module[{u = param["u"]}, (* copy the whole field *)
u[[3]] = 99; (*update local copy *)
param["u"] = u (*now update the field, ok *)
];
param["u"] = Table[0, {5}];
foo[param];
Well. It would have been better if one can update part of the field, so no 'special' handling would be needed. But at least the work around is not that bad.
Update
For completeness, I thought I mention another tiny thing about using indexed objects and a work around.
I wrote
param[u] = {1, 2, 3}
param[u][[1 ;; -1]]
Which returns {1,2,3} as expected.
Then I found that I can use param#u instead of param[u], which really helped as too many solid brackets are starting to give me a headache.
But then when I typed
param#u[[1 ;; -1]]
expecting to get back the same answer as before, it did not. one gets an error, (I think I know why, operator precedence issue, but that is not the point now), just wanted to say that the workaround is easy, either one can use param[u][[1 ;; -1]] or use (param#u)[[1 ;; -1]]
I like the (param#u)[[1 ;; -1]] more, so that is what I am using now to access lists inside indexed objects.
The notation param#u is as close I can get to the standard record notation which is param.u so I am now happy now with indexed objects, and it seems to be working really well. Just couple of minor things to watch out for.
I am not going to respond to the question of low level data structures as I am simply not qualified to do so.
You are creating "indexed objects" and using immutable strings as indices. Generally speaking I believe that these are similar to hash tables.
Taking your code example, let us remove a possible ambiguity by using a unique name for the argument of foo:
Remove[parms, foo]
foo[thing_] :=
Module[{u},
u = Table[99, {10}];
thing["p"] = u
];
parms["p"] = 0;
foo[parms];
parms["p"]
DownValues[parms]
{HoldPattern[parms["p"]] :> {99, 99, 99, 99, 99, 99, 99, 99, 99, 99}}
This shows how the data is stored in terms of high level Mathematica structures.
parms is a global symbol. As you have observed, it can be safely "passed" without anything happening, because it is just a symbol without OwnValues and nothing is triggered when it is evaluated. This is exactly the same as writing/evaluating foo by itself.
The replacement that takes place in foo[thing_] := ... is analogous to With. If we write:
With[{thing = parms}, thing[x]]
thing in thing[x] is replaced with parms before thing[x] evaluates. Likewise, in the code above we get parms["p"] = u before thing["p"] or the Set evaluates. At that time, the HoldFirst attribute of Set takes over, and you get what you want.
Since you are using an immutable string as your index, it is not in danger of changing. As long as you are aware of how to handle non-localized symbols such as parms then there is also no new danger that I see in using this method.

May I write {x,a,b}//Do[...,#]& instead of Do[...,{x,a,b}]?

I'm in love with Ruby. In this language all core functions are actually methods. That's why I prefer postfix notation – when the data, which I want to process is placed left from the body of anonymous processing function, for example: array.map{...}. I believe, that it has advantages in how easy is this code to read.
But Mathetica, being functional (yeah, it can be procedural if you want) dictates a style, where Function name is placed left from the data. As we can see in its manuals, // is used only when it's some simple Function, without arguments, like list // MatrixForm. When Function needs a lot of arguments, people who wrote manuals, use syntax F[data].
It would be okay, but my problem is the case F[f,data], for example Do[function, {x, a, b}]. Most of Mathematica functions (if not all) have arguments in exactly this order – [function, data], not [data, function]. As I prefer to use pure functions to keep namespace clean instead of creating a lot of named functions in my notebook, the argument function can be too big – so big, that argument data would be placed on the 5-20th line of code after the line with Function call.
This is why sometimes, when evil Ruby nature takes me under control, I rewrite such functions in postfix way:
Because it's important for me, that pure function (potentially big code) is placed right from processing data. Yeah I do it and I'm happy. But there are two things:
this causes Mathematica's highlighting parser problem: the x in postfix notation is highlighted with blue color, not turquoise;
everytime when I look into Mathematica manuals, I see examples like this one: Do[x[[i]] = (v[[i]] - U[[i, i + 1 ;; n]].x[[i + 1 ;; n]])/ U[[i, i]], {i, n, 1, -1}];, which means... hell, they think it's easy to read/support/etc.?!
So these two things made me ask this question here: am I so bad boy, that use my Ruby-style, and should I write code like these guys do, or is it OK, and I don't have to worry, and should write as I like to?
The style you propose is frequently possible, but is inadvisable in the case of Do. The problem is that Do has the attribute HoldAll. This is important because the loop variable (x in the example) must remain unevaluated and be treated as a local variable. To see this, try evaluating these expressions:
x = 123;
Do[Print[x], {x, 1, 2}]
(* prints 1 and 2 *)
{x, 1, 2} // Do[Print[x], #]&
(* error: Do::itraw: Raw object 123 cannot be used as an iterator.
Do[Print[x], {123, 1, 2}]
*)
The error occurs because the pure function Do[Print[x], #]& lacks the HoldAll attribute, causing {x, 1, 2} to be evaluated. You could solve the problem by explicitly defining a pure function with the HoldAll attribute, thus:
{x, 1, 2} // Function[Null, Do[Print[x], #], HoldAll]
... but I suspect that the cure is worse than the disease :)
Thus, when one is using "binding" expressions like Do, Table, Module and so on, it is safest to conform with the herd.
I think you need to learn to use the styles that Mathematica most naturally supports. Certainly there is more than one way, and my code does not look like everyone else's. Nevertheless, if you continue to try to beat Mathematica syntax into your own preconceived style, based on a different language, I foresee nothing but continued frustration for you.
Whitespace is not evil, and you can easily add line breaks to separate long arguments:
Do[
x[[i]] = (v[[i]] - U[[i, i + 1 ;; n]].x[[i + 1 ;; n]]) / U[[i, i]]
, {i, n, 1, -1}
];
This said, I like to write using more prefix (f # x) and infix (x ~ f ~ y) notation that I usually see, and I find this valuable because it is easy to determine that such functions are receiving one and two arguments respectively. This is somewhat nonstandard, but I do not think it is kicking over the traces of Mathematica syntax. Rather, I see it as using the syntax to advantage. Sometimes this causes syntax highlighting to fail, but I can live with that:
f[x] ~Do~ {x, 2, 5}
When using anything besides the standard form of f[x, y, z] (with line breaks as needed), you must be more careful of evaluation order, and IMHO, readability can suffer. Consider this contrived example:
{x, y} // # + 1 & ## # &
I do not find this intuitive. Yes, for someone intimate with Mathematica's order of operations, it is readable, but I believe it does not improve clarity. I tend to reserve // postfix for named functions where reading is natural:
Do[f[x], {x, 10000}] //Timing //First
I'd say it is one of the biggest mistakes to try program in a language B in ways idiomatic for a language A, only because you happen to know the latter well and like it. There is nothing wrong in borrowing idioms, but you have to make sure to understand the second language well enough so that you know why other people use it the way they do.
In the particular case of your example, and generally, I want to draw attention to a few things others did not mention. First, Do is a scoping construct which uses dynamic scoping to localize its iterator symbols. Therefore, you have:
In[4]:=
x=1;
{x,1,5}//Do[f[x],#]&
During evaluation of In[4]:= Do::itraw: Raw object
1 cannot be used as an iterator. >>
Out[5]= Do[f[x],{1,1,5}]
What a surprise, isn't it. This won't happen when you use Do in a standard fashion.
Second, note that, while this fact is largely ignored, f[#]&[arg] is NOT always the same as f[arg]. Example:
ClearAll[f];
SetAttributes[f, HoldAll];
f[x_] := Print[Unevaluated[x]]
f[5^2]
5^2
f[#] &[5^2]
25
This does not affect your example, but your usage is close enough to those cases affected by this, since you manipulate the scopes.
Mathematica supports 4 ways of applying a function to its arguments:
standard function form: f[x]
prefix: f#x or g##{x,y}
postfix: x // f, and
infix: x~g~y which is equivalent to g[x,y].
What form you choose to use is up to you, and is often an aesthetic choice, more than anything else. Internally, f#x is interpreted as f[x]. Personally, I primarily use postfix, like you, because I view each function in the chain as a transformation, and it is easier to string multiple transformations together like that. That said, my code will be littered with both the standard form and prefix form mostly depending on whim, but I tend to use standard form more as it evokes a feeling of containment with regards to the functions parameters.
I took a little liberty with the prefix form, as I included the shorthand form of Apply (##) alongside Prefix (#). Of the built in commands, only the standard form, infix form, and Apply allow you easily pass more than one variable to your function without additional work. Apply (e.g. g ## {x,y}) works by replacing the Head of the expression ({x,y}) with the function, in effect evaluating the function with multiple variables (g##{x,y} == g[x,y]).
The method I use to pass multiple variables to my functions using the postfix form is via lists. This necessitates a little more work as I have to write
{x,y} // f[ #[[1]], #[[2]] ]&
to specify which element of the List corresponds to the appropriate parameter. I tend to do this, but you could combine this with Apply like
{x,y} // f ## #&
which involves less typing, but could be more difficult to interpret when you read it later.
Edit: I should point out that f and g above are just placeholders, they can, and often are, replaced with pure functions, e.g. #+1& # x is mostly equivalent to #+1&[x], see Leonid's answer.
To clarify, per Leonid's answer, the equivalence between f#expr and f[expr] is true if f does not posses an attribute that would prevent the expression, expr, from being evaluated before being passed to f. For instance, one of the Attributes of Do is HoldAll which allows it to act as a scoping construct which allows its parameters to be evaluated internally without undo outside influence. The point is expr will be evaluated prior to it being passed to f, so if you need it to remain unevaluated, extra care must be taken, like creating a pure function with a Hold style attribute.
You can certainly do it, as you evidently know. Personally, I would not worry about how the manuals write code, and just write it the way I find natural and memorable.
However, I have noticed that I usually fall into definite patterns. For instance, if I produce a list after some computation and incidentally plot it to make sure it's what I expected, I usually do
prodListAfterLongComputation[
args,
]//ListPlot[#,PlotRange->Full]&
If I have a list, say lst, and I am now focusing on producing a complicated plot, I'll do
ListPlot[
lst,
Option1->Setting1,
Option2->Setting2
]
So basically, anything that is incidental and perhaps not important to be readable (I don't need to be able to instantaneously parse the first ListPlot as it's not the point of that bit of code) ends up being postfix, to avoid disrupting the already-written complicated code it is applied to. Conversely, complicated code I tend to write in the way I find easiest to parse later, which, in my case, is something like
f[
g[
a,
b,
c
]
]
even though it takes more typing and, if one does not use the Workbench/Eclipse plugin, makes it more work to reorganize code.
So I suppose I'd answer your question with "do whatever is most convenient after taking into account the possible need for readability and the possible loss of convenience such as code highlighting, extra work to refactor code etc".
Of course all this applies if you're the only one working with some piece of code; if there are others, it is a different question alltogether.
But this is just an opinion. I doubt it's possible for anybody to offer more than this.
For one-argument functions (f#(arg)), ((arg)//f) and f[arg] are completely equivalent even in the sense of applying of attributes of f. In the case of multi-argument functions one may write f#Sequence[args] or Sequence[args]//f with the same effect:
In[1]:= SetAttributes[f,HoldAll];
In[2]:= arg1:=Print[];
In[3]:= f#arg1
Out[3]= f[arg1]
In[4]:= f#Sequence[arg1,arg1]
Out[4]= f[arg1,arg1]
So it seems that the solution for anyone who likes postfix notation is to use Sequence:
x=123;
Sequence[Print[x],{x,1,2}]//Do
(* prints 1 and 2 *)
Some difficulties can potentially appear with functions having attribute SequenceHold or HoldAllComplete:
In[18]:= Select[{#, ToExpression[#, InputForm, Attributes]} & /#
Names["System`*"],
MemberQ[#[[2]], SequenceHold | HoldAllComplete] &][[All, 1]]
Out[18]= {"AbsoluteTiming", "DebugTag", "EvaluationObject", \
"HoldComplete", "InterpretationBox", "MakeBoxes", "ParallelEvaluate", \
"ParallelSubmit", "Parenthesize", "PreemptProtect", "Rule", \
"RuleDelayed", "Set", "SetDelayed", "SystemException", "TagSet", \
"TagSetDelayed", "Timing", "Unevaluated", "UpSet", "UpSetDelayed"}

SaveDefinitions considered dangerous

SaveDefinitions is a nice option of Manipulate. It causes Manipulate to store any definitions used for its creation inside the Manipulate panel. A Manipulate made this way can be copied to an empty notebook and will still work on its own. Additionally, your working notebook containing many such Manipulates also doesn't turn into a flurry of pink boxes with printed error messages below it upon opening. Great!
However, all this goodness has its dark side which can bite you real hard if you are not aware of it. I've had this in a notebook I had been working on for a few days, but I present you with a step-by-step toy example scenario which recreates the problem.
In this scenario you want to create a Manipulate showing a plot of a nice wavy function, so you define this (please make a window size like this, this is important):
The definition is nice, so we keep it for the next time and make it an initialization cell. Next we add the Manipulate, and execute it too.
f[x_] := x^2
Manipulate[
Plot[n f[x], {x, -3, 3}],
{n, 1, 4},
SaveDefinitions -> True
]
All works great, the Manipulate really shines, it is a good day.
Just being your paranoid self you check whether the definition is OK:
Yeah, everything still checks out. Fine. But now it occurs to you that a better wavy function would be a sine, so you change the definition, execute, and being paranoid, check:
Everything still fine. You're ready from a day's hard work you save your work and quit. [Quit kernel]
Next day. You start your work again. You evaluate the initialization cells in your notebook. Definition still good? Check.
Now, you scroll down to your Manipulate box (no need to re-execute thanks to the SaveDefinitions), play a little with the slider. And scroll back up.
Being the paranoid you, you once more check the definition of f:
Lo and behold, someone has changed the definition behind your back! And nothing executed between your first and second Information(?) check according to the In[] numbers (In[1]: def of f, In[2] first ?, In[3] second ?).
What happened? Well, it's the Manipulate of course. A FullForm reveals its internal structure:
Manipulate[Plot[n*f[x],{x, -3, 3}],{{n, 2.44}, 1, 4},Initialization:>{f[x_] := x^2}]
There you have the culprit. The initialization part of the box defines f again, but it's the old version because we didn't re-evaluate the Manipulate after modifying its definition. As soon as the manipulate box gets on the screen, it is evaluated and you've got your old definition back. Globally!
Of course, in this toy example it is immediately clear something strange is happening. In my case, I had a larger module in a larger notebook in which I, after some debugging, had changed a small part. It seemed to work, but the next day, the same bug that had bugged me before hit again. It took me a couple of hours before I realized that one of the several Manipulates that I used to study the problem at hand from all sides was doing this.
Clearly, I'm tempted to say, this is unwanted behavior. Now, for the obligatory question: what can we do to prevent this behind-your-back behavior of Manipulate from occurring other than re-executing every Manipulate in your notebook each time you change a definition that might be used by them?
Here is an attempt. The idea is to identify symbols with DownValues or some other ...Values inside your manipulated code, and automatically rename them using unique variables / symbols in place of them. The idea here can be executed rather elegantly with the help of cloning symbols functionality, which I find useful from time to time. The function clone below will clone a given symbol, producing a symbol with the same global definitions:
Clear[GlobalProperties];
GlobalProperties[] :=
{OwnValues, DownValues, SubValues, UpValues, NValues, FormatValues,
Options, DefaultValues, Attributes};
Clear[unique];
unique[sym_] :=
ToExpression[
ToString[Unique[sym]] <>
StringReplace[StringJoin[ToString /# Date[]], "." :> ""]];
Attributes[clone] = {HoldAll};
clone[s_Symbol, new_Symbol: Null] :=
With[{clone = If[new === Null, unique[Unevaluated[s]], ClearAll[new]; new],
sopts = Options[Unevaluated[s]]},
With[{setProp = (#[clone] = (#[s] /. HoldPattern[s] :> clone)) &},
Map[setProp, DeleteCases[GlobalProperties[], Options]];
If[sopts =!= {}, Options[clone] = (sopts /. HoldPattern[s] :> clone)];
HoldPattern[s] :> clone]]
There are several alternatives of how to implement the function itself. One is to introduce the function with another name, taking the same arguments as Manipulate, say myManipulate. I will use another one: softly overload Manipulate via UpValues of some custom wrapper, that I will introduce. I will call it CloneSymbols. Here is the code:
ClearAll[CloneSymbols];
CloneSymbols /:
Manipulate[args___,CloneSymbols[sd:(SaveDefinitions->True)],after:OptionsPattern[]]:=
Unevaluated[Manipulate[args, sd, after]] /.
Cases[
Hold[args],
s_Symbol /; Flatten[{DownValues[s], SubValues[s], UpValues[s]}] =!= {} :>
clone[s],
Infinity, Heads -> True];
Here is an example of use:
f[x_] := Sin[x];
g[x_] := x^2;
Note that to use the new functionality, one has to wrap the SaveDefinitions->True option in CloneSymbols wrapper:
Manipulate[Plot[ f[n g[x]], {x, -3, 3}], {n, 1, 4},
CloneSymbols[SaveDefinitions -> True]]
This will not affect the definitions of original symbols in the code inside Manipulate, since it were their clones whose definitions have been saved and used in initialization now. We can look at the FullForm for this Manipulate to confirm that:
Manipulate[Plot[f$37782011751740542578125[Times[n,g$37792011751740542587890[x]]],
List[x,-3,3]],List[List[n,1.9849999999999999`],1,4],RuleDelayed[Initialization,
List[SetDelayed[f$37782011751740542578125[Pattern[x,Blank[]]],Sin[x]],
SetDelayed[g$37792011751740542587890[Pattern[x,Blank[]]],Power[x,2]]]]]
In particular, you can change the definitions of functions to say
f[x_]:=Cos[x];
g[x_]:=x;
Then move the slider of the Manipulate produced above, and then check the function definitions
?f
Global`f
f[x_]:=Cos[x]
?g
Global`g
g[x_]:=x
This Manipulate is reasonably independent of anything and can be copied and pasted safely. What happens here is the following: we first find all symbols with non-trivial DownValues, SubValues or UpValues (one can probably add OwnValues as well), and use Cases and clone to create their clones on the fly. We then replace lexically all the cloned symbols with their clones inside Manipulate, and then let Manipulate save the definitions for the clones. In this way, we make a "snapshot" of the functions involved, but do not affect the original functions in any way.
The uniqueness of the clones (symbols) has been addressed with the unique function. Note however, that while the Manipulate-s obtained in this way do not threaten the original function definitions, they will generally still depend on them, so one can not consider them totally independent of anything. One would have to walk down the dependency tree and clone all symbols there, and then reconstruct their inter-dependencies, to construct a fully standalone "snapshot" in Manipulate. This is doable but more complicated.
EDIT
Per request of #Sjoerd, I add code for a case when we do want our Manipulate-s to update to the function's changes, but do not want them to actively interfere and change any global definitions. I suggest a variant of a "pointer" technique: we will again replace function names with new symbols, but, rather than cloning those new symbols after our functions, we will use the Manipulate's Initialization option to simply make those symbols "pointers" to our functions, for example like Initialization:>{new1:=f,new2:=g}. Clearly, re-evaluation of such initialization code can not harm the definitions of f or g, and at the same time our Manipulate-s will become responsive to changes in those definitions.
The first thought is that we could just simply replace function names by new symbols and let Manipulate initialization automatically do the rest. Unfortunately, in that process, it walks the dependency tree, and therefore, the definitions for our functions would also be included - which is what we try to avoid. So, instead, we will explicitly construct the Initialize option. Here is the code:
ClearAll[SavePointers];
SavePointers /:
Manipulate[args___,SavePointers[sd :(SaveDefinitions->True)],
after:OptionsPattern[]] :=
Module[{init},
With[{ptrrules =
Cases[Hold[args],
s_Symbol /; Flatten[{DownValues[s], SubValues[s], UpValues[s]}] =!= {} :>
With[{pointer = unique[Unevaluated[s]]},
pointer := s;
HoldPattern[s] :> pointer],
Infinity, Heads -> True]},
Hold[ptrrules] /.
(Verbatim[HoldPattern][lhs_] :> rhs_ ) :> (rhs := lhs) /.
Hold[defs_] :>
ReleaseHold[
Hold[Manipulate[args, Initialization :> init, after]] /.
ptrrules /. init :> defs]]]
With the same definitions as before:
ClearAll[f, g];
f[x_] := Sin[x];
g[x_] := x^2;
Here is a FullForm of produced Manipulate:
In[454]:=
FullForm[Manipulate[Plot[f[n g[x]],{x,-3,3}],{n,1,4},
SavePointers[SaveDefinitions->True]]]
Out[454]//FullForm=
Manipulate[Plot[f$3653201175165770507872[Times[n,g$3654201175165770608016[x]]],
List[x,-3,3]],List[n,1,4],RuleDelayed[Initialization,
List[SetDelayed[f$3653201175165770507872,f],SetDelayed[g$3654201175165770608016,g]]]]
The newly generated symbols serve as "pointers" to our functions. The Manipulate-s constructed with this approach, will be responsive for updates in our functions, and at the same time harmless for the main functions' definitions. The price to pay is that they are not self-contained and will not display correctly if the main functions are undefined. So, one can use either CloneSymbols wrapper or SavePointers, depending on what is needed.
The answer is to use initialization cell as initialization for the Manipulate:
Manipulate[
Plot[n f[x], {x, -3, 3}], {n, 1, 4},
Initialization :> FrontEndTokenExecute["EvaluateInitialization"]]
You can also use DynamicModule:
DynamicModule[{f},
f[x_] := x^2;
Manipulate[Plot[n f[x], {x, -3, 3}], {n, 1, 4}]]
You do not need SaveDefinitions -> True in this case.
EDIT
In response to Sjoerd's comment. With the following simple technique you do not need to copy the definition everywhere and update all copies if you change the definition (but you still need to re-evaluate your code to get updated Manipulate):
DynamicModule[{f}, f[x_] := x^2;
list = Manipulate[Plot[n^# f[x], {x, -3, 3}], {n, 2, 4}] & /# Range[3]];
list // Row

Fixing Combinatorica redefinition of Element

My code relies on version of Element which works like MemberQ, but when I load Combinatorica, Element gets redefined to work like Part. What is the easiest way to fix this conflict? Specifically, what is the syntax to remove Combinatorica's definition from DownValues? Here's what I get for DownValues[Element]
{HoldPattern[
Combinatorica`Private`a_List \[Element] \
{Combinatorica`Private`index___}] :>
Combinatorica`Private`a[[Combinatorica`Private`index]],
HoldPattern[Private`x_ \[Element] Private`list_List] :>
MemberQ[Private`list, Private`x]}
If your goal is to prevent Combinatorica from installing the definition in the first place, you can achieve this result by loading the package for the first time thus:
Block[{Element}, Needs["Combinatorica`"]]
However, this will almost certainly make any Combinatorica features that depend upon the definition fail (which may or may not be of concern in your particular application).
You can do several things. Let us introduce a convenience function
ClearAll[redef];
SetAttributes[redef, HoldRest];
redef[f_, code_] := (Unprotect[f]; code; Protect[f])
If you are sure about the order of definitions, you can do something like
redef[Element, DownValues[Element] = Rest[DownValues[Element]]]
If you want to delete definitions based on the context, you can do something like this:
redef[Element, DownValues[Element] =
DeleteCases[DownValues[Element],
rule_ /; Cases[rule, x_Symbol /; (StringSplit[Context[x], "`"][[1]] ===
"Combinatorica"), Infinity, Heads -> True] =!= {}]]
You can also use a softer way - reorder definitions rather than delete:
redef[Element, DownValues[Element] = RotateRight[DownValues[Element]]]
There are many other ways of dealing with this problem. Another one (which I already recommended) is to use UpValues, if this is suitable. The last one I want to mention here is to make a kind of custom dynamic scoping construct based on Block, and wrap it around your code. I personally find it the safest variant, in case if you want strictly your definition to apply (because it does not care about the order in which various definitions could have been created - it removes all of them and adds just yours). It is also safer in that outside those places where you want your definitions to apply (by "places" I mean parts of the evaluation stack), other definitions will still apply, so this seems to be the least intrusive way. Here is how it may look:
elementDef[] := Element[x_, list_List] := MemberQ[list, x];
ClearAll[elemExec];
SetAttributes[elemExec, HoldAll];
elemExec[code_] := Block[{Element}, elementDef[]; code];
Example of use:
In[10]:= elemExec[Element[1,{1,2,3}]]
Out[10]= True
Edit:
If you need to automate the use of Block, here is an example package to show one way how this can be done:
BeginPackage["Test`"]
var;
f1;
f2;
Begin["`Private`"];
(* Implementations of your functions *)
var = 1;
f1[x_, y_List] := If[Element[x, y], x^2];
f2[x_, y_List] := If[Element[x, y], x^3];
elementDef[] := Element[x_, list_List] := MemberQ[list, x];
(* The following part of the package is defined at the start and you don't
touch it any more, when adding new functions to the package *)
mainContext = StringReplace[Context[], x__ ~~ "Private`" :> x];
SetAttributes[elemExec, HoldAll];
elemExec[code_] := Block[{Element}, elementDef[]; code];
postprocessDefs[context_String] :=
Map[
ToExpression[#, StandardForm,
Function[sym,DownValues[sym] =
DownValues[sym] /.
Verbatim[RuleDelayed][lhs_,rhs_] :> (lhs :> elemExec[rhs])]] &,
Select[Names[context <> "*"], ToExpression[#, StandardForm, DownValues] =!= {} &]];
postprocessDefs[mainContext];
End[]
EndPackage[]
You can load the package and look at the DownValues for f1 and f2, for example:
In[17]:= DownValues[f1]
Out[17]= {HoldPattern[f1[Test`Private`x_,Test`Private`y_List]]:>
Test`Private`elemExec[If[Test`Private`x\[Element]Test`Private`y,Test`Private`x^2]]}
The same scheme will also work for functions not in the same package. In fact, you could separate
the bottom part (code-processing package) to be a package on its own, import it into any other
package where you want to inject Block into your functions' definitions, and then just call something like postprocessDefs[mainContext], as above. You could make the function which makes definitions inside Block (elementDef here) to be an extra parameter to a generalized version of elemExec, which would make this approach more modular and reusable.
If you want to be more selective about the functions where you want to inject Block, this can also be done in various ways. In fact, the whole Block-injection scheme can be made cleaner then, but it will require slightly more care when implementing each function, while the above approach is completely automatic. I can post the code which will illustrate this, if needed.
One more thing: for the less intrusive nature of this method you pay a price - dynamic scope (Block) is usually harder to control than lexically-scoped constructs. So, you must know exactly the parts of evaluation stack where you want that to apply. For example, I would hesitate to inject Block into a definition of a higher order function, which takes some functions as parameters, since those functions may come from code that assumes other definitions (like for example Combinatorica` functions relying on overloaded Element). This is not a big problem, just requires care.
The bottom line of this seems to be: try to avoid overloading built-ins if at all possible. In this case you faced this definitions clash yourself, but it would be even worse if the one who faces this problem is a user of your package (may be yourself a few months later), who wants to combine your package with another one (which happens to overload same system functions as yours). Of course, it also depends on who will be the users of your package - only yourself or potentially others as well. But in terms of design, and in the long term, you may be better off assuming the latter scenario from the start.
To remove Combinatorica's definition, use Unset or the equivalent form =.. The pattern to unset you can grab from the Information output you show in the question:
Unprotect[Element];
Element[a_List, {index___}] =.
Protect[Element];
The worry would be, of course, that Combinatorica depends internally on this ill-conceived redefinition, but you have reason to believe this to not be the case as the Information output from the redefined Element says:
The use of the function
Element in Combinatorica is now
obsolete, though the function call
Element[a, p] still gives the pth
element of nested list a, where p is a
list of indices.
HTH
I propose an entirely different approach than removing Element from DownValues. Simply use the full name of the Element function.
So, if the original is
System`Element[]
the default is now
Combinatorica`Element[]
because of loading the Combinatorica Package.
Just explicitly use
System`Element[]
wherever you need it. Of course check that System is the correct Context using the Context function:
Context[Element]
This approach ensures several things:
The Combinatorica Package will still work in your notebook, even if the Combinatorica Package is updated in the future
You wont have to redefine the Element function, as some have suggested
You can use the Combinatorica`Element function when needed
The only downside is having to explicitly write it every time.

Programmatically creating multivariate functions in Mathematica

This is a split from discussion on earlier question.
Suppose I need to define a function f which checks if given labeling of a graph is a proper coloring. In other words, we have an integer assigned to every node and no two adjacent nodes get the same answer. For instance, for {"Path",3}, f[{1,2,3}] returns True and f[{1,1,2}] returns False. How would I go about creating such a function for arbitrary graph?
The following does essentially what I need, but generates Part warnings.
g[edges_] := Function ## {{x}, And ## (x[[First[#]]] != x[[Last[#]]] & /# edges)}
f = g[GraphData[{"Path", 3}, "EdgeIndices"]];
f[{1, 2, 1}]==False
This is a toy instance problem I regularly come across -- I need to programmatically create a multivariate function f, and end up with either 1) part warning 2) deferring evaluation of g until evaluation of f
Here's something. When nothing else is working, Hold and rules can usually get the job done. I'm not sure it produces the correct results w.r.t. your graph-coloring question but hopefully gives you a starting place. I ended up using Slot instead of named variable because there were some scoping issues (also present in my previous suggestion, x$ vs. x) when I used a named variable that I didn't spend the time trying to work around.
g[edges_] :=
With[{ors = (Hold ## edges) /. {a_, b_} :> #[[a]] == #[[b]]},
Function[!ors] /. Hold -> Or
]
In[90]:= f = g[GraphData[{"Path", 3}, "EdgeIndices"]]
Out[90]= !(#1[[1]] == #1[[2]] || #1[[2]] == #1[[3]]) &
In[91]:= f[{1, 2, 3}]
Out[91]= True
In[92]:= f[{1, 1, 2}]
Out[92]= False
I feel like it lacks typical Mathematica elegance, but it works. I'll update if I'm inspired with something more beautiful.
There are a couple other solutions to this kind of problem which don't require you to use Hold or ReleaseHold, but instead rely on the fact that Function already has the HoldAll attribute. You first locally "erase" the definitions of Part using Block, so the expression you're interested in can be safely constructed, and then uses With to interpolate that into a Function which can then be safely returned outside of the Block, and also uses the fact that Slot doesn't really mean anything outside of Function.
Using your example:
coloringChecker[edges_List] :=
Block[{Part},
With[{body = And ## Table[#[[First#edge]] != #[[Last#edge]], {edge, edges}]},
body &]]
I don't know if this is all that much less cumbersome than using Hold, but it is different.
I'm still puzzled by the difficulties that you seem to be having. Here's a function which checks that no 2 consecutive elements in a list are identical:
f[l_List] := Length[Split[l]] == Length[l]
No trouble with Part, no error messages for the simple examples I've tried so far including OP's 'test' cases. I also contend that this is more concise and more readable than either of the other approaches seen so far.

Resources