Problem
My web front-end calls back-end queries with complex arguments. During development, to avoid time-consuming manual replication of those arguments, I want to capture their values in Vars, which can be used in REPL.
Research
This article shows that inline def is a suitable solution, but I couldn't make it work. After a call from the front-end happened, the Var remained unbound.
I launched the backend with REPL through VS Code + Calva, with the following code:
(defn get-analytics-by-category [params]
(def params params)
...)
And here's the evaluation of the Var in the REPL:
#object[clojure.lang.Var$Unbound 0x1298f89e "Unbound: #'urbest.db.queries/params"]
Question
Why the code above didn't bound value of the argument to the Var? Is there another solution?
The best way that I found is to use scope-capture library. It captures all the local variables with addition of 1 line in a function, and then with another 1-liner you can define all those variables as global, which allows you to evaluate in REPL any sub-expression in the function using runtime values.
If you ever spent a lot of time reproducing complex runtime values, I strongly recommend watching their 8-min demo.
My issue with inline-def was likely caused by reloading namespace after the Var was bound to a value. After restarting VS Code and carefully doing everything again, the issue went away.
Another way to look at the runtime is to use a debugger.
It is more flexible than scope-capture, but requires a bit more work to make variables available outside an execution.
VS Code's Calva extension comes with one, there's also Emacs packages.
Related
I've got a large Rails 5 app (Ruby 2.6.x at present) that makes crucial use of Kernel::eval (please don't tell me to try to refactor this out because eval is dangerous - I didn't write the original code, and this is not in the cards for any time soon).
There are a very wide variety of Ruby expressions (coming out of the db) that can be passed to eval, sometimes of great complexity, making extensive use of classes and resources of the app.
(you might want to jump straight down to BIG EDIT below)
What I want is to be able to set a global value ($global) that will be seen within the scope of the eval execution, but that will not "infect" any of the execution context outside of that. I can't try to interpolate this into the string and pass it down though method params and such, because, as I say, the code being eval'ed is complex and stacks can get very deep, and I want the value to potentially be accessed (though never modified) anywhere within.
I understand about Bindings. I have played around with setting local and instance vars in a binding, and passing this to eval, but inevitably these are not seen inside any method calls within the eval, especially if I'm inside a method of some random class (which I always am). Seems like global is the only possibility. But experimentation shows that a global set inside an eval remains in the code that calls the eval:
2.6.3 :002 > $foo
=> nil
2.6.3 :003 > eval("$foo = 12")
=> 12
2.6.3 :004 > $foo
=> 12
Although I might find some hacky way to deal with this situation, I'm sure you can see where I'd really rather not.
The Binding class offers methods to set local and instance vars dynamically within a Binding object, but nothing for globals (apparently). I've thought about something like this:
...
eval code_string, get_binding()
...
def get_binding
$global = :special_value
binding
end
but I'm really worried, with a Rails app that might be servicing lots of requests at the same time, that these settings of $global will step on each other in unpredictable ways. Related clarifying question: Is a global value in a Rails app global to the entire thing, readable and writable within the scope of all the requests whose servicing may be overlapping in time? (I'm running under Passenger, if that means anything)
So this is a fairly simple and straightforward problem when you understand it, although oddly not addressed in anything I can google about it, and I think I've written enough words. Thanks for any help or ideas to try.
BIG EDIT:
Ok, let me try to refocus this in a different way. I'm getting that the scope of a global can never, no-how, be constrained (duh, right?), but how about this strategy (similar to above):
...
eval code_string, get_binding()
...
def get_binding
luaapg = :special_value ## local used as a pseudo-global
binding
end
So, now I've got this Binding that includes the local var luaapg. I've confirmed that. I eval code_string with this Binding. When I am somewhere inside the execution of code_string, where do I find luaapg - how do I access it? If you look at pretty much every tutorial on this stuff on the web, they show you puts eval("luaapg", get_binding) and voila, the assigned value comes out! But this is too simplistic for real life. When I am in the middle of my code_string, in some method scope of some class, luaapg is not there. I had great hope that this would work, even deep down the stack:
TOPLEVEL_BINDING.local_variable_get(:luaapg)
but it doesn't (I learned about TOPLEVEL_BINDING from here - thanks to that author). So this is the new question: what does it mean to say that I have executed (eval'ed) my code_string in the context of that Binding, which contains a local variable, if I have no way to access that variable, other than with the most simpleminded code? (incidentally I played around with instance vars too - same thing). I'm still hoping there's some magic incantation...
I think you've put your finger on it in the name of the type of variable - it's global - common to all the code in the executing program. I'm not sure exactly how Passenger works but I suspect it runs several copies of your program, so it won't be common between the copies.
To get reliable shared information I think you're going to have to use your database or some sort of information cache like memcached. You choose how you save/name it there.
I'm wondering what benefit discriminating between local and global variables provides. It seems to me that if everything were made a global variable, there would be a lot less confusion.
Wouldn't declaring everything a global variable result in fewer errors because one wouldn't mistakenly call a local variable in a global instance, thereby encountering fewer errors?
Where is my logic wrong on this?
Some of this boils down to good coding practices. Keeping variables local also means it becomes simpler to share code from one application to another without having to worry about code conflicts. While its simpler to make everything global, getting into the habit of only using global variables when you actually have to will force you to code more efficiently and will make your code more structured.
I think your key oversight is thinking that an error telling you a local variable doesn't exist is a bad thing - it isn't. You've made a mistake and ruby is telling you so. This type of mistake is usually easy to fix: you've misspelled something or you're using something that you forgot to create.
Global variables everywhere might remove those errors but they would replace them with a far harder set of errors to reason about: accidentally using a variable that another bit of code is using. Imagine if every time you called a function (one of your own or a standard library one or one from a gem) you had to check which global variables it might change (and which functions it called, since it might also change global variables) If you make a mistake then you might get an error message (if the class of the object in the variable changes enough) but often you would just silently get incorrect results (if the value of a variable you were using changes unexpectedly).
In general global variables are much harder to work with and people avoid them when possible.
If all variables are global, every line of code in every program (including those which haven't been written yet) written by every programmer on the planet (including those who haven't been born yet or are already dead) must universally, uniquely agree on the names of variables. If you use a variable name that someone else on a different continent two years from now will also use, both of your programs will break, when used together.
Can someone provide a better explanation of the xdmp:eval() and xdmp:value() functions?
I had tried to follow the Developer API. However, I am not really satisfied with the instances and it's a bit vague for me. I would really appreciate if someone could help me understand those functions and their differences with examples.
Both functions are for executing strings of code dynamically, but xdmp:value is evaluated against the current context, such that if you have variables defined in the current scope or modules declared, you can reference them without redeclaring them.
xdmp:eval necessitates the creation of an entirely new context that has no knowledge of the context calling xdmp:eval. One must define a new XQuery prolog, and variables from the main context are passed to the xdmp:eval call as parameters and declared as external variables in the eval script.
Generally, if you can use xdmp:value, it's probably the best choice; however, xdmp:eval has some capabilities that xdmp:value doesn't, namely everything defined in the <options> argument. Through these options, it's possible to control the user executing the query, the database it's executed against, transaction mode, etc.
There is another function for executing dynamic strings: xdmp:unpath, and it's similar to xdmp:value, but more limited in that it can only execute XPath.
As a continuation of this question and the subsequent answer, does anyone know how to have a job created using the Parallel Computing Toolbox (using createJob and createTask) access external toolboxes? Is there a configuration parameter I can specify when creating the function to specify toolboxes that should be loaded?
According to this section of the documentation, one way you can do this is to specify either the 'PathDependencies' property or the 'FileDependencies' property of the job object so that it points to the functions you need the job's workers to be able to use.
You should be able to point the way to the KbCheck function in PsychToolbox, along with any other functions or directories needed for KbCheck to work properly. It would look something like this:
obj = createJob('PathDependencies',{'path_to_KbCheck',...
'path_to_other_PTB_functions'});
A few comments, based on my work troubleshooting this:
It appears that there are inconsistencies with how well nested functions and anonymous functions work with the Parallel Computation toolkit. I was unable to get them to work, while others have been able to. (Also see here.) As such, I would recommend having each function stored in it's own file, and including those files using the PathDependencies or FileDependencies properties, as described by gnovice above.
It is very hard to troubleshoot the Parallel Computation toolkit, as everything happens outside your view. Use breakpoints liberally in your code, and the inspect command is your friend. Also note that if there is an error, task objects will contain an error parameter, which in turn will contain ErrorMessage string, and possibly the Error.causes MException object. Both of these were immensely useful in debugging.
When including Psychtoolbox, you need to do it as follows. First, create a jobStartup.m file with the following lines:
PTB_path = '/Users/eliezerk/Documents/MATLAB/Psychtoolbox3/';
addpath( PTB_path );
cd( PTB_path );
SetupPsychtoolbox;
However, since the Parallel Computation toolkit can't handle any graphics functionality, running SetupPsychtoolbox as-is will actually cause your thread to crash. To avoid this, you need to edit the PsychtoolboxPostInstallRoutine function, which is called at the very end of SetupPsychtoolbox. Specifically, you want to comment out the line AssertOpenGL (line 496, as of the time of this answer; this may change in future releases).
Peter Norvig in PAIP says:
in modern lisps...eval is used less often (in fact, in Scheme there is
no eval at all). If you find yourself using eval, you are probably
doing the wrong thing
What are some of the ways to circumvent using eval in scheme? Arent there case where eval is absolutely necessary?
There are cases where eval is necessary, but they always involve advanced programs that do things like dynamically loading some code (eg, a servlet in a web server). As for a way to "circumvent" using it -- that depends on the actual problem you're trying to solve, there's no magic solution to avoiding eval except for ... eval.
(BTW, my guess is that PAIP was written a long time ago, before eval was added to the Scheme Report.)
He's wrong. Of course there is eval in Scheme.
You'll need eval in some very rare case. The case that comes into mind first is when you'll build program with a program and then execute it. This happens mainly with genetic algorithm for example. In this case you build a lot a randomized programs that you'll need to execute. Having eval in conjunction with code being data make lisp the easiest programming language to do genetic algorithm.
Having these properties comes at a great cost (in term of speed and size of your program) because you'll remove all possibility to do compile time optimization on the code that will be evaled and you must keep the full interpreter in your resulting binary.
As a result it is considered poor design to use eval when it can be avoided.
The claim that Scheme has no eval is inaccurate at least for the most recent versions of the Scheme standard (R5RS and later). Usually, what you want is a macro instead, which will generate code at compilation time.
It is true that eval should be avoided. For starters, I've never seen a satisfactory definition of how should it behave, for example:
What environment expressions should be evaluated in when no environment is passed?
When you do pass in an environment, how do those work? For example, the standards specify no way you can pre-bind a value in that environment object.
That said, I've worked with a Scheme application that uses eval to generate code dynamically at runtime for cases where the structure of the computation cannot be known at compilation time. The intent has been to get the Scheme system to compile the code at runtime for performance reasons—and the difficulty is that there is no standard way to tell a Scheme system "compile this code."
It should go without saying also that eval can be a huge security risk. You should never eval anything that doesn't have a huge wall of separation from user input. Basically, if you want to use eval safely, you should be doing so in the context of the code-generation phase of a compiler-like system, after you've parsed some input (using a comprehensively defined grammar!).
First, PAIP is written for Common Lisp, not Scheme, so I don't know that he'd say the same thing. CL macros do much the same thing as eval, although at compile time instead of run time, and there's other things you could do. If you'd show me an example of using eval in Common Lisp, I could try to come up with other methods of doing the same thing.
I'm not a Scheme programmer. I can only speak from Norvig's perspective, as a Common Lisp programmer. I don't think he was talking about Scheme, and I don't know if he knew or knows Scheme particularly well.
Second, Norvig says "you are probably doing the wrong thing" rather than "you're doing the wrong thing". This implies that, for all he knows, there's times when eval is the correct thing to use. In a language like C, I'd say the same thing about goto, although they're quite useful in some restricted circumstances, but most goto use is by people who don't know any better.
One use I've seen for 'eval' in scripting environments is to parameterize some code with runtime values. for instance, in psuedo-C:
param = read_integer();
fn = eval("int lambda(int x) {
int param = " + to_string(param) + ";
return param*x; }");
I hope you find that really ugly. String pasting to create code at runtime? Ick. In Scheme and other lexically scoped Lisps, you can make parameterized functions without using eval.
(define make-my-fn
(lambda (param)
(lambda (x) (* param x)))
(let* ([ param (read-integer) ]
[ fn (make-my-fn param ])
;; etc.
)
Like was mentioned, dynamic code loading and such still need eval, but parameterized code and code composition can be produced with first class functions.
You could write a scheme interpreter in scheme. Such is certainly possible, but it is not practical.
Granted, this is a general answer, as I have no used scheme, but it may help you nonetheless. :)