Forcing a package's function to use user-provided function - debugging
I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions
Namespaces are sealed once they are loaded. Sealing means that imports
and exports cannot be changed and that internal variable bindings
cannot be changed. Sealing allows a simpler implementation strategy
for the namespace mechanism. Sealing also allows code analysis and
compilation tools to accurately identify the definition corresponding
to a global variable reference in a function body.
The namespace controls the search strategy for variables used by
functions in the package. If not found locally, R searches the package
namespace first, then the imports, then the base namespace and then
the normal search path.
Related
How to debug with PureScript?
Issue Following is a minimal, contrived example: read :: FilePath -> Aff String read f = do log ("File: " <> f) -- (1) readTextFile UTF8 f -- (2) I would like to do some debug logging in (1), before a potential error on (2) occurs. Executing following code in Spago REPL works for success cases so far: $ spago repl > launchAff_ $ read "test/data/tree/root.txt" File: test/data/tree/root.txt unit Problem: If there is an error with (2) - file is directory here - , (1) seems to be not executed at all: $ spago repl > launchAff_ $ read "test/data/tree" ~/purescript-book/exercises/chapter9/.psci_modules/node_modules/Effect.Aff/foreign.js:532 throw util.fromLeft(step); ^ [Error: EISDIR: illegal operation on a directory, read] { errno: -21, code: 'EISDIR', syscall: 'read' } The original problem is more complex including several layers of recursions (see E-Book exercise 3), where I need logging to debug above error. Questions How can I properly log regardless upcoming errors here? (Optional) Is there a more sophisticated, well-established debugging alternative - purescript-debugger? A decicated VS Code debug extension/functionality would be the cherry on the cake.
First of all, the symptoms you observe do not mean that the first line doesn't execute. It does always execute, you're just not seeing output from it due to how console works in the PureScript REPL. The output gets swallowed. Not the only problem with REPL, sadly. You can verify that the first line is always executed by replacing log with throwError and observing that the error always gets thrown. Or, alternatively, you can make the first line modify a mutable cell instead of writing to the console, and then examine the cell's contents. Finally, this only happens in REPL. If you put that launchAff_ call inside main and run the program, you will always get the console output. Now to the actual question at hand: how to debug trace. Logging to console is fine if you can afford it, but there is a more elegant way: Debug.trace. This function has a hidden effect - i.e. its type says it's pure, but it really produces an effect when called. This little lie lets you use trace in a pure setting and thus debug pure code. No need for Effect! This is ok as long as used for debugging only, but don't put it in production code. The way it works is that it takes two parameters: the first one gets printed to console and the second one is a function to be called after printing, and the result of the whole thing is whatever that function returns. For example: calculateSomething :: Int -> Int -> Int calculateSomething x y = trace ("x = " <> show x) \_ -> x + y main :: Effect Unit main = log $ show $ calculateSomething 37 5 > npx spago run 'x = 37' 42 The first parameter can be anything at all, not just a string. This lets you easily print a lot of stuff: calculateSomething :: Int -> Int -> Int calculateSomething x y = trace { x, y } \_ -> x + y > npx spago run { x: 37, y: 5 } 42 Or, applying this to your code: read :: FilePath -> Aff String read f = trace ("File: " <> f) \_ -> do readTextFile UTF8 f But here's a subtle detail: this tracing happens as soon as you call read, even if the resulting Aff will never be actually executed. If you need tracing to happen on effectful execution, you'll need to make the trace call part of the action, and be careful not to make it the very first action in the sequence: read :: FilePath -> Aff String read f = do pure unit trace ("File: " <> f) \_ -> pure unit readTextFile UTF8 f It is, of course, a bit inconvenient to do this every time you need to trace in an effectful context, so there is a special function that does it for you - it's called traceM: read :: FilePath -> Aff String read f = do traceM ("File: " <> f) readTextFile UTF8 f If you look at its source code, you'll see that it does exactly what I did in the example above. The sad part is that trace won't help you in REPL when an exception happens, because it's still printing to console, so it'll still get swallowed for the same reasons. But even when it doesn't get swallowed, the output is a bit garbled, because trace actually outputs in color (to help you make it out among other output), and PureScript REPL has a complicated relationship with color: > calculateSomething 37 5 ←[32m'x = 37'←[39m 42
In addition to Fyodor Soikin's great answer, I found a variant using VS Code debug view. 1.) Make sure to build with sourcemaps: spago build --purs-args "-g sourcemaps" 2.) Add debug configuration to VS Code launch.json: { "version": "0.2.0", "configurations": [ { "type": "pwa-node", "request": "launch", "name": "Launch Program", "skipFiles": ["<node_internals>/**"], "runtimeArgs": ["-e", "require('./output/Main/index.js').main()"], "smartStep": true // skips files without (valid) source map } ] } Replace "./output/Main/index.js" / .main() with the compiled .js file / function to be debugged. 3.) Set break points and step through the .purs file via sourcemap support.
Unable to Call Function in Go debugger
I am following the "Little Go Book" by Karl Seguin, in order to learn Go. My working environment is Visual Studio Code. Upon debugging, when I try to call a function from the debug console, i get the following error: "function calls not allowed without using 'call'", if I try using "call fib(10)", i get "Unable to eval expression: "1:6: expected 'EOF', found fib". This is the function I am trying to evaluate: //Fibonnaci func fib(n int) int64 { if n == 0 { return 0 } else if n == 1 { return 1 } else { return fib(n-1) + fib(n-2) } } If i try to call the function from the code itself ( from the main() for instance, it works perfectly). However, if I set a breakpoint and try to call the same function from the debugger console, I get the below error: Eval error: function calls not allowed without using 'call' call fib(10) Unable to eval expression: "1:6: expected 'EOF', found fib" Failed to eval expression: { "Expr": "call fib(10)", "Scope": { "goroutineID": 1, "frame": 0 }, "Cfg": { "followPointers": true, "maxVariableRecurse": 1, "maxStringLen": 64, "maxArrayValues": 64, "maxStructFields": -1 } }
Looks like "Function calls via delve 'call' are not supported" yet github issue in microsoft/vscode-go repo :(
The issue vscode-go issue 100 "debug: support function calls via delve 'call'" just got closed with PR 101 and commit 5a7752c / CL 249377 Delve supports function calls. Even though it is still experimental and can be applied only to a limited set of functions, this is a useful feature, many vscode-go users long for. Unlike other javascript/typescript debuggers, delve treats function calls specially and requires different call paths than usual expression evaluation. That is because Go is a compiled, runtime-managed GC language, calling a function safely from debugger is complex. DAP and VS Code UI does not distinguish function calls and other expression evaluation either, so we have to implement this in the same evaluateRequest context. We use a heuristic to guess which route (call or expression evaluation) we need to take based on evaluateRequest's request. This is part of the 0.17.0 milestone, yet to be released, and available for now in the nightly build.
VIF values in R
I got a question: Someone have run the corvif function with the code HighstatLibV10.R available in the page http://www.highstat.com/index.php/mixed-effects-models-and-extensions-in-ecology-with-r? I can't get the VIF values because the output gives me this error: Error in myvif(lm_mod) : object 'tmp_cor' not found! I have 6 physical variables and I'm looking for collinearity among variables. Any help more than welcome!
If working with the corvif() is not of utmost importance you can use the vif() in the R package 'car' to get VIF values for your linear models.
So tmp_cor is an object that is supposed to be created in corvif tmp_cor is created using the cor function (in the base stats package that comes with R install) via: tmp_cor <- cor(dataz,use="complete.obs"). However, I noticed that with both v1 and v10 of Zurr et al's HighstatLib.R code this error occurs: Error in myvif(lm_mod) : object 'tmp_cor' not found! First I checked V10: It seems that the "final" version of corvif created when sourcing HighstatLibV10.R actually neglects to create tmp_cor at all! > print(corvif) function(dataz) { dataz <- as.data.frame(dataz) #vif part form <- formula(paste("fooy ~ ",paste(strsplit(names(dataz)," "),collapse=" + "))) dataz <- data.frame(fooy=1 + rnorm(nrow(dataz)) ,dataz) lm_mod <- lm(form,dataz) cat("\n\nVariance inflation factors\n\n") print(myvif(lm_mod)) } But, I noticed that the error in the OP's post also occurred when using V1 (i.e., HighstatLib.R associated with Zuur et al 2010). Although the code file creates 2 versions of corvif, they (and especially the latter of the two which would supercede the first) include a line to create tmp_cor: corvif <- function(dataz) { dataz <- as.data.frame(dataz) #correlation part cat("Correlations of the variables\n\n") tmp_cor <- cor(dataz,use="complete.obs") print(tmp_cor) #vif part form <- formula(paste("fooy ~ ",paste(strsplit(names(dataz)," "),collapse=" + "))) dataz <- data.frame(fooy=1,dataz) lm_mod <- lm(form,dataz) cat("\n\nVariance inflation factors\n\n") print(myvif(lm_mod)) } So even though the code for corvif creates tmp_cor in the V1 code file, it appears that the helper function myvif (which actually uses the tmp_cor object) is not accessing it. This suggests that we have a scoping problem... Sure enough, if I just quickly change the tmp_cor line to create a global object, the code works fine: tmp_cor <<- cor(dataz,use="complete.obs") Specifically: corvif <- function(dataz) { dataz <- as.data.frame(dataz) #correlation part cat("Correlations of the variables\n\n") tmp_cor <<- cor(dataz,use="complete.obs") print(tmp_cor) #vif part form <- formula(paste("fooy ~ ",paste(strsplit(names(dataz)," "),collapse=" + "))) dataz <- data.frame(fooy=1,dataz) lm_mod <- lm(form,dataz) cat("\n\nVariance inflation factors\n\n") print(myvif(lm_mod)) } A more complete "fix" could be done by manipulating environments.
Docpad: confused about extending template data
I'm totally confused about adding mongo data to template data. I haven't even started trying to get the data from a database as I can't get my templates to see test data (see below). This is in docpad.coffee for the moment, but ultimately g will be the output of mongoDB. events: extendTemplateData: (opts) -> # {templateData} = opts getGigsData: -> g = { "date" : "3-4-2013", "location" : "Gent" } return g opts.templateData["getGigsData"] = getGigsData And I hope to access it with <%= #getGigsData().date %> Thanks so much for some guidance I should add that this design is based on wanting to make it easy for the band to add gigs, without letting them edit the page content itself as I fear they would mess up the markup - if there are other ways to achieve this goal, I'd be pleased to hear.
Tried this locally. And hit the issue: debug: Emitting the event: extendTemplateData → [2014-02-14 01:38:50.030] [/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/out/lib/docpad.js:1184] [DocPad.emitSerial] error: Something went wrong with the action → [2014-02-14 01:38:50.037] [/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/out/lib/interfaces/console.js:107] [ConsoleInterface.destroyWithError] error: An error occured: ReferenceError: getGigsData is not defined at Object.docpadConfig.events.extendTemplateData (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/docpad.coffee:42:44) at ambi (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/ambi/out/lib/ambi.js:25:27) at DocPad.<anonymous> (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/out/lib/docpad.js:995:25) at ambi (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/ambi/out/lib/ambi.js:23:18) at Task.<anonymous> (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/event-emitter-grouped/out/lib/event-emitter-grouped.js:45:23) at ambi (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/ambi/out/lib/ambi.js:23:18) at fire (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/taskgroup/out/lib/taskgroup.js:163:25) at b (domain.js:183:18) at Domain.run (domain.js:123:23) at Task.fire (/Users/balupton/Projects/docpad-extras/skeletons/so-21747504/node_modules/docpad/node_modules/taskgroup/out/lib/taskgroup.js:173:25) at processImmediate [as _immediateCallback] (timers.js:330:15) Which indicates that the error is actually inside our event handler, rather than inside our code. That for some reason getGigsData is not being set, despite our: getGigsData: -> g = { "date" : "3-4-2013", "location" : "Gent" } return g Examining the code, as a CoffeeScript user, I found the issue. As a non-coffeescript user, you can use the coffeescript compiler on the coffeescript website http://coffeescript.org to see the compiled javascript, which is: ({ events: { extendTemplateData: function(opts) { ({ getGigsData: function() { var g; g = { "date": "3-4-2013", "location": "Gent" }; return g; } }); return opts.templateData["getGigsData"] = getGigsData; } } }); As we can see that is definitely not what we expected. We are just defining getGigsData inside an object, then doing nothing with it. The issue is that we used a colon instead of an equals sign, so getGigsData: -> instead of getGigsData = ->. This is not a coffeescript thing, but you would have run into the same issue if this was javascript too, albeit javascript may be a bit more obvious due to the necessary squiggly braces around object definitions. As a sidenote, if you prefer to use JavaScript with DocPad for whatever reason, that is totally supported. You could use a docpad.json or docpad.js file for your docpad configuration file. Another option, is to continue using CoffeeScript then just wrap JavaScript code within the backtick, see: http://coffeescript.org/#embedded
Convert markdown italics and boldface to latex
I want to be able to convert markdown italics and boldface to latex versions on the fly (i.e., give a text string(s) return a text string(s)). I thought easy. Wrong! (Which it still may be). See the sill buisness and error I tried at the bottom. What I have (note the starting asterisk that's been escaped as in markdown): x <- "\\*note: I *like* chocolate **milk** too ***much***!" What I would like: "*note: I \\emph{like} chocolate \\textbf{milk} too \\textbf{\\emph{much}}!" I'm not attached to regex but would prefer a base solution (though not essential). Silly business: helper <- function(ins, outs, x) { gsub(paste0(ins[1], ".+?", ins[2]), paste0(outs[1], ".+?", outs[2]), x) } helper(rep("***", 2), c("\\textbf{\\emph{", "}}"), x) Error in gsub(paste0(ins[1], ".+?", ins[2]), paste0(outs[1], ".+?", outs[2]), : invalid regular expression '***.+?***', reason 'Invalid use of repetition operators' I have this toy that Ananda Mahto helped me make if it's helpful. You could access it from reports via wheresPandoc <- reports:::wheresPandoc EDIT Per Ben's comments I tried: action <- paste0(" echo ", x, " | ", wheresPandoc(), " -t latex ") system(action) *note: I *like* chocolate **milk** too ***much***! | C:\PROGRA~2\Pandoc\bin\pandoc.exe -t latex EDIT2 Per Dason's comments I tried: out <- paste("echo", shQuote(x), "|", wheresPandoc(), " -t latex"); system(out) system(out, intern = T) > system(out, intern = T) \*note: I *like* chocolate **milk** too ***much***! | C:\PROGRA~2\Pandoc\bin\pandoc.exe -t latex
The lack of pipes on Windows made this tricky, but you can get around it using input to provide the stdin: > x = system("pandoc -t latex", intern=TRUE, input="\\*note: I *like* chocolate **milk** too ***much***!") > x [1] "*note: I \\emph{like} chocolate \\textbf{milk} too \\textbf{\\emph{much}}!"
Noting I am working on windows and from ?system This means that redirection, pipes, DOS internal commands, ... cannot be used and the note from ?system2 Note system2 is a more portable and flexible interface than system, introduced in R 2.12.0. It allows redirection of output without needing to invoke a shell on Windows, a portable way to set environment variables for the execution of command, and finer control over the redirection of stdout and stderr. Conversely, system (and shell on Windows) allows the invocation of arbitrary command lines. Using system2 system2('pandoc', '-t latex', input = '**em**', stdout = TRUE)