My expectation would be for debug-on-error to trap most anomalous events and show me what went wrong. However, this is not the case with json.el which uses signal instead of error.
The documentation cautions against using debug-on-signal and my brief experimentation with it pretty much confirms this -- there are signals all over the place, so I would have to trap on just the json-* signals to get anything useful out of this. But I can't get even that to work.
I was about to submit a bug report for json.el but I feel that I may be missing something. Is there a good, established reason why json.el prefers signal over error?
Steps to repro:
(require 'json)
(setq debug-on-error t
debug-on-signal t) ; "json" ; '("json") ;; Still nothing
(json-read-from-string "")
Somewhat more realistically, you would have an actual, real-life problem when you pass the output from something to json-read (or one of its convenience wrappers json-read-from-string, json-read-file, json-read-object, etc) and it doesn't contain what you expect.
The result, even with debug-on-signal set to t, is the following text in the *Messages* buffer, and not much else:
with no indication as to the source of the error, or anything. (In real code with a similar problem, I was seeing if: End of file during parsing .)
In fact, the default value of debug-ignore-errors includes end-of-file so you have to remove that to be able to debug this particular problem.
(delete 'end-of-file debug-ignored-errors)
or perhaps more realistically
(let (debug-ignored-errors)
(json-something))
to temporarily override the value of debug-ignored-errors for the duration of the json-something call.
... and then you don't need debug-on-signal at all.
So my speculation that the use of signal instead of error causes the problematic behavior turned out to be false.
How can I debug font lock keywords I write? For example
(progn
(font-lock-add-keywords
nil
'(
;; ("hi1" . 'success)
("hi2" . (intern (concat "warn" "ing")))
))
(font-lock-fontify-buffer))
will produce the following message in Messages buffer:
Error during redisplay: (jit-lock-function 1) signaled (wrong-type-argument stringp 22)
In order to see the call stack upon wrong-type-argument error, I invoke toggle-debug-on-error and Emacs still doesn't enter debugger upon font lock error.
font-lock
can be applied in different ways. By default it is applied using
jit-lock-mode, which applies it "lazily" (aka "just-in-time"), which has
the disadvantage that it is applied at a time where we can't display
message (or enter the debugger) because that would make us inf-loop. So
there are two ways to work around the problem:
Use jit-lock-debug-mode (recently added to Emacs's development
code).
Set font-lock-support-mode to nil, then turn font-lock off and then
back on.
Both options should change font-lock so that it is applied at a time
where debugging can be used.
I have a personal elisp function that performs an multi-directory grep. It uses compilation-start, which creates a compilation-mode buffer with the results, in which I can press RET (bound to compile-goto-error) to jump to the corresponding location.
However, compile-goto-error always visits the location in another window, leaving the compilation buffer up. Half the time I am just searching for one particular location, so what I would like to do is bind some other key (say C-RET) to also visit the corresponding location in a buffer, but stay in the current window, replacing the compilation buffer with the location's buffer.
I've traced the relevant execution from compile-goto-error to next-error-internal to next-error-function to compilation-next-error-function to compilation-find-file, but can't find a nice place to hook in my differing behavior. Is there a simple way (or, failing that, a complicated one) to create a compile-goto-error variant that switches to the new buffer in-place in the window that held the compilation buffer?
I suggest just advising the compile-goto-error function to do the right thing, rather than creating a custom function and keybinding. The following does the right thing for me (following Stefan's suggestion):
(defadvice compile-goto-error (around my-compile-goto-error activate)
(let ((display-buffer-overriding-action '(display-buffer-reuse-window (inhibit-same-window . nil))))
ad-do-it))
I think you should be able to get what you want by let-binding display-buffer-overriding-action, something like:
(defun my-next-error ()
(interactive)
(let ((display-buffer-overriding-action '(display-buffer-same-window)))
(next-error)))
This question comes from the recent question "Correct way to cap Mathematica memory use?"
I wonder, is it possible to programmatically restart MathKernel keeping the current FrontEnd process connected to new MathKernel process and evaluating some code in new MathKernel session? I mean a "transparent" restart which allows a user to continue working with the FrontEnd while having new fresh MathKernel process with some code from the previous kernel evaluated/evaluating in it?
The motivation for the question is to have a way to automatize restarting of MathKernel when it takes too much memory without breaking the computation. In other words, the computation should be automatically continued in new MathKernel process without interaction with the user (but keeping the ability for user to interact with the Mathematica as it was originally). The details on what code should be evaluated in new kernel are of course specific for each computational task. I am looking for a general solution how to automatically continue the computation.
From a comment by Arnoud Buzing yesterday, on Stack Exchange Mathematica chat, quoting entirely:
In a notebook, if you have multiple cells you can put Quit in a cell by itself and set this option:
SetOptions[$FrontEnd, "ClearEvaluationQueueOnKernelQuit" -> False]
Then if you have a cell above it and below it and select all three and evaluate, the kernel will Quit but the frontend evaluation queue will continue (and restart the kernel for the last cell).
-- Arnoud Buzing
The following approach runs one kernel to open a front-end with its own kernel, which is then closed and reopened, renewing the second kernel.
This file is the MathKernel input, C:\Temp\test4.m
Needs["JLink`"];
$FrontEndLaunchCommand="Mathematica.exe";
UseFrontEnd[
nb = NotebookOpen["C:\\Temp\\run.nb"];
SelectionMove[nb, Next, Cell];
SelectionEvaluate[nb];
];
Pause[8];
CloseFrontEnd[];
Pause[1];
UseFrontEnd[
nb = NotebookOpen["C:\\Temp\\run.nb"];
Do[SelectionMove[nb, Next, Cell],{12}];
SelectionEvaluate[nb];
];
Pause[8];
CloseFrontEnd[];
Print["Completed"]
The demo notebook, C:\Temp\run.nb contains two cells:
x1 = 0;
Module[{},
While[x1 < 1000000,
If[Mod[x1, 100000] == 0, Print["x1=" <> ToString[x1]]]; x1++];
NotebookSave[EvaluationNotebook[]];
NotebookClose[EvaluationNotebook[]]]
Print[x1]
x1 = 0;
Module[{},
While[x1 < 1000000,
If[Mod[x1, 100000] == 0, Print["x1=" <> ToString[x1]]]; x1++];
NotebookSave[EvaluationNotebook[]];
NotebookClose[EvaluationNotebook[]]]
The initial kernel opens a front-end and runs the first cell, then it quits the front-end, reopens it and runs the second cell.
The whole thing can be run either by pasting (in one go) the MathKernel input into a kernel session, or it can be run from a batch file, e.g. C:\Temp\RunTest2.bat
#echo off
setlocal
PATH = C:\Program Files\Wolfram Research\Mathematica\8.0\;%PATH%
echo Launching MathKernel %TIME%
start MathKernel -noprompt -initfile "C:\Temp\test4.m"
ping localhost -n 30 > nul
echo Terminating MathKernel %TIME%
taskkill /F /FI "IMAGENAME eq MathKernel.exe" > nul
endlocal
It's a little elaborate to set up, and in its current form it depends on knowing how long to wait before closing and restarting the second kernel.
Perhaps the parallel computation machinery could be used for this? Here is a crude set-up that illustrates the idea:
Needs["SubKernels`LocalKernels`"]
doSomeWork[input_] := {$KernelID, Length[input], RandomReal[]}
getTheJobDone[] :=
Module[{subkernel, initsub, resultSoFar = {}}
, initsub[] :=
( subkernel = LaunchKernels[LocalMachine[1]]
; DistributeDefinitions["Global`"]
)
; initsub[]
; While[Length[resultSoFar] < 1000
, DistributeDefinitions[resultSoFar]
; Quiet[ParallelEvaluate[doSomeWork[resultSoFar], subkernel]] /.
{ $Failed :> (Print#"Ouch!"; initsub[])
, r_ :> AppendTo[resultSoFar, r]
}
]
; CloseKernels[subkernel]
; resultSoFar
]
This is an over-elaborate setup to generate a list of 1,000 triples of numbers. getTheJobDone runs a loop that continues until the result list contains the desired number of elements. Each iteration of the loop is evaluated in a subkernel. If the subkernel evaluation fails, the subkernel is relaunched. Otherwise, its return value is added to the result list.
To try this out, evaluate:
getTheJobDone[]
To demonstrate the recovery mechanism, open the Parallel Kernel Status window and kill the subkernel from time-to-time. getTheJobDone will feel the pain and print Ouch! whenever the subkernel dies. However, the overall job continues and the final result is returned.
The error-handling here is very crude and would likely need to be bolstered in a real application. Also, I have not investigated whether really serious error conditions in the subkernels (like running out of memory) would have an adverse effect on the main kernel. If so, then perhaps subkernels could kill themselves if MemoryInUse[] exceeded a predetermined threshold.
Update - Isolating the Main Kernel From Subkernel Crashes
While playing around with this framework, I discovered that any use of shared variables between the main kernel and subkernel rendered Mathematica unstable should the subkernel crash. This includes the use of DistributeDefinitions[resultSoFar] as shown above, and also explicit shared variables using SetSharedVariable.
To work around this problem, I transmitted the resultSoFar through a file. This eliminated the synchronization between the two kernels with the net result that the main kernel remained blissfully unaware of a subkernel crash. It also had the nice side-effect of retaining the intermediate results in the event of a main kernel crash as well. Of course, it also makes the subkernel calls quite a bit slower. But that might not be a problem if each call to the subkernel performs a significant amount of work.
Here are the revised definitions:
Needs["SubKernels`LocalKernels`"]
doSomeWork[] := {$KernelID, Length[Get[$resultFile]], RandomReal[]}
$resultFile = "/some/place/results.dat";
getTheJobDone[] :=
Module[{subkernel, initsub, resultSoFar = {}}
, initsub[] :=
( subkernel = LaunchKernels[LocalMachine[1]]
; DistributeDefinitions["Global`"]
)
; initsub[]
; While[Length[resultSoFar] < 1000
, Put[resultSoFar, $resultFile]
; Quiet[ParallelEvaluate[doSomeWork[], subkernel]] /.
{ $Failed :> (Print#"Ouch!"; CloseKernels[subkernel]; initsub[])
, r_ :> AppendTo[resultSoFar, r]
}
]
; CloseKernels[subkernel]
; resultSoFar
]
I have a similar requirement when I run a CUDAFunction for a long loop and CUDALink ran out of memory (similar here: https://mathematica.stackexchange.com/questions/31412/cudalink-ran-out-of-available-memory). There's no improvement on the memory leak even with the latest Mathematica 10.4 version. I figure out a workaround here and hope that you may find it's useful. The idea is that you use a bash script to call a Mathematica program (run in batch mode) multiple times with passing parameters from the bash script. Here is the detail instruction and demo (This is for Window OS):
To use bash-script in Win_OS you need to install cygwin (https://cygwin.com/install.html).
Convert your mathematica notebook to package (.m) to be able to use in script mode. If you save your notebook using "Save as.." all the command will be converted to comments (this was noted by Wolfram Research), so it's better that you create a package (File->New-Package), then copy and paste your commands to that.
Write the bash script using Vi editor (instead of Notepad or gedit for window) to avoid the problem of "\r" (http://www.linuxquestions.org/questions/programming-9/shell-scripts-in-windows-cygwin-607659/).
Here is a demo of the test.m file
str=$CommandLine;
len=Length[str];
Do[
If[str[[i]]=="-start",
start=ToExpression[str[[i+1]]];
Pause[start];
Print["Done in ",start," second"];
];
,{i,2,len-1}];
This mathematica code read the parameter from a commandline and use it for calculation.
Here is the bash script (script.sh) to run test.m many times with different parameters.
#c:\cygwin64\bin\bash
for ((i=2;i<10;i+=2))
do
math -script test.m -start $i
done
In the cygwin terminal type "chmod a+x script.sh" to enable the script then you can run it by typing "./script.sh".
You can programmatically terminate the kernel using Exit[]. The front end (notebook) will automatically start a new kernel when you next try to evaluate an expression.
Preserving "some code from the previous kernel" is going to be more difficult. You have to decide what you want to preserve. If you think you want to preserve everything, then there's no point in restarting the kernel. If you know what definitions you want to save, you can use DumpSave to write them to a file before terminating the kernel, and then use << to load that file into the new kernel.
On the other hand, if you know what definitions are taking up too much memory, you can use Unset, Clear, ClearAll, or Remove to remove those definitions. You can also set $HistoryLength to something smaller than Infinity (the default) if that's where your memory is going.
Sounds like a job for CleanSlate.
<< Utilities`CleanSlate`;
CleanSlate[]
From: http://library.wolfram.com/infocenter/TechNotes/4718/
"CleanSlate, tries to do everything possible to return the kernel to the state it was in when the CleanSlate.m package was initially loaded."
Some sample, straight forward gtk2 within-main-loop
I'm currently trying to add code at key-press-event
If I make an error in the guy code, the gui thread will hang in the repl, I can still get the repl back but I didn't find a way to continue with gui development without restarting sbcl and reloading all the packages
(defun run ()
(within-main-loop
; match the controls that matter to vars
(let ((window (builder-get-object builder "window"))
;....
; on window close keep the gtk running. helps with debugging
(g-signal-connect search-field "key-press-event" (lambda (w) (declare (ignore w)) (format "~a" "danut")))
(g-signal-connect window "destroy" (lambda (w) (declare (ignore w)) (leave-gtk-main)))
(widget-show window)))))
The error I get when I run this function is :
debugger invoked on a SIMPLE-ERROR in thread #:
There is no applicable method for the generic function
#
when called with arguments
(NIL).
With enough work I can fix the error, but everytime I get something broken in the gui I can't make the backend thread to continue from where it hanged.
I tried all kinds of thread functions. list-all-treads will show:
(#<SB-THREAD:THREAD "cl-gtk2 main thread" RUNNING {CF48EF1}>
#<SB-THREAD:THREAD "initial thread" RUNNING {AA5F591}>)
This is all I tried until now:
(sb-thread:list-all-threads)
(sb-thread:interrupt-thread (first (sb-thread:list-all-threads)) #'leave-gtk-main)
(sb-thread:terminate-thread (first (sb-thread:list-all-threads)))
I just can't get it unstuck.
What's your normal workflow with cl-gtk2-gtk, how do you avoid this problem?
Use sb-thread:release-foreground to switch between threads waiting for input. See Threads in SBCL for an example session.