Can you compare values across probes in a multi-CPU safe way in DTrace? - macos

I'm trying to write a DTrace script which does the following:
Whenever a new thread is started, increment a count.
Whenever one of these threads exits, decrement the count, and exit the script if the count is now zero.
I have something like this:
BEGIN {
threads_alive = 0;
}
proc:::lwp-start /execname == $$1/ {
self->started = timestamp;
threads_alive += 1;
}
proc:::lwp-exit /self->started/ {
threads_alive -= 1;
if (threads_alive == 0) {
exit(0);
}
}
However, this doesn't work, because threads_alive is a scalar variable and thus it is not multi-cpu safe. As a result, multiple threads will overwrite each other's changes to the variable.
I have also tried using an aggregate variable instead:
#thread_count = sum(1)
//or
#threads_entered = count();
#threads_exitted = count();
Unfortunately, I haven't found syntax to be able to do something like #thread_count == 0 or #threads_started == #threads_stopped.

DTrace doesn't have facilities for doing the kind of thread-safe data sharing you're proposing, but you have a few options depending on precisely what you're trying to do.
If the executable name is unique, you can use the proc:::start and proc:::exit probes for the start of the first thread and the exit of the last thread respectively:
proc:::start
/execname == $$1/
{
my_pid = pid;
}
proc:::exit
/pid == my_pid/
{
exit(0);
}
If you're using the -c option to dtrace, the BEGIN probe fires very shortly after the corresponding proc:::start. Internally, dtrace -c starts the specified forks the specified command and then starts tracing at one of four points: exec (before the first instruction of the new program), preinit (after ld has loaded all libraries), postinit (after each library's _init has run), or main (right before the first instruction of the program's main function, though this is not supported in macOS).
If you use dtrace -x evaltime=exec -c <program> BEGIN will fire right before the first instruction of the program executes:
# dtrace -xevaltime=exec -c /usr/bin/true -n 'BEGIN{ts = timestamp}' -n 'pid$target:::entry{printf("%dus", (timestamp - ts)/1000); exit(0); }'
dtrace: description 'BEGIN' matched 1 probe
dtrace: description 'pid$target:::entry' matched 1767 probes
dtrace: pid 1848 has exited
CPU ID FUNCTION:NAME
10 16757 _dyld_start:entry 285us
The 285us is due to the time it takes dtrace to resume the process via /proc or ptrace(2) on macOS. Rather than proc:::start or proc:::lwp-start you may be able to use BEGIN, pid$target::_dyld_start:entry, or pid$target::main:entry.

Related

How can I setup an LLDB breakpoint firing every 10th time?

To debug the values of high frequency timers or sensors it would be useful to configure a breakpoint that only fires every x times. What's the best way to accomplish this?
I tried the "Ignore x times before stopping" option in Xcode but that only works for the first time. Can I reset this counter using an LLDB command?
You can reset the ignore counter at any time with:
(lldb) break modify -i <NEW_VALUE> <BKPT_SPECIFICATION>
Note, a breakpoint which doesn't satisfy its "ignore count" is not considered to be hit, so its breakpoint command does NOT get run. So if you wanted to stop every tenth the time you hit the breakpoint automatically, just do:
(lldb) break set -l 10 -i 10 -N my_bkpt
Breakpoint 1: where = foo`main + 46 at foo.c:10, address = 0x0000000100000f5e
(lldb) break com add
Enter your debugger command(s). Type 'DONE' to end.
> break modify -i 10 my_bkpt
> DONE
(lldb)
Then just hit "continue" at each stop and you will hit the breakpoint once every 10 times.
Note, I used the ability to name the breakpoint (the -N option) so I didn't have to know the breakpoint number in the breakpoint command that I added. That's convenient if you're going to store these breakpoints in a command file, etc.
Ref: Apple docs on Managing breakpoints. You can also do help breakpoint set command for a complete list of available options
I'm not sure you can define a persistent variable(counter) in lldb. You can always have one global var that you use as a counter helper and simply not include it in the release builds.
class BrCounter{
static var freq = 10
}
Edit the breakpoint and add the following condition:
BrCounter.freq--;
if(BrCounter.freq == 0){
BrCounter.freq = 10;
return true;
}else{
return false;
}
Oneliner:
BrCounter.freq--; if(BrCounter.freq == 0){ BrCounter.freq = 10; return true; }else{ return false; }

How do I use stack content in an LLDB breakpoint condition?

The problem:
I've got a situation where we have a media playback during launch, and objc_exception_throw() hits about 5 times during that period, but is always caught, and it's way south of the media player object.
I'm tired of either (a) having to manually continue n times, or (b) having to leave breakpoints disabled until after the playback is complete.
What I've tried:
making the breakpoint ignore the first five hits (problem: it's not always exactly five times)
creating my own symbolic breakpoint using my target as the module (problem: nothing changed)
What I'd like to do:
One solution that comes to mind is to evaluate the stack when the breakpoint hits, and continue if a particular method or function is listed therein. But I have no idea how to do this.
Other ideas welcome as well.
You do it using Python.
The following defines an ignore list and a function you can attach as a command to a breakpoint.
The function grabs the names of functions in the backtrace and set-intersects those names with the ignore list. If any names match, it continues running the process. This effectively skips dropping into the debugger for unwanted stacks.
(lldb) b objc_exception_throw
Breakpoint 1: where = libobjc.A.dylib`objc_exception_throw, address = 0x00000000000113c5
(lldb) script
Python Interactive Interpreter. To exit, type 'quit()', 'exit()' or Ctrl-D.
>>> ignored_functions = ['recurse_then_throw_and_catch']
def continue_ignored(frame, bp_loc, dict):
global ignored_functions
names = set([frame.GetFunctionName() for frame in frame.GetThread()])
all_ignored = set(ignored_functions)
ignored_here = all_ignored.intersection(names)
if len(ignored_here) > 0:
frame.GetThread().GetProcess().Continue()
quit()
(lldb) br comm add -F continue_ignored 1
(lldb) r
I tried it against the following file, and it successfully skips the first throw inside recurse_then_throw_and_catch and drops into the debugger during the throw inside throw_for_real.
#import <Foundation/Foundation.h>
void
f(int n)
{
if (n <= 0) #throw [NSException exceptionWithName:#"plugh" reason:#"foo" userInfo:nil];
f(n - 1);
}
void
recurse_then_throw_and_catch(void)
{
#try {
f(5);
} #catch (NSException *e) {
NSLog(#"Don't care: %#", e);
}
}
void
throw_for_real(void)
{
f(2);
}
int
main(void)
{
recurse_then_throw_and_catch();
throw_for_real();
}
I imagine you could add this function to your .lldbinit and then connect it to breakpoints as needed from the console. (I don't think you can set a script command from within Xcode.)
break command add -s python -o "return any('xyz' in f.name for f in frame.thread)"
If a python breakpoint command returns False, lldb will keep going. So this is saying: if any frame in the stack has the string 'xyz' in its name, then return True (to stop). Otherwise if no frame has that name, this any expression will return False (to keep going).

Same command, with different parameters, on a while true loop with bash or something else

I always become crazy with bash, i don't understand it.
I basically want to do this (i'm not using some specific syntax, it's just to explain my problem):
processes_count = 20;
for (i = 0; i < processes_count; i++)
{
php -f file.php "{$i}-{$processes_count}" &
proc_id[i] = $!
}
The above cycle start the processes. The next one should keep the processes "alive for ever"!
while(true)
{
foreach(proc_id as id)
{
if(!exist(proc_id[id]))
{
php -f file.php "{$id}-{$processes_count}" &
proc_id[id] = $!
}
}
sleep 5
}
If someone can help translating this into bash, python or something, thank you :)
I don't think you can do that because bash doesn't provide a method to 'wait for any one child process to die and let me know which one it was that died'. The nearest approach is wait:
wait
wait [jobspec or pid ...]
Wait until the child process specified by each process id pid or job specification
jobspec exits and return the exit status of the last command waited for. If a
job spec is given, all processes in the job are waited for. If no arguments are
given, all currently active child processes are waited for, and the return status
is zero. If neither jobspec nor pid specifies an active child process of the shell,
the return status is 127.
This means you can wait for a specific child to die, or you can wait for all children to die, but you can't do what you want.
If you drop into Perl or Python, you can do it, using the wait system call.

How do I do a non-blocking read from a pipe in Perl?

I have a program which is calling another program and processing the child's output, ie:
my $pid = open($handle, "$commandPath $options |");
Now I've tried a couple different ways to read from the handle without blocking with little or no success.
I found related questions:
perl-win32-how-to-do-a-non-blocking-read-of-a-filehandle-from-another-process
why-does-my-perl-sysread-block-when-reading-from-a-socket
But they suffer from the problems:
ioctl consistently crashes perl
sysread blocks on 0 bytes (a common occurrence)
I'm not sure how to go about solving this problem.
Pipes are not as functional on Windows as they are on Unix-y systems. You can't use the 4-argument select on them and the default capacity is miniscule.
You are better off trying a socket or file based workaround.
$pid = fork();
if (defined($pid) && $pid == 0) {
exit system("$commandPath $options > $someTemporaryFile");
}
open($handle, "<$someTemporaryFile");
Now you have a couple more cans of worms to deal with -- running waitpid periodically to check when the background process has stopped creating output, calling seek $handle,0,1 to clear the eof condition after you read from $handle, cleaning up the temporary file, but it works.
I have written the Forks::Super module to deal with issues like this (and many others). For this problem you would use it like
use Forks::Super;
my $pid = fork { cmd => "$commandPath $options", child_fh => "out" };
my $job = Forks::Super::Job::get($pid);
while (!$job->is_complete) {
#someInputToProcess = $job->read_stdout();
... process input ...
... optional sleep here so you don't consume CPU waiting for input ...
}
waitpid $pid, 0;
#theLastInputToProcess = $job->read_stdout();

Monitor Cocoa apps for execution of external utilities (e.g., ffmpeg) on Mac OS X?

There are Mac GUI applications which provide a front-end to more geeky commandline tools (often included as a part of the application package). I would like to look at what is happening under the hood of such GUIs.
How to "attach" to an application, monitor it for calls to command line utilities and log a filename and command line parameters of these calls?
A solution can also be an application that logs execution of all applications on Mac OS X (filtering out the most common system calls).
Example GUI frontend: http://xact.sourceforge.net/ (since it is open source one can just debug it, but xACT is just an example. let's pretend we have just a ready-made *.app to monitor).
Update: dtrace can monitor exec calls and print name of the command called. that's a half of the solution, the other half is getting its command line arguments. that's unsolved yet (until someone confirms they have got dtrace to do this).
DTrace can do the job. Based on the discussion I had with Joey Hagedorn in the comments elsewhere on this question, the script that comes with 10.6 can be improved to work with a reasonable number of arguments (50+). Because the script has a lot of repetition, I'll include here a script which outputs the DTrace script which works well. This one does up to 50 arguments; you may want to extend the number of arguments by changing the for-loop.
#!/bin/bash
cat <<HEADER
#!/usr/sbin/dtrace -s
/*
* newproc.d - snoop new processes as they are executed. DTrace OneLiner.
*
* This is a DTrace OneLiner from the DTraceToolkit.
*
* 15-May-2005 Brendan Gregg Created this.
*/
/*
* Updated to capture arguments in OS X. Unfortunately this isn't straight forward...
*/
#pragma D option quiet
this unsigned long long argv_ptr; /* Wide enough for 64 bit user procs */
proc:::exec-success
{
print_pid[pid] = 1; /* This pid emerged from an exec, make a note of that. */
}
/*
* The "this" variables are local to (all) of the following syscall::mmap:return probes,
* and only those probes. They must be initialized before use in each new firing.
*/
syscall::mmap:return
{
this->argc = 0; /* Disable argument collection until we notice an exec-success */
}
syscall::mmap:return
/ print_pid[pid] /
{
print_pid[pid] = 0;
this->is64Bit = curpsinfo->pr_dmodel == PR_MODEL_ILP32 ? 0 : 1;
this->wordsize = this->is64Bit ? 8 : 4;
this->argc = curpsinfo->pr_argc;
this->argc = (this->argc < 0) ? 0 : this->argc; /* Safety */
this->argv_ptr = curpsinfo->pr_argv;
printf("%d %s ", pid, this->is64Bit ? "64b" : "32b");
}
HEADER
for ((i=0;i<50;++i)); do
cat <<REPEAT
syscall::mmap:return
/ this->argc /
{
this->here_argv = copyin(this->argv_ptr, this->wordsize);
this->arg = this->is64Bit ? *(unsigned long long*)(this->here_argv) : *(unsigned long*)(this->here_argv);
printf("%s ", copyinstr(this->arg));
this->argv_ptr += this->wordsize;
this->argc--;
}
REPEAT
done
cat <<FOOTER
syscall::mmap:return
/ this->argv_ptr /
{
printf("%s\n", this->argc > 0 ? "(...)" : "");
this->argc = 0;
this->argv_ptr = 0;
}
FOOTER
newproc.d is yet another dtrace script which does also output the command line arguments of the processes in the to the process names. Running it is simple:
sudo newproc.d
This works for me on OS X Mountain Lion. Older versions may have various problems; see the comments thread on this ServerFault answer for some discussion of newproc.d on Leopard and Snow Leopard.
Also, you should be aware of a few minor limitations. If you take a look at the script's source code, it indicates that it will not display more than 5 arguments, and it will not display arguments that are longer than 128 characters in length:
/*
* Updated to capture arguments in OS X. Unfortunately this isn't straight forward... nor inexpensive ...
* Bound the size of copyinstr()'s and printf incrementally to prevent "out of scratch space errors"
* print "(...)" if the length of an argument exceeds COPYINSTRLIMIT.
* print "<...>" if argc exceeds 5.
*/
inline int COPYINSTRLIMIT = 128;
You could use dtrace to monitor the exec*() system calls and display the arguments when they're invoked. dtrace is documented here:
https://wikis.oracle.com/display/DTrace/Documentation
Graham: dtrace would be perfect here. could you (or anyone else here) show a dtrace script that would print the commandline of the process?
This oneliner prints names of processes being executed:
dtrace -qn 'syscall::exec*:return { printf("%Y %s\n",walltimestamp,curpsinfo->pr_psargs); }'
But how to get / print their command line arguments?

Resources