Tracing system calls using dtrace - macos

I am running an application that runs with a process Id 423.
Basically want to debug this process.
The problem is that,
using the command sudo dtruss -a -t open_nocancel -p 423 I dont see print messages executed and also systems signals like sudo kill -30 423 dont seem to show in the stack trace. Am I missing something?. How do I achieve this?.
Sample Stack trace below
PID/THRD RELATIVE ELAPSD CPU SYSCALL(args) = return
423/0xcf5: 109498638 14 9 open_nocancel("/Users/krishna/.rstudio-desktop/sdb/s-3F25A09C/373AE888\0", 0x0, 0x1B6) = 21 0
423/0xcf5: 109509540 20 16 open_nocancel("/Users/krishna/.rstudio-desktop/history_database\0", 0x209, 0x1B6) = 20 0
423/0xcf5: 109510342 56 44 open_nocancel(".\0", 0x0, 0x1) = 20 0
423/0xcf5: 109516113 19 15 open_nocancel("/Users/krishna/.rstudio-desktop/history_database\0", 0x209, 0x1B6) = 20 0
423/0xcf5: 109517099 35 30 open_nocancel(".\0", 0x0, 0x1) = 20 0
423/0xcf5: 109576820 16 11 open_nocancel("/Users/krishna/.rstudio-desktop/sdb/s-3F25A09C/373AE888\0", 0x0, 0x1B6) = 21 0
423/0xcf5: 109673038 16 10 open_nocancel("/Users/krishna/.rstudio-desktop/sdb/s-3F25A09C/373AE888\0", 0x0, 0x1B6) = 21 0

The command sudo dtruss -a -t open_nocancel -p 423 will trace only the open_nocancel system call. Per the OS X man page for dtruss:
NAME
dtruss - process syscall details. Uses DTrace.
SYNOPSIS
dtruss [-acdeflhoLs] [-t syscall] { -p PID | -n name | command }
...
-t syscall
examine this syscall only
If you want to trace other system calls, you need to either change the -t ... argument, or remove it.

Related

Gearman worker in shell hangs as a zombie

I have a Gearman worker in a shell script started with perp in the following way:
runuid -s gds \
/usr/bin/gearman -h 127.0.0.1 -t 1000 -w -f gds-rel \
-- xargs /home/gds/gds-rel-worker.sh < /dev/null 2>/dev/null
The worker only does some input validation and calls another shell script run.sh that invokes bash, curl, Terragrunt, Terraform, Ansible and gcloud to provision and update resources in GCP like this:
./run.sh --release 1.2.3 2>&1 >> /var/log/gds-release
The script is intended to run unattended. The problem I have is that after the job finishes successfully (that's both shell scripts run.sh and gds-rel-worker.sh) the Gearman job remains executing, because the child process becomes zombie (see last line below).
root 144748 1 0 Apr29 ? 00:00:00 perpboot -d /etc/perp
root 144749 144748 0 Apr29 ? 00:00:00 \_ tinylog -k 8 -s 100000 -t -z /var/log/perp/perpd-root
root 144750 144748 0 Apr29 ? 00:00:00 \_ perpd /etc/perp
root 2492482 144750 0 May14 ? 00:00:00 \_ tinylog (gearmand) -k 10 -s 100000000 -t -z /var/log/perp/gearmand
gearmand 2492483 144750 0 May14 ? 00:00:08 \_ /usr/sbin/gearmand -L 127.0.0.1 -p 4730 --verbose INFO --log-file stderr --keepalive --keepalive-idle 120 --keepalive-interval 120 --keepalive-count 3 --round-robin --threads 36 --worker-wakeup 3 --job-retries 1
root 2531800 144750 0 May14 ? 00:00:00 \_ tinylog (gds-rel-worker) -k 10 -s 100000000 -t -z /var/log/perp/gds-rel-worker
gds 2531801 144750 0 May14 ? 00:00:00 \_ /usr/bin/gearman -h 127.0.0.1 -t 1000 -w -f gds-rel -- xargs /home/gds/gds-rel-worker.sh
gds 2531880 2531801 0 May14 ? 00:00:00 \_ [xargs] <defunct>
So far I have traced the problem to run.sh, because if I replace its call with something simpler (e.g. echo "Hello"; sleep 5) the worker does not hang. Unfortunately, I have no clue what is causing the problem. The script run.sh is rather long and complex, but has been working without a problem so far. Tracing the worker process I see this:
getpid() = 2531801
write(2, "gearman: ", 9) = 9
write(2, "gearman_worker_work", 19) = 19
write(2, " : ", 3) = 3
write(2, "gearman_wait(GEARMAN_TIMEOUT) ti"..., 151) = 151
write(2, "\n", 1) = 1
sendto(5, "\0REQ\0\0\0'\0\0\0\0", 12, MSG_NOSIGNAL, NULL, 0) = 12
recvfrom(5, "\0RES\0\0\0\n\0\0\0\0", 8192, MSG_NOSIGNAL, NULL, NULL) = 12
sendto(5, "\0REQ\0\0\0\4\0\0\0\0", 12, MSG_NOSIGNAL, NULL, 0) = 12
poll([{fd=5, events=POLLIN}, {fd=3, events=POLLIN}], 2, 1000) = 1 ([{fd=5, revents=POLLIN}])
sendto(5, "\0REQ\0\0\0'\0\0\0\0", 12, MSG_NOSIGNAL, NULL, 0) = 12
recvfrom(5, "\0RES\0\0\0\6\0\0\0\0\0RES\0\0\0(\0\0\0QH:terra-"..., 8192, MSG_NOSIGNAL, NULL, NULL) = 105
pipe([6, 7]) = 0
pipe([8, 9]) = 0
clone(child_stack=NULL, flags=CLONE_CHILD_CLEARTID|CLONE_CHILD_SETTID|SIGCHLD, child_tidptr=0x7fea38480a50) = 2531880
close(6) = 0
close(9) = 0
write(7, "1.2.3\n", 18) = 6
close(7) = 0
read(8, "which: no terraform-0.14 in (/us"..., 1024) = 80
read(8, "Identity added: /home/gds/.ssh/i"..., 1024) = 54
read(8, 0x7fff6251f5b0, 1024) = ? ERESTARTSYS (To be restarted if SA_RESTART is set)
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=2531880, si_uid=1006, si_status=0, si_utime=0, si_stime=0} ---
read(8,
So the worker continues reading standard output even though the child has finished successfully and presumably closed it. Any ideas how to catch what causes this problem?
I was able to solve it. The script run.sh was starting ssh-agent, which opens a socket and since Gearman redirects all outputs the worker continued reading the open file descriptor even after the script successfully completed.
I found it by examining the open file descriptors for the Gearman worker process after it hang:
# ls -l /proc/2531801/fd/*
lr-x------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/0 -> /dev/null
l-wx------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/1 -> 'pipe:[9356665]'
l-wx------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/2 -> 'pipe:[9356665]'
lr-x------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/3 -> 'pipe:[9357481]'
l-wx------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/4 -> 'pipe:[9357481]'
lrwx------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/5 -> 'socket:[9357482]'
lr-x------. 1 gds devops 64 May 17 11:26 /proc/2531801/fd/8 -> 'pipe:[9369888]'
Then identified the processes using file node for the pipe in file descriptor 8 that German worker continued reading:
# lsof | grep 9369888
gearman 2531801 gds 8r FIFO 0,13 0t0 9369888 pipe
ssh-agent 2531899 gds 9w FIFO 0,13 0t0 9369888 pipe
And finally listed files opened by ssh-agent and found what stands behind file descriptor 3:
# ls -l /proc/2531899/fd/*
lrwx------. 1 root root 64 May 17 11:14 /proc/2531899/fd/0 -> /dev/null
lrwx------. 1 root root 64 May 17 11:14 /proc/2531899/fd/1 -> /dev/null
lrwx------. 1 root root 64 May 17 11:14 /proc/2531899/fd/2 -> /dev/null
lrwx------. 1 root root 64 May 17 11:14 /proc/2531899/fd/3 -> 'socket:[9346577]'
# lsof | grep 9346577
ssh-agent 2531899 gds 3u unix 0xffff89016fd34000 0t0 9346577 /tmp/ssh-0b14coFWhy40/agent.2531898 type=STREAM
As a solution I added kill of the ssh-agent before exit from run.sh script and now there are no jobs hanging due to zombie process.

Why are there not any execve calls in my dtruss trace?

I have a script like this:
script.sh
#!/bin/bash
clang -v
If I do a dtruss on it then I would expect to see an execve call to clang.
$ sudo dtruss -f -a -e ./script.sh
However, the trace does not contain an execve. Instead there is an error:
...
1703/0x16931: 856 4 0 sigaction(0x15, 0x7FFEE882A3B8, 0x7FFEE882A3F8) = 0 0
1703/0x16931: 858 4 0 sigaction(0x16, 0x7FFEE882A3C8, 0x7FFEE882A408) = 0 0
1703/0x16931: 874 4 0 sigaction(0x2, 0x7FFEE882A3C8, 0x7FFEE882A408) = 0 0
1703/0x16931: 881 4 0 sigaction(0x3, 0x7FFEE882A3C8, 0x7FFEE882A408) = 0 0
1703/0x16931: 883 4 0 sigaction(0x14, 0x7FFEE882A3C8, 0x7FFEE882A408) = 0 0
dtrace: error on enabled probe ID 2149 (ID 280: syscall::execve:return): invalid address (0x7fc2b5502c30) in action #12 at DIF offset 12
1703/0x16932: 2873: 0: 0 fork() = 0 0
1703/0x16932: 2879 138 5 thread_selfid(0x0, 0x0, 0x0) = 92466 0
1703/0x16932: 2958 8 0 issetugid(0x0, 0x0, 0x0) = 0 0
1703/0x16932: 2975 8 1 csrctl(0x0, 0x7FFEEE21DC3C, 0x4) = 0 0
1703/0x16932: 2985 12 6 csops(0x0, 0x0, 0x7FFEEE21E550) = 0 0
1703/0x16932: 3100 13 3 shared_region_check_np(0x7FFEEE21DA98, 0x0, 0x0)
...
What is causing this error?
How can I get the execve command to show so that I can see the program called and its arguments?
This means that the DTrace script that dtruss is using internally is accessing an invalid memory address, which is happening while it's trying to trace the execve call you're curious about. So basically, dtruss (or possibly DTrace itself) appears to have a bug which is preventing you from getting the information you want. Apple hasn't been the best about keeping DTrace and the tools that depend on it working well on macOS, unfortunately :-/.
For Bash / shell scripts in particular, you can make it print every command that it runs by adding set -x at the top of your script (more info in this other answer).
If you want, you could also try using DTrace directly instead -- this is a pretty simple one-liner (haven't tried running this myself so apologies if there are typos):
sudo dtrace -n 'proc:::exec-success /ppid == $target/ { trace(curpsinfo->pr_psargs); }' -c './script.sh'
The way this works is:
proc:::exec-success: Trace all exec-success events in the system, which fire in the subprocess when an exec*-family syscall returns successfully.
/ppid == $target/: Filter which means this only fires when the parent process's PID (ppid) matches the PID returned for the process started by the -c option we passed to the dtrace command ($target).
{ trace(curpsinfo->pr_psargs); }: This is the action to take when the event fires and it matches our filter. We simply print (trace) the arguments passed to the process, which is stored in the curpsinfo variable.
(If that fails with a similar-looking error, it's likely that the bug is in macOS's implementation of curpsinfo somewhere.)

How to decrease TCP connect() system call timeout?

In command below I enable file /dev/tcp/10.10.10.1/80 both for reading and writing and associate it with file descriptor 3:
$ time exec 3<>/dev/tcp/10.10.10.1/80
bash: connect: Operation timed out
bash: /dev/tcp/10.10.10.1/80: Operation timed out
real 1m15.151s
user 0m0.000s
sys 0m0.000s
This automatically tries to perform TCP three-way handshake. If 10.10.10.1 is not reachable as in example above, then connect system call tries to connect for 75 seconds. Is this 75 second timeout determined by bash? Or is this system default? Last but not least, is there a way to decrease this timeout value?
It's not possible in Bash without modifying the source as already mentioned, although here is the workaround by using timeout command, e.g.:
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/80" && echo Port open. || echo Port closed.
Port open.
$ timeout 1 bash -c "</dev/tcp/stackoverflow.com/81" && echo Port open. || echo Port closed.
Port closed.
Using this syntax, the timeout command will kill the process after the given time.
See: timeout --help for more options.
It is determined by TCP. It can be decreased on a per-socket basis by application code.
NB The timeout only takes effect if there is no response at all. If there is a connection refusal, the error occurs immediately.
No: there is no way of changing timeout by using /dev/tcp/
Yes, you could change default timeout for TCP connection in any programming language.
But, bash is not a programming language!
You could have a look into source code (see: Bash Homepage), you may find lib/sh/netopen.c file where you could read in _netopen4 function:
s = socket(AF_INET, (typ == 't') ? SOCK_STREAM : SOCK_DGRAM, 0);
You could read this file carefully, there are no consideration of connection timeout.
Without patching bash sources, there is no way of changing connection timeout by a bash script.
Simple HTTP client using netcat (near pure bash)
There is a little sample HTTP client written in pure bash, but using netcat:
#!/bin/bash
tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
exec 6<$tmpfile
rm $tmpfile
printf >&7 "GET %s HTTP/1.0\r\nHost: stackoverflow.com\r\n\r\n" \
/questions/24317341/how-to-decrease-tcp-connect-system-call-timeout
timeout=100;
while ! read -t .001 -u 6 status ; do read -t .001 foo;done
echo STATUS: $status
[ "$status" ] && [ -z "${status//HTTP*200 OK*}" ] || exit 1
echo HEADER:
while read -u 6 -a head && [ "${head//$'\r'}" ]; do
printf "%-20s : %s\n" ${head%:} "${head[*]:1}"
done
echo TITLE:
sed '/<title>/s/<[^>]*>//gp;d' <&6
exec 7>&-
exec 6<&-
This could render:
STATUS: HTTP/1.1 200 OK
HEADER:
Cache-Control : private
Content-Type : text/html; charset=utf-8
X-Frame-Options : SAMEORIGIN
X-Request-Guid : 46d55dc9-f7fe-425f-a560-fc49d885a5e5
Content-Length : 91642
Accept-Ranges : bytes
Date : Wed, 19 Oct 2016 13:24:35 GMT
Via : 1.1 varnish
Age : 0
Connection : close
X-Served-By : cache-fra1243-FRA
X-Cache : MISS
X-Cache-Hits : 0
X-Timer : S1476883475.343528,VS0,VE100
X-DNS-Prefetch-Control : off
Set-Cookie : prov=ff1129e3-7de5-9375-58ee-5f739eb73449; domain=.stackoverflow.com; expires=Fri, 01-Jan-2055 00:00:00 GMT; path=/; HttpOnly
TITLE:
bash - How to decrease TCP connect() system call timeout? - Stack Overflow
Some explanations:
We create first a temporary file (under private directory for security reason), bind and delete before using them.
$ tmpfile=$(mktemp -p $HOME .netbash-XXXXXX)
$ exec 7> >(nc -w 3 -q 0 stackoverflow.com 80 >$tmpfile)
$ exec 6<$tmpfile
$ rm $tmpfile
$ ls $tmpfile
ls: cannot access /home/user/.netbash-rKvpZW: No such file or directory
$ ls -l /proc/self/fd
lrwx------ 1 user user 64 Oct 19 15:20 0 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 1 -> /dev/pts/1
lrwx------ 1 user user 64 Oct 19 15:20 2 -> /dev/pts/1
lr-x------ 1 user user 64 Oct 19 15:20 3 -> /proc/30237/fd
lr-x------ 1 user user 64 Oct 19 15:20 6 -> /home/user/.netbash-rKvpZW (deleted)
l-wx------ 1 user user 64 Oct 19 15:20 7 -> pipe:[2097453]
$ echo GET / HTTP/1.0$'\r\n\r' >&7
$ read -u 6 foo
$ echo $foo
HTTP/1.1 500 Domain Not Found
$ exec 7>&-
$ exec 6>&-

Mac OS X: GNU parallel can't find the number of cores on a remote server

I used homebrew to install GNU parallel on my mac so I can run some tests remotely on my University's servers. I was quickly running through the tutorials, but when I ran
parallel -S <username>#$SERVER1 echo running on ::: <username>#$SERVER1
I got the message
parallel: Warning: Could not figure out number of cpus on <username#server> (). Using 1.
Possibly related, I never added parallel to my path and got the warning that "parallel" wasn't a recognized command, but parallel ran anyways and still echo'd correctly. This particular server has 16 cores, how can I get parallel to recognize them?
GNU Parallel is less tested on OS X as I do not have access to an OS X installation, so you have likely found a bug.
GNU Parallel has since 20120322 used these to find the number of CPUs:
sysctl -n hw.physicalcpu
sysctl -a hw 2>/dev/null | grep [^a-z]physicalcpu[^a-z] | awk '{ print \$2 }'
And the number of cores:
sysctl -n hw.logicalcpu
sysctl -a hw 2>/dev/null | grep [^a-z]logicalcpu[^a-z] | awk '{ print \$2 }'
Can you test what output you get from those?
Which version of GNU Parallel are you using?
As a work around you can force GNU Parallel to detect 16 cores:
parallel -S 16/<username>#$SERVER1 echo running on ::: <username>#$SERVER1
Since version 20140422 you have been able to export your path to the remote server:
parallel --env PATH -S 16/<username>#$SERVER1 echo running on ::: <username>#$SERVER1
That way you just need to add the dir where parallel lives on the server to your path on local machine. E.g. parallel on the remote server is in /home/u/user/bin/parallel:
PATH=$PATH:/home/u/user/bin parallel --env PATH -S <username>#$SERVER1 echo running on ::: <username>#$SERVER1
Information for Ole
My iMac (OSX MAvericks on Intel core i7) gives the following, which all looks correct:
sysctl -n hw.physicalcpu
4
sysctl -a hw
hw.ncpu: 8
hw.byteorder: 1234
hw.memsize: 17179869184
hw.activecpu: 8
hw.physicalcpu: 4
hw.physicalcpu_max: 4
hw.logicalcpu: 8
hw.logicalcpu_max: 8
hw.cputype: 7
hw.cpusubtype: 4
hw.cpu64bit_capable: 1
hw.cpufamily: 1418770316
hw.cacheconfig: 8 2 2 8 0 0 0 0 0 0
hw.cachesize: 17179869184 32768 262144 8388608 0 0 0 0 0 0
hw.pagesize: 4096
hw.busfrequency: 100000000
hw.busfrequency_min: 100000000
hw.busfrequency_max: 100000000
hw.cpufrequency: 3400000000
hw.cpufrequency_min: 3400000000
hw.cpufrequency_max: 3400000000
hw.cachelinesize: 64
hw.l1icachesize: 32768
hw.l1dcachesize: 32768
hw.l2cachesize: 262144
hw.l3cachesize: 8388608
hw.tbfrequency: 1000000000
hw.packages: 1
hw.optional.floatingpoint: 1
hw.optional.mmx: 1
hw.optional.sse: 1
hw.optional.sse2: 1
hw.optional.sse3: 1
hw.optional.supplementalsse3: 1
hw.optional.sse4_1: 1
hw.optional.sse4_2: 1
hw.optional.x86_64: 1
hw.optional.aes: 1
hw.optional.avx1_0: 1
hw.optional.rdrand: 0
hw.optional.f16c: 0
hw.optional.enfstrg: 0
hw.optional.fma: 0
hw.optional.avx2_0: 0
hw.optional.bmi1: 0
hw.optional.bmi2: 0
hw.optional.rtm: 0
hw.optional.hle: 0
hw.cputhreadtype: 1
hw.machine = x86_64
hw.model = iMac12,2
hw.ncpu = 8
hw.byteorder = 1234
hw.physmem = 2147483648
hw.usermem = 521064448
hw.pagesize = 4096
hw.epoch = 0
hw.vectorunit = 1
hw.busfrequency = 100000000
hw.cpufrequency = 3400000000
hw.cachelinesize = 64
hw.l1icachesize = 32768
hw.l1dcachesize = 32768
hw.l2settings = 1
hw.l2cachesize = 262144
hw.l3settings = 1
hw.l3cachesize = 8388608
hw.tbfrequency = 1000000000
hw.memsize = 17179869184
hw.availcpu = 8
sysctl -n hw.logicalcpu
8

Why does there appear text on my command line even though I've redirected both STDOUT and STDERR to /dev/null?

I'm trying to unmount a encfs-filesystem from a script, but no matter how I try I seem unable to prevent the fuse error below to appear on screen/in crontab emails.
# exec 3>&1 1>/dev/null 4>&2 2>/dev/null; setsid fusermount -u /data/encfs; exec 1>&3 2>&4 3>&- 4>&-
# fuse failed. Common problems:
- fuse kernel module not installed (modprobe fuse)
- invalid options -- see usage message
The error itself I have to live with. The unmount is successfull and the error is false and due to a bug that is long gone in modern versions of fuse. I'm stuck with the older version since I'm on special hardware running a semi-ancient version of debian.
What annoys me is that I cannot tell the system to toss the nonsense message in /dev/null.
How does the message even appear on my screen after me using both setsid and redirects in my best efforts to prevent it?
EDIT:
# exec 3>&1 1>/dev/null 4>&2 2>/dev/null; setsid fusermount -u /data/encfs > /dev/null 2>&1; EXIT=$?; exec 1>&3 2>&4 3>&- 4>&-; echo $EXIT
0
# fuse failed. Common problems:
- fuse kernel module not installed (modprobe fuse)
- invalid options -- see usage message
I've even tried things like:
perl -e "`fusermount -u /data/encfs`"
But the error remain the same.
My /etc/syslog.conf:
auth,authpriv.* -/var/log/auth.log
*.*;auth,authpriv,cron.none -/var/log/syslog
cron.* -/var/log/cron.log
daemon.* -/var/log/daemon.log
kern.* -/var/log/kern.log
lpr.* -/var/log/lpr.log
mail.* -/var/log/mail.log
user.* -/var/log/user.log
*.=debug;\
auth,authpriv.none;\
mail.none -/var/log/debug
*.=info;*.=notice;*.=warn;\
auth,authpriv.none;\
cron,daemon.none;\
mail.none -/var/log/messages
EDIT2:
I don't think fusermount is the program actually generating the text. It pokes something else that does:
# strace -o ~/trash/strace.txt fusermount -u /data/encfs; EXIT=$?; echo $EXIT; grep write ~/trash/strace.txt
fuse failed. Common problems:
- fuse kernel module not installed (modprobe fuse)
- invalid options -- see usage message
0
write(5, "/dev/hdc1 / ext3 rw,noatime 0 0\n", 32) = 32
write(5, "proc /proc proc rw 0 0\n", 23) = 23
write(5, "devpts /dev/pts devpts rw 0 0\n", 30) = 30
write(5, "sysfs /sys sysfs rw 0 0\n", 24) = 24
write(5, "tmpfs /ramfs ramfs rw 0 0\n", 26) = 26
write(5, "tmpfs /USB tmpfs rw,size=16k 0 0"..., 33) = 33
write(5, "/dev/c/c /c ext3 rw,noatime,acl,"..., 65) = 65
write(5, "nfsd /proc/fs/nfsd nfsd rw 0 0\n", 31) = 31
write(5, "usbfs /proc/bus/usb usbfs rw 0 0"..., 33) = 33
write(5, "//localhost/smb /root/folder"..., 55) = 55
If I let strace log to stdout I get the error message in the middle of the umount system call:
# strace fusermount -u /data/encfs
execve("/usr/bin/fusermount", ["fusermount", "-u", "/data/encfs"], [/* 16 vars */]) = 0
[... abbreviating ...]
close(5) = 0
munmap(0x20020000, 16384) = 0
profil(0, 0, 0x2010c168, 0x4) = 0
umount("/data/encfs", 0fuse failed. Common problems:
- fuse kernel module not installed (modprobe fuse)
- invalid options -- see usage message
) = 0
profil(0, 0, 0x1177c, 0x20179f98) = 0
stat64("/etc/mtab", {st_mode=S_IFREG|0644, st_size=407, ...}) = 0
ftime(0x13840) = 0
Use strace on the command. It will show you details about what's going on, including the number of the descriptor to which the message is written
strace fsusermount -u /data/encfs
If the message comes from fsusermount you should see a line like
write(0, "- fuse kernel module not installed (modprobe fuse)\n")
somewhere in the output. The number (not necessarily 0) is the number of the file descriptor to which the message is written. Redirecting the descriptor with that number should get you rid of the message:
fsusermount -u /data/encfs 0>/dev/null
Figured it out.
The error message does not come from the fusermount, it comes from the mount command when fusermount runs.
Doing it like this fixes the problem:
# encfs --extpass=echo_key.sh /data/.encfs /data/encfs 2>/dev/null; sleep 3; fusermount -u /data/encfs
#
Feels so obvious now that I know...

Resources