Behaviour of /dev/stdout in Cygwin - shell

I have a script (executed with zsh 5.8; but this should not be relevant in this case) in a Cygwin environment, which takes as parameter the name of some output file and writes to this files via redirection in various places, like this:
outfile=$1
: >$outfile # Ensure that the file exists and is empty.
.... do some work
command_x >>$outfile
.... do more work
command_y >>$outfile
... and so on
I would like to modify the behviour of the script in that if no parameter is supplied, the output of the commands goes to standard output instead. I thought that it would be sufficient to modify the script in one line:
outfile=${1:-/dev/stdout}
But nothing is written to stdout. Investigating the case further, I found that instead a regular file named stdout had been created in the /dev directory. It seems that in the Cygwin environment, /dev/stdout does not represent the standard output of the process.
How would I achieve my goal under Cygwin?
UPDATE
As requested by #matzeri, here is a simple testcase:
echo x >/dev/stdout
Expected behaviour: Seeing x on stdout
Real behaviour: A regular file /dev/stdout has been created

on a standard windows installation the /dev/std* are a symlink to the /proc/self/fd/*
ls -l /dev/std*
lrwxrwxrwx 1 Marco Kein 15 Jun 19 2018 /dev/stderr -> /proc/self/fd/2
lrwxrwxrwx 1 Marco Kein 15 Jun 19 2018 /dev/stdin -> /proc/self/fd/0
lrwxrwxrwx 1 Marco Kein 15 Jun 19 2018 /dev/stdout -> /proc/self/fd/1
if for any reason that is not anymore true they can be recreated
by /etc/postinstall/bash.sh.done script
$ grep self /etc/postinstall/bash.sh.done
/bin/test -h /dev/stdin || ln -sf /proc/self/fd/0 /dev/stdin || result=1
/bin/test -h /dev/stdout || ln -sf /proc/self/fd/1 /dev/stdout || result=1
/bin/test -h /dev/stderr || ln -sf /proc/self/fd/2 /dev/stderr || result=1
/bin/test -h /dev/fd || ln -sf /proc/self/fd /dev/fd || result=1
In that condition the command
$ echo x > /dev/stdout
x
produces the expected output on both Bash and Zsh

Related

Process substitution not working with sudo

From a main bash script run as root, I want to execute a subprocess using sudo as unpriviledge user nobody; that subprocess should source a file, which content is provided by the main script.
I am trying to solve this using bash process substitution. But I cannot manage to get this to work.
Can someone tell me why the following script, ...
#! /bin/bash
sudo -u nobody \
bash -c 'source /dev/stdin || ls -l /dev/stdin /proc/self/fd/0 /proc/$$/fd/0; echo "A=$A"' \
< <(echo "A=$(ls /root/.profile)")
... when run as root, produces the following ouput ?
root#raspi:~# ./test3.sh
bash: line 1: /dev/stdin: Permission denied
lrwxrwxrwx 1 root root 15 Mar 20 20:55 /dev/stdin -> /proc/self/fd/0
lr-x------ 1 nobody nogroup 64 Aug 21 14:38 /proc/3243/fd/0 -> 'pipe:[79069]'
lr-x------ 1 nobody nogroup 64 Aug 21 14:38 /proc/self/fd/0 -> 'pipe:[79069]'
A=
I would expect reading from stdin to work because, as indicated by ls -l, read access to stdin is granted to nobody (which makes sense).
So why this does not work ? And is there any way to get this to work ?
Answers to this question did not help: as sample above shows, code in the <(...) bloc should access data that only root can.
To see why you have Permission denied, use ls -lL
sudo -u nobody \
bash -c 'source /dev/stdin || ls -lL /dev/stdin /proc/self/fd/0 /proc/$$/fd/0; echo "A=$A"' \
< <(echo "A=$(ls /root/.profile)")
To get around the error, use cat |
sudo -u nobody \
bash -c 'cat | { source /dev/stdin || ls -lL /dev/stdin /proc/self/fd/0 /proc/$$/fd/0; echo "A=$A"; }' \
< <(echo "A=$(ls /root/.profile)")

How to follow symlinks for which output?

I have this handy function for showing symlinks from the which command.
function whichl() {
ls -la $(which -a $#)
}
It works great for things that are symlinked from the path, like python.
$ whichl python
-rwxr-xr-x 1 root wheel 66880 Mar 27 23:02 /usr/bin/python
lrwxr-xr-x 1 me admin 43 Dec 19 2017 /usr/local/bin/python -> /usr/local/Cellar/python/2.7.14/bin/python2
$ ls -la $(which -a python3)
lrwxr-xr-x 1 me admin 34 Jun 8 10:42 /usr/local/bin/python3 -> ../Cellar/python/3.6.5/bin/python3
It does not work so great when which doesn't find anything; the ls -la command runs against the current directory. This has confused me for the last time!
So, looking for two answers here.
Is there a better way to get which results to show symlinks?
I'm on OS X (if it wasn't obvious) and man which says my version is BSD December 13, 2006.
What is the best way to get my helper function to halt on when which returns no results? I've confirmed the return code in this case is 1, but a simple set -e doesn't change the behavior.
As I mentioned in the question, this doesn't work:
function whichl() {
set -e
ls -la $(which -a $#)
}
The explanation from the comments:
You call set -e in the current shell (where the function runs) but which is executed in a subshell (because of $(...)). You can store the value produced by $(which -a $#) in a variable and run ls -la only when it is not empty.
Following that suggestion, this is what I arrived at.
function whichl() {
RESULT=$(which -a $#)
if [ -z "$RESULT" ]; then
echo "No results for \"$#\""
return 1
fi
ls -la -- ${RESULT}
}
And it works perfectly from all I can test right now!
$ whichl python python3
-rwxr-xr-x 1 root wheel 66880 Mar 27 23:02 /usr/bin/python
lrwxr-xr-x 1 me admin 43 Dec 19 2017 /usr/local/bin/python -> /usr/local/Cellar/python/2.7.14/bin/python2
lrwxr-xr-x 1 me admin 34 Jun 8 10:42 /usr/local/bin/python3 -> ../Cellar/python/3.6.5/bin/python3
Notice it returns 3 results from 2 inputs, the system python and brew python.
$ whichl foo bar
No results for "foo bar"

bash "wc -l" command output differs if call or through tee

When I issued two equivalent commands in Bash I got different output (from "wc -l" command), see below:
root#devel:~# ls /usr/bin -lha | tee >(wc -l) >(head) > /dev/null
total 76M
drwxr-xr-x 2 root root 20K Nov 11 18:58 .
drwxr-xr-x 10 root root 4.0K Oct 8 15:31 ..
-rwxr-xr-x 1 root root 51K Feb 22 2017 [
-rwxr-xr-x 1 root root 96 Jan 19 2017 2to3-3.5
-rwxr-xr-x 1 root root 23K Mar 22 2017 addpart
lrwxrwxrwx 1 root root 26 May 10 2017 addr2line -> x86_64-linux-gnu- addr2line
lrwxrwxrwx 1 root root 6 Dec 13 2016 apropos -> whatis
-rwxr-xr-x 1 root root 15K Sep 13 19:47 apt
-rwxr-xr-x 1 root root 79K Sep 13 19:47 apt-cache
137
root#devel:~# ls /usr/bin -lha | wc -l
648
what am I missing?
it's strange, but when I call it this way it gets even stranger output:
root#devel:~# ls /usr/bin -lha | tee >(wc) >(wc) > /dev/null
648 6121 39179
648 6121 39179
root#devel:~# ls /usr/bin -lha | tee >(wc) >(wc) > /dev/null
648 6121 39179
648 6121 39179
root#devel:~# ls /usr/bin -lha | tee >(wc) >(wc -l) > /dev/null
648
root#devel:~# 648 6121 39179
seems like commands running asynchronously and ends in different time... or what it can be?
Simple answer:
how to fix:
ls /usr/bin -lha | tee --output-error=exit-nopipe >(wc -l) >(head) > /dev/null
Details:
The command head only prints the head of input, so it can finish its job as long as it gets enough input, then exits without waiting for all input.
So let's replace command head with the simple "head".
ls /usr/bin -lha | tee >(wc -l) >(read l; echo $l) > /dev/null
The simple "head" will read only one line, then exit, which causes that the pipe file gets closed immediately before tee finishes transferring all data to it.
So no doubt, you'll get same result with the simple "head". wc still prints wrong number.
The root reason of your issue, I think you can conclude yourself, is that one of the output pipes of tee is closed earlier, tee hits a write error, and then stops writing to other output files.
After understanding the root reason, I think it would be very easy for you to understand the following section in man page.
MODE determines behavior with write errors on the outputs:
'warn' diagnose errors writing to any output
'warn-nopipe'
diagnose errors writing to any output not a pipe
'exit' exit on error writing to any output
'exit-nopipe'
exit on error writing to any output not a pipe
The default MODE for the -p option is 'warn-nopipe'. The default operation
when --output-error is not specified, is to exit immediately on error writing to
a pipe, and diagnose errors writing to non pipe outputs.
Some extra words
Actually if you replace >(wc -l) with a regular file in your problematic command line, you will find the file size will always be 16384 or 20480 or 32768 or 36864 or 28672 or ..., all of which are the multiple of 4096. (The writing to the regular file is incomplete because tee aborts earlier. If the writing was complete, the file size would be any value.)
4096 is the value of PIPE_BUF for most UNIX-like system. If you know what PIPE_BUF is, you will easily understand why the file size is always the multiple of 4096.

Print out the alias'd command when using the alias

I am trying to get alias's setup so that they print out the command, then run the command.
Ex:
> alias ls='ls -alh'
> ls
Running "ls -alh"
total 1.8G
drwxr-x--- 36 root root 4.0K Apr 23 09:44 ./
drwxr-xr-x 28 root root 4.0K Mar 6 17:24 ../
Is this possible? I was thinking of using a wrapper function, but I am unsure as to how one would acomplish this.
Thanks!
Just add an echo command in your alias before the actual command:
alias ls='echo "Running ls -alh"; ls -alh'
alias ls='echo "Running ls -alh" && ls -alh'
This runs two commands one after the other. The first command is echo "Running ls -alh", the && checks the return value of the echo command, if that's 0, then the command ls -alh is run. However, if for some reason there is a problem with the echo command and its return value is not 0 then the ls command won't be run.
The && command can come in very handy when writing scripts to run one command only when another is successful.

How to use shell command to check SVN check-out files and create links in an directory

Basically my question is how to use bash shell command to do following automatically, so I can track modified files easily.
list svn check-out files
create link files to above files in an directory called "change"
laptop$ svn status -q
M rcms/src/config/ta_show.c
M rcms/src/config/ta_config.c
laptop$ cd change
laptop$ link -s ../rcms/src/config/ta_show.c ta_show.c
laptop$ link -s ../rcms/src/config/ta_config.c ta_config.c
laptop$ ls
lrwxrwxrwx 1 root root 59 Nov 27 12:24 ta_show.c -> ../rcms/src/config/ta_show.c
lrwxrwxrwx 1 root root 59 Nov 27 12:24 ta_config.c -> ../rcms/src/config/ta_config.c
I am thinking to use shell command like below:
$ svn status -q | sed 's/M //' | xargs -I xxx ln -s ***BETWEEN REAL FILE AND BASE FILENAME***
you have two things need to be concerned:
the empty line between each file with svn status 'M'
extract the file name
the awk one liner could do it:
awk '$0{x=$2;gsub(".*/","",x);print "ln -s ../"$2" "x}'
so if you pipe your svn status output to the line above, it print the ln -s command lines for you.
if you want the ln -s lines to get executed, you could either pipe the output to sh (svn status|awk ...|sh) or replace the print with system
at the end i would like to show the output below as an exmple:
kent$ echo "M rcms/src/config/ta_show.c
M rcms/src/config/ta_config.c"|awk '$0{x=$2;gsub(".*/","",x);print "ln -s .."$2" "x}'
ln -s ../rcms/src/config/ta_show.c ta_show.c
ln -s ../rcms/src/config/ta_config.c ta_config.c

Resources