/dev/stdin with herestring - bash

I would like a Bash script that can take input from a file or stdin, much like grep, for example
$ cat hw.txt
Hello world
$ grep wor hw.txt
Hello world
$ echo 'Hello world' | grep wor
Hello world
$ grep wor <<< 'Hello world'
Hello world
all works beautifully. However with the following script
read b < "${1-/dev/stdin}"
echo $b
It fails if using a herestring
$ hw.sh hw.txt
Hello world
$ echo 'Hello world' | hw.sh
Hello world
$ hw.sh <<< 'Hello world'
/opt/a/hw.sh: line 1: /dev/stdin: No such file or directory

Using /dev/stdin in this manner can be problematic because you are attempting to get a handle to stdin using a name in the filesystem (/dev/stdin) rather than using the file descriptor which bash has already handed you as stdin (file descriptor 0).
Here's a small script for you to test:
#!/bin/bash
echo "INFO: Listing of /dev"
ls -al /dev/stdin
echo "INFO: Listing of /proc/self/fd"
ls -al /proc/self/fd
echo "INFO: Contents of /tmp/sh-thd*"
cat /tmp/sh-thd*
read b < "${1-/dev/stdin}"
echo "b: $b"
On my cygwin installation this produces the following:
./s <<< 'Hello world'
$ ./s <<< 'Hello world'
INFO: Listing of /dev
lrwxrwxrwx 1 austin None 15 Jan 23 2012 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-xr-xr-x 2 austin None 0 Mar 11 14:27 .
dr-xr-xr-x 3 austin None 0 Mar 11 14:27 ..
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 0 -> /tmp/sh-thd-1362969584
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 1 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 2 -> /dev/tty0
lrwxrwxrwx 1 austin None 0 Mar 11 14:27 3 -> /proc/5736/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
./s: line 12: /dev/stdin: No such file or directory
b:
What this output shows is that bash is creating a temporary file to hold your HERE document (/tmp/sh-thd-1362969584) and making it available on file descriptor 0, stdin. However, the temporary file has already been unlinked from the file system and so is not accessible by reference through a file system name such as /dev/stdin. You can get the contents by reading file descriptor 0, but not by trying to open /dev/stdin.
On Linux, the ./s script above gives the following, showing that the file has been unlinked:
INFO: Listing of /dev
lrwxrwxrwx 1 root root 15 Mar 11 09:26 /dev/stdin -> /proc/self/fd/0
INFO: Listing of /proc/self/fd
total 0
dr-x------ 2 austin austin 0 Mar 11 14:30 .
dr-xr-xr-x 7 austin austin 0 Mar 11 14:30 ..
lr-x------ 1 austin austin 64 Mar 11 14:30 0 -> /tmp/sh-thd-1362965400 (deleted) <---- /dev/stdin not found
lrwx------ 1 austin austin 64 Mar 11 14:30 1 -> /dev/pts/12
lrwx------ 1 austin austin 64 Mar 11 14:30 2 -> /dev/pts/12
lr-x------ 1 austin austin 64 Mar 11 14:30 3 -> /proc/10659/fd
INFO: Contents of /tmp/sh-thd*
cat: /tmp/sh-thd*: No such file or directory
b: Hello world
Change your script to use the stdin supplied, rather than trying to reference through /dev/stdin.
if [ -n "$1" ]; then
read b < "$1"
else
read b
fi

bash parses some file names (like /dev/stdin) specially, so that they are recognized even if they are not actually present in the file system. If your script doesn't have #!/bin/bash at the top, and /dev/stdin isn't in your file system, your script may be run using /bin/sh, which would expect /dev/stdin to actually be a file.
(This should, perhaps, not be an answer, but rather a comment to Austin's answer.)

$ cat ts.sh
read b < "${1-/dev/stdin}"
echo $b
$ ./ts.sh <<< 'hello world'
hello world
No problem for me. I'm using bash 4.2.42 on Mac OS X.

You got a typo here
read b < "${1-/dev/stdin}"
Try
read b < "${1:-/dev/stdin}"

Related

Calling a bash script function from another bash script doesn't display my shell command

I have 2 bash scripts, one calling another but depending on how I call it, it does or does not display my ls command.
script2.sh
#!/bin/bash
function test() {
i=0
while IFS= read -r line; do
IFS=',' read -ra ITEM <<<"$line"
printf "\n[${i}] ${ITEM}"
((i = i + 1))
done <<<$(ls $1)
printf "\nPress any other keys to abort.\n\n"
read -p "Please enter your selection: " ANSWER
echo $ANSWER
}
script1a.sh WORKS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
(test $PARAM)
Returns:
[0] total 5
[1] drwxr-xr-x 1 1049089 0 Oct 29 09:10 .
[2] drwxr-xr-x 1 1049089 0 Oct 9 23:11 ..
[3] -rw-r--r-- 1 1049089 87 Jul 6 14:19 .eslintignore
[4] -rw-r--r-- 1 1049089 449 Jul 10 13:56 .forceignore
[5] drwxr-xr-x 1 1049089 0 Oct 29 09:11 .git
Press any other keys to abort.
Please enter your selection:
script1b.sh FAILS
#!/bin/bash
. ./scripts/bash/script2.sh
PARAM='-lag'
myanswer=$(test $PARAM)
Returns:
Please enter your selection:
Anyone knows why this odd behavior and how to get around it? Thanks in advance.

How to iterate through multiple directories with multiple ifs in bash?

unfortunately I'm quite new at bash, and I want to write a script that will start in a main directory, and check all subdirectories one by one for the presence of certain files, and if those files are present, perform an operation on them. For now, I have written a simplified version to test whether I can do the first part (checking for the files in each directory). This code runs without any errors that I can tell, but it does not echo anything to say that it has successfully found the files which I know are there.
#!/bin/bash
runlist=(1 2 3 4 5 6 7 8 9)
for f in *; do
if [[ -d {$f} ]]; then
#if f is a directory then cd into it
cd "{$f}"
for b in $runlist; do
if [[ -e "{$b}.png" ]]; then
echo "Found {$b}"
#if the file exists then say so
fi
done
cd -
fi
done
'''
Welcome to stackoverflow.
The following will do the trick (a combination of find, array, and if then else):
# list of files we are looking for
runlist=(1 2 4 8 16 32 64 128)
#find each of above anywhere below current directory
# using -maxdepth 1 because, based on on your exam you want to look one level only
# if that's not what you want then take out -maxdepth 1 from the find command
for b in ${runlist[#]}; do
echo
PATH_TO_FOUND_FILE=`find . -name $b.png`
if [ -z "$PATH_TO_FOUND_FILE" ]
then
echo "nothing found" >> /dev/null
else
# You wanted a postive confirmation, so
echo found $b.png
# Now do something with the found file. Let's say ls -l: change that to whatever
ls -l $PATH_TO_FOUND_FILE
fi
done
Here is an example run:
mamuns-mac:stack foo$ ls -lR
total 8
drwxr-xr-x 4 foo 1951595366 128 Apr 11 18:03 dir1
drwxr-xr-x 3 foo 1951595366 96 Apr 11 18:03 dir2
-rwxr--r-- 1 foo 1951595366 652 Apr 11 18:15 find_file_and_do_something.sh
./dir1:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 8.png
./dir2:
total 0
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 64.png
mamuns-mac:stack foo$ ./find_file_and_do_something.sh
found 1.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/1.png
found 8.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 17:58 ./dir1/8.png
found 64.png
-rw-r--r-- 1 foo 1951595366 0 Apr 11 18:03 ./dir2/64.png

For loop with if statements isn't working as expected in bash

It only prints the "else" statement for everything but I know for a fact the files exist that it's looking for. I've tried adapting some of the other answers but I thought this should definitely work.
Does anyone know what's wrong with my syntax?
# Contents of script
for ID_SAMPLE in $(cut -f1 metadata.tsv | tail -n +2);
do if [ -f ./output/${ID_SAMPLE} ]; then
echo Skipping ${ID_SAMPLE};
else
echo Processing ${ID_SAMPLE};
fi
done
Additional information
# Output directory
(base) -bash-4.1$ ls -lhS output/
total 170K
drwxr-xr-x 8 jespinoz tigr 185 Jan 3 16:16 ERR1701760
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 18:03 ERR315863
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 23:23 ERR599042
drwxr-xr-x 8 jespinoz tigr 185 Jan 17 00:10 ERR599072
drwxr-xr-x 8 jespinoz tigr 185 Jan 16 13:00 ERR599078
# Example of inputs
(base) -bash-4.1$ cut -f1 metadata.tsv | tail -n +2 | head -n 10
ERR1701760
ERR599078
ERR599079
ERR599070
ERR599071
ERR599072
ERR599073
ERR599074
ERR599075
ERR599076
# Output of script
(base) -bash-4.1$ bash test.sh | head -n 10
Processing ERR1701760
Processing ERR599078
Processing ERR599079
Processing ERR599070
Processing ERR599071
Processing ERR599072
Processing ERR599073
Processing ERR599074
Processing ERR599075
Processing ERR599076
# Checking a directory
(base) -bash-4.1$ ls -l ./output/ERR1701760
total 294
drwxr-xr-x 2 jespinoz tigr 386 Jan 15 21:00 checkpoints
drwxr-xr-x 2 jespinoz tigr 0 Jan 10 01:36 tmp
-f is for checking whether the name is a file, but all your names are directories. Use -d to check that.
if [ -d "./output/$ID_SAMPLE" ]
then
If you want to check whether the name exists with any type, use -e.

I need to run the same script multiple times in parallel with log management

I need to run the same script multiple times in parallel with log management of each script execution
I need an optimal method please?
script name : script.sh
logs : log_execution1.log log_execution2.log ...
or
logs : log_execution-PID.log log_execution-PID.log ...
thanks
This can be done around a loop:
$ ntimes=42
$ for i in $(seq 1 $ntimes) ; do (script.sh > log_execution$i.log &) ; done
I did it like that :
it's work fine now.
while [ -e file.dat ]
do
./a.ksh
sleep 5
done
file : a.ksh
#!/bin/ksh
echo "Hello StackOverFlow" > log_execution-$$.log
exit 0
Result
-rw-r--r-- 1 aai aai 22 Apr 19 11:11 log_execution-27310.log
-rw-r--r-- 1 aai aai 0 Apr 19 11:11 log_execution-26005.log
-rw-r--r-- 1 aai aai 22 Apr 19 11:11 log_execution-27327.log
thank you for your help

Unix shell - How to have a prompt that change randomly

I was wondering if there is any way to have a prompt that change every time i press Enter, like:
[jtouzea - such prompt] $>
[jtouzea - much style] $> ls -l
total 0
drwx------+ 5 jtouzea 2013 170 Mar 11 16:50 Desktop
drwx------+ 2 jtouzea 2013 68 Mar 11 16:49 Documents
drwx------+ 2 jtouzea 2013 68 Mar 11 16:58 Downloads
drwxr-xr-x# 26 jtouzea 2013 884 Mar 11 17:20 Library
drwx------+ 2 jtouzea 2013 68 Mar 11 16:50 Movies
drwx------+ 2 jtouzea 2013 68 Mar 11 16:50 Music
drwx------+ 2 jtouzea 2013 68 Mar 11 16:50 Pictures
drwxr-xr-x+ 2 jtouzea 2013 68 Mar 11 16:50 Public
[jtouzea - wow] $> echo "test"
test
[jtouzea - 10/10] $>
i have already found the $RANDOM command, that allow to do so, but i need to do:
source ~/.zshrc
or else i doesn't change my prompt.
Any idea?
EDIT:
Currently i have:
PROMPT="[jtouzea - $RANDOM] $> "
in my .zshrc
EDIT2:
I use zsh, so unfortunately, PROMPT_COMMAND doesn't seems to work
EDIT3:
Here is my final code now that the question is solved:
function precmd()
{
sentence[1]="much prompt";
sentence[2]="such style";
sentence[3]="wow";
nb=$[$RANDOM % 3 + 1];
PROMPT="[jtouzea - ${sentence[$nb]}] $> ";
}
Try using the PROMPT_COMMAND hook:
PROMPT_COMMAND() { randPromptNum=$[ $RANDOM % 3 ]; PS1=${POSSIBLE_PROMPTS[$randPromptNum]};}
For zsh, you must use precmd instead of PROMPT_COMMAND.
Before you use this, you must set up a variable with all possible prompts:
POSSIBLE_PROMPTS[0]="such prompt: "
POSSIBLE_PROMPTS[1]="much style: "
POSSIBLE_PROMPTS[2]="wow: "
If you want to adjust the number of prompts possible, remember to edit the % 3.
EDIT - Result on cygwin
much style: echo hi
hi
such prompt: echo blah
blah
much style: pwd
/usr/bin
such prompt: yes y | head
y
y
y
y
y
y
y
y
y
y
wow: echo foo
foo
such prompt:
It's not the same shell as yours, but you could try this:
I edited my .kshrc file like this:
PS1="Look, it's random: \$RANDOM \$ "
And then sourced the file, and pressed Enter several times:
/home/user $ . .kshrc
Look, it's random: 2155 $
Look, it's random: 6032 $
Look, it's random: 13065 $
Put all your phrases in a text file or an array and then use $RANDOM to look into that.
Note: remember to escape the $.

Resources