I was wondering is there a way that i can enter
./myscript.sh FILENAME
and the file will link into
for a in $(cat FILENAME) ; do
done
Calling your script with:
./myscript.sh babynames
You can process each line of you file with read:
while read -r line; do
echo "$line"
done < "$1"
$ cat > myscript.sh # create myscript.sh
for i in "$(cat $1)" ; do echo "$i" ; done # in the end CTRL-d
$ cat > babynames # create babynames
primo
secundo # in the end CTRL-d
$ bash myscript.sh babynames # execute the script with babynames as parameter
primo
secundo
Related
From a local machine I am running a shell script on a remote server and passing some arguments to the scripts. Like test.sh "name" "age"
This is my script:
#!/bin/bash
echo $1
echo $2
On the remote server while the script is executing and if I run ps aux | grep .sh i could see the value of the two parameters. Like bash -s name age
Is there a way to suppress or hide the values in the running shell process so that one can see the parameters ?
I have an idea. You could create a global environment variable with unique name and save there the positional arguments, then re-exec your process and get the arguments:
#!/bin/bash
if [[ -z "$MYARGS" ]]; then
export MYARGS="$(printf "%q " "$#")"
exec "$0"
fi
eval set -- "$MYARGS"
printf -- "My arguments:\n"
printf -- "-- %s\n" "$#"
sleep infinity
It will hide it from ps aux:
$ ps aux | grep 1.sh
kamil 196704 0.4 0.0 9768 2084 pts/1 S+ 16:49 0:00 /bin/bash /tmp/1.sh
kamil 196777 0.0 0.0 8924 1640 pts/2 S+ 16:49 0:00 grep 1.sh
The environment variable could be still extracted from /proc:
$ cat /proc/196704/environ | sed -z '/MYARGS/!d'; echo
MYARGS=1 2 3 54 5
Another way might be writing the positional arguments as a string on stdin and pass it to outselves with original input:
#!/bin/bash
if [[ -z "$MYARGS" ]]; then
export MYARGS=1 # just so it's set
# restart outselves with no arguments
exec "$0" < <(
# Stream arguments on stdin on one line
printf "%q " "$#" | xxd -p | tr -d '\n'
echo
exec cat
)
fi
IFS= read -r args # read _one line_ of input - it's our arguments
args=$(xxd -r -p <<<"$args") # encoded with xxd
eval set -- "$args"
printf -- "My arguments:\n"
printf -- "-- %s\n" "$#"
sleep infinity
Here's a way to take the cmd line args and read args from stdin:
#!/usr/bin/env bash
args=()
for arg; do
printf "%d\t%s\n" $((++c)) "$arg"
args+=("$arg")
done
if ! [[ -t 0 ]]; then
while IFS= read -r arg; do
args+=("$arg")
done
fi
declare -p args
Do you can do:
script.sh hello world
printf "%s\n" hello world | script.sh
echo world | script.sh hello
I have an executable that accepts queries from stdin and responds to them, reading until EOF. Additionally I have an input file and a special command, let's call those EXEC, FILE and CMD respectively.
What I need to do is:
Pass FILE to EXEC as input.
Disregard all the output corresponding to commands read from FILE (/dev/null/).
Pass CMD as the last command.
Fetch output for the last command and save it in a variable.
EXEC's output can be multiline for each query.
I know how to pass FILE + CMD into the EXEC:
echo ${CMD} | cat ${FILE} - | ${EXEC}
but I have no idea how to fetch only output resulting from CMD.
Is there a magical one-liner that does this?
After looking around I've found the following partial solution:
mkfifo mypipe
(tail -f mypipe) | ${EXEC} &
cat ${FILE} | while read line; do
echo ${line} > mypipe
done
echo ${CMD} > mypipe
This allows me to redirect my input, but now the output gets printed to screen. I want to ignore all the output produced by EXEC in the while loop and get only what it prints for the last line.
I tried what first came into my mind, which is:
(tail -f mypipe) | ${EXEC} > somefile &
But it didn't work, the file was empty.
This is race-prone -- I'd suggest putting in a delay after the kill, or using an explicit sigil to determine when it's been received. That said:
#!/usr/bin/env bash
# route FD 4 to your output routine
exec 4> >(
output=; trap 'output=1' USR1
while IFS= read -r line; do
[[ $output ]] && printf '%s\n' "$line"
done
); out_pid=$!
# Capture the PID for the process substitution above; note that this requires a very
# new version of bash (4.4?)
[[ $out_pid ]] || { echo "ERROR: Your bash version is too old" >&2; exit 1; }
# Run your program in another process substitution, and close the parent's handle on FD 4
exec 3> >("$EXEC" >&4) 4>&-
# cat your file to FD 3...
cat "$file" >&3
# UGLY HACK: Wait to let your program finish flushing output from those commands
sleep 0.1
# notify the subshell writing output to disk that the ignored input is done...
kill -USR1 "$out_pid"
# UGLY HACK: Wait to let the subprocess actually receive the signal and set output=1
sleep 0.1
# ...and then write the command for which you actually want content logged.
echo "command" >&3
In validating this answer, I'm doing the following:
EXEC=stub_function
stub_function() {
local count line
count=0
while IFS= read -r line; do
(( ++count ))
printf '%s: %s\n' "$count" "$line"
done
}
cat >file <<EOF
do-not-log-my-output-1
do-not-log-my-output-2
do-not-log-my-output-3
EOF
file=file
export -f stub_function
export file EXEC
Output is only:
4: command
You could pipe it into a sed:
var=$(YOUR COMMAND | sed '$!d')
This will put only the last line into the variable
I think, that your proram EXEC does something special (open connection or remember state). When that is not the case, you can use
${EXEC} < ${FILE} > /dev/null
myvar=$(echo ${CMD} | ${EXEC})
Or with normal commands:
# Do not use (printf "==%s==\n" 1 2 3 ; printf "oo%soo\n" 4 5 6) | cat
printf "==%s==\n" 1 2 3 | cat > /dev/null
myvar=$(printf "oo%soo\n" 4 5 6 | cat)
When you need to give all input to one process, perhaps you can think of a marker that you can filter on:
(printf "==%s==\n" 1 2 3 ; printf "%s\n" "marker"; printf "oo%soo\n" 4 5 6) | cat | sed '1,/marker/ d'
You should examine your EXEC what could be used. When it is running SQL, you might use something like
(cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d'
and write this in a var with
myvar=$( (cat ${FILE}; echo 'select "DamonMarker" from dual;' ; echo ${CMD} ) |
${EXEC} | sed '1,/DamonMarker/ d' )
I wrote a script, including this loop:
#!/bin/bash
cat "$1" | while read -r line; do
echo "$line"; sleep 2;
done
A shellcheck run put out the following message:
SC2002: Useless cat. Consider 'cmd < file | ..' or 'cmd file | ..' instead.
I changed the script to:
#!/bin/bash
cmd < "$1" | while read -r line; do
echo "$line"; sleep 2;
done
but now bash exits with:
cmd: command not found
what have I done wrong?
Your cmd is the whole while cond; do ... done compound statement and in this case the redirection needs to come at the end:
while read -r line; do
echo "$line"; sleep 0.2
done < "$1"
Remove the | and have the end line as :
done < "$1"
In a bash script, I see several while statements with those redirect signs when closing the loop.
I know that if I end it with "done < file", I am redirecting the file to the stdin of the command in the while statement. But what the others means?
I would appreciate if someone could give an explanation with examples.
With the file text.txt
1aa
2bb
3cc
Redirection:
$ cat < text.txt
1aa
2bb
3cc
Here document:
$ cat << EOF
> 1AA
> 2BB
> EOF
1AA
2BB
Here string:
$ cat <<< 1aaa
1aaa
The first form, <, is an input redirection. It somewhat different than << and <<< which are two variants of a here document.
The first form, <, is primarily used to redirect the contents of a file to a command or process. It is a named FIFO, and therefor a file that is passed to a command that accepts file arguments.
cmd < file
will open the file named file and create a new file name to open and read. The difference between cmd file and cmd < file is the name passed to cmd in the second case is the name of a named pipe.
You can also do process substitution:
cmd <(process)
An example use would be comparing two directories:
diff <(ls dir1) <(ls dir2)
In this case, the command ls dir1 and ls dir2 has output redirected to a file like stream that is then read by diff as if those were two files.
You can see the name of the file device by passing to echo a process substitution:
$ echo <(ls)
/dev/fd/63
Since echo does not support opening files, it just prints the name of the FIFO.
Here documents are easier to demonstrate. The << form has a 'limit string' that is not included in the output:
$ cat <<HERE
> line 1
> line 2
> line 3
> HERE
line 1
line 2
line 3
The HERE is a unique string that must be on its own line.
The 'here string' or <<< form does not require the delimiting string of the << form and is on a single line:
$ cat <<< 'line 1'
line 1
You can also expand parameters:
$ v="some text"
$ cat <<< "$v"
some text
But not other forms of shell expansions:
Brace expansion:
$ echo a{b,c,d}e
abe ace ade
$ cat <<< a{b,c,d}e
a{b,c,d}e
Given a 'generic' Bash while loop that reads input line by line:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done
There are several ways that you can feed input into that loop.
First example, you can redirect a file. For demo, create a 6 line file:
$ seq 6 > /tmp/6.txt
Redirect the input of the file into the loop:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done </tmp/6.txt
'1'
'2'
'3'
'4'
'5'
'6'
Or, second example, you can directly read from the output of seq using redirection:
$ while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done < <(seq 3)
'1'
'2'
'3'
(Please note the extra < with a space for this form)
Or, third example, you can use a 'HERE' doc separated by CR:
while IFS= read -r line || [[ -n $line ]]; do printf "'%s'\n" "$line"; done <<HERE
1
2 3
4
HERE
'1 '
'2 3'
' 4'
Going back to diff which will only work on files, you can use process substitution and a HERE doc or process substitution and redirection to use diff on free text or the output of a program.
Given:
$ cat /tmp/f1.txt
line 1
line 2
line 3
Normally you would need to have a second file to compare free text with that file. You can use a HERE doc and process substitution to skip creating a separate file:
$ diff /tmp/f1.txt <(cat <<HERE
line 1
line 2
line 5
HERE
)
3c3
< line 3
---
> line 5
command < foo
Redirect the file foo to the standard input of command.
command << foo
blah 1
blah 2
foo
Here document: send the following lines up to foo to the standard input of command.
command <<< foo
Here-string. The string foo is sent to the standard input of command.
I am creating a shell script. Now, I want to create a flag to print the output of script on the screen if flag is ON otherwise script will not print the output if flag is OFF
Thanks
This might work for you:
#!/bin/bash
# assuming your first argument is the printing flag
[[ "${1}" = "ON" ]] && OUTPUT="/dev/stdout" || OUTPUT="/dev/null"
# from now on:
echo "Something" > $OUTPUT
# will work as expected...
test.sh code below:
#!/bin/sh
while IFS= read -r line
do
cat "$line"
done < $1
Test it:
$ ls
myflags testfile0 testfile1 testfile2 test.py test.sh
$ cat myflags
testfile0
testfile1
test.py
$ cat testfile0
some test
$ sh test.sh myflags
some test
#!/usr/bin/python
import sys
if sys.version_info[0] == 2:
sys.stdout.write("ls -l")
$