I write many simple scripts for testing or processing CSV files. Most of scripts have same logic: read one file and write the output to a console or another file. For such case I usually use the simplest for implementation approach: read from stdin, write to stdout, so invocation is like:
script < input.csv > output.csv
Can you give some points why I shall prefer this style:
script input.csv > output.csv
Or even:
script input.csv output.csv
Here's how you can have a script that handles both (not guaranteed to be 100% safe, maybe someone can give better options or pinpoint serious issues in a comment):
#!/bin/bash
if [[ -n $1 ]]; then
exec "$0" < "$1" || exit $?
fi
# your program starts here, reading stdin, e.g.,
while read; do
echo "$REPLY"
done
This is how I deal with the same task now, similar to what #gniourf_gniourf's answer does (except my scripts are in Haskell).
When the program is invoked with arguments, like CMD INPUT1 [INPUT2 [...]], treat them as input files and ignore standard input, except for special case when one or more args are -.
When no args are given, read standard input. So CMD call without args is equivalent to CMD -.
Related
I am creating a script (myscript.sh) in BASH that reads from STDOUT, typically a stream of data that comes from cat, or from a file and outputs the stream of data (amazing!), like this:
$cat myfile.txt
hello world!
$cat myfile.txt | myscript.sh
hello world!
$myscript.sh myfile.txt
hello world!
But I also would like the following behaviour: if I call the script without arguments I'd like it to output a brief help:
$myscript.sh
I am the help: I just print what you say.
== THE PROBLEM ==
The problem is that I am capturing the stream of data like this:
if [[ $# -eq 0 ]]; then
stream=$(cat <&0)
elif [[ -n "$stream" ]]; then
echo "I am the help: I just print what you say."
else
echo "Unknown error."
fi
And when I call the script with no arguments like this:
$myscript.sh
It SHOULD print the "help" part, but it just keep waiting for a stream of data in line 2 of code above...
Is there any way to tell bash that if nothing comes from STDOUT just break and continue executing?
Thanks in advance.
There's always a standard input stream; if no arguments are given and input isn't redirected, standard input is the terminal.
If you want to treat that specially, use test -t to test if standard input is connected to a terminal.
if [[ $# -eq 0 && -t 0 ]]; then
echo "I am the help: I just print what you say."
else
stream=$(cat -- "$#")
fi
There's no need to test $#. Just pass your arguments to cat; if it gets filenames it will read from them, otherwise it will read from standard input.
I agree to #Barmar's solution.
However, it might be better to entirely avoid a situation where your program behavior depends on whether the input file descriptor is a terminal (there are situations where a terminal is mimicked even though there's none -- in such a situation, your script would just produce the help string).
You could instead introduce a special - argument to explicitly request reading from stdin. This will result in simpler option handling and uniform behavior of your script, no matter what's the environment.
First answer is to help yourself - try running the script with bash -x myscript.sh. It will include lot of information to help you.
If you specific case, the condition $# -eq 0 was flipped. As per requirement, you want to print the help message is NOT ARGUMENT ARE PROVIDED:
if [[ $# -eq 0 ]] ; then
echo "I am the help: I just print what you say."
exit 0
fi
# Rest of you script, read data from file, etc.
cat -- "$#"
Assuming this approach is taken, and if you want to process standard input or a file, simple pass '-' as parameter: cat foobar.txt | myscript.sh -
I need to read command line arguments. First arg is script name. second one is redirection operator i.e. "<" and third one is input filename. When I tried to use "$#", I got 0. When I used "$*", it gave me nothing. I have to use "<" this operator. My input file consists of all user input data. If I don't use the operator, It asks user for the input. Can someone please help me? Thank you !
Command Line :
./script_name < input_file
Script:
echo "$*" # gave nothing
echo "$#" # gave me 0
I need to read input filename and store it to some variable. Then I have to change the extension of it. Any help/suggestions should be appreciated.
When a user runs:
./script_name <input_file
...that's exactly equivalent to if they did the following:
(exec <input_file; exec ./script_name)
...first redirecting stdin from input_file, then invoking the script named ./script_name without any arguments.
There are operating-system-specific interfaces you can use to get the filename associated with a handle (when it has one), but to use one of these would make your script only able to run on an operating system providing that interface; it's not worth it.
# very, very linux-specific, won't work for "cat foo | ./yourscript", generally evil
if filename=$(readlink /proc/self/fd/0) && [[ -e $filename ]]; then
set -- "$#" "$filename" # append filename to the end of the argument list
fi
If you want to avoid prompting for input when an argument is given, and to have the filename of that argument, then don't take it on stdin but as an argument, and do the redirection yourself within the script:
#!/bin/bash
if [[ $1 ]]; then
exec <"$1" # this redirects your stdin to come from the file
fi
# ...put other logic here...
...and have users invoke your script as:
./script_name input_file
Just as ./yourscript <filename runs yourscript with the contents of filename on its standard input, a script invoked with ./yourscript filename which invokes exec <"$1" will have the contents of filename on its stdin after executing that command.
< is used for input redirection. And whatever is at the right side of < is NOT a command line argument.
So, when you do ./script_name < input_file , there will be zero (0) command line arguments passed to the script, hence $# will be zero.
For your puprpose you need to call your script as:
./script_name input_file
And in your script you can change the extension with something like:
mv -- "$1" "${1}_new_extension"
Edit: This was not what OP wanted to do.
Altough, there is already another spot on answer, I will write this for the sake of completeness. If you have to use the '<' redirection you can do something like this in your script.
while read filename; do
mv -- "$filename" "${filename}_bak"
done
And call the script as, ./script < input_file. However, note that you will not be able to take inputs from stdin in this case.
Unfortunately, if you're hoping to take redirection operators as arguments to your script, you're not going to be able to do that without surrounding your command line arguments in quotes:
./script_name "<input_file"
The reason for this is that the shell (at least bash or zsh) processes the command before ever invoking your script. When the shell interprets your command, it reads:
[shell command (./script_name)][shell input redirection (<input_file)]
invoking your script with quotes effectively results in:
[shell command (./script_name)][script argument ("<input_file")]
Sorry this is a few years late; hopefully someone will find this useful.
I want to make sure my script will work when the user uses a syntax like this:
script.sh firstVariable < SecondVariable
For some reason I can't get this to work.
I want $1=firstVariable
And $2=SecondVariable
But for some reason my script thinks only firstVariable exists?
This is a classic X-Y problem. The goal is to write a utility in which
utility file1 file2
and
utility file1 < file2
have the same behaviour. It seems tempting to find a way to somehow translate the second invocation into the first one by (somehow) figuring out the "name" of stdin, and then using that name the same way as the second argument would be used. Unfortunately, that's not possible. The redirection happens before the utility is invoked, and there is no portable way to get the "name" of an open file descriptor. (Indeed, it might not even have a name, in the case of other_cmd | utility file1.)
So the solution is to focus on what is being asked for: make the two behaviours consistent. This is the case with most standard utilities (grep, cat, sort, etc.): if the input file is not specified, the utility uses stdin.
In many unix implementations, stdin does actually have a name: /dev/stdin. In such systems, the above can be achieved trivially:
utility() {
utility_implementation "$1" "${2:-/dev/stdin}"
}
where utility_implementation actually does whatever is required to be done. The syntax of the second argument is normal default parameter expansion; it represents the value of $2 if $2 is present and non-empty, and otherwise the string /dev/stdin. (If you leave out the - so that it is "${2:/dev/stdin}", then it won't do the substitution if $2 is present and empty, which might be better.)
Another way to solve the problem is to ensure that the first syntax becomes the same as the second syntax, so that the input is always coming from stdin even with a named file. The obvious simple approach:
utility() {
if (( $# < 2 )); then
utility_implementation "$1"
else
utility_implementation "$1" < "$2"
fi
}
Another way to do this uses the exec command with just a redirection to redirect the shell's own stdin. Note that we have to do this inside a subshell ((...) instead of {...}) so that the redirection does not apply to the shell which invokes the function:
utility() (
if (( $# > 1 )) then; exec < "$2"; fi
# implementation goes here. $1 is file1 and stdin
# is now redirected to $2 if $2 was provided.
# ...
)
To make the stdin of the second variable the final argument to the script(so if you have one arg then < second arg, it will be the second), you can use the below
#!/bin/bash
##read loop to read in stdin
while read -r line
do
## This just checks if the variable is empty, so a newline isn't appended on the front
[[ -z $Vars ]] && Vars="$line" && continue
## Appends every line read to variable
Vars="$Vars"$'\n'"$line"
## While read loop using stdin
done < /dev/stdin
##Set re-sets the arguments to the script to the original arguments and then the new argument we derived from stdin
set - "$#" "$Vars"
## Echo the new arguments
echo "$#"
I am trying to read contents of a file given from standard input into a script. Any ideas how to do that?
Basically what I want is:
someScript.ksh < textFile.txt
Inside the ksh, I am using a binary which will read data from "textFile.txt" if the file is given on the standard input.
Any ideas how do I "pass" the contents of the given input file, if any, to another binary inside the script?
You haven't really given us enough information to answer the question, but here are a few ideas.
If you have a script that you want to accept data on stdin, and that script calls something else that expects data to be passed in as a filename on the command line, you can take stdin and dump it to a temporary file. Something like:
#!/bin/sh
tmpfile=$(mktemp tmpXXXXXX)
cat > $tmpfile
/some/other/command $tmpfile
rm -f $tmpfile
(In practice, you would probably use trap to clean up the temporary file on exit).
If instead the script is calling another command that also expects input on stdin, you don't really have to do anything special. Inside your script, stdin of anything you call will be connected to stdin of the calling script, and as long as you haven't previously consumed the input you should be all set.
E.g., given a script like this:
#!/bin/sh
sed s/hello/goodbye/
I can run:
echo hello world | sh myscript.sh
And get:
goodbye world
Is there any bash trick that allows giving some parameters in command line to a program that gets its inputs via input stream? Something like this:
program < 'a=1;b=a*2;'
but < needs a file input stream.
For very short here-documents, there are also here-strings:
program <<< "a=1;b=a*2"
I think
echo 'a=1;b=a*2;' | program
is what you need. This process is called "piping"
As a side note: doing the opposite (i.e. piping other programs output as arguments) could be done with xargs
echo works great. The other answer is Here-documents [1]
program <<EOF
a=1;b=a*2;
EOF
I use echo when I have one very short thing on one line, and heredocs when I have something that requires newlines.
[1] http://tldp.org/LDP/abs/html/here-docs.html
shopt -s expand_aliases
alias 'xscript:'='<<:ends'
xscript: bc | anotherprog | yetanotherprog ...
a=1;b=a*2;
:ends
Took me a year to hack this one out. Premium bash script here fellas. Give respect where due please :)
I call this little 'diddy' xscript because you can expand bash variables and substitutions inside of the here document.
alias 'script:'='<<":ends"'
The above version does not expand substitutions.
xscript: cat
The files in our path are: `ls -A`
:ends
script: cat
The files in our path are: `ls -A`
:ends
I'm not finished!
source <(xscript: cat
echo \$BASH "hello world, I'mma script genius!"
echo You can thank me now $USER
:ends
)