Good day.
At work i have a script that receives a number as an argument and it shows me some information related to that number as an output.
The thing is some times i receive a huge list of numbers and executing this script numerous times during the day takes a lot of my time.
I would like to know if there is a way to save all those numbers on a file and direct each line of said file to be executed by the script.
Lets say my file contains the following numbers:
999999999
888888888
777777777
666666666
555555555
and the script is executed as follows:
sh script.sh 55(Number) go
Thanks in advance.
Assuming your inputs are in input.txt, a simple while read loop will do:
while read -r line <&3; do
sh script.sh "55${line}" go
done 3<input.txt
See BashFAQ #1 for discussion of this technique.
A few notes:
sh script.sh is actually very poor form: It ignores the shebang line in that script, such that it can no longer specify its own interpreter. It's much better to run ./script.sh after making your script executable; that way if it has #!/usr/bin/env bash or #!/usr/bin/ksh or any other shebang that will be honored, so you don't risk causing your script to fail by running it with a POSIX sh interpreter when it was written for a more capable shell language.
The 3<input.txt puts the input file on descriptor 3, rather than the default <input.txt redirection of using stdin; using an alternate file descriptor number ensures that your script can read from the user (prompting from stdin), if it wants to.
One way to accomplish that is using xargs, for example:
xargs -I{} sh script.sh 55{} go < numbers.txt
This assumes that you put your numbers in a file named numbers.txt.
For each line in the file, the {} in sh script.sh 55{} go will be replaced with the value in the line.
For example:
xargs -I{} echo number is: {} < input
Given your sample input, the above will produce:
number is: 999999999
number is: 888888888
number is: 777777777
number is: 666666666
number is: 555555555
Related
I have a simple Bash script:
#!/usr/bin/env bash
read X
echo "X=$X"
When I execute it with ./myscript.sh it works. But when I execute it with cat myscript.sh | bash it actually puts echo "X=$X" into $X.
So this script prints Hello World executed with cat myscript.sh | bash:
#!/usr/bin/env bash
read X
hello world
echo "$X"
What's the benefit of executing a script with cat myscript.sh | bash? Why doesn't do it the same things as if I execute it with ./myscript.sh?
How can I avoid Bash to execute line by line but execute all lines after the STDIN reached the end?
Instead of just running
read X
...instead replace it with...
read X </dev/tty || {
X="some default because we can't read from the TTY here"
}
...if you want to read from the console. Of course, this only works if you have a /dev/tty, but if you wanted to do something robust, you wouldn't be piping from curl into a shell. :)
Another alternative, of course, is to pass in your value of X on the command line.
curl https://some.place/with-untrusted-code-only-idiots-will-run-without-reading \
| bash -s "value of X here"
...and refer to "$1" in your script when you want X.
(By the way, I sure hope you're at least using SSL for this, rather than advising people to run code they download over plain HTTP with no out-of-band validation step. Lots of people do it, sure, but that's making sites they download from -- like rvm.io -- big targets. Big, easy-to-man-in-the-middle-or-DNS-hijack targets).
When you cat a script to bash the code to execute is coming from standard input.
Where does read read from? That's right also standard input. This is why you can cat input to programs that take standard input (like sed, awk, etc.).
So you are not running "a script" per-se when you do this. You are running a series of input lines.
Where would you like read to read data from in this setup?
You can manually do that (if you can define such a place). Alternatively you can stop running your script like this.
I have a simple Bash script:
#!/usr/bin/env bash
read X
echo "X=$X"
When I execute it with ./myscript.sh it works. But when I execute it with cat myscript.sh | bash it actually puts echo "X=$X" into $X.
So this script prints Hello World executed with cat myscript.sh | bash:
#!/usr/bin/env bash
read X
hello world
echo "$X"
What's the benefit of executing a script with cat myscript.sh | bash? Why doesn't do it the same things as if I execute it with ./myscript.sh?
How can I avoid Bash to execute line by line but execute all lines after the STDIN reached the end?
Instead of just running
read X
...instead replace it with...
read X </dev/tty || {
X="some default because we can't read from the TTY here"
}
...if you want to read from the console. Of course, this only works if you have a /dev/tty, but if you wanted to do something robust, you wouldn't be piping from curl into a shell. :)
Another alternative, of course, is to pass in your value of X on the command line.
curl https://some.place/with-untrusted-code-only-idiots-will-run-without-reading \
| bash -s "value of X here"
...and refer to "$1" in your script when you want X.
(By the way, I sure hope you're at least using SSL for this, rather than advising people to run code they download over plain HTTP with no out-of-band validation step. Lots of people do it, sure, but that's making sites they download from -- like rvm.io -- big targets. Big, easy-to-man-in-the-middle-or-DNS-hijack targets).
When you cat a script to bash the code to execute is coming from standard input.
Where does read read from? That's right also standard input. This is why you can cat input to programs that take standard input (like sed, awk, etc.).
So you are not running "a script" per-se when you do this. You are running a series of input lines.
Where would you like read to read data from in this setup?
You can manually do that (if you can define such a place). Alternatively you can stop running your script like this.
I am trying to read contents of a file given from standard input into a script. Any ideas how to do that?
Basically what I want is:
someScript.ksh < textFile.txt
Inside the ksh, I am using a binary which will read data from "textFile.txt" if the file is given on the standard input.
Any ideas how do I "pass" the contents of the given input file, if any, to another binary inside the script?
You haven't really given us enough information to answer the question, but here are a few ideas.
If you have a script that you want to accept data on stdin, and that script calls something else that expects data to be passed in as a filename on the command line, you can take stdin and dump it to a temporary file. Something like:
#!/bin/sh
tmpfile=$(mktemp tmpXXXXXX)
cat > $tmpfile
/some/other/command $tmpfile
rm -f $tmpfile
(In practice, you would probably use trap to clean up the temporary file on exit).
If instead the script is calling another command that also expects input on stdin, you don't really have to do anything special. Inside your script, stdin of anything you call will be connected to stdin of the calling script, and as long as you haven't previously consumed the input you should be all set.
E.g., given a script like this:
#!/bin/sh
sed s/hello/goodbye/
I can run:
echo hello world | sh myscript.sh
And get:
goodbye world
I'm writing a bash script called 'run' that tests programs with pre-defined inputs.
It takes in a file as the first parameter, then a program as a second parameter.
The call would look like
./run text.txt ./check
for example, the program 'run' would run 'check' with text.txt as the input. This will save me lots of testing time with my programs.
right now I have
$2 < text.txt > text.created
So it takes the text.txt and redirects it as input in to the program specified, which is the second argument. Then dumps the result in text.created.
I have the input in text.txt and I know what the output should look like, but when I cat text.created, it's empty.
Does anybody know the proper way to run a program with a file as the input? This seems intuitive to me, but could there be something wrong with the 'check' program rather than what I'm doing in the 'run' script?
Thanks! Any help is always appreciated!
EDIT: the file text.txt contains multiple lines of files that each have an input for the program 'check'.
That is, text.txt could contain
asdf1.txt
asdf2.txt
asdf3.txt
I want to test check with each file asdf1.txt, asdf2.txt, asdf3.txt.
A simple test with
#!/bin/sh
# the whole loop reads $1 line by line
while read
do
# run $2 with the contents of the file that is in the line just read
xargs < $REPLY $2
done < $1
works fine. Call that file "run" and run it with
./run text.txt ./check
I get the program ./check executed with text.txt as the parameters. Don't forget to chmod +x run to make it executable.
This is the sample check program that I use:
#!/bin/sh
echo "This is check with parameters $1 and $2"
Which prints the given parameters.
My file text.txt is:
textfile1.txt
textfile2.txt
textfile3.txt
textfile4.txt
and the files textfile1.txt, ... contain one line each for every instance of "check", for example:
lets go
or
one two
The output:
$ ./run text.txt ./check
This is check with parameters lets and go
This is check with parameters one and two
This is check with parameters uno and dos
This is check with parameters eins and zwei
The < operator redirects the contents of the file to the standard input of the program. This is not the same as using the file's contents for the arguments of the file--which seems to be what you want. For that do
./program $(cat file.txt)
in bash (or in plain old /bin/sh, use
./program `cat file.txt`
).
This won't manage multiple lines as separate invocations, which your edit indicates is desired. For that you probably going to what some kind scripting language (perl, awk, python...) which makes parsing a file linewise easy.
I'm not familiar with shell scripting, so I'm not sure how to do it or if it is possible. If you can give me links or advice, that would be great.
What I want to do is:
Create a file, simple text file
EX:
param1 (RANDOMVALUE)
Where randomvalue is a random number generated.
run a program with that file we just created and output to a file
./program filewejustcreated > A
The program has already been created and it takes a filename as a parameter, no need to worry about that stuff.
run another program with the file we just created, the program already exists and out put it to a file
./Anotherprogram filewejustcreated > B
run a diff comamand on A, B
diff A B
Display what diff returns...
Thanks
[Edit] I am using shell: tcsh
I am not sure about the function to generate a random number in tcsh. However, in a more common shell like BASH, references to the variable $RANDOM, generates random numbers.
So, in your shell script (a BASH shell script here), the contents would be:
#Pick the first argument to the call as the file name
FILE_NAME=shift
echo "param1 $RANDOM" > $FILE_NAME
./program $FILE_NAME > $FILE1
./Anotherprogram $FILE_NAME > $FILE2
diff $FILE1 $FILE2
You have almost written the script already. The only missing thing is the random number; I'll do it with Perl. Here is a quick & dirty solution in sh (or bash; I'm presuming you're on a Linux/Unix system):
#!/bin/sh
perl -e 'print "TheWord (", int(rand(1000)), ")\n"' > tempfile
./program tempfile > A
./Anotherprogram tempfile > B
# rm tempfile # this would delete 'tempfile' if uncommented
diff A B
Now save this in a file (say, script.sh) and in a shell execute:
chmod +x script.sh
to make it executable, and
./script.sh
to run it.
Shell scripting is mostly just putting together different programs in ways that get the job done. There are a lot of programs that do just one simple thing and can be combined to accomplish larger tasks that you will learn of as you get into the shell scripting world.
An example of a large shell script is perl's Configure script. In the first bit you see (along with some humorous comments) cat, true, sh, rm, test, sed, uname, and grep used.