I have a command that I need to use repeatedly within a shell script. This is command contains a pipe and the output of the whole command will be piped to other commands.
e.g. Let's say for simplicity sake the command is ls | tee. Then I might pipe it to other commands, says ls | tee | someprogram or ls | tee | anotherprogram.
So naturally I'll want to keep ls | tee is a variable. The problem is that I can't seem to execute a variable with a pipe in it.
#!/bin/sh
TEST="ls | tee"
$TEST
Gives the following output
ls: cannot access |: No such file or directory
ls: cannot access tee: No such file or directory
How do I execute a variable like $TEST above, whist being able to pipe the output to other commands?
The short answer is eval.
eval $TEST somefile
eval $TEST otherfile | more
However, you need to be aware that eval means problems with quoting special characters and blanks and the like. If everything is simple (TEST="ls -l | tee"), then it is easy. If you have spaces in arguments or shell metacharacters, then it is hard — very hard — to do it right. At that point, you'd be better off creating a function or separate shell script.
You might well be better off with a function or shell script even so.
If the string you eval comes from a user, you have to worry even more!
Related
Take the following example:
ls -l | grep -i readme | ./myscript.sh
What I am trying to do is get ls -l | grep -i readme as a string variable in myscript.sh. So essentially I am trying to get the whole command before the last pipe to use inside myscript.sh.
Is this possible?
No, it's not possible.
At the OS level, pipelines are implemented with the mkfifo(), dup2(), fork() and execve() syscalls. This doesn't provide a way to tell a program what the commands connected to its stdin are. Indeed, there's not guaranteed to be a string representing a pipeline of programs being used to generate stdin at all, even if your stdin really is a FIFO connected to another program's stdout; it could be that that pipeline was generated by programs calling execve() and friends directly.
The best available workaround is to invert your process flow.
It's not what you asked for, but it's what you can get.
#!/usr/bin/env bash
printf -v cmd_str '%q ' "$#" # generate a shell command representing our arguments
while IFS= read -r line; do
printf 'Output from %s: %s\n' "$cmd_str" "$line"
done < <("$#") # actually run those arguments as a command, and read from it
...and then have your script start the things it reads input from, rather than receiving them on stdin.
...thereafter, ./yourscript ls -l, or ./yourscript sh -c 'ls -l | grep -i readme'. (Of course, never use this except as an example; see ParsingLs).
It can't be done generally, but using the history command in bash it can maybe sort of be done, provided certain conditions are met:
history has to be turned on.
Only one shell has been running, or accepting new commands, (or failing that, running myscript.sh), since the start of myscript.sh.
Since command lines with leading spaces are, by default, not saved to the history, the invoking command for myscript.sh must have no leading spaces; or that default must be changed -- see Get bash history to remember only the commands run with space prefixed.
The invoking command needs to end with a &, because without it the new command line wouldn't be added to the history until after myscript.sh was completed.
The script needs to be a bash script, (it won't work with /bin/dash), and the calling shell needs a little prep work. Sometime before the script is run first do:
shopt -s histappend
PROMPT_COMMAND="history -a; history -n"
...this makes the bash history heritable. (Code swiped from unutbu's answer to a related question.)
Then myscript.sh might go:
#!/bin/bash
history -w
printf 'calling command was: %s\n' \
"$(history | rev |
grep "$0" ~/.bash_history | tail -1)"
Test run:
echo googa | ./myscript.sh &
Output, (minus the "&" associated cruft):
calling command was: echo googa | ./myscript.sh &
The cruft can be halved by changing "&" to "& fg", but the resulting output won't include the "fg" suffix.
I think you should pass it as one string parameter like this
./myscript.sh "$(ls -l | grep -i readme)"
I think that it is possible, have a look at this example:
#!/bin/bash
result=""
while read line; do
result=$result"${line}"
done
echo $result
Now run this script using a pipe, for example:
ls -l /etc | ./script.sh
I hope that will be helpful for you :)
In my program I need to know the maximum number of process I can run. So I write a script. It works when I run it in shell but but when in program using system("./limit.sh"). I work in bash.
Here is my code:
#/bin/bash
LIMIT=\`ulimit -u\`
ACTIVE=\`ps -u | wc -l \`
echo $LIMIT > limit.txt
echo $ACTIVE >> limit.txt
Anyone can help?
Why The Original Fails
Command substitution syntax doesn't work if escaped. When you run:
LIMIT=\`ulimit -u\`
...what you're doing is running a command named
-u`
...with the environment variable named LIMIT containing the value
`ulimit
...and unless you actually have a command that starts with -u and contains a backtick in its name, this can be expected to fail.
This is because using backticks makes characters which would otherwise be syntax into literals, and running a command with one or more var=value pairs preceding it treats those pairs as variables to export in the environment for the duration of that single command.
Doing It Better
#!/bin/bash
limit=$(ulimit -u)
active=$(ps -u | wc -l)
printf '%s\n' "$limit" "$active" >limit.txt
Leave off the backticks.
Use modern $() command substitution syntax.
Avoid multiple redirections.
Avoid all-caps names for your own variables (these names are used for variables with meaning to the OS or system; lowercase names are reserved for application use).
Doing It Right
#!/bin/bash
exec >limit.txt # open limit.txt as output for the rest of the script
ulimit -u # run ulimit -u, inheriting that FD for output
ps -u | wc -l # run your pipeline, likewise with output to the existing FD
You have a typo on the very first line: #/bin/bash should be #!/bin/bash - this is often known as a "shebang" line, for "hash" (#) + "bang" (!)
Without that syntax written correctly, the script is run through the system's default shell, which will see that line as just a comment.
As pointed out in comments, that also means only the standardised options available to the builtin ulimit command, which doesn't include -u.
I have made a program that acts as a shell and for testing purposes, I try and use the < operator, but receive this error in my bash
the purpose is to take ls as an input and run it in my mock shell
Is there a specific reason I am receiving this error, could it be from my code?
the name of my program is rshell:
[xx#xx rshell]$ ./bin/rshell < ls
bash: ls: No such file or directory
The right-hand side of the input redirection operator < requires a filename. So the shell interprets "ls" in your example as a filename. You get an error because there is no such file.
If you want to pass the output of ls to your shell, use a pipe:
ls | ./bin/rshell
Or process substitution:
./bin/rshell < <(ls)
If you want to pass the text "ls" on standard input to your shell, use a here-string:
./bin/rshell <<< ls
The syntax you have, ./bin/rshell < ls, means "read from a file named 'ls'".
If you are trying to send the output of the ls command into /bin/rshell, you need to either pipeline it with a pipe:
ls | ./bin/rshell
or read from an anonymous filehandle:
./bin/rshell < <(ls)
Basically, > and < are for dealing with files (file i/o), and | is what you want for dealing with passing output from one command to another (command i/o); but, you can combine the two with anonymous filehandles using the <(command) syntax.
Inside of a shell script, you can do fancier things, such as storing the output of a command in a string variable, then using that string as input, parsing it up, and all kinds of fun stuff. First things first, though.
An excellent resource for learning all things bash is the Advanced Bash-Scripting Guide: http://www.tldp.org/LDP/abs/html/index.html
I'm working with Mac OS X's pbpaste command, which returns the clipboard's contents. I'd like to create a shell script that executes each line returned by pbpaste as a separate bash command. For example, let's say that the clipboard's contents consists of the following lines of text:
echo 1234 >~/a.txt
echo 5678 >~/b.txt
I would like a shell script that executes each of those lines, creating the two files a.txt and b.txt in my home folder. After a fair amount of searching and trial and error, I've gotten to the point where I'm able to assign individual lines of text to a variable in a while loop with the following construct:
pbpaste | egrep -o [^$]+ | while read l; do echo $l; done
which sends the following to standard out, as expected:
echo 1234 >~/a.txt
echo 5678 >~/b.txt
Instead of simply echoing each line of text, I then try to execute them with the following construct:
pbpaste | egrep -o [^$]+ | while read l; do $l; done
I thought that this would execute each line (thus creating two text files a.txt and b.txt in my home folder). Instead, the first term (echo) seems to be interpreted as the command, and the remaining terms (nnnn >~/...) seem to get lumped together as if they were a single parameter, resulting in the following being sent to standard out without any files being created:
1234 >~/a.txt
5678 >~/b.txt
I would be grateful for any help in understanding why my construct isn't working and what changes might get it to work.
[…] the remaining terms (nnnn >~/...) seem to get lumped together as if they were a single parameter, […]
Not exactly. The line actually gets split on whitespace (or whatever $IFS specifies), but the problem is that the redirection operator > cannot be taken from a shell variable. For example, this snippet:
gt='>'
echo $gt foo.txt
will print > foo.txt, rather than printing a newline to foo.txt.
And you'll have similar problems with various other shell metacharacters, such as quotation marks.
What you need is the eval builtin, which takes a string, parses it as a shell command, and runs it:
pbpaste | egrep -o [^$]+ | while IFS= read -r LINE; do eval "$LINE"; done
(The IFS= and -r and the double-quotes around $LINE are all to prevent any other processing besides the processing performed by eval, so that e.g. whitespace inside quotation marks will be preserved.)
Another possibility, depending on the details of what you need, is simply to pipe the commands into a new instance of Bash:
pbpaste | egrep -o [^$]+ | bash
Edited to add: For that matter, it occurs to me that you can pass everything to eval in a single batch; just as you can (per your comment) write pbpaste | bash, you can also write eval "$(pbpaste)". That will support multiline while-loops and so on, while still running in the current shell (useful if you want it to be able to reference shell parameters, to set environment variables, etc., etc.).
I'm trying to write a bash script which would locate a single file in the current directory. The file will be used later but I don't need help there. I tried using ls and grep but it doesn't work, I'm a newbie using bash.
#!/bin/sh
#Here I need smt like
#trFile = ls | grep myString (but I get file not found error)
echo $trFile
Use shell wildcards, as in
ls *${pattern}*
And, to store the result in a variable, put it inside a $() structure (you can also use deprecated backticks if you like using deprecated functionality that doesn't nest well)
var=$( ls *${pattern}* )
Or, put your ls | grep in there (but that's bad practice, IMHO):
var=$( ls | grep -- "$pattern" )
#!/bin/sh
#
trfile=$( ls | grep myString )
echo $trfile
The $( xxx ) causes the commands within to be executed and the output returned.
I believe you are looking for something like this:
#!/bin/sh
trFile=`ls | grep "$myString"`
In order to run a command and redirect/store its output, you need put the command between backticks. The variable that will store the output, equal sign and the backtick need to be together, as in my example.
Hope this helps.
Try this, if I guess what you're trying to do is, get the capture of the filename from grepping via the output of the ls into a shell variable, try this:
#!/bin/sh
trFile=`ls | grep "name_of_file"`
echo $trFile
Notice the usage of the back-tick operator surrounding the command, what-ever is the output, in this case, will get captured.
using output of ls will bite when you least expect. Better use Globbing.
http://tldp.org/LDP/abs/html/globbingref.html