Suppose I have the following shell program.
#!/bin/sh
FOO="foo"
echo $FOO | cat
I want to generate another shell program that does the same thing as this one, except that all shell variables have been substituted. For example,
#!/bin/sh
echo "foo" | cat
I know that I can get close if I run the above program using #!/bin/sh -x, but that output does not preserve redirections. Instead, I get
+ FOO=foo
+ echo foo
+ cat
foo
Any ideas?
The following shell:
$ cat eval.sh
echo "#!/bin/sh"
FOO="foo"
echo "echo $FOO | cat"
will write a shell:
$ sh eval.sh
#!/bin/sh
echo foo | cat
which does what you need.
Related
I'm struggling to understand the difference between | and > operators
I've looked in places like:
https://www.gnu.org/software/bash/manual/html_node/Redirections.html
and
Pipe vs redirect into process
But can't make enough sense of the explanations.
Here is my practical example:
test-a.sh:
alias testa='echo "alias testa here"'
echo "testa echo"
echo "testa echo2"
test-b.sh:
alias testb='echo "alias testb here"'
echo "testa echo"
echo "testa echo2"
test-pipes.sh:
function indent() {
input=$(cat)
echo "$input" | perl -p -e 's/(.*)/ \1/'
}
source test-a.sh | indent
testa
source test-b.sh > >(indent)
testb
output:
$ source test-pipes.sh
testa echo
testa echo2
test-pipes.sh:10: command not found: testa
testa echo
testa echo2
alias testb here
Piping doesn't allow the alias to be set in the current process, but the redirection does.
Can someone give a simple explanation?
From the bash man page:
Each command in a pipeline is executed as a separate process (i.e., in a sub‐shell).
Many things child processes do are isolated from the parent. Among the list are: changing the current directory, setting shell variables, setting environment variables, and aliases.
$ alias foo='echo bar' | :
$ foo
foo: command not found
$ foo=bar | :; echo $foo
$ export foo=bar | :; echo $foo
$ cd / | :; $ pwd
/home/jkugelman
Notice how none of the changes took effect. You can see the same thing with explicit subshells:
$ (alias foo='echo bar')
$ foo
foo: command not found
$ (foo=bar); echo $foo
$ (export foo=bar); echo $foo
$ (cd /); pwd
/home/jkugelman
Redirections, on the other hand, do not create subshells. They merely change where the input and output of a command go. The same goes with function calls. Functions are executed in the current shell, no subshell, so they're able to create aliases.
I am trying to run a command with a variable which holds another command that suppresses warning messages of the jar. However, it is not working as expected and I can't figure out what I am doing wrong.
TEST=${TEST:-2> /dev/null}
java -jar ~/bin/aw.jar ${Test}
Redirection is performed only if it is unquoted and present in the command line literally rather than originating from any kind of expansion:
$ ls
$ echo hello >out
$ ls
out
$ cat out
hello
$ rm *
$ echo hello '>out'
hello >out
$ ls
$ x='>out'
$ echo hello $x
hello >out
$ ls
In order to interpret redirection operator coming from a quoted string or an expansion you must execute the command through eval (note, however, that this may result in undesired expansions in other parts of your command):
$ ls
$ x='>out'
$ eval echo hello $x
$ ls
out
$ cat out
hello
How do I use the file redirected to the standard input of a bash script?
$ script.sh < file_to_use_in_script
what do I have to put in my script, so that I can write the filename in a variable from this input, without knowing the pathname beforehand.
FILENAME=$file_to_use_in_script
You can use the filename /dev/stdin. Make sure to only read it once.
$ cat myscript
#!/bin/bash
file=${1:-/dev/stdin}
echo "Reading from $file"
nl "$file"
$ cat myfile
hello world
$ ./myscript myfile
Reading from myfile
1 hello world
$ ./myscript < myfile
Reading from /dev/stdin
1 hello world
$ echo "something" | ./myscript
Reading from /dev/stdin
1 something
i would suggest to pass the filename as a an argument, you obviously know it anyway
echo "name $1"
while read line
do
echo $line
done
and then:
./test.sh foo/bar.txt < foo/bar.txt
gives
name foo/bar.txt
1
2
3
if foo/bar.txt contains
1
2
3
I have two bash scripts. Script1 does the following (doesn't matter why I'm using two scripts; just assume it's for a good reason):
export RUN=1
And script2:
. script1
echo ${RUN}
sed -n ${RUN}p mytext.txt > mytextnew.txt
In script 2, echo returns "1" as I expect. However, the sed command (or any other command I try and use the RUN variable with) returns an error, as if RUN doesn't exist. If I simply run script2 with the following:
RUN=2p
sed -n ${RUN} mytext.txt > mytextnew.txt
then everything works fine. This only happens with global variables. If I do the exact same thing I do with local variables that I do with global variables, everything works. But the minute a global variable is thrown in there, everything goes haywire.
Any insight into the problem?
The following:
$ cat script1
export RUN=1
$cat script2
. script1
echo ${RUN}
sed -n ${RUN}p /etc/passwd
after the
$ bash -x script2
prints
+ . script1
++ export RUN=1
++ RUN=1
+ echo 1
1
+ sed -n 1p /etc/passwd
if adding the space to
sed -n ${RUN} p /etc/passwd
prints
sed: -e expression #1, char 1: missing command
check the content of the $RUN with
echo "${RUN}" | od -bc #note the double quotes
#or
echo "${RUN}" | xxd
to see, what really contains your $RUN variable...
This bug appears when using shell commands in a ruby script. The commands are not executed as what would happen in the bash terminal.
The first two commands are reasonable, the third is not.
`echo foo` # => "foo\n"
`echo -n foo` # => "foo"
`echo -n foo | cat` # => "-n foo\n"
In bash, we would have the following:
$ echo foo # => "foo\n"
$ echo -n foo # => "foo"
$ echo -n foo | cat # => "foo"
Is there a way I am messing up the parameter passing to the calls to echo in the Ruby commands? If you're unfamiliar, the command echo returns the string it is given. Without the -n flag, it appends a newline character, when you add the -n, it does not. cat repeats whatever it is given.
First, Ruby uses /bin/sh, not bash, for the back tick operator. It is likely that on your system, /bin/sh is a link to /bin/bash, but bash behaves differently when invoked as sh, for better POSIX compliance. However, it's not perfect, as your example shows. The POSIX specification states that echo "shall not support any options", but leaves handling of an initial argument -n to be implementation-specific.
In a quick test with /bin/sh as a link to /bin/bash,
bash -c "echo -n foo" and bash -c "echo -n foo | cat" produced identical, newline-free results, but sh -c "echo -n foo" and sh -c "echo -n foo | cat" showed the results you report. (I am not sure how other shells, such as ksh, dash, or zsh, behave when invoked as sh, but by default they all treat -n as a newline suppressor.)
For predictable results, use printf instead, which never prints a newline unless a \n is included in the format string.
printf 'foo\n'
printf 'foo'
printf 'foo' | cat
UPDATE: This appears to be a bug in bash 3.2 that was fixed at some point in the 4.x series. With 4.1 or later, sh -c "echo -n foo | cat and sh -c "echo -n foo" produce the same output.