Can anyone tell me what the big difference here is and why the latter doesn't work?
test="ls -l"
Both now work fine:
eval $test
echo `$test`
But in this case:
test="ls -l >> test.log"
eval $test
echo `$test`
The latter will not work. Why is that? I know that eval is just executing a script while the apostrophes are executing it and return the result as a string. What makes it not possible to use >> or simmilar stuff inside the command to execute? Maybe is there a way to make it work with apostrophes and I'm doing something wrong?
When you're using backticks to execute your command, the command being sent to the shell is:
ls -l '>>' test.log
which makes both >> and test.log arguments to ls (note the quotes around >>).
While using eval, the command being executed is:
ls -l >> test.log
(Execute your script by saying bash -vx scriptname to see what's happening.)
eval is 'expression value' i.e.
test="ls -l >> test.log"
eval $test
is execute in same way in terminal as
ls -l >> test.log
whether
echo is for display purpose only.
Related
I want to be able to add newline characters before every occurences of some tokens appearing in some .tex files that I possess, some of those tokens are '\itemQ', '\pagebreakQ'. I created a procedure that ends up creating a command for sed stored in $sedInpt:
~$ echo "$sedInpt"
-e s/\(\\itemQ\)/\n\1/ -e s/\(\\pagebreakQ\)/\n\1/
I want to use "$sedInpt" as a command for sed:
echo "$inputText" | eval "sed ${sedInpt}"
but if I do the following as a test:
echo 'hello\itemQ' | eval "sed ${sedInpt}"
hello\itemQ
you can see there ain't any newline that has been added before \itemQ.
So I've tried debugging this way of doing thing by calling bash -x to see what's happened in detail:
~$ bash -x
~$ echo "hello\itemQ" | eval "sed ${sedInpt}"
+ echo 'hello\itemQ'
+ eval 'sed -e s/\(\\itemQ\)/\n\1/ -e s/\(\\pagebreakQ\)/\n\1/'
++ sed -e 's/(\itemQ)/n1/' -e 's/(\pagebreakQ)/n1/'
hello\itemQ
you can see that the backslashes of \n and \1 and even the ones before ( and ) that I had placed in "$sedInpt" seem to have disappeared when parsed by eval.
So I am bit lost on what to do next to do what I want.. any ideas?
You could also just combine them into a single command, which in my opinion is more straightforward:
$ cat /tmp/sed.sh
sedInpt='s/\(\\itemQ\)/\n\1/; s/\(\\pagebreakQ\)/\n\1/'
echo "hello\itemQ" | sed "$sedInpt"
$ /tmp/sed.sh
hello
\itemQ
Edit: As #123 rightly points out, storing commands in variables is dangerous and should be avoided if possible. If you have complete control over what is stored, it should be safe, but if it comes from any sort of user input, it is a "Command Injection" vulnerability.
Following #Inian advice I managed to achieve what I wanted to do in this way:
~$ sedInpt=( -e 's/\(\\itemQ\)/\n\1/' -e 's/\(\\pagebreakQ\)/\n\1/' )
~$ echo "hello\itemQ" | sed "${sedInpt[#]}"
hello
\itemQ
I came across an interesting thing in Shell Scripting and not 100% sure why the behaviour is like this
I tried the below script:
#!/bin/sh
CMD="curl -XGET http://../endpoint";
var1=eval $CMD | sed -e 's/find/replace/g';
echo $var1; # Output: printed the value on this line
echo $var1; # Output: blank/no data printed (Why it is blank?)
I had to change the command in variable enclosing with back-tick ` to print the variable as many time as I wanted.
CMD="curl -XGET http://../endpoint";
var1=`eval $CMD | sed -e 's/find/replace/g'`;
echo $var1; # Output: printed the value on this line
echo $var1; # Output: printed the value on this line
Why I have to surround my command with ` to assign it's o/p to the variable in subsequent variable usage?
I have a feeling that it has something to do with the variable-command scope.
Shedding light on my understanding will be appreciated!
UPDATE:
I tried the below command and it is working in my env.
#!/bin/sh
CMD="curl -XGET http://www.google.com/";
var1=eval $CMD | sed -e 's/find/replace/g';
echo $var1; # Output: printed the value on this line
echo "######";
echo $var1; # Output: blank/no data printed (Why it is blank?)
sh/bash allows you to run a command with a variable in its environment, without permanently modifying the variable in the shell. This is great, because you can e.g. run a command in a certain language just one time without having to change your entire user's or system's language:
$ LC_ALL=en_US.utf8 ls foo
ls: cannot access foo: No such file or directory
$ LC_ALL=nb_NO.utf8 ls foo
ls: cannot access foo: Ingen slik fil eller filkatalog
However, this means that when you try to do
var=this is some command
you're trigger this syntax.
It means "run the command is a command and tell it that the variable var is set to this"
It does not assign "this is my string" to the variable, and it definitely does not evaluate "this is a string" as a command, and then assign its output to var.
Given this, we can look at what actually happened:
CMD="curl -XGET http://../endpoint";
var1=eval $CMD | sed -e 's/find/replace/g'; # No assignment, output to screen
echo $var1; # Output: blank/no data printed
echo $var1; # Output: blank/no data printed
There is no scope issue and no inconsistency: the variable is never assigned, and is never written by an echo statement.
var=`some command` (or preferably, var=$(some command)) works because this is valid syntax to assign output from a program to a variable.
The first example isn't doing what you think it is.
Neither echo is printing anything. Make them echo "[$var1]" to see that.
You need the backticks to run the command and capture its output.
Your first attempt was running the $CMD | sed -e 's/find/replace/g'; pipeline with the environment of $CMD containing var1 set to a value of eval.
You also shouldn't be putting commands inside strings (or using eval in general). See http://mywiki.wooledge.org/BashFAQ/001 for more on why.
I need my bashscript to cat all of its parameters into a file. I tried to use cat for this because I need to add a lot of lines:
#!/bin/sh
cat > /tmp/output << EOF
I was called with the following parameters:
"$#"
or
$#
EOF
cat /tmp/output
Which leads to the following output
$./test.sh "dsggdssgd" "dsggdssgd dgdsdsg"
I was called with the following parameters:
"dsggdssgd dsggdssgd dgdsdsg"
or
dsggdssgd dsggdssgd dgdsdsg
I want neither of these two things: I need the exact quoting which was used on the command line. How can I achieve this? I always thought $# does everything right in regards to quoting.
Well, you are right that "$#" has the args including the whitespace in each arg. However, since the shell performs quote removal before executing a command, you can never know how exactly the args were quoted (e.g. whether with single or double quotes, or backslashes or any combination thereof--but you shouldn't need to know, since all you should care for are the argument values).
Placing "$#" in a here-document is pointless because you lose the information about where each arg starts and ends (they're joined with a space inbetween). Here's a way to see just this:
$ cat test.sh
#!/bin/sh
printf 'I was called with the following parameters:\n'
printf '"%s"\n' "$#"
$ ./test.sh "dsggdssgd" "dsggdssgd dgdsdsg"
I was called with the following parameters:
"dsggdssgd"
"dsggdssgd dgdsdsg"
Try:
#!/bin/bash
for x in "$#"; do echo -ne "\"$x\" "; done; echo
To see what's interpreted by Bash, use:
bash -x ./script.sh
or add this to the beginning of your script:
set -x
You might want add this on the parent script.
Can you explain the output of the following test script to me:
# prepare test data
echo "any content" > myfile
# set bash to inform me about the commands used
set -x
cat < myfile
output:
+cat
any content
Namely why does the line starting with + not show the "< myfile" bit?
How to force bash to do that. I need to inform the user of my script's doings as in:
mysql -uroot < the_new_file_with_a_telling_name.sql
and I can't.
EDIT: additional context: I use variables. Original code:
SQL_FILE=`ls -t $BACKUP_DIR/default_db* | head -n 1` # get latest db
mysql -uroot mydatabase < ${SQL_FILE}
-v won't expand variables and cat file.sql | mysql will produce two lines:
+mysql
+cat file.sql
so neither does the trick.
You could try set -v or set -o verbose instead which enables command echoing.
Example run on my machine:
[me#home]$ cat x.sh
echo "any content" > myfile
set -v
cat < myfile
[me#home]$ bash x.sh
cat < myfile
any content
The caveat here is that set -v simply echos the command literally and does not do any shell expansion or iterpolation. As pointed out by Jonathan in the comments, this can be a problem if the filename is defined in a variable (e.g. command < $somefile) making it difficult to identify what $somefile refers to.
The difference there is quite simple:
in the first case, you're using the program cat, and you're redirecting the contents of myfile to the standard input of cat. This means you're executing cat, and that's what bash shows you when you have set -x;
in a possible second case, you could use cat myfile, as pointed by #Jonathan Leffler, and you'd see +cat myfile, which is what you're executing: the program cat with the parameter myfile.
From man bash:
-x After expanding each simple command, for command, case command,
select command, or arithmetic for command, display the expanded
value of PS4, followed by the command and its expanded arguments or
associated word list.
As you can see, it simply displays the command line expanded, and its argument list -- redirections are neither part of the expanded command cat nor part of its argument list.
As pointed by #Shawn Chin, you may use set -v, which, as from man bash:
-v Print shell input lines as they are read.
Basically, that's the way bash works with its -x command. I checked on a Solaris 5.10 box, and the /bin/sh there (which is close to a genuine Bourne shell) also omits I/O redirection.
Given the command file (x3.sh):
echo "Hi" > Myfile
cat < Myfile
rm -f Myfile
The trace output on the Solaris machine was:
$ sh -x x3.sh
+ echo Hi
+ cat
Hi
+ rm -f Myfile
$ /bin/ksh -x x3.sh
+ echo Hi
+ 1> Myfile
+ cat
+ 0< Myfile
Hi
+ rm -f Myfile
$ bash -x x3.sh
+ echo Hi
+ cat
Hi
+ rm -f Myfile
$
Note that bash and sh (which are definitely different executables) produce the same output. The ksh output includes the I/O redirection information — score 1 for the Korn shell.
In this specific example, you can use:
cat myfile
to see the name of the file. In the general case, it is hard, but consider using ksh instead of bash to get the I/O redirection reported.
I wrote a simple shell script to get the version of Perl modules installed
on a server and I keep receiving the following error:
Can't find string terminator "'" anywhere before EOF at -e line 1.
Here is my script:
#!/bin/sh
#
mod_name="Sub::Uplevel"
tmp1="perl -M$mod_name -e 'print \"\$$mod_name::VERSION\"'"
echo $tmp1
$tmp1
If I just directly run the echo'd line (perl -MSub::Uplevel -e 'print "$Sub::Uplevel::VERSION"'), it works. Why doesn't the line work when its run from the variable $tmp1?
In place of just $tmp1, eval works:
eval "$tmp1"
That's because splitting a variable into words (for arguments) is done strictly by splitting on $IFS, not the normal input-parsing. eval forces the normal input parsing.
How did I figure this out?
Change your tmp1= line to put an echo in front, and you get:
perl -MSub::Uplevel -e 'print "$Sub::Uplevel::VERSION"'
Note that the ' are still there, which you wouldn't expect. If you write a quick script:
#!/bin/sh
for a in "$#"; do
echo "arg: $a"
done
and put a call to that in place of echo, you find how the arguments are really split:
arg: perl
arg: -MSub::Uplevel
arg: -e
arg: 'print
arg: "$Sub::Uplevel::VERSION"'
So, you can see that's splitting on spaces, so IFS.
It's always better to construct commands using bash arrays. That will keep arguments with whitespace properly grouped:
#!/bin/bash
mod_name="Sub::Uplevel"
perl_script=$(printf 'print "$%s::VERSION"' $mod_name)
tmp1=(perl -M$mod_name -e "$perl_script")
echo "${tmp1[#]}"
output=$( "${tmp1[#]}" )
Arrays are a bash feature, so the shebang line must reference bash not sh.
I'd usually write what you are doing with backticks, to run the command inside the shell:
#!/bin/sh
#
mod_name="Sub::Uplevel"
tmp1=`perl -M$mod_name -e 'print \"\$$mod_name::VERSION\"'`
echo $tmp1
Then you can work on $tmp1 as needed. It also avoids dealing with escaping.
Try to execute the script the below way(debugging the script):
sh -vx your_script.sh
Then you would be able to see where exactly the problem is.
I donot have the shell to execute it right now.