echo does not print command output as expected - bash

This following command prints nothing on my machine
echo `python3.8 -c 'print ("*"*10)'`
whereas
python3.8 -c 'print ("*"*10)'
does. Why?

The first example does command substitution for the argument to echo and is equivalent to the command echo **********. This happens to output a list of directory contents, since ********** seems to be equivalent to *1 and is expanded by the shell.
If you want to prevent the shell to expand **********, you need to quote it:
echo "`python3.8 -c 'print ("*"*10)'`"
which is equivalent to echo "**********".
1don't quote me on that (pun intended)

`expr` means to evaluate the contents between ' as a command and replace it with the result.
`print(*)` is equal to ` * ` which is evaluated as all the files in the current directory
The output really depends on the machine used

Related

Bash: Assign command to a variable

Below I have an example which confuses me a bit, any help would be appreciated.
I bind a normal command line command (ls) to a new variable. If I echo it it the output is just the command (ls) but if I just use the variable without echo i get the result of the command but why?? Is it because $listdir gets translated to ls so I just get the output? And if I use the echo command it will be interpreted as a string?
router#test:~/scripting$ listdir=ls
router#test:~/scripting$ echo "$listdir"
ls
----- VS ----
router#test:~/scripting$ $listdir
basicLoop.sh fileflood.sh .......
Thank you for any help!
By doing listdir=ls you literally assign a string "ls" to the $listdir variable. So if you run echo $listdir now it will just expand into echo ls, which (as you may have guessed) will just print "ls" onto a screen. If you want to store a result of a command into a variable you can wrap the command in `` or $() (eg. listdir=$(ls) or listdir=`ls`).
jarmusz#emacs~$ listdir=`ls`
jarmusz#emacs~$ echo "$listdir"
dls
docs
music
...
If you want to store just a name of the command and run it later you can do it like this:
jarmusz#emacs~$ listdir=ls
<some other commands...>
jarmusz#emacs~$ echo `$listdir`
dls docs music ...
In this example, echo `$listdir` will expand into echo `ls` and then into echo dls docs music...
When bash is interpreting the commands you feed to it, the first thing it will do is expand any expansions it is given. So when you give it $listdir by the time bash starts to execute the value it is given, all it knows is that it was given the value ls. It does not care where the value came from, only what the value is.
Lets look at the trace given after running set -x, which instructs bash to prints to stderr after expansion and before execution:
$> echo $listdir
+ echo ls
ls
$> $listdir
+ ls
file_0 file_1
As you can see, in the second line, bash will attempt to run the command ls just as if you have explicity called ls or even /usr/bin/ls
Edit
Expansion isn't the first step in in shell evaluation, see #Gordon Davisson's comment for details

How to run "source" command (Linux) from a perl script?

I am trying to source a script from a Perl script (script.pl).
system ("source /some/generic/script");
Please note that this generic script could be a shell, python or any other script. Also, I cannot replicate the logic present inside this generic script into my Perl script. I tried replacing system with ``, exec, and qx//. Each time I got the following error:
Can't exec "source": No such file or directory at script.pl line 18.
I came across many forums on the internet, which discussed various reasons for this problem. But none of them provided a solution. Is there any way to run/execute source command from a Perl script?
In bash, etc, source is a builtin that means read this file, and interpret it locally (a little like a #include).
In this context that makes no sense - you either need to remove source from the command and have a shebang (#!) line at the start of the shell script that tells the system which shell to use to execute that script, or you need to explicitly tell system which shell to use, e.g.
system "/bin/sh", "/some/generic/script";
[with no comment about whether it's actually appropriate to use system in this case].
There are a few things going on here. First, a child process can't change the environment of its parent. That source would only last as long as its process is around.
Here's a short program that set and export an environment variable.
#!/bin/sh
echo "PID" $$
export HERE_I_AM="JH";
Running the file does not export the variable. The file runs in its own proces. The process IDs ($$) are different in set_stuff.sh and the shell:
$ chmod 755 set_stuff.sh
$ ./set_stuff.sh
PID 92799
$ echo $$
92077
$ echo $HERE_I_AM # empty
source is different. It reads the file and evaluates it in the shell. The process IDs are the same in set_stuff.sh and the shell, so the file is actually affecting its own process:
$ unset HERE_I_AM # start over
$ source set_stuff.sh
PID 92077
$ echo $$
92077
$ echo $HERE_I_AM
JH
Now on to Perl. Calling system creates a child process (there's an exec in there somewhere) so that's not going to affect the Perl process.
$ perl -lwe 'system( "source set_stuff.sh; echo \$HERE_I_AM" );
print "From Perl ($$): $ENV{HERE_I_AM}"'
PID 92989
JH
Use of uninitialized value in concatenation (.) or string at -e line 1.
From Perl (92988):
Curiously, this works even though your version doesn't. I think the different is that in this there are no special shell metacharacters here, so it tries to exec the program directory, skipping the shell it just used for my more complicated string:
$ perl -lwe 'system( "source set_stuff.sh" ); print $ENV{HERE_I_AM}'
Can't exec "source": No such file or directory at -e line 1.
Use of uninitialized value in print at -e line 1.
But, you don't want a single string in that case. The list form is more secure, but source isn't a file that anything can execute:
$ which source # nothing
$ perl -lwe 'system( "source", "set_stuff.sh" ); print "From Perl ($$): $ENV{HERE_I_AM}"'
Can't exec "source": No such file or directory at -e line 1.
Use of uninitialized value in concatenation (.) or string at -e line 1.
From Perl (93766):
That is, you can call source, but as something that invokes the shell.
Back to your problem. There are various ways to tackle this, but we need to get the output of the program. Instead of system, use backticks. That's a double-quoted context so I need to protect some literal $s that I want to pass as part of the shell commans
$ perl -lwe 'my $o = `echo \$\$ && source set_stuff.sh && echo \$HERE_I_AM`; print "$o\nFrom Perl ($$): $ENV{HERE_I_AM}"'
Use of uninitialized value in concatenation (.) or string at -e line 1.
93919
From Shell PID 93919
JH
From Perl (93918):
Inside the backticks, you get what you like. The shell program can see the variable. Once back in Perl, it can't. But, I have the output now. Let's get more fancy. Get rid of the PID stuff because I don't need to see that now:
#!/bin/sh
export HERE_I_AM="JH";
And the shell command creates some output that has the name and value:
$ perl -lwe 'my $o = `source set_stuff.sh && echo HERE_I_AM=\$HERE_I_AM`; print $o'
HERE_I_AM=JH
I can parse that output and set variables in Perl. Now Perl has imported part of the environment of the shell program:
$ perl -lwe 'my $o = `source set_stuff.sh && echo HERE_I_AM=\$HERE_I_AM`; for(split/\R/,$o){ my($k,$v)=split/=/; $ENV{$k}=$v }; print "From Perl: $ENV{HERE_I_AM}"'
From Perl: JH
Let's get the entire environment, though. env outputs every value in the way I just processed it:
$ perl -lwe 'my $o = `source set_stuff.sh && env | sort`; print $o'
...
DISPLAY=:0
EC2_PATH=/usr/local/ec2/ec2-api-tools
EDITOR=/usr/bin/vi
...
I have a few hundred varaibles set in the shell, and I don't want to expose most of them. Those are all set by the Perl process, so I can temporarily clear out %ENV:
$ perl -lwe 'local %ENV=(); my $o = `source set_stuff.sh && env | sort`; print $o'
HERE_I_AM=JH
PWD=/Users/brian/Desktop/test
SHLVL=1
_=/usr/bin/env
Put that together with the post processing code and you have a way to pass that information back up to the parent.
This is, by the way, similar to how you'd pass variables back up to a parent shell process. Since that output is already something the shell understands, you use the shell's eval instead of parsing it.
You can't. source is a shell function that 'imports' the contents of that script into your current environment. It's not an executable.
You can replicate some of it's functionality by rolling your own - run or parse whatever you're 'sourcing' and capture the result:
print `. file_to_source; echo $somevar`;
or similar.

Finding text from file and assigning to a variable

I am trying to find a text from file and assign it to a variable. I used grep with regex option to find a text. I need the output to get assigned to a variable so that I can use the output wherever I need. Filename: wgettesting.html
cat wgettesting.html
productvale:productvalues,productinfo:productinformation,product_group_type,value:product_genderledger,productcategories:smallcategories
Command I used in a script:
gl=grep -o --perl-regexp "(?<=product_group_type,value:product_)[a-zA-Z0-9_]+" wgettesting.html
echo $gl
When I run the script, I got: -o: command not found for the above command.
I also tried like:
gl=cat wgettesting.html|grep -o --perl-regexp ""(?<=product_group_type,value:product_)[a-zA-Z0-9_]+"
echo $gl
For the above, I received: wgettesting.html: command not found
My intended output has to be:
genderledger
Someone guide me on the issue.
You failed to put the grep command within $(),
$ gl=$(grep -o --perl-regexp "(?<=product_group_type,value:product_)[a-zA-Z0-9_]+" wgettesting.html)
$ echo $gl
genderledger
Commands should be enclosed within $() while assigning it's output to a variable.

Passing a variable into awk within a shell script

I have a shell script that I'm writing to search for a process by name and return output if that process is over a given value.
I'm working on finding the named process first. The script currently looks like this:
#!/bin/bash
findProcessName=$1
findCpuMax=$2
#echo "parameter 1: $findProcessName, parameter2: $findCpuMax"
tempFile=`mktemp /tmp/processsearch.XXXXXX`
#echo "tempDir: $tempFile"
processSnapshot=`ps aux > $tempFile`
findProcess=`awk -v pname="$findProcessName" '/pname/' $tempFile`
echo "process line: "$findProcess
`rm $tempFile`
The error is occuring when I try to pass the variable into the awk command. I checked my version of awk and it definitely does support the -v flag.
If I replace the '/pname/' portion of the findProcess variable assignment the script works.
I checked my syntax and it looks right. Could anyone point out where I'm going wrong?
The processSnapshot will always be empty: the ps output is going to the file
when you pass the pattern as a variable, use the pattern match operator:
findProcess=$( awk -v pname="$findProcessName" '$0 ~ pname' $tempFile )
only use backticks when you need the output of a command. This
`rm $tempFile`
executes the rm command, returns the output back to the shell and, it the output is non-empty, the shell attempts to execute that output as a command.
$ `echo foo`
bash: foo: command not found
$ `echo whoami`
jackman
Remove the backticks.
Of course, you don't need the temp file at all:
pgrep -fl $findProcessName

Why do I get a “Can't find string terminator” error only when the command is in a variable?

I wrote a simple shell script to get the version of Perl modules installed
on a server and I keep receiving the following error:
Can't find string terminator "'" anywhere before EOF at -e line 1.
Here is my script:
#!/bin/sh
#
mod_name="Sub::Uplevel"
tmp1="perl -M$mod_name -e 'print \"\$$mod_name::VERSION\"'"
echo $tmp1
$tmp1
If I just directly run the echo'd line (perl -MSub::Uplevel -e 'print "$Sub::Uplevel::VERSION"'), it works. Why doesn't the line work when its run from the variable $tmp1?
In place of just $tmp1, eval works:
eval "$tmp1"
That's because splitting a variable into words (for arguments) is done strictly by splitting on $IFS, not the normal input-parsing. eval forces the normal input parsing.
How did I figure this out?
Change your tmp1= line to put an echo in front, and you get:
perl -MSub::Uplevel -e 'print "$Sub::Uplevel::VERSION"'
Note that the ' are still there, which you wouldn't expect. If you write a quick script:
#!/bin/sh
for a in "$#"; do
echo "arg: $a"
done
and put a call to that in place of echo, you find how the arguments are really split:
arg: perl
arg: -MSub::Uplevel
arg: -e
arg: 'print
arg: "$Sub::Uplevel::VERSION"'
So, you can see that's splitting on spaces, so IFS.
It's always better to construct commands using bash arrays. That will keep arguments with whitespace properly grouped:
#!/bin/bash
mod_name="Sub::Uplevel"
perl_script=$(printf 'print "$%s::VERSION"' $mod_name)
tmp1=(perl -M$mod_name -e "$perl_script")
echo "${tmp1[#]}"
output=$( "${tmp1[#]}" )
Arrays are a bash feature, so the shebang line must reference bash not sh.
I'd usually write what you are doing with backticks, to run the command inside the shell:
#!/bin/sh
#
mod_name="Sub::Uplevel"
tmp1=`perl -M$mod_name -e 'print \"\$$mod_name::VERSION\"'`
echo $tmp1
Then you can work on $tmp1 as needed. It also avoids dealing with escaping.
Try to execute the script the below way(debugging the script):
sh -vx your_script.sh
Then you would be able to see where exactly the problem is.
I donot have the shell to execute it right now.

Resources