Passing a variable into awk within a shell script - bash

I have a shell script that I'm writing to search for a process by name and return output if that process is over a given value.
I'm working on finding the named process first. The script currently looks like this:
#!/bin/bash
findProcessName=$1
findCpuMax=$2
#echo "parameter 1: $findProcessName, parameter2: $findCpuMax"
tempFile=`mktemp /tmp/processsearch.XXXXXX`
#echo "tempDir: $tempFile"
processSnapshot=`ps aux > $tempFile`
findProcess=`awk -v pname="$findProcessName" '/pname/' $tempFile`
echo "process line: "$findProcess
`rm $tempFile`
The error is occuring when I try to pass the variable into the awk command. I checked my version of awk and it definitely does support the -v flag.
If I replace the '/pname/' portion of the findProcess variable assignment the script works.
I checked my syntax and it looks right. Could anyone point out where I'm going wrong?

The processSnapshot will always be empty: the ps output is going to the file
when you pass the pattern as a variable, use the pattern match operator:
findProcess=$( awk -v pname="$findProcessName" '$0 ~ pname' $tempFile )
only use backticks when you need the output of a command. This
`rm $tempFile`
executes the rm command, returns the output back to the shell and, it the output is non-empty, the shell attempts to execute that output as a command.
$ `echo foo`
bash: foo: command not found
$ `echo whoami`
jackman
Remove the backticks.
Of course, you don't need the temp file at all:
pgrep -fl $findProcessName

Related

echo does not print command output as expected

This following command prints nothing on my machine
echo `python3.8 -c 'print ("*"*10)'`
whereas
python3.8 -c 'print ("*"*10)'
does. Why?
The first example does command substitution for the argument to echo and is equivalent to the command echo **********. This happens to output a list of directory contents, since ********** seems to be equivalent to *1 and is expanded by the shell.
If you want to prevent the shell to expand **********, you need to quote it:
echo "`python3.8 -c 'print ("*"*10)'`"
which is equivalent to echo "**********".
1don't quote me on that (pun intended)
`expr` means to evaluate the contents between ' as a command and replace it with the result.
`print(*)` is equal to ` * ` which is evaluated as all the files in the current directory
The output really depends on the machine used

grep output different in bash script

I am creating a bash script that will simply use grep to look through a bunch of logs for a certain string.
Something interesting happens though.
For the purpose of testing all of the log files the files are named test1.log, test2.log, test3.log, etc.
When using the grep command:
grep -oHnR TEST Logs/test*
The output contains all instances from all files in the folder as expected.
But when using a command but contained in the bash script below:
#!/bin/bash
#start
grep -oHnR $1 $2
#end
The output displays the instances from only 1 file.
When running the script I am using the following command:
bash test.bash TEST Logs/test*
Here is an example of the expected output (what occurs when simply using grep):
Logs/test2.log:8:TEST
Logs/test2.log:20:TEST
Logs/test2.log:41:TEST
Logs/test.log:2:TEST
Logs/test.log:18:TEST
and here is an example of the output received when using the bash script:
Logs/test2.log:8:TEST
Logs/test2.log:20:TEST
Logs/test2.log:41:TEST
Can someone explain to me why this happens?
When you call the line
bash test.bash TEST Logs/test*
this will be translated by the shell to
bash test.bash TEST Logs/test1.log Logs/test2.log Logs/test3.log Logs/test4.log
(if you have four log files).
The command line parameters TEST, Logs/test1.log, Logs/test2.log, etc. will be given the names $1, $2, $3, etc.; $1 will be TEST, $2 will be Logs/test1.log.
You just ignore the remaining parameters and use just one log file when you use $2 only.
A correct version would be this:
#!/bin/bash
#start
grep -oHnR "$#"
#end
This will pass all the parameters properly and also take care of nastinesses like spaces in file names (your version would have had trouble with these).
To understand what's happening, you can use a simpler script:
#!/bin/bash
echo $1
echo $2
That outputs the first two arguments, as you asked for.
You want to use the first argument, and then use all the rest as input files. So use shift like this:
#!/bin/bash
search=$1
shift
echo "$1"
echo "$#"
Notice also the use of double quotes.
In your case, because you want the search string and the filenames to be passed to grep in the same order, you don't even need to shift:
#!/bin/bash
grep -oHnR -e "$#"
(I added the -e in case the search string begins with -)
The unquoted * is being affected by globbing when you are calling the script.
Using set -x to output what is running from the script makes this more clear.
$ ./greptest.sh TEST test*
++ grep -oHnR TEST test1.log
$ ./greptest.sh TEST "test*"
++ grep -oHnR TEST test1.log test2.log test3.log
In the first case, bash is expanding the * into the list of file names versus the second case it is being passed to grep. In the first case you actually have >2 args (as each filename expanded would become an arg) - adding echo $# to the script shows this too:
$ ./greptest.sh TEST test*
++ grep -oHnR TEST test1.log
++ echo 4
4
$ ./greptest.sh TEST "test*"
++ grep -oHnR TEST test1.log test2.log test3.log
++ echo 2
2
You probably want to escape the wildcard on your bash invocation:
bash test.bash TEST Logs/test\*
That way it'll get passed through to grep as an *, otherwise the shell will have expanded it to every file in the Logs dir whose name starts with test.
Alternatively, change your script to allow more than one file on the command line:
#!/bin/bash
hold=$1
shift
grep -oHnR $hold $#

Expansion of variable does not work when calling bash functions

See also my previous question.
So... I have a script:
function go_loop (){
for i in `grep -v ^# $1`; do
$2
done
}
go_loop "/tmp/text.txt" "echo $i"
I should have in a result:
9
20
21
...
But apparently I only get an empty result. How can I feed the second input parameter to the loop?
Please don't advice me do this:
for i in `grep -v ^# $1`; do
echo $i
done
I need to make 2 input parameters, first - name of file, second - name of execution command
You need to eval the second parameter like this:
eval $2
and pass it like this:
go_loop "/tmp/text.txt" 'echo $i'
You can do this using exec inside your loop, which will run the $2 as bash command:
[root#box ~]# ./test.sh 1 ls
test.sh tests_passed.txt
[root#box ~]# cat test
exec $2
The exec builtin command is used to
replace the shell with a given program (executing it, not as new process)
set redirections for the program to execute or for the current shell

Finding text from file and assigning to a variable

I am trying to find a text from file and assign it to a variable. I used grep with regex option to find a text. I need the output to get assigned to a variable so that I can use the output wherever I need. Filename: wgettesting.html
cat wgettesting.html
productvale:productvalues,productinfo:productinformation,product_group_type,value:product_genderledger,productcategories:smallcategories
Command I used in a script:
gl=grep -o --perl-regexp "(?<=product_group_type,value:product_)[a-zA-Z0-9_]+" wgettesting.html
echo $gl
When I run the script, I got: -o: command not found for the above command.
I also tried like:
gl=cat wgettesting.html|grep -o --perl-regexp ""(?<=product_group_type,value:product_)[a-zA-Z0-9_]+"
echo $gl
For the above, I received: wgettesting.html: command not found
My intended output has to be:
genderledger
Someone guide me on the issue.
You failed to put the grep command within $(),
$ gl=$(grep -o --perl-regexp "(?<=product_group_type,value:product_)[a-zA-Z0-9_]+" wgettesting.html)
$ echo $gl
genderledger
Commands should be enclosed within $() while assigning it's output to a variable.

bash set -x and stream

Can you explain the output of the following test script to me:
# prepare test data
echo "any content" > myfile
# set bash to inform me about the commands used
set -x
cat < myfile
output:
+cat
any content
Namely why does the line starting with + not show the "< myfile" bit?
How to force bash to do that. I need to inform the user of my script's doings as in:
mysql -uroot < the_new_file_with_a_telling_name.sql
and I can't.
EDIT: additional context: I use variables. Original code:
SQL_FILE=`ls -t $BACKUP_DIR/default_db* | head -n 1` # get latest db
mysql -uroot mydatabase < ${SQL_FILE}
-v won't expand variables and cat file.sql | mysql will produce two lines:
+mysql
+cat file.sql
so neither does the trick.
You could try set -v or set -o verbose instead which enables command echoing.
Example run on my machine:
[me#home]$ cat x.sh
echo "any content" > myfile
set -v
cat < myfile
[me#home]$ bash x.sh
cat < myfile
any content
The caveat here is that set -v simply echos the command literally and does not do any shell expansion or iterpolation. As pointed out by Jonathan in the comments, this can be a problem if the filename is defined in a variable (e.g. command < $somefile) making it difficult to identify what $somefile refers to.
The difference there is quite simple:
in the first case, you're using the program cat, and you're redirecting the contents of myfile to the standard input of cat. This means you're executing cat, and that's what bash shows you when you have set -x;
in a possible second case, you could use cat myfile, as pointed by #Jonathan Leffler, and you'd see +cat myfile, which is what you're executing: the program cat with the parameter myfile.
From man bash:
-x After expanding each simple command, for command, case command,
select command, or arithmetic for command, display the expanded
value of PS4, followed by the command and its expanded arguments or
associated word list.
As you can see, it simply displays the command line expanded, and its argument list -- redirections are neither part of the expanded command cat nor part of its argument list.
As pointed by #Shawn Chin, you may use set -v, which, as from man bash:
-v Print shell input lines as they are read.
Basically, that's the way bash works with its -x command. I checked on a Solaris 5.10 box, and the /bin/sh there (which is close to a genuine Bourne shell) also omits I/O redirection.
Given the command file (x3.sh):
echo "Hi" > Myfile
cat < Myfile
rm -f Myfile
The trace output on the Solaris machine was:
$ sh -x x3.sh
+ echo Hi
+ cat
Hi
+ rm -f Myfile
$ /bin/ksh -x x3.sh
+ echo Hi
+ 1> Myfile
+ cat
+ 0< Myfile
Hi
+ rm -f Myfile
$ bash -x x3.sh
+ echo Hi
+ cat
Hi
+ rm -f Myfile
$
Note that bash and sh (which are definitely different executables) produce the same output. The ksh output includes the I/O redirection information — score 1 for the Korn shell.
In this specific example, you can use:
cat myfile
to see the name of the file. In the general case, it is hard, but consider using ksh instead of bash to get the I/O redirection reported.

Resources