I have this bash script:
How I can call it the grep's output in the if else statement? If I suppress the output of the grep command with -q, also will work?
#!/usr/bin/env bash
DOMAINS=( '.com' '.biz' )
while read input; do
for (( i=0;i<${#DOMAINS[#]};i++)); do
jwhois --force-lookup --disable-cache --no-redirect -c jwhois.conf "$input${DOMAINS[$i]}" | MATCH="$(grep -oPa '^.*\b(clientTransferProhibited)\b.*$')"
if [ $? -eq 0 ]; then
echo -e "$input${DOMAINS[$i]}\tregistered\t" $(date +%y/%m/%d_%H:%M:%S) "\t" "$MATCH" |& tee --append output/registered.txt
else
echo -e "$input${DOMAINS[$i]}\tavailable\t" $(date +%y/%m/%d_%H:%M:%S) "\t" "$MATCH" |& tee --append output/available.txt
fi
done
done < "$1"
So MATCH="$(grep -oPa '^.*\b(clientTransferProhibited) used by "$MATCH" in the if else statement not outputting anything, but if I just use grep -oPa '^.*\b(clientTransferProhibited), it's printing the line without problem. The reason why I want to use as a variable, because I want to put in specific places in the if else statements.
Actual output:
$ domain1.com available 15/11/16_14:13:05
$ domain1.biz available 15/11/16_14:13:05
$ domain2.com registered 15/11/16_14:13:05
$ domain2.biz registered 15/11/16_14:13:05
Output that I want:
$ domain1.com available 15/11/16_14:13:05
$ domain1.biz available 15/11/16_14:13:05
$ domain2.com registered 15/11/16_14:13:05 Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited
$ domain2.biz registered 15/11/16_14:13:05 Status: clientTransferProhibited http://www.icann.org/epp#clientTransferProhibited
So the result of this line is what you want stored in a variable...
jwhois --force-lookup --disable-cache --no-redirect -c jwhois.conf "$input${DOMAINS[$i]}" | grep -oPa '^.*\b(clientTransferProhibited)\b.*$'
In that case, you want to evaluate that line and set the entire thing to your desired variable. The type of assignment you attempt in the middle of a pipeline will not be valid.
MATCH=$(jwhois --force-lookup --disable-cache --no-redirect -c jwhois.conf "$input${DOMAINS[$i]}" | grep -oPa '^.*\b(clientTransferProhibited)\b.*$')
Those are two different MATCH variables, because pipes execute subshells:
Bash subshell/pipelines - which parts are executing in subshells?
Try it this way instead:
MATCH="$(jwhois --force-lookup --disable-cache --no-redirect -c jwhois.conf "$input${DOMAINS[$i]}" | grep -oPa '^.*\b(clientTransferProhibited)\b.*$')"
Related
Trying to run a command as a variable but I am getting strange results
Expected result "1" :
grep -i nosuid /etc/fstab | grep -iq nfs
echo $?
1
Unexpected result as a variable command:
cmd="grep -i nosuid /etc/fstab | grep -iq nfs"
$cmd
echo $?
0
It seems it returns 0 as the command was correct not actual outcome. How to do this better ?
You can only execute exactly one command stored in a variable. The pipe is passed as an argument to the first grep.
Example
$ printArgs() { printf %s\\n "$#"; }
# Two commands. The 1st command has parameters "a" and "b".
# The 2nd command prints stdin from the first command.
$ printArgs a b | cat
a
b
$ cmd='printArgs a b | cat'
# Only one command with parameters "a", "b", "|", and "cat".
$ $cmd
a
b
|
cat
How to do this better?
Don't execute the command using variables.
Use a function.
$ cmd() { grep -i nosuid /etc/fstab | grep -iq nfs; }
$ cmd
$ echo $?
1
Solution to the actual problem
I see three options to your actual problem:
Use a DEBUG trap and the BASH_COMMAND variable inside the trap.
Enable bash's history feature for your script and use the hist command.
Use a function which takes a command string and executes it using eval.
Regarding your comment on the last approach: You only need one function. Something like
execAndLog() {
description="$1"
shift
if eval "$*"; then
info="PASSED: $description: $*"
passed+=("${FUNCNAME[1]}")
else
info="FAILED: $description: $*"
failed+=("${FUNCNAME[1]}")
done
}
You can use this function as follows
execAndLog 'Scanned system' 'grep -i nfs /etc/fstab | grep -iq noexec'
The first argument is the description for the log, the remaining arguments are the command to be executed.
using bash -x or set -x will allow you to see what bash executes:
> cmd="grep -i nosuid /etc/fstab | grep -iq nfs"
> set -x
> $cmd
+ grep -i nosuid /etc/fstab '|' grep -iq nfs
as you can see your pipe | is passed as an argument to the first grep command.
#!/bin/bash
set_bash_profile()
{
local bash_profile="$HOME/.profile"
if [[ -w $bash_profile ]]; then
if (grep 'MY_VAR' $bash_profile 2>&1); then
sed -i '/MY_VAR/d' $bash_profile
fi
echo "export MY_VAR=foo" >>$bash_profile
fi
}
set_bash_profile
Here is the first run:
bash-4.1$ ./set_bash.sh
No output --which is great! And cat shows export MY_VAR=foo was appended to the file. But when executing a second time, I want sed to silently edit $bash_profile without outputting the matching string, like it does here:
bash-4.1$ ./set_bash.sh
export MY_VAR=foo
You get the output from grep on grep 'MY_VAR' $bash_profile 2>&1. grep outputs the matched line in your profile:
export MY_VAR=foo
on stdout. The 2>&1 only forwards stderr to stdout. It's good to use -q option with grep. Also the subshell (...) around the grep is not needed. Try this:
#!/bin/bash
set_bash_profile()
{
local bash_profile="$HOME/.profile"
if [ -w $bash_profile ]; then
if grep -q 'MY_VAR' $bash_profile; then
sed -i '/MY_VAR/d' $bash_profile
fi
echo "export MY_VAR=foo" >>$bash_profile
fi
}
set_bash_profile
I am trying to check the output of a command and run different commands depending on the output.
count="1"
for f in "$#"; do
BASE=${f%.*}
# if [ -e "${BASE}_${suffix}_${suffix2}.mp4" ]; then
echo -e "Reading GPS metadata using MediaInfo file ${count}/${##} "$(basename "${BASE}_${suffix}_${suffix2}.mp4")"
mediainfo "${BASE}_${suffix}_${suffix2}.mp4" | grep "©xyz" | head -n 1
if [[ $? != *xyz* ]]; then
echo -e "WARNING!!! No GPS information found! File ${count}/${##} "$(basename "${BASE}_${suffix}_${suffix2}.mp4")" || exit 1
fi
((count++))
done
MediaInfo is the command I am checking the output of.
If a video file has "©xyz" atom written into it the output looks like this:
$ mediainfo FILE | grep "©xyz" | head -n 1
$ ©xyz : +60.9613-125.9309/
$
otherwise it is null
$ mediainfo FILE | grep "©xyz" | head -n 1
$
The above code does not work and echos the warning even when ©xyz presented.
Any ideas of what I am doing wrong?
The syntax you are using the capture the output of the mediainfo command is plain wrong. When using grep you can use its return code (the output of $?) directly in the if-conditional
if mediainfo "${BASE}_${suffix}_${suffix2}.mp4" | grep -q "©xyz" 2> /dev/null;
then
..
The -q flag in grep instructs it to run the command silently without throwing any results to stdout, and the part 2>/dev/null suppresses any errors thrown via stderr, so you will get the if-conditional pass when the string is present and fail if not present
$? is the exit code of the command: a number between 0 and 255. It's not related to stdout, where your value "xyz" is written.
To match in stdout, you can just use grep:
if mediainfo "${BASE}_${suffix}_${suffix2}.mp4" | grep -q "©xyz"
then
echo "It contained that thing"
else
echo "It did not"
fi
I have a file with many service names, some of them are running, some of them aren't.
foo.service
bar.service
baz.service
I would like to find an efficient way to get the PIDs of the running processes started by the services (for the not running ones a 0, -1 or empty results are valid).
Desired output example:
foo.service:8484
bar.server:
baz.service:9447
(bar.service isn't running).
So far I've managed to do the following: (1)
cat t.txt | xargs -I {} systemctl status {} | grep 'Main PID' \
| awk '{print $3}'
With the following output:
8484
9447
But I can't tell which service every PID belongs to.
(I'm not bound to use xargs, grep or awk.. just looking for the most efficient way).
So far I've managed to do the following: (2)
for f in `cat t.txt`; do
v=`systemctl status $f | grep 'Main PID:'`;
echo "$f:`echo $v | awk '{print \$3}'`";
done;
-- this gives me my desired result. Is it efficient enough?
I ran into similar problem and fount leaner solution:
systemctl show --property MainPID --value $SERVICE
returns just the PID of the service, so your example can be simplified down to
for f in `cat t.txt`; do
echo "$f:`systemctl show --property MainPID --value $f`";
done
You could also do:
while read -r line; do
statuspid="$(sudo service $line status | grep -oP '(?<=(process|pid)\s)[0-9]+')"
appendline=""
[[ -z $statuspid ]] && appendline="${line}:${statuspid}" || appendline="$line"
"$appendline" >> services-pids.txt
done < services.txt
To use within a variable, you could also have an associative array:
declare -A servicearray=()
while read -r line; do
statuspid="$(sudo service $line status | grep -oP '(?<=(process|pid)\s)[0-9]+')"
[[ -z $statuspid ]] && servicearray[$line]="statuspid"
done < services.txt
# Echo output of array to command line
for i in "${!servicearray[#]}"; do # Iterating over the keys of the associative array
# Note key-value syntax
echo "service: $i | pid: ${servicearray[$i]}"
done
Making it more efficient:
To list all processes with their execution commands and PIDs. This may give us more than one PID per command, which might be useful:
ps -eo pid,comm
So:
psidcommand=$(ps -eo pid,comm)
while read -r line; do
# Get all PIDs with the $line prefixed
statuspids=$(echo $psidcommand | grep -oP '[0-9]+(?=\s$line)')
# Note that ${statuspids// /,} replaces space with commas
[[ -z $statuspids ]] && appendline="${line}:${statuspids// /,}" || appendline="$line"
"$appendline" >> services-pids.txt
done < services.txt
OUTPUT:
kworker:5,23,28,33,198,405,513,1247,21171,22004,23749,24055
If you're confident your file has the full name of the process, you can replace the:
grep -oP '[0-9]+(?=\s$line)'
with
grep -oP '[0-9]+(?=\s$line)$' # Note the extra "$" at the end of the regex
to make sure it's an exact match (in the grep without trailing $, line "mys" would match with "mysql"; in the grep with trailing $, it would not, and would only match "mysql").
Building up on Yorik.sar's answer, you first want to get the MainPID of a server like so:
for SERVICE in ...<service names>...
do
MAIN_PID=`systemctl show --property MainPID --value $SERVICE`
if test ${MAIN_PID} != 0
than
ALL_PIDS=`pgrep -g $MAIN_PID`
...
fi
done
So using systemctl gives you the PID of the main process controlled by your daemon. Then the pgrep gives you the daemon and a list of all the PIDs of the processes that daemon started.
Note: if the processes are user processes, you have to use the --user on the systemctl command line for things to work:
MAIN_PID=`systemctl --user show --property MainPID --value $SERVICE`
Now you have the data you are interested in the MAIN_PID and ALL_PIDS variables, so you can print the results like so:
if test -n "${ALL_PID}"
then
echo "${SERVICE}: ${ALL_PIDS}"
fi
I am checking to see if a process on a remote server has been killed. The code I'm using is:
if [ `ssh -t -t -i id_dsa headless#remoteserver.com "ps -auxwww |grep pipeline| wc -l" | sed -e 's/^[ \t]*//'` -lt 3 ]
then
echo "PIPELINE STOPPED SUCCESSFULLY"
exit 0
else
echo "PIPELINE WAS NOT STOPPED SUCCESSFULLY"
exit 1
fi
However when I execute this I get:
: integer expression expected
PIPELINE WAS NOT STOPPED SUCCESSFULLY
1
The actual value returned is "1" with no whitespace. I checked that by:
vim <(ssh -t -t -i id_dsa headless#remoteserver.com "ps -auxwww |grep pipeline| wc -l" | sed -e 's/^[ \t]*//')
and then ":set list" which showed only the integer and a line feed as the returned value.
I'm at a loss here as to why this is not working.
If the output of the ssh command is truly just an integer preceded by optional tabs, then you shouldn't need the sed command; the shell will strip the leading and/or trailing whitespace as unnecessary before using it as an operand for the -lt operator.
if [ $(ssh -tti id_dsa headless#remoteserver.com "ps -auxwww | grep -c pipeline") -lt 3 ]; then
It is possible that result of the ssh is not the same when you run it manually as when it runs in the shell. You might try saving it in a variable so you can output it before testing it in your script:
result=$( ssh -tti id_dsa headless#remoteserver.com "ps -auxwww | grep -c pipeline" )
if [ $result -lt 3 ];
The return value you get is not entirely a digit. Maybe some shell-metacharacter/linefeed/whatever gets into your way here:
#!/bin/bash
var=$(ssh -t -t -i id_dsa headless#remoteserver.com "ps auxwww |grep -c pipeline")
echo $var
# just to prove my point here
# Remove all digits, and look wether there is a rest -> then its not integer
test -z "$var" -o -n "`echo $var | tr -d '[0-9]'`" && echo not-integer
# get out all the digits to use them for the arithmetic comparison
var2=$(grep -o "[0-9]" <<<"$var")
echo $var2
if [[ $var2 -lt 3 ]]
then
echo "PIPELINE STOPPED SUCCESSFULLY"
exit 0
else
echo "PIPELINE WAS NOT STOPPED SUCCESSFULLY"
exit 1
fi
As user mbratch noticed I was getting a "\r" in the returned value in addition to the expected "\n". So I changed my sed script so that it stripped out the "\r" instead of the whitespace (which chepner pointed out was unnecessary).
sed -e 's/\r*$//'