I have the following bash script:
#!/bin/bash
KEY=$(./get-aws-profile.sh --profile=$1 --key)
SECRET=$(./get-aws-profile.sh --profile=$1 --secret)
ES="https://search-****.eu-west-1.es.amazonaws.com"
INDEXES=$(AWS_ACCESS_KEY_ID="$KEY" AWS_SECRET_ACCESS_KEY="$SECRET" aws-es-curl $ES/_cat/indices | awk '{print $3}')
YESTERDAY=$(date --date="yesterday" +"%Y.%m.%d")
for i in $INDEXES
do
# $i is like report-processing-2019.10.10
if grep "$YESTERDAY" $i; then
echo "Pulling documents from $i to $i.ndjson"
echo "docker run -it blaze blaze --host=$ES --index=$i --insecure > /tmp/$i.ndjson"
echo "Index $i saved"
else
echo no
fi
done
exit 0
where $i is like report-processing-2019.10.10
when i run this, i am not able to get the the lines which contain only lines with 2019.10.10 in them, what am i missing?
any advice is much appreciated
Related
I'm trying to format my output based on the result of a certain command. However, it's not printing out. Can you guys let me know what I am missing out.
VID=$(grep -iE $vvwwn ${TPAR_TEMP}/vvid_${TPAR_NAME} > ${TPAR_TEMP}/tvvid 2> /dev/null)
TVVID=$(cat ${TPAR_TEMP}/tvvid |awk '{print$3}' 2> /dev/null)
if [ "${TVVID}" = "32" ]; then
sh templ1
echo "$VID"
else
sh templ2
echo "$VID"
fi
templ1 is
#!/bin/bash
echo "========================================================================================================"
awk '
BEGIN {printf "%-27s %-6s %-32s %-8s %-8s %-9s %-4s\n" , "Name", "State", "VV_WWN", "VSize_MB", "Usr_Used_MB", "UsrCPG", "Prov"}'
echo "========================================================================================================"
The problem is with $VID been empty due to the redirect in the command grep -iE $vvwwn ${TPAR_TEMP}/vvid_${TPAR_NAME} > ${TPAR_TEMP}/tvvid, To make the problem easier fill in the variables and run the command in your terminal.
I have a problem with a bash script in Nagios. this is a script to get the space disk. When I used an IF section it returns (null) and when I don't set the variable in IF section, nagios display the variable correctly. I tried to run the script with nagios user and the result is good. Ex:
TOTAL=`/srv/eyesofnetwork/nagios/plugins/check_nt -H $2 -p 12489 -s "" -v USEDDISKSPACE -l $4 |awk -F"-" '{print $2}' |awk '{print $2}'`
if [ $TOTAL -gt 2 ] && [ $TOTAL -le 99 ];then
RUN=`/srv/eyesofnetwork/nagios/plugins/check_nrpe -H $2 -c ``check_drivesize -a drive=$4 'warning=free<2G' 'critical=free<1G' show-all 'perf-config=*(unit:G)' top-syntax='${status} : ${problem_list}'`
VAR=$(echo $RUN |grep -i ok |wc -l)
if [ $VAR -eq 1 ];then
echo "$RUN"
exit 0
fi
I tried all possibilities (for me..), with "", with '', with nothing. The variable $RUN is not displayed.
Thanks
Finally, I wrote a script in python and now it works correctly
I am trying to write a script in bash to read in arguments (usernames) and then return information about that username by searching the /etc/passwd file. I am not the root on the server that I am scripting this on, but I don't need to be. Here is the code that I have so far, but I dont think I am using the grep command right.
#!/bin/bash
if [ $# -eq 1 ]; then #if num of args =1; then continue on
UNAME=$1 #set arg to var UNAME
grep UNAME=$(grep $/etc/passwd)
if [ $UNAME == /etc/passwd ]; then #if UNAME exists then display info below
echo "-------------------------------------------"
echo "Username: $(grep UNAME/etc/passwd/ $f1) "
echo "User ID (UID): $(grep UNAME/etc/passwd/) "
echo "Group ID (GID): $(grep UNAME/etc/passwd/) "
echo "User info: $(grep UNAME/etc/passwd/) "
echo "Home directory: $(grep UNAME/etc/passwd/) "
echo "Command shell: $(grep UNAME/etc/passwd/) "
echo "-------------------------------------------"
else #if UNAME is nonexistant then display this error message
echo "-------------------------------------------"
echo ""$UNAME" does not exist."
echo "-------------------------------------------"
fi
fi
if [ $# -eq 0 ]; then #if num of args =0; then display this
echo "Not enough arguments provided."
echo "USAGE: $0 <user_name> [user_name] ..."
exit 1
fi
You're doing too much. grep is the wrong tool for the type of parsing you are trying to do. The solution given here is pretty sloppy since it reads the passwd file once for each user you are querying instead of reporting everything in one pass, but it's not unreasonable:
#!/bin/sh
for UNAME; do
while IFS=: read login passwd uid gid name home shell; do
if test "$login" = "$UNAME"; then
echo "-------------------------------------------"
echo "Username: $login"
echo "User ID (UID): $uid"
echo "Group ID (GID): $gid"
echo "User info: $name"
echo "Home directory: $home"
echo "Command shell: $shell"
echo "-------------------------------------------"
continue 2
fi
done < /etc/passwd
echo "No entry for user $UNAME" >&2
done
Before I get to the explanation, here is how I would have written your script:
cat ./test.sh
#!/bin/bash
UNAME=${1} #set arg to var UNAME
if [ -z ${UNAME} ]; then #if no argument is provided; then display this
echo "Not enough arguments provided."
echo "USAGE: $0 <user_name>"
exit 1
fi
if grep ${UNAME} /etc/passwd >/dev/null; then #if UNAME exists then display info below
echo "-------------------------------------------"
echo "Username: $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $1}' /etc/passwd)"
echo "User ID (UID): $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $3}' /etc/passwd)"
echo "Group ID (GID): $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $4}' /etc/passwd)"
echo "User info: $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $5}' /etc/passwd)"
echo "Home directory: $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $6}' /etc/passwd)"
echo "Command shell: $(awk -F ':' -v uname=${UNAME} '$0 ~ uname {print $7}' /etc/passwd)"
echo "-------------------------------------------"
else #if UNAME is nonexistant then display this error message
echo "-------------------------------------------"
echo "\"${UNAME}\" does not exist."
echo "-------------------------------------------"
fi
Explanation:
if can use grep directly to check for the existence of
your argument in passwd
UNAME= is how arguments are assigned, not
called - they are called with the $ - for example $UNAME
/etc/passwd is an absolute file name and not an argument, thus
$/etc/passwd doesn't mean anything to bash
argument=$(any old bash command) is telling bash that you want to
assign the output of "any old bash command" to "argument", so $()
has to contain a command with output for "argument" to be assigned
anything.
the >/dev/null sends the output of grep to the waste basket because we only care at this point whether or not it's successful so that if knows what to do
grep works like this: grep <the thing you want to find> <file> (note that spaces matter and to use "" if you want to include spaces in your search)
to get specific fields, awk is more useful. awk -F':' tells awk that I want : to define my field separators; -v uname=${UNAME} passes my argument to awk; within awk, $0 ~ uname checks the line for my argument (like grep); and {print $1} prints the first field, and so on
Also consider the limitation of partial matches. If, for example, I have a user JSmith and Smith, searching Smith will return both. This can be addressed with regex but will add a great deal of complexity to the script.
Here is a version with lighter awk usage which works with multiple arguments:
#!/bin/bash
for username; do
if grep ${username} /etc/passwd >/dev/null; then #if username exists then display info below
echo "---------------------------------------"
awk -F':' -v uname="${username}" '\
$0 ~ uname \
{print " Username: "$1\
"\n User ID (UID): "$3\
"\n Group ID (GID): "$4\
"\n User info: "$5\
"\n Home directory: "$6\
"\n Command shell: "$7}' /etc/passwd
echo "---------------------------------------"
else #if username is nonexistant then display this error message
echo "---------------------------------------"
echo "\"${username}\" does not exist."
echo "---------------------------------------"
fi
done
if [ -z ${1} ]; then #if no argument is provided; then display this
echo " Not enough arguments provided."
echo " USAGE: $0 <user name>"
exit 1
fi
*Credit to William Pursell for explaining the for loop
The best tool to read databases such as passwd is getent. This will fetch entries from a database supported by Name Service Switch Libraries. If you really need to limit reading the files databases, you can tell getent to use files database with -s files.
Your code is unnecessarily long. Fetching a record and parsing them into multiple variables should be enough.
#!/bin/bash
if [ $# -eq 1 ]; then #if num of args =1; then continue on
if IFS=: read username ignore uid gid gecos home shell < <(getent -s files passwd $1); then
echo "-------------------------------------------"
echo "Username: $username"
echo "User ID (UID): $uid"
echo "Group ID (GID): $gid"
echo "User info: $gecos"
echo "Home directory: $home"
echo "Command shell: $shell"
echo "-------------------------------------------"
else #if $1 is nonexistant then display this error message
echo "-------------------------------------------"
echo "'$1' does not exist."
echo "-------------------------------------------"
fi
fi
if [ $# -eq 0 ]; then #if num of args =0; then display this
echo "Not enough arguments provided."
echo "USAGE: $0 <user_name> [user_name] ..."
exit 1
fi
I'm working on a bash script that will check +1000 domains if they are expired. I use a a for loop to iterate over all users in /var/cpanel/users/*. It works great for like the 10 first users (loops) then it just hangs.
A weird thing is that I can stop the script with Ctrl+Z and then start the script again with fg and it continues to work normal for about +10 users but then it hangs again.
This is my scirpt:
# File that will have the result.
file="domain-result.txt"
printf "USER\t\tDOMAIN\t\t\tREPORT\n" > "$file"
printf "\n" >> "$file"
# For loop to iterate over all users in cpanel.
for z in /var/cpanel/users/*;
do
# Only files can be used.
if [[ -f "$z" ]]
then
# Get the domain name.
awk -F'=' '/DNS=/ {print $2}' "$z" | while read row;
do
# If there's no domain name than skip to next account.
if [[ -z "$row" ]]; then continue; fi
printf "Checking domain: %s...done\n" "$row"
# Execute whois command on the domain.
whois=$( /usr/bin/whois $row | grep 'not found' )
# Get the username.
user=$( echo "$z" | awk -F'/' '{print $5}' )
if [[ -n "$whois" ]]
then
printf "%s\t\t%s\t\t%s - EXPIRED\n" "$user" "$row" "$whois" >> "$file"
break
else
continue
fi
done
else
continue
fi
done
printf "\n"
printf "Total: $( sed '1,2d' "$file" | wc -l ) expired domains.\n"
This is a sample of how the files in /var/cpanel/users/* look like:
DNS=stackoverflow.com
Thank you Ignacio Vazquez-Abrams for pointing out WHOIS abuse. I got it to work by adding a sleep 2 to the for loop. Now it works great.
I have a script which is checking a key in one file against a key in another to see if it exists in both. However in the script the grep never returns anything has been found but on the command line it does.
#!/bin/bash
# First arg is the csv file of repo keys separated by line and in
# this manner 'customername,REPOKEY'
# Second arg is the log file to search through
log_file=$2
csv_file=$1
while read line;
do
customer=`echo "$line" | cut -d ',' -f 1`
repo_key=`echo "$line" | cut -d ',' -f 2`
if [ `grep "$repo_key" $log_file` ]; then
echo "1"
else
echo "0"
fi
done < $csv_file
The CSV file is formatted as follows:
customername,REPOKEY
and the log file is as follows:
REPOKEY
REPOKEY
REPOKEY
etc
I call the script by doing ./script csvfile.csv logfile.txt
Rather then checking output of grep command use grep -q to check its return status:
if grep -q "$repo_key" "$log_file"; then
echo "1"
else
echo "0"
fi
Also your script can be simplified to:
log_file=$2
csv_file=$1
while IFS=, read -r customer repo_key; do
if grep -q "$repo_key" "$log_file"; then
echo "1"
else
echo "0"
fi
done < "$csv_file"
use the exit status of the grep command to print 1 or 0
repo_key=`echo "$line" | cut -d ',' -f 2`
grep -q "$repo_key" $log_file
if [ $? -eq 1 ]; then
echo "1"
else
echo "0"
fi
-q supresses the output so that no output is printed
$? is the exit status of grep command 1 on successfull match and 0 on unsuccessfull
you can have a much simpler version as
grep -q "$repo_key" $log_file
echo $?
which will produce the same output