Bash getopt check parameter for every single user in system - bash

Ive done a script which works as follows
bash users.sh -u "user" [-f | -d | -g]
-f , shows the number of files of the specified user directory.
-d, shows the number of folders of the specified user directory.
-g , shows the groups to which the specified user belongs.
#!/bin/bash
user="" ;
directory="" ;
while getopts u:fgd options; do
case "$options" in
u)
if grep -q "^$OPTARG" /etc/passwd ; then
user="$OPTARG";
echo "user existe";
else
echo "user no existe"
fi
;;
g)
if [ -n "$user" ] ; then
echo "los grupos del usuario son: ";
groups "$user" ;
else
echo "user missing"
fi
;;
d)
if [ -n "$user" ] ; then
directory=$(grep $user /etc/passwd | cut -d : -f 6);
directory+=/*/;
echo "numero de carpetas";
ls -d $directory | wc -l
else
echo "user missing"
fi
;;
f)
if [ -n "$user" ] ; then
directory=$(grep $user /etc/passwd | cut -d : -f 6);
echo "numero de archivos";
ls -p $directory | grep -v / | wc -l
else
echo "user missing"
fi
;;
*) echo "opcion invalida $options" ;;
esac
done
The point is that my current script "must" include an user as an arguemnt. But I would want that if the username argument is missing
bash users.sh -u [-f | -d | -g]
It would show the specified option for each user of the system instead of a specific user

Related

Invalidate metadata gives TableNotFoundException although I can see the table in the list of tables

I am trying to run a script that validates if all the tables that are supposed to be created as a part of my deployment. This is my script:
set -x
ENV=$1
. /user/setenv.sh
ticket=""
wlf_kinit ticket
echo "Checking if Kerberos Ticket is available.."
klist $ticket
if [ $? -eq 1 ]; then
echo "Kerberos Ticket not found..Exiting"
exit 1
fi
echo "Validating Hadoop tables" > /tmp/psk1/db_validation_log.txt
db_dir=/user/db
cd $db_dir
for current_directory in `find . -maxdepth 1 -type d`
do
#current_directory=`echo $current_directory | awk -F '/' '{print $2}'`
echo $current_directory
if [ "$current_directory" != "." ]; then
current_directory=`echo $current_directory | awk -F '/' '{print $2}'`
if [ "${current_directory:0:1}" = "v" ]; then
dir=$db_dir/$current_directory/views
else
dir=$db_dir/$current_directory/tables
fi
cd $dir
pwd
#echo "Validating tables in "$current_directory >> /tmp/psk1/db_validation_log.txt
find . -name '*.hql' | while read rec; do
echo $rec
tbl_name=`echo $rec | awk -F '/' '{print $2}' | awk -F '.' '{print $1}'`
result=$(impala-shell --quiet --delimited --ssl -i ${impala_host} -ku ${user_id}${impala_realm} -q "set request_pool = ${request_pool}; use $current_directory$ENV; invalidate metadata $tbl_name; show tables like '$tbl_name';") 2>> /tmp/psk1/db_validation_log.txt
#echo $result
if [ $? -eq 0 ]; then
if [ ${result} == ${tbl_name} ]; then
echo "$tbl_name exists" #>> /tmp/psk1/db_validation_log.txt
else
echo "$tbl_name does not exist" #>> /tmp/psk1/db_validation_log.txt
fi
else
echo $current_directory$ENV"."$tbl_name" Query error" >> /tmp/psk1/db_validation_log.txt
impala-shell --quiet --delimited --ssl -i ${impala_host} -ku ${user_id}${impala_realm} -q "set request_pool = ${request_pool}; use $current_directory$ENV; invalidate metadata $tbl_name; show tables like '$tbl_name';" 2>> /tmp/psk1/db_validation_log.txt
fi
done
fi
done
cat /tmp/psk1/db_validation_log.txt | mail -a /tmp/psk1/db_validation_log.txt -s 'Hadoop DB validation completed. Check the attached log.' -r ${from_email} ${to_email} 2>> /tmp/psk1/db_validation_log.txt
kdestroy -c $ticket
This script prints the failed queries to a .txt file and sends it in an email. In the text file I see some of the queries failing with TableNotFoundException. But when I open the impala shell and list out the tables, I am able to see the table in the list. I am not sure what is causing this inconsistency.
Any help would be appreciated. Thank you.

bash script, command "select" hold scrolling

I have this bash script I would like to pause so I can read what is echoed. I use this on a mobile phone so the screen is very small. The continent output is in 2 colums by default, can I do this with the city output aswell
I tried putting " | more " before && but it didn't work.
normaly I put everything in functions() but finding the select command here on stackoverflow.com made it much faster.
#!/bin/bash
PS3=" Enter a Number "
dir=/usr/share/zoneinfo
echo
echo " Please select your Continent "
echo
select continent in $dir/* ; do
test -n "$continent" && break;
echo ">>> Invalid Selection";
done
selected_cont=($(echo "$continent" | grep -Eo '[^/]+/?$' | cut -d'/' -f2 ))
echo
echo " Selected $selected_cont as your Continent "
echo
echo " Please select nearest city "
select city in $continent/* ; do
test -n "$city" && break;
echo ">>> Invalid Selection";
done
echo
selected_city=($(echo "$city" | grep -Eo '[^/]+/?$' | cut -d'/' -f2 ))
timezone="$selected_cont/$selected_city"
echo
echo
echo " $timezone is selected as your timezone "
#echo "show Continent: $continent"
#echo "show City: $city"
sleep 2
#echo export TZ=$city >> $HOME/.bashrc
#sudo ln -sf $city /etc/localtime
This was my first working try
#!/bin/bash
dir=("/usr/share/zoneinfo/")
continent=$(dialog --stdout --title "Select Continent" --dselect $dir 0 0 --no-mouse --aspect 9 --ascii-lines)
city=$(dialog --stdout --title "Please choose a file" --fselect $continent/ 14 48)
#echo ${continent} selected
#echo ${city} selected
selected_cont=($(echo ${continent} | grep -Eo '[^/]+/?$' | cut -d'/' -f2))
# Strip full path to last directory to get the continent. use it with $selected_continent
selected_city=($(echo ${city} | grep -Eo '[^/]+/?$' | cut -d'/' -f2))
# Strip full path to last directory to get the nearest city. use it with $selected_city
# echo "export TZ=(${continent/${city})" >> $HOME/.bashrc
# sudo ln -sf ${city} /etc/localtime
echo $selected_cont selected as continent
echo selected $selected_city as nearest city
echo "set TZ in bashrc TZ=$selected_cont/$selected_city "
echo "link ${city} to /etc/localtime "
########EOF###########
This is my second try with directory and file check
#!/bin/bash
start() {
if [[ -z $selected_cont ]]; then
continent=($(dialog --stdout --title "Select Continent" --dselect $dir 0 0 --no-mouse --aspect 9 --ascii-lines));
selected_cont=($(echo ${continent} | grep -Eo '[^/]+/?$' | cut -d'/' -f2));
if [[ -d $dir$selected_cont ]]; then
echo $dir$selected_cont
start
else
selected_cont=
start
fi
elif [[ -z $selected_city ]]; then
city=($(dialog --stdout --title "Please choose a file" --fselect $continent/ 14 48));
selected_city=($(echo ${city} | grep -Eo '[^/]+/?$' | cut -d'/' -f2));
if [[ -f ${city} ]]; then
echo $selected_city
start
else
selected_city=
start
fi
else
# echo "export TZ=(${continent/${city})" >> $HOME/.bashrc
# sudo ln -sf ${city} /etc/localtime
echo $selected_cont selected as continent
echo selected $selected_city as nearest city
echo "set TZ in bashrc TZ=$selected_cont/$selected_city "
echo "link ${city} to /etc/localtime "
fi
}
start
######EOF######
This 2nd one is preferred but it errors on line 6 continent= Error: Expected a box option. The script continuous as wanted and the result is as wanted.
Can anyone clarify the error as I use the same variable in script one without errors?
I've updated to output to a temp file no errors in this script
#!/bin/bash
dir=("/usr/share/zoneinfo/")
temp=/tmp/test.$$
start() {
if [[ -z $selected_cont ]]; then
continent=($(dialog --stdout --title "Select Continent" --dselect $dir 0 0 --no-mouse --aspect 9 --ascii-lines 2>&1 > $temp));
selected_cont=($(cat $temp | grep -Eo '[^/]+/?$' ));
echo
if [[ -d $dir$selected_cont ]]; then
start
else
selected_cont=
start
fi
elif [[ -z $selected_city ]]; then
city=($(dialog --stdout --title "Please choose a file" --fselect $dir$selected_cont/ 0 0 2>&1 > $temp));
selected_city=($(cat $temp | grep -Eo '[^/]+/?$' ));
if [[ -f $(cat $temp) ]]; then
start
else
selected_city=
start
fi
else
# echo "export TZ=(${continent/${city})" >> $HOME/.bashrc
# sudo ln -sf $(cat $temp) /etc/localtime
echo $selected_cont selected as continent
echo selected $selected_city as nearest city
echo "set TZ in bashrc TZ=$selected_cont/$selected_city "
echo link $(cat $temp) to /etc/localtime
fi
}
start

Syntax error: “(” unexpected (expecting “fi”)

filein="users.csv"
IFS=$'\n'
if [ ! -f "$filein" ]
then
echo "Cannot find file $filein"
else
#...
groups=(`cut -d: -f 6 "$filein" | sed 's/ //'`)
fullnames=(`cut -d: -f 1 "$filein"`)
userid=(`cut -d: -f 2 "$filein"`)
usernames=(`cut -d: -f 1 "$filein" | tr [A-Z] [a-z] | awk '{print substr($1,1,1) $2}'`)
#...
for group in ${groups[*]}
do
grep -q "^$group" /etc/group ; let x=$?
if [ $x -eq 1 ]
then
groupadd "$group"
fi
done
#...
x=0
created=0
for user in ${usernames[*]}
do
useradd -n -c ${fullnames[$x]} -g "${groups[$x]}" $user 2> /dev/null
if [ $? -eq 0 ]
then
let created=$created+1
fi
#...
echo "${userid[$x]}" | passwd --stdin "$user" > /dev/null
#...
echo "Welcome! Your account has been created. Your username is $user and temporary
password is \"$password\" without the quotes." | mail -s "New Account for $user" -b root $user
x=$x+1
echo -n "..."
sleep .25
done
sleep .25
echo " "
echo "Complete. $created accounts have been created."
fi
I'm guessing the problem is that you're trying to capture command output in arrays without actually using command substitution. You want something like this:
groups=( $( cut... ) )
Note the extra set of parentheses with $ in front of the inner set.

Bash script help/evaluation

I'm trying to learn some scripting however I can't find solution for one functionality.
Basically I would like to ask to evaluate my script as it's probably possible to reduce the complexity and number of lines.
The purpose of this script is to download random, encrypted MySQL backups from Amazon S3, restore the dump and run some random MySQL queries.
I'm not sure how to email the output from printf statements - one is for headers and second one for actual data. I've tried to format the output so it looks like below but I had to exclude the headers from the loop:
Database: Table: Entries:
database1 random_table 0
database2 random_table 0
database3 random_table 0
database4 random_table 0
I would like to include this output in the email and also change the email subject based on the success/failure of the script.
I probably use to much if loops and MySQL queries are probably to complicated.
Script:
#!/usr/bin/env bash
# DB Details:
db_user="user"
db_pass="password"
db_host="localhost"
# Date
date_stamp=$(date +%d%m%Y)
# Initial Setup
data_dir="/tmp/backup"
# Checks
if [ ! -e /usr/bin/s3cmd ]; then
echo "Required package (http://s3tools.org/s3cmd)"
exit 2
fi
if [ -e /usr/bin/gpg ]; then
gpg_key=$(gpg -K | tr -d "{<,>}" | awk '/an#example.com/ { print $4 }')
if [ "$gpg_key" != "an#example.com" ]; then
echo "No GPG key"
exit 2
fi
else
echo "No GPG package"
exit 2
fi
if [ -d $data_dir ]; then
rm -rf $data_dir/* && chmod 700 $data_dir
else
mkdir $data_dir && chmod 700 $data_dir
fi
# S3 buckets
bucket_1=s3://test/
# Download backup
for backup in $(s3cmd ls s3://test/ | awk '{ print $2 }')
do
latest=$(s3cmd ls $backup | awk '{ print $2 }' | sed -n '$p')
random=$(s3cmd ls $latest | shuf | awk '{ print $4 }' | sed -n '1p')
s3cmd get $random $data_dir >/dev/null 2>&1
done
# Decrypting Files
for file in $(ls -A $data_dir)
do
filename=$(echo $file | sed 's/\.e//')
gpg --out $data_dir/$filename --decrypt $data_dir/$file >/dev/null 2>&1 && rm -f $data_dir/$file
if [ $? -eq 0 ]; then
# Decompressing Files
bzip2 -d $data_dir/$filename
if [ $? -ne 0 ]; then
echo "Decompression Failed!"
fi
else
echo "Decryption Failed!"
exit 2
fi
done
# MySQL Restore
printf "%-40s%-30s%-30s\n\n" Database: Table: Entries:
for dump in $(ls -A $data_dir)
do
mysql -h $db_host -u $db_user -p$db_pass < $data_dir/$dump
if [ $? -eq 0 ]; then
# Random DBs query
db=$(echo $dump | sed 's/\.sql//')
random_table=$(mysql -h $db_host -u $db_user -p$db_pass $db -e "SHOW TABLES" | grep -v 'Tables' | shuf | sed -n '1p')
db_entries=$(mysql -h $db_host -u $db_user -p$db_pass $db -e "SELECT * FROM $random_table" | grep -v 'id' | wc -l)
printf "%-40s%-30s%-30s\n" $db $random_table $db_entries
mysql -h $db_host -u $db_user -p$db_pass -e "DROP DATABASE $db"
else
echo "The system was unable to restore backups!"
rm -rf $data_dir
exit 2
fi
done
#Remove backups
rm -rf $data_dir
You'll get the best answers if you ask specific questions (rather than, "please review my code")...and if you limit each post to a single question. Regarding emailing the output of your printf statements:
You can group statements into a block and then pipe the output of a block into another program. For example:
{
echo "This is a header"
echo
for x in {1..10}; do
echo "This is row $x"
done
} | mail -s "Here is my output" lars#example.com
If you want to make the email subject conditional upon the success or
failure of something elsewhere in the script, you can (a) save your
output to a file, and then (b) email the file after building the
subject line:
{
echo "This is a header"
echo
for x in {1..10}; do
echo "This is row $x"
done
} > output
if is_success; then
subject="SUCCESS: Here is your output"
else
subject="FAILURE: Here are your errors"
fi
mail -s "$subject" lars#example.com < output

Search for user files in relative and absolute path

I have cript which search for user files in absolute path, but now I need to add for relative also. How can I do this?
#!/usr/local/bin/bash
if [ $# -eq 2 ]; then
DIR=$1
USERNAME=$2
while [ ! $(id -g $USERNAME) ]; do
echo "non-existing username, please add again "
read USERNAME
done
while [ ! -d "/home/$DIR" ]; do
echo "non-existing '$DIR' directory, please add again "
read DIR
done
echo "variable username is now $USERNAME"
echo "variable DIR is now $DIR"
echo -e "username group file"
ls -1Apl /home/$DIR | grep -v /\$ | awk -v user=${USERNAME} '$3==user{print}{}' | tr -s ' ' | cut -d ' ' -f3 -f4 -f9
else
echo "bla"
fi
i don't need damn abolute and relative path with freakin' ls :/
so
while [ ! -d "$DIR" ]; do
and
ls -1Apl $DIR | grep -v /\$
and it is working

Resources