I try to make a script to connect with MySQL.
Reading hosts from one file and the MySQL password from another file, but I have a problem.
When I try to execute the script it's returning me this error:
./do: line 15: syntax error: unexpected end of file
The code is like this:
#!/bin/bash
FILE=$1
INFO=$2
cat $FILE | while read HOST;
cat $INFO | while read INFO;do
DBS=`mysql -u root -p $INFO -h $HOST --connect_timeout=4 -Bse'show databases' | wc -l`
if [ "$DBS" -gt "0" ]; then
echo $HOST - mysql - $DBS >> log.sql
fi
sleep 0.1
done
Where is my mistake ?
Salut Dragoș,
You can't use two while loops (even after you fix the syntax error) to read from two different files at the same time.
Instead, you can use paste to combine the two files first, then execute your loop:
#!/bin/bash
hostnames="$1"
passwords="$2"
while IFS=$'\t' read host password; do
dbs=$(mysql -u root -p "$password" -h "$host" --connect_timeout=4 -Bse'show databases' | wc -l)
[ $dbs -gt 0 ] && echo "$host - mysql - $dbs" >> log.sql
done <<<"$(paste "$hostnames" "$passwords")"
This script will correctly handle filenames with spaces, as well as hostnames and passwords containing spaces.
Related
I have a set of search strings in a file (File1) and a content file (File2). I am trying to loop through all the search strings within File1 and get a count of each of the search string within File2 and output it - I want to automate this and make it generic so I can search through multiple content files. However, I dont seem to be able to get the exact count when I execute this loop. I get a "0" count for each of the strings although I have those strings in the file. Unable to figure out what I am doing wrong and can use some help !
Below is the script I came up with:
#!/bin/bash
while IFS='' read -r line || [[ -n "$line" ]]; do
count=$(echo cat "$2" | grep -c "$line")
echo "$count - $line"
done < "$1"
Command I am using to run this script:
./scanscript.sh File1.log File2.log
I say this since I searched this command separately and get the right value. This command works by itself but I want to put this in a loop
cat File2.log | grep -c "Search String"
Sample Data for File 1 (Search Strings):
/SERVER_NAME/Root/DEV/Database/NJ-CONTENT/Procs/
/SERVER_NAME3/Root/DEV/Database/NJ-CONTENT/Procs/
Sample Data for File 2 (Content File):
./SERVER_NAME/Root/DEV/Database/NJ-CONTENT/Procs/test.test_proc.sql:29:
./SERVER_NAME2/Root/DEV/Database/NJ-CONTENT/Procs/test.test_proc.sql:100:
./SERVER_NAME3/Root/DEV/Database/NJ-CONTENT/Procs/test.test_proc.sql:143:
./SERVER_NAME4/Root/DEV/Database/NJ-CONTENT/Procs/test.test_proc.sql:223:
./SERVER_NAME5/Root/DEV/Database/NJ-CONTENT/Procs/test.test_proc.sql:5589:
Problem is this line:
count=$(echo cat "$2" | grep -c "$line")
That should be changed to:
count=$(grep -Fc "$line" "$2")
Also note -F is to be used for fixed string search instead of regex search.
Full code:
while IFS='' read -r line || [[ -n "$line" ]]; do
count=$(grep -Fc "$line" "$2");
echo "$count - $line";
done < "$1"
Run it as:
./scanscript.sh File1.log File2.log
Output:
1 - /SERVER_NAME/Root/DEV/Database/NJ-CONTENT/Procs/
1 - /SERVER_NAME3/Root/DEV/Database/NJ-CONTENT/Procs/
I'm trying to deploy a cluster with a script which uses a yaml file. Except for an entry called "RaftFS" each yaml entry represents a machine to deploy. I don't understand why the script does only one while cycle if the ssh command is executed (even if the command is a simple ls !) but if I delete it then everything is fine and it does a number of cycle equals to the number of machines defined in the yaml file!
cat RaftFS/servers.yaml | shyaml keys-0 |
while read -r -d $'\0' value; do
if [ ! $value == "RaftArgs" ]; then
address=$(cat RaftFS/servers.yaml | shyaml get-value $value.machineIP | xargs -0 -n 1 echo)
username=$(cat RaftFS/servers.yaml | shyaml get-value $value.username | xargs -0 -n 1 echo)
password=$(cat RaftFS/servers.yaml | shyaml get-value $value.password | xargs -0 -n 1 echo)
#uploading my fingerprint (in order to use pssh)
echo $address $username $password
echo "uploading my fingerprint on $username#$address $password"
sshpass -p $password ssh-copy-id -oStrictHostKeyChecking=no $username#$address
echo "creating RaftFS"
ssh $username#$address echo "MACHINE=$value vagrant up">>vagrantscript.sh
fi
echo $address $username $password
done
I think there is no issue with the ssh command but it is a delimiter's issue
I've played a little with read -r -d $'\0' and these are the results
echo "a\0b\0c" | while read -r -d $'\0' var; do echo $var; done
prints
a
b
and
echo "a\0b\0c\0" | while read -r -d $'\0' var; do echo $var; done
prints
a
b
c
I assume there is some difference in the end line when the $value == "RaftArgs"
The standard input to the while loop is also the standard input to every command within the while loop. ssh reads from standard input in order to pipe the data to the remote command. It's probably consuming the data intended for the read statement.
You can redirect the ssh command's input:
ssh $username#$address ... >>vagrantscript.sh < /dev/null
Or you can run ssh with the "-n" flag to prevent reading from stdin:
ssh -n $username#$address ... >>vagrantscript.sh
I need to check if the recipient username is in file /etc/passwd which contains all the users in my class, but I have tried a few different combinations of if statements and grep without success. The best I could come up with is below, but I don't think it's working properly.
My logic behind it is that if the grep is null, the user is invalid.
send_email()
{
message=
address=
attachment=
validuser=1
until [ "$validuser" = "0" ]
do
echo "Enter the email address: "
read address
if [ -z grep $address /etc/passwd ]
then
validuser=0
else
validuser=1
fi
echo -n "Enter the subject of the message: "
read message
echo ""
echo "Enter the file you want to attach: "
read attachment
mail -s "$message" "$address"<"$attachment"
done
press_enter
}
Just do a simple if like this:
if grep -q $address /etc/passwd
then
echo "OK";
else
echo "NOT OK";
fi
The -q option is used here just to make grep quiet (don't output...)
Use getent and check for grep's exit code. Avoid using /etc/passwd. Equivalent in the shell:
getent passwd | grep -q valid_user
echo $?
Output:
0
And:
getent passwd | grep -q invalid_user
echo $?
Output:
1
Your piece of code:
if [ -z grep $address /etc/passwd ]
You haven't saved the results of grep $address /etc/passwd in a variable. Before putting it in the if statement and then testing the variable to see if it is empty.
You can try it like this:
check_address=`grep $address /etc/passwd`
if [ -z "$check_address" ]
then
validuser=0
else
validuser=1
fi
The -z check is for variable strings, which your grep isn't giving. To give a value from your grep command, enclose it in $():
if [ -z $(grep $address /etc/passwd) ]
The easiest one will be this:
cat test123
# Output: 12345678
cat test123 | grep 123 >/dev/null && echo "grep result exist" || echo "grep result doesn't exist"
# Output: grep result exist
cat test123 | grep 999 >/dev/null && echo "grep result exist" || echo "grep result doesn't exist"
# Output: grep result doesn't exist
My problem was that the file I was trying to grep was a binary file. On windows, the first two characters in the file were little squares. On mac, the first two characters were question marks. When I used more or less on the file, I could see it was binary and when I used diff, it responded that the "Binary files foo.log and requirements.txt differ".
I used cat to display the contents of the file, highlighted and copied the text (minus the two question marks at the top, deleted the file, created a new file with touch and then used vi to paste the text back into the new file.
After that, grep worked fine.
Shorter:
until ! grep $address /etc/passwd ; do {
do_your_stuff
}
I'm trying to learn some scripting however I can't find solution for one functionality.
Basically I would like to ask to evaluate my script as it's probably possible to reduce the complexity and number of lines.
The purpose of this script is to download random, encrypted MySQL backups from Amazon S3, restore the dump and run some random MySQL queries.
I'm not sure how to email the output from printf statements - one is for headers and second one for actual data. I've tried to format the output so it looks like below but I had to exclude the headers from the loop:
Database: Table: Entries:
database1 random_table 0
database2 random_table 0
database3 random_table 0
database4 random_table 0
I would like to include this output in the email and also change the email subject based on the success/failure of the script.
I probably use to much if loops and MySQL queries are probably to complicated.
Script:
#!/usr/bin/env bash
# DB Details:
db_user="user"
db_pass="password"
db_host="localhost"
# Date
date_stamp=$(date +%d%m%Y)
# Initial Setup
data_dir="/tmp/backup"
# Checks
if [ ! -e /usr/bin/s3cmd ]; then
echo "Required package (http://s3tools.org/s3cmd)"
exit 2
fi
if [ -e /usr/bin/gpg ]; then
gpg_key=$(gpg -K | tr -d "{<,>}" | awk '/an#example.com/ { print $4 }')
if [ "$gpg_key" != "an#example.com" ]; then
echo "No GPG key"
exit 2
fi
else
echo "No GPG package"
exit 2
fi
if [ -d $data_dir ]; then
rm -rf $data_dir/* && chmod 700 $data_dir
else
mkdir $data_dir && chmod 700 $data_dir
fi
# S3 buckets
bucket_1=s3://test/
# Download backup
for backup in $(s3cmd ls s3://test/ | awk '{ print $2 }')
do
latest=$(s3cmd ls $backup | awk '{ print $2 }' | sed -n '$p')
random=$(s3cmd ls $latest | shuf | awk '{ print $4 }' | sed -n '1p')
s3cmd get $random $data_dir >/dev/null 2>&1
done
# Decrypting Files
for file in $(ls -A $data_dir)
do
filename=$(echo $file | sed 's/\.e//')
gpg --out $data_dir/$filename --decrypt $data_dir/$file >/dev/null 2>&1 && rm -f $data_dir/$file
if [ $? -eq 0 ]; then
# Decompressing Files
bzip2 -d $data_dir/$filename
if [ $? -ne 0 ]; then
echo "Decompression Failed!"
fi
else
echo "Decryption Failed!"
exit 2
fi
done
# MySQL Restore
printf "%-40s%-30s%-30s\n\n" Database: Table: Entries:
for dump in $(ls -A $data_dir)
do
mysql -h $db_host -u $db_user -p$db_pass < $data_dir/$dump
if [ $? -eq 0 ]; then
# Random DBs query
db=$(echo $dump | sed 's/\.sql//')
random_table=$(mysql -h $db_host -u $db_user -p$db_pass $db -e "SHOW TABLES" | grep -v 'Tables' | shuf | sed -n '1p')
db_entries=$(mysql -h $db_host -u $db_user -p$db_pass $db -e "SELECT * FROM $random_table" | grep -v 'id' | wc -l)
printf "%-40s%-30s%-30s\n" $db $random_table $db_entries
mysql -h $db_host -u $db_user -p$db_pass -e "DROP DATABASE $db"
else
echo "The system was unable to restore backups!"
rm -rf $data_dir
exit 2
fi
done
#Remove backups
rm -rf $data_dir
You'll get the best answers if you ask specific questions (rather than, "please review my code")...and if you limit each post to a single question. Regarding emailing the output of your printf statements:
You can group statements into a block and then pipe the output of a block into another program. For example:
{
echo "This is a header"
echo
for x in {1..10}; do
echo "This is row $x"
done
} | mail -s "Here is my output" lars#example.com
If you want to make the email subject conditional upon the success or
failure of something elsewhere in the script, you can (a) save your
output to a file, and then (b) email the file after building the
subject line:
{
echo "This is a header"
echo
for x in {1..10}; do
echo "This is row $x"
done
} > output
if is_success; then
subject="SUCCESS: Here is your output"
else
subject="FAILURE: Here are your errors"
fi
mail -s "$subject" lars#example.com < output
I've created a script for account creation that reads from a csv file, i do not need the first line in the csv as it has the titles for the colums, im trying to delete the first line using sed 1d $file, but it doesnt seem to work.
#!/bin/bash
FILE="applicants.csv"
sed 1d $FILE |while IFS=: read USERNAME PASSWORD SCHOOL PROGRAM STATUS; do
#------------------------------------------
groupadd -f $SCHOOL
useradd $USERNAME -p $PASSWORD -g $SCHOOL
if [ $? == 0 ]; then
echo
echo "Success! $USERNAME created"
grep $USERNAME /etc/passwd
echo
#------------------------------------------
else
echo "Failed to create account for $USERNAME"
fi
done < $FILE
here is the csv file
Full Name:DOB:School:Program:Status
JoseDavid:22-08-86:ACE:Bsc Computing:Unfinished
YasinAhmed:22-07-85:ACE:Bsc Networking:Complete
MohammedAli:21-04-84:ACE:Bsc Forensics:Complete
UtahKing:22-09-84:ACE:BSC IT:Unfinished
UsmanNaeem:21-09-75:ACE:BSC Computing:Complete
Here is a screenshot of the output
http://i.stack.imgur.com/R5zPN.jpg
is there anyway to skip the first line?
Try using tail -n +2 $FILE instead of sed.
You're reading from the unedited file with the redirect at the end: done < $FILE. Try changing that line to just done.