i have a created a code to parse hostname,username and privilege from files in a dir..how ever i am not able to exit at the end of the code .the loop gives perfact data output.but not giving shell prompt back.
array arg[] contain name of file value
c=0
while((c<=${#arg[*]}))
do
hname=`grep -e "^hostname" ${arg[$c]} |awk '{ print $2 }'| sort |uniq`
if [[ -n $hname ]]
then
logDebug "data found for the $hname"
while read -r line; do
for term in "echo $line"; do
if [[ "$term" =~ (username)[[:space:]](.*?) ]]; then
userid=`echo "${BASH_REMATCH[2]}" | awk '{print $1}'`
logDebug "username: $userid"
if [[ "$term" =~ (privilege)[[:space:]](.*?) ]]; then
Priv=`echo "${BASH_REMATCH[2]}" | awk '{print $1}'`
PRIV=`echo "level=$Priv"`
logDebug "Privilege: $PRIV"
echo"$CUSTOMER|S|$hname|$OS|$userid|$Uid_Conv|$Uic_mode|$State|$i_login|$group|$PRIV|" >> "$OutputFile"
fi
fi
done
done < "${arg[$c]}"
fi
let c=c+1
done
For your first while clause...I would use
for ((c=0;c<${#arg[*];c++)); do
.....
done
Getting rid of the c=0 and let c=c+1
Related
Complete novice in Bash. Trying to iterate thru 1000 gzip files, may be GNU parallel is the solution??
#!/bin/bash
ctr=0
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
for f in "$dir"/*.gz; do
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr" >> $1
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym
else
ctr=$((ctr+1))
fi
done
done
Any help to speed the process will be greatly appreciated !!!
#!/bin/bash
ctr=0
export ctr
echo "file_name,symbol,record_count" > $1
dir="/data/myfolder"
export dir
doit() {
f="$1"
gunzip -c $f | while read line;
do
str=`echo $line | cut -d"|" -f1`
if [ "$str" == "H" ]; then
if [ $ctr -gt 0 ]; then
echo "$f,$sym,$ctr"
fi
ctr=0
sym=`echo $line | cut -d"|" -f3`
echo $sym >&2
else
ctr=$((ctr+1))
fi
done
}
export -f doit
parallel doit ::: *gz 2>&1 > $1
The Bash while read loop is probably your main bottleneck here. Calling multiple external processes for simple field splitting will exacerbate the problem. Briefly,
while IFS="|" read -r first second third rest; do ...
leverages the shell's built-in field splitting functionality, but you probably want to convert the whole thing to a simple Awk script anyway.
echo "file_name,symbol,record_count" > "$1"
for f in "/data/myfolder"/*.gz; do
gunzip -c "$f" |
awk -F "\|" -v f="$f" -v OFS="," '
/H/ { if(ctr) print f, sym, ctr
ctr=0; sym=$3;
print sym >"/dev/stderr"
next }
{ ++ctr }'
done >>"$1"
This vaguely assumes that printing the lone sym is just for diagnostics. It should hopefully not be hard to see how this can be refactored if this is an incorrect assumption.
I tried with below script. But, not working for cut the first field of each line and to be executed for "chmod".
#!/bin/bash
if [ -z "$1" ]; then
echo -e "Usage: $(basename $0) FILE\n"
exit 1
fi
if [ ! -e "$1" ]; then
echo -e "$1: File doesn't exist.\n"
exit 1
fi
while read -r line; do
awk '{print $1}'
[ -n "$line" ] && chown root "$line" && echo -e "$line Ownership changed"
done < "$1"
If field separator is space, try this:
while read -r line; do
FILE_TO_CHANGE=$(echo $line | awk '{print $1}')
[ -n "$line" ] && chown root "$FILE_TO_CHANGE" && echo -e "$line Ownership changed"
done < "$1"
awk read $line and print first token on standard output, the result is saved in FILE_TO_CHANGE variable and then it is used to run chown.
Another way could be:
awk '{print $1}' $1 | while read line; do
chown root "$line" && echo -e "$line Ownership changed"
done
awk read your file and print the first field of each line, in this case, while loop read awk output line by line and run chown on field.
You could extract the first word on each line with awk and pipe to xargs, invoking chown only as few times as possible:
awk '{print $1}' "$1" | xargs chown root
I'm working on a bash script that will check +1000 domains if they are expired. I use a a for loop to iterate over all users in /var/cpanel/users/*. It works great for like the 10 first users (loops) then it just hangs.
A weird thing is that I can stop the script with Ctrl+Z and then start the script again with fg and it continues to work normal for about +10 users but then it hangs again.
This is my scirpt:
# File that will have the result.
file="domain-result.txt"
printf "USER\t\tDOMAIN\t\t\tREPORT\n" > "$file"
printf "\n" >> "$file"
# For loop to iterate over all users in cpanel.
for z in /var/cpanel/users/*;
do
# Only files can be used.
if [[ -f "$z" ]]
then
# Get the domain name.
awk -F'=' '/DNS=/ {print $2}' "$z" | while read row;
do
# If there's no domain name than skip to next account.
if [[ -z "$row" ]]; then continue; fi
printf "Checking domain: %s...done\n" "$row"
# Execute whois command on the domain.
whois=$( /usr/bin/whois $row | grep 'not found' )
# Get the username.
user=$( echo "$z" | awk -F'/' '{print $5}' )
if [[ -n "$whois" ]]
then
printf "%s\t\t%s\t\t%s - EXPIRED\n" "$user" "$row" "$whois" >> "$file"
break
else
continue
fi
done
else
continue
fi
done
printf "\n"
printf "Total: $( sed '1,2d' "$file" | wc -l ) expired domains.\n"
This is a sample of how the files in /var/cpanel/users/* look like:
DNS=stackoverflow.com
Thank you Ignacio Vazquez-Abrams for pointing out WHOIS abuse. I got it to work by adding a sleep 2 to the for loop. Now it works great.
I have these diff results saved to a file:
bash-3.00$ cat /tmp/voo
18633a18634
> sashabrokerSTP
18634a18636
> sashatraderSTP
21545a21548
> yheemustr
I just really need the logins:
bash-3.00$ cat /tmp/voo | egrep ">|<"
> sashaSTP
> sasha
> yhee
bash-3.00$
But when I try to iterate through them and just print the names I get errors.
I just do not understand the fundamentals of using "if" with "while loops".
Ultimately, I want to use the while loop because I want to do something to the lines - and apparently while only loads one line into memory at a time, as opposed to the whole file at once.
bash-3.00$ while read line; do if [[ $line =~ "<" ]] ; then echo $line ; fi ; done < /tmp/voo
bash-3.00$
bash-3.00$
bash-3.00$ while read line; do if [[ egrep "<" $line ]] ; then echo $line ; fi ; done < /tmp/voo
bash: conditional binary operator expected
bash: syntax error near `"<"'
bash-3.00$
bash-3.00$ while read line; do if [[ egrep ">|<" $line ]] ; then echo $line ; fi ; done < /tmp/voo
bash: conditional binary operator expected
bash: syntax error near `|<"'
bash-3.00$
There has to be a way to loop through the file and then do something to each line. Like this:
bash-3.00$ while read line; do if [[ $line =~ ">" ]];
then echo $line | tr ">" "+" ;
if [[ $line =~ "<" ]];
then echo $line | tr "<" "-" ;
fi ;
fi ;
done < /tmp/voo
+ sashab
+ sashat
+ yhee
bash-3.00$
You should be checking for >, not <, no?
while read line; do
if [[ $line =~ ">" ]]; then
echo $line
fi
done < /tmp/voo
Do you really need regex here? The following shell glob can also work:
while read line; do [[ "$line" == ">"* ]] && echo "$line"; done < /tmp/voo
OR use AWK:
awk '/^>/ { print "processing: " $0 }' /tmp/voo
grep will do:
$ grep -oP '> \K\w+' <<END
18633a18634
> sashabrokerSTP
18634a18636
> sashatraderSTP
21545a21548
> yheemustr
END
sashabrokerSTP
sashatraderSTP
yheemustr
I have the following code
for ip in $(ifconfig | awk -F ":" '/inet addr/{split($2,a," ");print a[1]}')
do
bytesin=0; bytesout=0;
while read line
do
if [[ $(echo ${line} | awk '{print $1}') == ${ip} ]]
then
increment=$(echo ${line} | awk '{print $4}')
bytesout=$((${bytesout} + ${increment}))
else
increment=$(echo ${line} | awk '{print $4}')
bytesin=$((${bytesin} + ${increment}))
fi
done < <(pmacct -s | grep ${ip})
echo "${ip} ${bytesin} ${bytesout}" >> /tmp/bwacct.txt
done
Which I would like to print the incremented values to bwacct.txt, but instead the file is full of zeroes:
91.227.223.66 0 0
91.227.221.126 0 0
127.0.0.1 0 0
My understanding of Bash is that a redirected for loop should preserve variables. What am I doing wrong?
First of all, simplify your script! Usually there are many better ways in bash. Also most of the time you can rely on pure bash solutions instead of running awk or other tools.
Then add some debbuging!
Here is a bit refactored script with debugging
#!/bin/bash
for ip in "$(ifconfig | grep -oP 'inet addr:\K[0-9.]+')"
do
bytesin=0
bytesout=0
while read -r line
do
read -r subIp _ _ increment _ <<< "$line"
if [[ $subIp == "$ip" ]]
then
((bytesout+=increment))
else
((bytesin+=increment))
fi
# some debugging
echo "line: $line"
echo "subIp: $subIp"
echo "bytesin: $bytesin"
echo "bytesout: $bytesout"
done <<< "$(pmacct -s | grep "$ip")"
echo "$ip $bytesin $bytesout" >> /tmp/bwacct.txt
done
Much clearer now, huh? :)