Bash Code with while, list and crontab command - bash

I have something like this :
all_scripts=()
for i in "${OS_USER[#]}"; do
list_script=$(crontab -u $i -l | grep -o '[^ ]*\.sh' | grep -o '^[^ ]*' | sort -u)
I want to create a list ( all_scripts) which contain all scripts from all users
For example
if in user 1 crontab there are script1.sh , script2.sh
in user 2 there are scrip3.sh, script4.sh, script5.sh
and in user 3 script6.sh
I want a list with all scripts : script1,sh, script2.sh, script3,...script6.sh

What you presented is incomplete. What you have is almost all correct.
You need to include the action ... to build the list ... within the loop.
You also need to initialize list_script as an array.
You probably want to suppress the "no crontab for ..." messaging.
You also need to be selective regarding which fields are "harvested" from the crontab, so ... ignore comments and ignore parameter definitions.
I recognize that the below may not correctly handle parameter definitions that may include spaces, but it is a good start for what you are looking for. So, the end result would look something like this:
#!/bin/bash
## You may have a specific list instead of this next line.
OS_USER=( $(cut -f1 -d\: /etc/passwd) )
all_scripts=()
list_script=()
for i in ${OS_USER[#]}
do
list_script=$(crontab -u $i -l | grep -v '^#' | awk '{
pos=index($0, $6) ;
$0=substr($0, pos) ;
print NF ;
for( i=1 ; i<=NF ; i++){
if( index($i, "=") == 0 ){
print $i ;
break ;
} ;
} ;
}' | grep -o '[^ ]*\.sh' | grep -o '^[^ ]*' | sort -u )
all_scripts=( ${allscripts[#]} ${list_script[#]} )
done
print ${allscripts[#]}

Related

Numbered options for bash completion

Can we get numbered completion options with bash when we have started typing part of the option? For example with ksh I am able to get a numbered list of options and then when I press 3 followed by 3 I get the 3rd option completed.
> cat local.
1) local.cshrc
2) local.login
3) local.profile
> cat local.profile
Is this kind of completion possible using bash programmable completion?
With something like this I am able to give numbers for all available options, but this breaks when the user starts to type the option and presses tab so a fallback to normal options without numbers has to be used.
(( COMP_CWORD > 2 )) && return #This stops completion at third word for z oe
if [[ "${COMP_WORDS[2]}" = "" || $( echo "${COMP_WORDS[2]}" | perl -ne 'use Scalar::Util qw(looks_like_number); if (looks_like_number($_)){print 1}else{print 0}' ) -eq 1 ]]; then
#Show all instances oratab + running
local suggestions=($( compgen -W "$( { z ot | grep -v ^$ | grep -v SIDs | grep -v : && z p1; } | perl -lpe 's/\s+$//' | sort | uniq | perl -ne 'print "$.) $_"' )" -- "${COMP_WORDS[2]}" ))
if [ "${#suggestions[#]}" == "1" ]; then #One match found
local cmd=$( echo "${suggestions[0]}" | perl -pe 's/[1-9]+\)[\s]+//' )
COMPREPLY=("$cmd ") #Space needed here to handle -o nospace
return
else
#More than one suggestions resolved, respond with the suggestions intact
COMPREPLY=("${suggestions[#]}")
return
fi
else
local suggestions=( $( compgen -W "$( { z ot | grep -v ^$ | grep -v SIDs | grep -v : && z p1; } | perl -lpe 's/\s+$//' | sort | uniq )" -- "${COMP_WORDS[2]}" ) )
if [ "${#suggestions[#]}" == "1" ]; then #One match found
local cmd="${suggestions[0]}"
COMPREPLY=("$cmd ") #Space needed here to handle -o nospace
return
else
#More than one suggestions resolved, respond with the suggestions intact
COMPREPLY=("${suggestions[#]}")
return
fi
fi
Is there way to do the ksh like completion number + tab with bash completion?

Get full path name of file and its size using awk

I want to get the file names followed by their size for all files having size in MB or GB. I have done this much so far :
LIST=$(ls -lh -d -1 $PWD/{*,} | awk '{ print $9":"$5 }')
for i in $LIST
do
if [[ $( echo "$i" | cut -f2 -d: | egrep "M|G" | wc -l) -ne 0 ]]
# egrep not working, only finds M
then
echo "$i" >> bigfiles
fi
done
What I am getting is :
amit#C0deDaedalus:~$ test/findbig
/home/amit/Batch:3.8M
/home/amit/Black:3.6M
What I want is :
amit#C0deDaedalus:~$ test/findbig
/home/amit/Batch File Programming.pdf:3.8M
/home/amit/Black Panther - Legend Has It ( Instrumental ).opus:3.6M
Basically, everything is working fine except filenames that I get are not complete. Only first word is shown. I can't figure out whether there is something wrong with logic or syntax but I think it has something to do with awk.
So, How do I get the full path names of files (having spaces in between) in the output ?
I have tried the loop trick in awk, but don't know how to get both of the columns to fit in.
You can use read and the convenient occurrence of the filename at the right-side of the ls -l listing. read puts all the "extra" fields into the final variable:
function f_getfields
{
local perm lnk uname grp size d1 d2 d3 filename
while read perm lnk uname grp size d1 d2 d3 filename
do
echo "$filename $size"
done < <(ls -l)
}
f_getfields
The problem is due to the spaces in your file names. The for loop uses spaces as delimeter. Therefore the first item in your list will be "/home/amit/Batch", second item "File" and so on.
You can use while loop instead of for, something like :
ls -lh -d -1 $PWD/{*,} | awk '{ print $9":"$5 }' | while read LINE
do
echo ${LINE}
# do your stuff here
done
As an aside, if your only intention is to find out large files, you may want to check out disk usage command :
$ du -a | sort -rn | head

Unix ksh - To print the PID number and repeat count

Log file abc.log
PID:6543 ……
…………………
PID:4325 ……
……………………
PID:6543 ……
Log file xyz.log
PID:8888 ……
…………………
PID:9992 ……
……………………
PID:6543 ……
Note: The PID numbers can repeat in a file. And also one PID number can appear in multiple log files.
This question is asked in an interview today that I have to return output with each PID number and the count of each PID number logged today.
Here is the script that I have written. Can you confirm if that would work or not. The interviewer didn't said if my answer is correct or not. Can someone review this for me. They want me to print each unique PID:number and its count with tab space like PID:5674 10
— If today’s and previous day’s log files are in same folder
#!/bin/ksh
cd /A/B/
for a in `ls -lrt | grep "Mar 24" | awk '{print $9}'`; — list of files generated today
do
grep "^PID:" $a | cut -d " " f1 >> /tmp/abc.log — saving first column which look like PID:23456
done
for b in `cat /tmp/abc.log | sort -u`;
do
x=grep $b /tmp/abc.log | grep -v grep | wc;
echo $b" "$x — will print like PID:23456 56(count)
done
#!/bin/ksh
— If today’s log files are in different folder
cd /A/B/
for a in `ls /A/B/*.log`
do
grep "^PID:" $a | cut -d " " f1 >> /tmp/abc.log
done
for b in `cat /tmp/abc.log | sort -u`;
do
x=grep $b /tmp/abc.log | grep -v grep | wc;
echo $b" "$x
done
You could use awk
grep -h 'PID:' *.log > all_pids.log
#put only PID lines to a file.This can also be done
#with pattern in awk and multiple files. I'm just separating it for clarity
awk '
{ a[$1]++ } #increments by 1 for corresponding PID
END {
for (i in a) {
printf "%s %s\n", i, a[i];
}
}
' all_pids.log

How to process values from for loop in shell script

I have below for loop in shell script
#!/bin/bash
#Get the year
curr_year=$(date +"%Y")
FILE_NAME=/test/codebase/wt.properties
key=wt.cache.master.slaveHosts=
prop_value=""
getproperty(){
prop_key=$1
prop_value=`cat ${FILE_NAME} | grep ${prop_key} | cut -d'=' -f2`
}
#echo ${prop_value}
getproperty ${key}
#echo "Key = ${key}; Value="${prop_value}
arr=( $prop_value )
for i in "${arr[#]}"; do
echo $i | head -n1 | cut -d "." -f1
done
The output I am getting is as below.
test1
test2
test3
I want to process the test2 from above results to below script in place of 'ABCD'
grep test12345 /home/ptc/storage/**'ABCD'**/apache/$curr_year/logs/access.log* | grep GET > /tmp/test.access.txt
I tried all the options but could not able to succeed as I am new to shell scripting.
Ignoring the many bugs elsewhere and focusing on the one piece of code you say you want to change:
for i in "${arr[#]}"; do
val=$(echo "$i" | head -n1 | cut -d "." -f1)
grep test12345 /dev/null "/home/ptc/storage/$val/apache/$curr_year/logs/access.log"* \
| grep GET
done > /tmp/test.access.txt
Notes:
Always quote your expansions. "$i", "/path/with/$val/"*, etc. (The * should not be quoted on the assumption that you want it to be expanded).
for i in $prop_value would have the exact same (buggy) behavior; using arr buys you nothing. If you want using arr to increase correctness, populate it correctly: read -r -a arr <<<"$prop_value"
The redirection is moved outside the loop -- that way the second iteration through the loop doesn't overwrite the file written by the first one.
The extra /dev/null passed to grep ensures that its behavior is consistent regardless of the number of matches; otherwise, it would display filenames only if more than one matching log file existed, and not otherwise.

BASH script - print sorted contents from all files in directory with no rep's

In the current directory there are files with names of the form "gradesXXX" (where XXX is a course number) which look like this:
ID GRADE (this line is not contained in the files)
123456789 56
213495873 84
098342362 77
. .
. .
. .
I want to write a BASH script that prints all the IDs that have a grade above a certain number, which is given as the first parameter to said script.
The requirements are that an ID must be printed once at most, and that no intermediate files are used.
I was guided to use two scripts - the first with length of one line, and the second with length of up to six lines (not including the "#!" line).
I'm quite lost with this one so any suggestions will be appreciated.
Cheers.
The answer I was looking for was
// internal script
#!/bin/bash
while read line; do
line_split=( $line )
if (( ${line_split[1]} > $1 )); then
echo ${line_split[0]}
fi
done
// external script
#!/bin/bash
cat grades* | sort -r -n -k 1 | internalScript $1 | cut -f1 -d" " | uniq
OK, a simple solution.
cat grades[0-9][0-9][0-9] | sort -nurk 2 | while read ID GRADE ; do if [ $GRADE -lt 60 ] ; then break ; fi ; echo $ID ; done | sort -u
I'm not sure why two scripts should be necessary. All in a script:
#!/bin/bash
threshold=$1
cat grades[0-9][0-9][0-9] | sort -nurk 2 | while read ID GRADE ; do if [ $GRADE -lt $threshold ] ; then break ; fi ; echo $ID ; done | sort -u
We first cat all the grade files, the sort them by grade in reverse order. The while loop breaks if grade is below threshold, so that only lines with higher grades get their ID printed. sort -u makes sure that every ID is sent only once.
You can use awk:
awk '{ if ($2 > 70) print $1 }' grades777
It prints the first column of every line which seconds column is greater than 70. If you need to change the threshold:
N=71
awk '{ if ($2 > '$N') print $1 }' grades777
That ' are required to pass shell variables in AWK. To work with all grade??? files in the current directory and remove duplicated lines:
awk '{ if ($2 > '$N') print $1 }' grades??? | sort -u
A simple one-line solution.
Yet another solution:
cat grades[0-9][0-9][0-9] | awk -v MAX=70 '{ if ($2 > MAX) foo[$1]=1 }END{for (id in foo) print id }'
Append | sort -n after that if you want the IDs in sorted order.
In pure bash :
N=60
for file in /path/*; do
while read id grade; do ((grade > N)) && echo "$id"; done < "$file"
done
OUTPUT
213495873
098342362

Resources