Trying to add options to this script, not quite working - bash

I started with this script called wd:
cat "$#" | tr -cs '[:alpha:]' '\n' | tr '[:upper:]' '[:lower:]'
| sort | uniq -c | sort -n | awk '{print $2 " " $1}' | sort
That takes any number of files as input and prints a distribution of the words in the file like this:
wd file1 file2
blue 2
cat 3
the 5
yes 1
Now I'm trying to add 2 options to it: s and t. s causes the script to take an input file called stopwords, and deletes those words from the input file before making the distribution. t takes a number n as an argument and only outputs the top n words. Default is all words.
So, so far I have this script. Currently, my problem is when I try to use a -t 10 option for example, it tells me it cannot find the file 10, but it should be a number anyway, not a file. And, when I try to use the -s option, it simply does nothing, but does not output any error. I know this question isn't very specific, but I would appreciate any ideas on what's wrong.
#!/bin/bash
stopwords=FALSE
stopfile=""
topwords=0
while getopts s:t: option
do
case "$option"
in
s) stopwords=TRUE
stopfile="$OPTARG";;
t) topwords=$OPTARG;;
\?) echo "Usage: wd [-s stopfile] [-t n] inputfile"
echo "-s takes words in stopfile and removes them from inputfile"
echo "-t means to output only top n words"
exit 1;;
esac
done
if [ "stopwords" = FALSE ]
then
cat "$#" | tr -cs '[:alpha:]' '\n' | tr '[:upper:]' '[:lower:]'
| sort | uniq -c | sort -nr | head -n $topwords | awk '{print $2 " " $1}' | sort
else
cat "$#" | grep -v -f "$stopfile" | tr -cs '[:alpha:]' '\n' | tr '[:upper:]' '[:lower:]'
| uniq -c | sort -nr | head -n $topwords | awk '{print $2 " " $1}' | sort
fi

Usually after the while getopts loop you need to shift $((OPTIND - 1)). Following is an example I wrote before for both ksh and bash:
PROGNAME=$0
function _echo
{
printf '%s\n' "$*"
}
function usage
{
cat << END
usage: $PROGNAME [-a] [-b arg] [-h] file...
END
exit $1
}
function parseargs
{
typeset opt v
[[ $# = 0 ]] && usage 1
while getopts ":ab:h" opt "$#"; do
case $opt in
a) _echo -$opt ;;
b) _echo -$opt $OPTARG ;;
h) usage ;;
:) _echo "! option -$OPTARG wants an argument" ;;
'?') _echo "! unkown option -$OPTARG" ;;
esac
done
shift $((OPTIND - 1))
for v in "$#"; do
_echo "$v"
done
}
parseargs "$#"

Related

Bash - Counter for multiple parameters in file

I created a command, which works, but not exactly as I want. So I would like to upgrade this command to right output.
My command:
awk '{print $1}' ios-example.com.access | sort | uniq -c | sort -nr
Output of my command:
8 192.27.69.191
2 82.202.69.253
Input file:
https://pajda.fit.vutbr.cz/ios/ios-19-1-logs/blob/master/ios-example.com.access.log
Output I need(hashtags instead of numbers):
198.27.69.191 (8): ########
82.202.69.253 (2): ##
cat ios-example.com.access | sort | uniq -c | awk 'ht="#"{for(i=1;i<$1;i++){ht=ht"#"} str=sprintf("%s (%d): %s", $2,$1, ht); print str}'
expecting file with content like:
ipadress1
ipadress1
ipadress1
ipadress2
ipadress2
ipadress1
ipadress2
ipadress1
Using xargs with sh and printf. Comments in between the lines. Live version at tutorialspoint.
# sorry cat
cat <<EOF |
8 192.27.69.191
2 82.202.69.253
EOF
# for each 2 arguments
xargs -n2 sh -c '
# format the output as "$2 ($1): "
printf "%s (%s): " "$2" "$1"
# repeat the character `#` $1 times
seq "$1" | xargs printf "#%.0s"
# lastly a newline
printf "\n"
' --
I think we could shorten that a bit with:
xargs -n2 sh -c 'printf "%s (%s): %s\n" "$2" "$1" $(printf "#%.0s" $(seq $1))' --
or maybe just echo, if the input is sufficiently safe:
xargs -n2 sh -c 'echo "$2 ($1): $(printf "#%.0s" $(seq $1))"' --
You can upgrade your command by adding another awk to the list, or you can just use a single awk for the whole thing:
awk '{a[$1]++}
END { for(i in a) {
printf "%s (%d):" ,i,a[i]
for(j=0;j<a[i];++j) printf "#"; printf "\n"
}
}' file

String comparison from nested for returns always false

The main issue is that i try to parse ls to do a mock "Compare directories" but when i do so since i use nested fors i cant properly compare the results from it since the comparison of two filenames/strings even if they are the same it always returns false
I tried erasing the white characters but no results.
var1=$(ls -l $1 | grep -v ^d | tail -n +2 | tr -s " "| cut -d " " -f 9)
var2=$(ls -l $2 | grep -v ^d | tail -n +2 | tr -s " "| cut -d " " -f 9)
for i in $var1 ; do
i=$(printf "$i" | tr -d '[:space:]')
flag=0
var3=$(ls -l $1 | grep -v ^d | tail -n +2 | tr -s " " | grep $i | cut -d " " -f 5)
for j in $var2 ; do
j=$(printf $j | tr -d '[:space:]')
var4=$(ls -l $2 | grep -v ^d | tail -n +2 | tr -s " " | grep $j | cut -d " " -f 5)
if [ "$i" == "$j" ] ; then
if [ "$var3" != "$var4" ] ; then
flag=1
fi
else
flag=1
fi
done
if [ $flag -eq 1 ] ; then
printf "$i file does not exist on the $2 catalog\n"
printf "It 's size is :$var3 \n"
let Sum=$Sum+$var3
fi
done
This is not a string comparison problem, it's a logic problem.
I wrote you a MCVE that demonstrates the same problem with less code and fewer dependencies:
flag=0
target="hello"
for candidate in "hello" "world"
do
if [ "$target" != "$candidate" ]
then
flag=1
fi
done
if [ "$flag" -eq 1 ]
then
echo "The string was not found"
fi
This prints The string was not found every time, just like your script, even though it's clearly there.
The problem here is that the script requires that ALL files match. It should only require that ANY file matches. The easiest way to fix this is to:
Set flag=1 when a MATCH is found (not a mismatch)
Make flag=1 signify that a match was found (rather than no match was found)
Here's the version which correctly finds the string:
flag=0
target="hello"
for candidate in "hello" "world"
do
if [ "$target" = "$candidate" ]
then
flag=1
fi
done
if [ "$flag" -eq 1 ]
then
echo "The string was found"
else
echo "The string was not found"
fi

GETOPTS command does not work in my shell script

Can anybody help me please?
i wrote a script and in my script i used GETOPTS to make options but it does not work
it had some error and i check it in shellcheck.net and fixed them but it's not working
#!/bin/bash
while getopts 'n:c2rFt' option; do
case "$option" in
n) export Field="$OPTARG"
;;
c) #Question 1
cat "$1" | awk '{print $1}' | sort | uniq -c | sort -nrk 1,1 > file1
awk 'NR=="$Field" {print}' file1
;;
2) #Question 2
cat "$1" | awk '{ if($9 == 200) print $1,$9 }' | sort | uniq -c | sort -nrk 1,1 > file1
awk 'NR=="$Field" {print}' file1
;;
r) #Question 3
cat "$1" | awk '{print $1,$9}' | sort | uniq -c | sort -nrk 1,1 > file1
awk 'NR=="$Field" {print}' file1
;;
F) #Question 4
cat "$1" | awk '{if($9 >= 400 && $9 <= 451)} {print $1,$9}' | sort | uniq -c | sort -nrk 1,1 > file1
awk 'NR=="$Field" {print}' file1
;;
t) #Question 5
cat "$1" | awk '{print $1,$10}' | sort | uniq -c | sort -nrk 3,3 > file1
awk 'NR=="$Field" {print}' file1
;;
?)
echo "You used wrong option"
echo "USAGE: log_sum.sh [-n N] (-c|-2|-r|-F|-t|-f) <filename>"
echo " -n: Limit the number of results to N"
echo " -c: shows th IP address makes the most number of connection attempts"
echo " -2: shows th most number of seccessful attempts "
echo " -r: shows th most common result codes and their IP addresses"
echo " -F: shows the most common result codes that indicate failure"
echo " -t: shows the IP addresses that get the most bytes sent to them"
exit 1
;;
esac
done
You have your "business logic" in the wrong place: your code assumes that the user will provide the -n option first. That's not required by getopts. You have to write this kind of program with 3 stages: option parsing, validation and actions:
#!/bin/bash
usage() {
local program=$(basename "$0")
cat <<END_USAGE >&2
USAGE: $program -n N (-c|-2|-r|-F|-t|-f) <filename>
-n: Limit the number of results to N
-c: shows th IP address makes the most number of connection attempts
-2: shows th most number of seccessful attempts
-r: shows th most common result codes and their IP addresses
-F: shows the most common result codes that indicate failure
-t: shows the IP addresses that get the most bytes sent to them
END_USAGE
}
# Option parsing
while getopts ':n:c2rFt' option; do
case "$option" in
n) num_results=$OPTARG ;;
c) show_connections=yes ;;
2) show_successful=yes ;;
r) show_common_results=yes ;;
F) show_common_failures=yes ;;
t) show_most_bytes=yes ;;
?) echo "Error: unknown option $OPTARG"; usage; exit 1 ;;
esac
done
shift $((OPTIND - 1))
filename=$1
# Validation
if [[ -z $num_results ]]; then
echo "Error: you must provide the -n option" >&2
usage >&2
exit 1
fi
if [[ -z $filename ]]; then
echo "Error: you must provide a filename" >&2
usage >&2
exit 1
fi
# Actions
# helper function to encapsulate repeated code
top_results() { sort | uniq -c | sort -nrk 1,1 | sed "${num_results}q"; }
if [[ $show_connections == yes ]]; then
awk '{print $1}' "$filename" | top_results
fi
if [[ $show_successful == yes ]]; then
awk '$9 == 200 {print $1,$9}' "$filename" | top_results
fi
if [[ $show_common_results == yes ]]; then
awk '{print $1,$9}' "$filename" | top_results
fi
if [[ $show_common_failures == yes ]]; then
awk '$9 >= 400 && $9 <= 451 {print $1,$9}' "$filename" | top_results
fi
if [[ $show_most_bytes == yes ]]; then
awk '{print $1,$10}' "$filename" | top_results
fi

Convert to uppercase in shell

I am reading a character from keyboard and converting it to uppercase and then displaying the character again.
But the following code produces an error:
read a;
a=echo $a | tr 'a-z' 'A-Z'
echo $a
I also tried this:
read option;
eval $(awk -v option=$option '{print "a="toupper(option);}')
echo $a
If you want to store the result of a back in a, then you can do use command substitution:
read a;
a=$(echo $a | tr 'a-z' 'A-Z')
echo $a
This can be done natively in Bash as follows:
read a;
a="${a^^}"
echo "$a"
There is no need to invoke other commands like tr, because Bash can do this itself.
See also: Bash parameter expansion.
AWK is the right way to convert upper/lower case with full Unicode Support ;-)
echo "öäüßè" | awk 'BEGIN { getline; print toupper($0) }'
Use command substitution:
a=`echo $a | tr 'a-z' 'A-Z'`
Note the ticks ` around echo and tr.
using a bash script
printf "Type your Message Here: " read message
echo Upper Case: $message | tr [:lower:] [:upper:];
echo Lower Case: $message | tr [:upper:] [:lower:]
awk is the wrong way to go here, but here's one way it could be done:
a=$(awk 'BEGIN { getline; print toupper($0) }')
echo $a
Could not get
a=`echo $a | tr 'a-z' 'A-Z'`
to work, but
a=`echo $a | tr '[a-z]' '[A-Z]'`
did (note additional regex [] brackets.
Within a /usr/bin/sh script this worked as in
...
while getopts ":l:c:" option; do
case "$option"
in
l) L_OPT=`echo ${OPTARG}| tr '[a-z]' '[A-Z]'`
;;
c) C_OPT=`echo ${OPTARG} | tr '[a-z]' [A-Z]'`
;;
\?)
echo $USAGE
exit 1
;;
esac
done
...

Remove one directory component from path (string manipulation)

I'm looking for the easiest and most readable way to remove a field from a path. So for example, I have /this/is/my/complicated/path/here, and I would like to remove the 5th field ("/complicated") from the string, using bash commands, so that it becomes /this/is/my/path.
I could do this with
echo "/this/is/my/complicated/path/here" | cut -d/ -f-4
echo "/"
echo "/this/is/my/complicated/path/here" | cut -d/ -f6-
but I would like this done in just one easy command, something that would like
echo "/this/is/my/complicated/path" | tee >(cut -d/ -f-4) >(cut -d/ -f6-)
except that this doesn't work.
With cut, you can specify a comma separated list of fields to print:
$ echo "/this/is/my/complicated/path/here" | cut -d/ -f-4,6-
/this/is/my/path/here
So, it's not really necessary to use two commands.
How about using sed?
$ echo "/this/is/my/complicated/path/here" | sed -e "s%complicated/%%"
/this/is/my/path/here
This removes the 5th path element
echo "/this/is/my/complicated/path/here" |
perl -F/ -lane 'splice #F,4,1; print join("/", #F)'
just bash
IFS=/ read -a dirs <<< "/this/is/my/complicated/path/here"
newpath=$(IFS=/; echo "${dirs[*]:0:4} ${dirs[*]:5}")
Anything wrong with a bash script?
#!/bin/bash
if [ -z "$1" ]; then
us=$(echo $0 | sed "s/^\.\///") # Get rid of a starting ./
echo " "Usage: $us StringToParse [delimiterChar] [start] [end]
echo StringToParse: string to remove something from. Required
echo delimiterChar: Character to mark the columns "(default '/')"
echo " "start: starting column to cut "(default 5)"
echo " "end: last column to cut "(default 5)"
exit
fi
# Parse the parameters
theString=$1
if [ -z "$2" ]; then
delim=/
start=4
end=6
else
delim=$2
if [ -z "$3" ]; then
start=4
end=6
else
start=`expr $3 - 1`
if [ -z "$4" ]; then
end=6
else
end=`expr $4 + 1`
fi
fi
fi
result=`echo $theString | cut -d$delim -f-$start`
result=$result$delim
final=`echo $theString | cut -d$delim -f$end-`
result=$result$final
echo $result

Resources