I am trying to compare two text in Unix. I tried the below. It didn't work. In need to compare the first and last lines of a file.
firstline=`head -1 test.txt`
echo $firstline
lastline=`tail -1 test.txt`
echo $lastline
if [ $firstline == $lastline ]
then
echo "Found"
fi
Surely, am missing something. Please help.
Perhaps simpler...
bash-3.2$ if [ "$(sed -n '1p' file)" = "$(sed -n '$p' file)" ]; then
echo 'First and last lines are the same'
else
echo 'First and last lines differ'
fi
Update to answer Jan's questions.
bash-3.2$ cat file
-z
-G
bash-3.2$ if [ "$(sed -n '1p' file)" = "$(sed -n '$p' file)" ]; then
> echo 'First and last lines are the same'
> else
> echo 'First and last lines differ'
> fi
First and last lines differ
I prefer sed for grabbing the first and last lines of a file because the same command-line works on Linux, Mac OS and Solaris. The head and tail command-lines are different between Linux and Solaris.
Assuming you are using "some sort" of bourne shell, you should (a) quote the variables and (b) need to use a single =:
if [ "$firstline" = "$lastline" ]
then
echo "Found"
fi
Update In response to some comments, this will also work if $firstline is -z. Even in this case the if statement is not interpreted as if [ -z ... ], at least in the ksh (Korn Shell) or in Bash (I don't have a system with a plain bourne shell sh available).
Should be if [ "$firstline" = "$lastline" ]
If you omit double quotes it will not work if the line(s) contain white characters.
At the very least, you have to quote the variable expansions. Plus you should add prefix to avoid problems if the strings start with -. And the correct operator is =. So it should be
if [ "x$firstline" = "x$lastline" ]
Related
I have a script that is trying to find the presence of a given string inside a file of arbitrary text.
I've settled on something like:
#!/bin/bash
file="myfile.txt"
for j in `cat blacklist.txt`; do
echo Searching for $j...
unset match
match=`grep -i -m1 -o "$j" $file`
if [ $match ]; then
echo "Match: $match"
fi
done
Blacklist.txt contains lines of potential matches, like so:
matchthis
"match this too"
thisisasingleword
"This is multiple words"
myfile.txt could be something like:
I would matchthis if I could match things with grep. I really wish I could.
When I ask it to match this too, it fails to matchthis. It should match this too - right?
If I run this at a bash prompt, like so:
j="match this too"
grep -i -m1 -o "$j" myfile.txt
...I get "match this too".
However, when the batch file runs, despite the variables being set correctly (verified via echo lines), it never greps properly and returns nothing.
Where am I going wrong?
Wouldn't
grep -owF -f blacklist.txt myfile.txt
instead of writing an inefficient loop, do what you want?
Would you please try:
#!/bin/bash
file="myfile.txt"
while IFS= read -r j; do
j=${j#\"}; j=${j%\"} # remove surrounding double quotes
echo "Searching for $j..."
match=$(grep -i -m1 -o "$j" "$file")
if (( $? == 0 )); then # if match
echo "Match: $match" # then print it
fi
done < blacklist.txt
Output:
Searching for matchthis...
Match: matchthis
Searching for match this too...
Match: match this too
match this too
Searching for thisisasingleword...
Searching for This is multiple words...
I wound up abandoning grep entirely and using sed instead.
match=`sed -n "s/.*\($j\).*/\1/p" $file
Works well, and I was able to use unquoted multiple word phrases in the blacklist file.
With this:
if [ $match ]; then
you are passing random arguments to test. This is not how you properly check for variable net being empty. Use test -n:
if [ -n "$match" ]; then
You might also use grep's exit code instead:
if [ "$?" -eq 0 ]; then
for ... in X splits X at spaces by default, and you are expecting the script to match whole lines.
Define IFS properly:
IFS='
'
for j in `cat blacklist.txt`; do
blacklist.txt contains "match this too" with quotes, and it is read like this by for loop and matched literally.
j="match this too" does not cause j variable to contain quotes.
j='"match this too"' does, and then it will not match.
Since whole lines are read properly from the blacklist.txt file now, you can probably remove quotes from that file.
Script:
#!/bin/bash
file="myfile.txt"
IFS='
'
for j in `cat blacklist.txt`; do
echo Searching for $j...
unset match
match=`grep -i -m1 -o "$j" "$file"`
if [ -n "$match" ]; then
echo "Match: $match"
fi
done
Alternative to the for ... in ... loop (no IFS= needed):
while read; do
j="$REPLY"
...
done < 'blacklist.txt'
I have bash script which checks presence of certain files and that the content has a valid format. It uses variable prefixes so i can easily add/remove new files w/o the need of further adjustments.
Problem is that i need to run this on AIX servers where bash is not present. I've adjusted the script except the part with variable prefixes. After some attempts i am lost and have no idea how to properly migrate the following piece of code so it runs under sh ( $(echo ${!ifile_#}) ). Alternatively i have ksh or csh if plain sh is not an option.
Thank you in advance for any help/hints
#!/bin/sh
# Source files
ifile_one="/path/to/file/one.csv"
ifile_two="/path/to/file/two.csv"
ifile_three="/path/to/file/three.csv"
ifile_five="/path/to/file/four.csv"
min_columns='10'
existing_files=""
nonexisting_files=""
valid_files=""
invalid_files=""
# Check that defined input-files exists and can be read.
for input_file in $(echo ${!ifile_#})
do
if [ -r ${!input_file} ]; then
existing_files+="${!input_file} "
else
nonexisting_files+="${!input_file} "
fi
done
echo "$existing_files"
echo "$nonexisting_files"
# Check that defined input files have proper number of columns.
for input_file_a in $(echo "$existing_files")
do
check=$(grep -v "^$" $input_file_a | sed 's/[^;]//g' | awk -v min_columns="$min_columns" '{ if (length == min_columns) {print "OK"} else {print "KO"} }' | grep -i KO)
if [ ! -z "$check" ]; then
invalid_files+="${input_file_a} "
else
valid_files+="${input_file_a} "
fi
done
echo "$invalid_files"
echo "$valid_files"
Bash returns expected output (of the four ECHOes):
/path/to/file/one.csv /path/to/file/two.csv /path/to/file/three.csv
/path/to/file/four.csv
/path/to/file/three.csv
/path/to/file/one.csv /path/to/file/two.csv
ksh/sh throws:
./report.sh[14]: "${!ifile_#}": 0403-011 The specified substitution is not valid for this command.
Thanks #Benjamin W. and #user1934428 , ksh93 arrays are the answer.
So bellow code works for me as desired.
#!/bin/ksh93
typeset -A ifile
ifile[one]="/path/to/file/one.csv"
ifile[two]="/path/to/file/two.csv"
ifile[three]="/path/to/file/three.csv"
ifile[whatever]="/path/to/file/something.csv"
existing_files=""
nonexisting_files=""
for input_file in "${!ifile[#]}"
do
if [ -r ${ifile[$input_file]} ]; then
existing_files+="${ifile[$input_file]} "
else
nonexisting_files+="${ifile[$input_file]} "
fi
done
The output comes from a command I run from our netscaler. It outputs the following ... One thing to note is that the middle two numbers change but the even/odd criteria is always on the last digit. We never have more than 2 digits, so we'll never hit 10.
WC-01-WEB1
WC-01-WEB4
WC-01-WEB3
WC-01-WEB5
WC-01-WEB8
I need to populate a file called "even" and "odds." If we're dealing with numbers I can figure it out, but having the number within a string is throwing me off.
Example code but I'm missing the part where I need to match the string.
if [ $even_servers -eq 0 ]
then
echo $line >> evenfile
else
echo $line >> oddfile
fi
This is a simple awk command:
awk '/[02468]$/{print > "evenfile"}; /[13579]$/{print > "oddfile"}' input.txt
There must be better way.
How about this version:
for v in `cat <my_file>`; do export type=`echo $v | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $v >> evenfile ; else echo $v >> oddfile; fi; done
I assume your list of servers is stored in the filename <my_file>. The basic idea is to tokenize on WEB using awk and process the chars after WEB to determine even-ness. Once this is known, we export the value to a variable type and use this to selectively dump to the appropriate file.
For the case when the name is the output of another command:
export var=`<another command>`; export type=`echo $var | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $var >> evenfile ; else echo $var >> oddfile; fi;
Replace <another command> with your perl script.
As always grep is your friend:
grep "[2468]$" input_file > evenfile
grep "[^2468]$" input_file > oddfile
I hope this helps.
A text file has the following structure:
paa pee pii poo puu
baa bee bii boo buu
gaa gee gii goo guu
maa mee mii moo muu
Reading it line by line in a script is done with
while read LINE; do
ACTION
done < FILE
I'd need to get parameters 3 and 4 of each line into variables for ACTION. If this was manual input, $3 and $4 would do the trick. I assume awk is the tool, but I just can't wrap my head around the syntax. Halp?
read does this just fine. Pass it multiple variables and it will split on $IFS into that many fields.
while read -r one two three four five; do
action "$three" "$four"
done <file
I added the -r option because that is usually what you want. The default behavior is a legacy oddity of limited use.
Thanks tripleee. In the meantime I managed a suitably versatile solution:
#!/bin/sh
if [ ! $1 ]; then
echo "Which inputfile?"
exit
elif [ ! $2 -o ! $3 ]; then
echo "Two position parameters required"
exit
fi
if [ -f outfile ]; then
mv outfile outfile.old
fi
while read -a LINE; do
STRING="${LINE[#]}"
if [ ${LINE[$2-1]} == ${LINE[$3-1]} ]; then # remove comment for strings
# if [ ${LINE[$(($2-1))]} -eq ${LINE[$(($3-1))]} ]; then # remove comment for integers
echo $STRING >> outfile
fi
done < $1
I am making a bash script where I want to find files that are equal to a variable. The equals will then be used.
I want to use "mogrify" to shrink a couple of image files that have the same name as the ones i gather from a list (similar to "dpkg -l"). It is not "dpkg -l" I am using but it is similar. My problem is that it prints all the files not just the equals. I am pretty sure this could be done with awk instead of a for-loop but I do not know how.
prog="`dpkg -l | awk '{print $1}'`"
for file in $dirone* $dirtwo*
do
if [ "basename ${file}" = "${prog}" ]; then
echo ${file} are equal
else
echo ${file} are not equal
fi
done
Could you please help me get this working?
First, I think there's a small typo. if [ "basename ${file}" =... should have backticks inside the double quotes, just like the prog=... line at the top does.
Second, if $prog is a multi-line string (like dpkg -l) you can't really compare a filename to the entire list. Instead you have to compare one item at a time to the filename.
Here's an example using dpkg and /usr/bin
#!/bin/bash
progs="`dpkg -l | awk '{print $2}'`"
for file in /usr/bin/*
do
base=`basename ${file}`
for prog in ${progs}
do
if [ "${base}" = "${prog}" ]; then
echo "${file}" matches "${prog}"
fi
done
done
The condition "$file = $prog" is a single string. You should try "$file" = "$prog" instead.
The following transcript shows the fix:
pax> ls -1 qq*
qq
qq.c
qq.cpp
pax> export xx=qq.cpp
pax> for file in qq* ; do
if [[ "${file} = ${xx}" ]] ; then
echo .....${file} equal
else
echo .....${file} not equal
fi
done
.....qq equal
.....qq.c equal
.....qq.cpp equal
pax> for file in qq* ; do
if [[ "${file}" = "${xx}" ]] ; then
echo .....${file} equal
else
echo .....${file} not equal
fi
done
.....qq not equal
.....qq.c not equal
.....qq.cpp equal
You can see in the last bit of output that only qq.cpp is shown as equal since it's the only one that matches ${xx}.
The reason you're getting true is because that's what non-empty strings will give you:
pax> if [[ "" ]] ; then
echo .....equal
fi
pax> if [[ "x" ]] ; then
echo .....equal
fi
.....equal
That's because that form is the string length checking variation. From the bash manpage under CONDITIONAL EXPRESSIONS:
string
-n string
True if the length of string is non-zero.
Update:
The new code in your question won't quite work as expected. You need:
if [[ "$(basename ${file})" = "${prog}" ]]; then
to actually execute basename and use its output as the first part of the equality check.
you can use case/esac
case "$file" in
"$prog" ) echo "same";;
esac