Deleting a PARTICULAR word from a file using shell script - shell

Hi I am having a problem in deleting a particular set of words from a file using Shell script. Here goes my problem,
My file: group.dat
Sample lines
ADMIN
ADMINISTRATION
ADMINISTRATOR
My Script: groupdel.sh
#!/bin/sh
groupf="<pathtofile>/group.dat"
tmp="<pathtofile>/te"
delgrp()
{
echo "Enter the group to be deleted"
read gname
echo "-------------------"
for gn in `cat $groupf`
do
if [ "$gname" = "$gn" ]
then
sed -e "s/$gname//g" $groupf > $tmp&&mv $tmp $groupf
echo "deleted group"
cat $groupf
exit 1
fi
done
}
echo "Welcome to Group delete wizard"
delgrp
Output:
Enter the group to be deleted
ADMIN
deleted group
ISTRATION
ISTRATOR
Problem: My problem is I dont want the script to delete ADMINISTRATION or ADMINISTRATOR but to delete only ADMIN, any help how to achieve it.
Thanks in Advance

#!/bin/sh
groupf="<pathtofile>/group.dat"
tmp="<pathtofile>/te"
delgrp()
{
echo "Enter the group to be deleted"
read gname
echo "-------------------"
sed -e "/^$gname[[:blank:]]/d" "$groupf" > "$tmp" && mv "$tmp" "$groupf"
echo "deleted group $gname"
cat "$groupf"
return 0
}
echo "Welcome to Group delete wizard"
delgrp
Assuming that the group name is at the beginning of the line and there are other things on the line and you want to delete the whole line, use the regular expression and command as shown.
There's no need for a loop since sed will iterate over the lines of the file for free.
You should return from a function rather than exit from it. Zero means success. One indicates an error or failure.
Always quote variable names that contain filenames.

If the file is one group per line, and the group name is the only thing on the line, use anchors in your regular expression:
s/^$gname:.*//g
If you have Perl installed, you can probably simplify this a bit with something like this:
if grep -q "^${gname}:" $groupf ; then
perl -ni -e "print unless /^${gname}:/" $groupf
echo "Group deleted."
else
echo "No such group $gname."
fi
Or even
grep -v "^${gname}:" $groupf > $tmp && \
cp -f $tmp $groupf && rm -f $tmp
which will copy all lines except the matching one to the temporary file, and then copy the tempfile over the original file, replacing it.
Note that I suggest using a cp rather than a mv in order to retain the permissions of the original file; mv will result in the edited file having permissions set according to your umask with no concern for the original permissions.
So, for the complete script:
#!/bin/sh
groupf="<pathtofile>/group.dat"
tmp="<pathtofile>/te"
delgrp()
{
echo -n "Enter the group to be deleted: "
read gname
echo "-------------------"
if grep -q "^${gname}:" $groupf ; then
grep -v "^${gname}:" $groupf > $tmp
cp -f $tmp $groupf
rm -f $tmp
else
echo "No such group '$gname'"
fi
}
echo "Welcome to Group delete wizard"
delgrp
That should work reliably.

You can use \W to denote the start and end of a word, if they are separated properly:
sed -e "s/\(\W\)$gname\(\W\)/\1\2/g" $groupf > $tmp&&mv $tmp $groupf

Awk is a readable alternative to sed:
awk -v to_delete="$gname" -F: '$1 == to_delete {next} {print}'

Why you don't use sed ?
sed 's/^word$//g'
Also you can use regex to specify multiple words
sed 's/word1|word2//g'
I didn't try this, but this is what you need. Just take a look on Internet on the sed syntax.
Regards

Related

Extract a line from a text file using grep?

I have a textfile called log.txt, and it logs the file name and the path it was gotten from. so something like this
2.txt
/home/test/etc/2.txt
basically the file name and its previous location. I want to use grep to grab the file directory save it as a variable and move the file back to its original location.
for var in "$#"
do
if grep "$var" log.txt
then
# code if found
else
# code if not found
fi
this just prints out to the console the 2.txt and its directory since the directory has 2.txt in it.
thanks.
Maybe flip the logic to make it more efficient?
f=''
while read prev
do case "$prev" in
*/*) f="${prev##*/}"; continue;; # remember the name
*) [[ -e "$f" ]] && mv "$f" "$prev";;
done < log.txt
That walks through all the files in the log and if they exist locally, move them back. Should be functionally the same without a grep per file.
If the name is always the same then why save it in the log at all?
If it is, then
while read prev
do f="${prev##*/}" # strip the path info
[[ -e "$f" ]] && mv "$f" "$prev"
done < <( grep / log.txt )
Having the file names on the same line would significantly simplify your script. But maybe try something like
# Convert from command-line arguments to lines
printf '%s\n' "$#" |
# Pair up with entries in file
awk 'NR==FNR { f[$0]; next }
FNR%2 { if ($0 in f) p=$0; else p=""; next }
p { print "mv \"" p "\" \"" $0 "\"" }' - log.txt |
sh
Test it by replacing sh with cat and see what you get. If it looks correct, switch back.
Briefly, something similar could perhaps be pulled off with printf '%s\n' "$#" | grep -A 1 -Fxf - log.txt but you end up having to parse the output to pair up the output lines anyway.
Another solution:
for f in `grep -v "/" log.txt`; do
grep "/$f" log.txt | xargs -I{} cp $f {}
done
grep -q (for "quiet") stops the output

Two "if" conditions in the same time

I am writing a script to bring me data from other nodes via ssh in a multi selection choice menu, and i want to display a message according to this data.
if [[ "$option" == "1" ]]
then
ssh skyusr#<IP> "export JAVA_HOME=/opt/mesosphere && /var/lib/mesos/slave/slaves/*/frameworks/*/executors/*/runs/latest/apache-cassandra-3.0.10/bin/nodetool -p 7199 status" | sed -n '6,10p' | awk '{print $1,$2}' | grep DN > $file_name
if [ -s $file_name ]
then
echo "All Cassandra Nodes are UP !"
else cat "$file_name"
fi
fi
When i execute the script, i see it does not see the second if condition to display the message .
What is the correct syntax ?
There is, as far as I can see, nothing wrong with the syntax. You might want to do something about spacing etc. to enhance readability, but that is it.
I assumed, that file_name is set somewhere before this part, as is option.
If things do not work as you expect them to work, you can add some statements for debugging purposes, which you must remove later on. For example, in this case, I would like to see the output of the ssh and add some echo's to see the flow-control:
if [[ "$option" == "1" ]] ; then
ssh skyusr#<IP> "export JAVA_HOME=/opt/mesosphere && /var/lib/mesos/slave/slaves/*/frameworks/*/executors/*/runs/latest/apache-cassandra-3.0.10/bin/nodetool -p 7199 status" > tempfile
cat tempfile |
sed -n '6,10p' |
awk '{print $1,$2}' |
grep DN > "$file_name"
if [ -s "$file_name" ] ; then
echo "All Cassandra Nodes are UP !"
else
echo "$file_name is not empty"
cat "$file_name"
fi
fi
You can use the tempfile to verify that your sed, awk, grep combination acts correctly.

Check if file has been modified

How can I validate that this replace command succeeded:
perl -pi -e 's/contoso/'"$hostname"'/g' /etc/inet/hosts
I have tried checking the return value:
if [ $? -eq 0 ]; then
echo "OK"
else
echo "Error."
fi
But the return value is not being set when the command fails.
Thanks
One option is to check, if file has been modified. You can achieve with adding extension of backup file to -i option:
perl -pi.orig -e 's/contoso/'"$hostname"'/g' /etc/inet/hosts
This command will store original content of /etc/inet/hosts into /etc/inet/hosts.orig. Then run the specified command. Then you can check if the files are different with, for example cmp command:
if ! cmp -s foo.txt foo.txt.orig; then
echo OK
else
echo ERROR
fi
Remove the .orig file after that.
The other option is to modify the script to read the content of the file, replace required entry, check is change actually happened and return proper status at the end to verify in the shell using $?. You have been given solution in this answer.
I don't know Perl, but you can manage multiple case of "error" (no match/no way to write in file) with a little Bash script like that :
#!/bin/bash
FILE="/etc/inet/hosts"
SEARCH="contoso"
REPLACE="$hostname"
NB=$(grep -c $SEARCH $FILE)
if [ $NB -ne 0 ]; then
perl -pi -e 's/${SEARCH}/'${REPLACE}'/g' "$FILE" && echo "${NB} replaced" || echo "Error (permission maybe)"
else
echo "No match in file"
fi
I think there is a better way by improving the Perl code or by using the sed command. But it should works.
If you expect your perl script to return a value that has some meaning, you will need to write your perl script to return a meaningful value. In your case, perhaps something as simple as:
perl -p -e 's/contoso/'"$hostname"'/g; $rv=1 if $&; END{ exit !$rv }'
Generally checksums is a very efficient way to detect changes in files.
md5sum [filename]
root#miaoulis:~# echo 'line 1' >>1.txt
root#miaoulis:~# md5sum 1.txt
5c2ce561e1e263695dbd267271b86fb8 1.txt
root#miaoulis:~# echo 'line 2' >>1.txt
root#miaoulis:~# md5sum 1.txt
c7253b64411b3aa485924efce6494bb5 1.txt
I guess the sum could be extracted from the output with AWK
root#miaoulis:~# echo $(md5sum 1.txt) | awk 'BEGIN{FS=" *"}{print "MD5:",$1}'
MD5: c7253b64411b3aa485924efce6494bb5
root#miaoulis:~# echo $(md5sum 1.txt) | awk 'BEGIN{FS=" *"}{print "filename:",$2}'
filename: 1.txt
FS=" *" instructs AWK to split the string on the occurrence of one or more spaces. $1 will be the MD5, $2 will be the filename.
MD5 checksum works fast for any size of file. The downside is that you don't really detect what exactly changed in the file, only the fact that it has changed. Should be good enough for most scenarios.

How do I use Bash to create a copy of a file with an extra suffix before the extension?

This title is a little confusing, so let me break it down. Basically I have a full directory of files with various names and extensions:
MainDirectory/
image_1.png
foobar.jpeg
myFile.txt
For an iPad app, I need to create copies of these with the suffix #2X appended to the end of all of these file names, before the extension - so I would end up with this:
MainDirectory/
image_1.png
image_1#2X.png
foobar.jpeg
foobar#2X.jpeg
myFile.txt
myFile#2X.txt
Instead of changing the file names one at a time by hand, I want to create a script to take care of it for me. I currently have the following, but it does not work as expected:
#!/bin/bash
FILE_DIR=.
#if there is an argument, use that as the files directory. Otherwise, use .
if [ $# -eq 1 ]
then
$FILE_DIR=$1
fi
for f in $FILE_DIR/*
do
echo "Processing $f"
filename=$(basename "$fullfile")
extension="${filename##*.}"
filename="${filename%.*}"
newFileName=$(echo -n $filename; echo -n -#2X; echo -n $extension)
echo Creating $newFileName
cp $f newFileName
done
exit 0
I also want to keep this to pure bash, and not rely on os-specific calls. What am I doing wrong? What can I change or what code will work, in order to do what I need?
#!/bin/sh -e
cd "${1-.}"
for f in *; do
cp "$f" "${f%.*}#2x.${f##*.}"
done
It's very easy to do that with awk in one line like this:
ls -1 | awk -F "." ' { print "cp " $0 " " $1 "#2X." $2 }' | sh
with ls -1 you get just the bare list of files, then you pipe awk to use the dot (.) as separator. Then you build a shell command to create a copy of each file.
I suggest to run the command without the last sh pipe before, in order to check the cp commands are correct. Like this:
ls -1 | awk -F "." ' { print "cp " $0 " " $1 "#2X." $2 }'

In a unix box, I am taking a list of files as input. If it is found, return the path otherwise return a message "filename file not found"

I have used the find command for this, but it doesnt return any message when a file is not found.
And I want the search to be recursive and return a message "not found" when a file is not found.
Here's the code I have done so far. Here "input.txt" contains the list of files to be searched.
set `cat input.txt`
echo $#
for i in $#
do
find $HOME -name $i
done
Try this:
listfile=input.txt
exec 3>&1
find | \
grep -f <( sed 's|.*|/&$|' "$listfile" ) | \
tee /dev/fd/3 | \
sed 's|.*/\([^/]*\)$|\1|' | \
grep -v -f - "$listfile" | \
sed 's/$/ Not found/'
exec 3>&-
open file descriptor 3
find the files
see if they're on the list (use sed to
send a copy of the found ones to file descriptor 3
strip off the directory name
get a list of the ones that don't appear
add the "Not found" message
close file descriptor 3
Output looks like:
/path/to/file1
/path/somewhere/file2
foo Not found
bar Not found
No loops necessary.
Whats wrong with using a script. I hope this will do.
#!/bin/bash -f
for i in $#
do
var=`find $HOME -name $i`
if [ -z "$var"]
then
var="File not found"
fi
echo $var
done
You can use the shell builtin 'test' to test the existence of a file. There is also an alternative syntax using square brackets:
if [ -f $a ]; then # Don't forget the semicolon.
echo $a
else
echo 'Not Found'
fi
Here is one way - create a list of all the files to grep against. If your implementation supports
grep -q otherwise use grep [pattern] 2&>1 >/dev/null....
find $HOME -type f |
while read fname
do
echo "$(basename $fname) $fname"
done > /tmp/chk.lis
while read fname
do
grep -q "^$fname" /tmp/chk.lis
[ $? -eq 0 ] && echo "$fname found" || echo "$fname not found"
done < /tmp/chk.lis
All of this is needed because POSIX find does not return an error when a file is not found
perl -nlE'say-f$_?$_:"not found: $_"' file

Resources