I'm trying to fix a bash script by adding in some error catching. I have a file (list.txt) that normally has content like this:
People found by location:
person: john [texas]
more info on john
Sometimes that file gets corrupted, and it only has that first line:
People found by location:
I'm trying to find a method to check that file to see if any data exists on line 2, and I want to include it in my bash script. Is this possible?
Simple and clean:
if test $(sed -n 2p < /path/to/file); then
# line 2 exists and it is not blank
else
# otherwise...
fi
With sed we extract the second line only. The test expression will evaluate to true only if there is a second non-blank line in the file.
I assume that you want to check whether line 2 of a given file contains any data or not.
[ "$(sed -n '2p' inputfile)" != "" ] && echo "Something present on line 2" || echo "Line 2 blank"
This would work even if the inputfile has just one line.
If you simply want to check whether the inputfile has one line or more, you can say:
[ "$(sed -n '$=' z)" == "1" ] && echo "Only one line" || echo "More than one line"
Sounds like you want to check if your file has more than 1 line
if (( $(wc -l < filename) > 1 )); then
echo I have a 2nd line
fi
Another approach which doesn't require external commands is:
if ( IFS=; read && read -r && [[ -n $REPLY ]]; ) < /path/to/file; then
echo true
else
echo false
fi
Related
I have a single txt file where there are several codes with its metadata. However, I have been seeing how to change those codes with their names.
For this I have a list in csv with two columns, where the codes are in one and the names are in the other. These are 79.
This is a draft of script I have made:
#Here, the variable "Nombre" are the values of column 1 and the variable "Codigo" are the values of column 2 of nombres_codigos.csv file
#
#Then, I have use command sed to replace values of "Codigo" with values of "Nombre" into read RAxML_bipartitions_newick.newick file
#!/bin/bash
read nombres_codigos.csv
while read -r line || [[ -n $line ]]
do
Nombre="${line%;*}"
Codigo="${line#*;}"
echo "$Nombre"
echo "$Codigo"
read RAxML_bipartitions_newick.newick
while read -r line || [ -n "$line" ]
do
if [ "$Nombre" == "$Codigo" ]
then
sed -i "s/$Nombre/$Codigo/g" RAxML_bipartitions_newick.newick
echo "reemplazar"
else
echo "no reemplazar"
fi
done
p.s: nombres_codigos.csv is like:
Nombre;Codigo
EU528205_Floripondio_A56y7;EU528205_F
FJ710459_Floripondio_ABCD;FJ710459_F
EF514777_Floripondio123;EF514777_F
EU528999_Floripondio1;EU528999_F
...
My result is:
nombres.sh: line 2: read: `nombres_codigos.csv': not a valid identifier
nombres.sh: line 21: syntax error: unexpected end of file
But I hope from the file RAxML_bipartitions_newick.newick:
((((((EU528205_F:1.0000005000176948E-6,FJ710459_F:1.0000005000176948E-6):1.0000005000176948E-6,EF514777_F:1.0000005000176948E-...
switch to this:
((((((EU528205_Floripondio_A56y7:1.0000005000176948E-6,FJ710459_Floripondio_ABCD:1.0000005000176948E-6):1.0000005000176948E-6,EF514777_Floripondio123:1.0000005000176948E-...
Could someone help me modifying this, please? Thanks very much.
I am not 100% certain what you are attempting to accomplish, but it appears you want to read each line from nombres_codigos.csv and then split the line on the semi-colon. Then you want to check if the two parts are the same (which does not make sense with the sed expression that comes next)
A directly, albeit cleaned up reformatting would be:
#!/bin/bash
while read -r line || [ -n "$line" ]
do
Nombre="${line%;*}" ## separate using bash built-in parameter expansions
Codigo="${line#*;}"
echo "$Nombre"
echo "$Codigo"
if [ "$Nombre" == "$Codigo" ]
then
sed -i "s/$Nombre/$Codigo/g" RAxML_bipartitions_newick.newick
echo "reemplazar"
else
echo "no reemplazar"
fi
done < nombres_codigos.csv
If you separate "$Nombre" and "$Codigo" and then check if they are the SAME, then ... having sed replace "$Nombre" with "$Codigo" doesn't do anything at all. (It's like replacing 'A' with 'A') Did you mean:
if [ "$Nombre" != "$Codigo" ]
That would make what the script appears to do -- consistent. (e.g. change all instances of "$Nombre" with "$Codigo" in the file RAxML_bipartitions_newick.newick) If that is the case, then just change the conditional and let me know if you have further questions.
I am trying to write a shell script that asks a user for number of lines they would like to display from a file and then display those lines.
I am trying to do this via the following:
#!/bin/bash
#author = johndoe
read -p "How many lines from /c/Users/johndoe/files/helpme.sh would you like to see? " USERLINEINPUT
LINE_NUM=1
while [ $LINE_NUM -lt $USERLINEINPUT ]
do
echo "$LINE_NUM: $USESRLINEINPUT"
((LINE_NUM++))
done < /c/Users/johndoe/files/helpme.sh
This code doesn't appear to do as I'd like, please see an example below:
$ ./linecount.sh
How many lines from /c/Users/johndoe/files/helpme.sh would you line to see? 10
1:
2:
3:
4:
5:
6:
7:
8:
9:
Your code does not satisfy your requirement. You need to read each line of code into a variable and print it. Your while loop is only satisfy with user input value and you are not printing the file line at all. See the correct code below and see you mistakes. Hope this will help you:-
#!/bin/bash
#author = johndoe
LINE_NUM=1
read -p "How many lines from /c/Users/johndoe/files/helpme.sh would you like to see? " USERLINEINPUT
while read -r line
do
echo "$LINE_NUM:$line"
if [ $LINE_NUM -ge $USERLINEINPUT ]; then
break;
fi
((LINE_NUM++))
done < "/c/Users/johndoe/files/helpme.sh"
#!/usr/bin/env bash
# file will be the first argument or helpme.sh by default
file=${1:-/c/Users/johndoe/files/helpme.sh}
# exit if the file is NOT readable
[[ -f $file && -r $file ]] || exit 1
# ask for a number and exit if the input is NOT a valid number
IFS= read -r -p "Number of lines from \"$file\": " num
[[ $num =~ ^[0-9]+$ ]] || exit 2
# 1st option: while/read
count=
while IFS= read -r line && ((count++<num)); do
printf '%d:%s\n' "$count" "$line"
done < "$file"
# 2nd option: awk
awk 'NR>ln{exit}{print NR":"$0}' ln="$num" "$file"
I am creating a simple phonebook using unix shell scripts. I have gotten all of my functions to work except the removal of a contact after it has been created. I have tried combining grep and sed in order to accomplish this, but cannot seem to get over the hump. The removal shell i've tried is as follows.
#!/bin/sh
#removeContact.sh
echo “Remove Submenu”
echo “Please input First Name:”
read nameFirst
echo “Please input Last Name:”
read nameLast
x=$(grep -e “$nameFirst” -e “$nameLast” ContactList)
echo $x
sed '/'$x'/ d' ContactList;
echo “$nameFirst $nameLast is removed from your contacts”
exit 0
I'm not sure if I am declaring x incorrectly, or if my syntax is wrong when sed is used.
Any help would be greatly appreciated. Thank you.
#!/bin/bash
ContactList="contacts.txt"
export ContactList
exit=0
while [ $exit -ne 1 ]
do
echo "Main Menu"
echo "(a) Add a Contact"
echo "(r) Remove a Contact"
echo "(s) Search a Contact"
echo "(d) Display All Contact’s Information"
echo "(e) Exit"
echo "Your Choice?"
read choice
if [ "$choice" = "a" ]
then
./addContact.sh
elif [ "$choice" = "r" ]
then
./removeContact.sh
elif [ "$choice" = "s" ]
then
./searchContact.sh
elif [ "$choice" = "d" ]
then
./displayContact.sh
elif [ "$choice" = "e" ]
then
exit=1
else
echo "Error"
sleep 2
fi
done
exit 0
#!/bin/sh
#addContact.sh
ContactList="contacts.txt"
echo “Please input First Name:”
read nameFirst
echo “Please input Last Name:”
read nameLast
echo “Please input Phone Number:”
read number
echo “Please Input Address”
read address
echo “Please input Email:”
read email
echo $nameFirst:$nameLast:$number:$address:$email>> ContactList;
echo "A new contact is added to your book."
exit 0
sed '/'$x'/ d' ContactList
won't remove anything from the file ContactList, it will simply output the changes to standard output.
If you want to edit the file in-place, you'll need the -i flag (easy) or to make a temporary file which is then copied back over ContactList (not so easy, but needed if your sed has no in-place editing option).
In addition, since ContactList is a shell variable referencing the real file contacts.txt, you'll need to use $ContactList.
And, as a final note, since you're using the full line content to do deletion, the presence of an address like 1/15 Station St is going to royally screw up your sed command by virtue of the fact it contains the / character.
I would suggest using awk rather than sed for this task since it's much better suited to field-based data. With the record layout:
$nameFirst:$nameLast:$number:$address:$email
you could remove an entry with something like (including my patented paranoid perfect protection policy):
cp contacts.txt contacts.txt.$(date +%Y.%m.%d.%H.%M.%S_$$)
awk <contacts.txt >tmp.$$ -F: "-vF=$nameFirst" "-vL=$nameLast" '
F != $1 || L != $2 {print}'
mv tmp.$$ contacts.txt
I have a problem with writing bash script. The problem is in comparison of strings. When I launch it, there's no errors. However in result, it is always changing the variable client.
So if for an example we have two lines in file
apple A
orange D
and if I give the who=A I expect to see in result apple, or if at D - orange
But no matter of what I choose A or D it is always giving me the result - orange
No matter of the strings, it always change the variable client, like ignoring the comparison. Please help.
while read line
do
IFS=" "
set -- $line
echo $2" "$who":"$1
if [[ "$2"="$who" ]]
then
echo "change"
client=$1
fi
done < $file
echo $client
So now I changed the code as in one of the comment below, but now the caparison always false therefore the variable client is always empty
while read -r line
do
#IFS=" "
#set -- $line
#echo $2" "$who":"$1
#if [[ "$2" = "$who" ]]
a="${line% *}"
l="${line#* }"
if [[ "$l" == "$who" ]]
then
echo "hi"
client="$a"
fi
done < $file
If you have data in a file with each line like apple D and you want to read the file and separate then items, the parameter expansion/substring extraction is the correct way to process the line. For example (note $who is taken from your problem statement):
while read -r line
do
fruit="${line% *}" # remove from end to space
letter="${line#* }" # remove from start to space
if [[ "$letter" == "$who" ]]
then
echo "change"
client="$fruit"
fi
done < $file
Short Example
Here is a quick example of splitting the words with parameter expansion/substring extraction:
#!/bin/bash
while read -r line
do
fruit="${line% *}"
letter="${line#* }"
echo "fruit: $fruit letter: $letter"
done
exit 0
input
$ cat dat/apple.txt
Apple A
Orange D
output
$ bash apple.sh <dat/apple.txt
fruit: Apple letter: A
fruit: Orange letter: D
Change if [[ "$2"="$who" ]] to
if [[ "$2" = "$who" ]]
spaces around =
Example (for clarification):
who=A
while read line
do
IFS=" "
set -- $line
echo $2" "$who":"$1
if [[ "$2" = "$who" ]]
then
echo "change"
client=$1
fi
done < file #this is the file I used for testing
echo $client
Output:
A A:apple
change
D A:orange
apple
For who=D:
A D:apple
D D:orange
change
orange
You do need spaces around that = operator.
However, I think you're facing yet another issue as you're trying to change the value of the client variable from inside the while loop (which executes in a subshell). I don't think that will work; see this quesion for details.
I need to validate my log files:
-All new log lines shall start with date.
-This date will respect the ISO 8601 standard. Example:
2011-02-03 12:51:45,220Z -
Using shell script, I can validate it looping on each line and verifying the date pattern.
The code is below:
#!/bin/bash
processLine(){
# get all args
line="$#"
result=`echo $line | egrep "[0-9]{4}-[0-9]{2}-[0-9]{2} [012][0-9]:[0-9]{2}:[0-9]{2},[0-9]{3}Z" -a -c`
if [ "$result" == "0" ]; then
echo "The log is not with correct date format: "
echo $line
exit 1
fi
}
# Make sure we get file name as command line argument
if [ "$1" == "" ]; then
echo "You must enter a logfile"
exit 0
else
file="$1"
# make sure file exist and readable
if [ ! -f $file ]; then
echo "$file : does not exists"
exit 1
elif [ ! -r $file ]; then
echo "$file: can not read"
exit 2
fi
fi
# Set loop separator to end of line
BAKIFS=$IFS
IFS=$(echo -en "\n\b")
exec 3<&0
exec 0<"$file"
while read -r line
do
# use $line variable to process line in processLine() function
processLine $line
done
exec 0<&3
# restore $IFS which was used to determine what the field separators are
IFS=$BAKIFS
echo SUCCESS
But, there is a problem. Some logs contains stacktraces or something that uses more than one line, in other words, stacktrace is an example, it can be anything. Stacktrace example:
2011-02-03 12:51:45,220Z [ERROR] - File not found
java.io.FileNotFoundException: fred.txt
at java.io.FileInputStream.<init>(FileInputStream.java)
at java.io.FileInputStream.<init>(FileInputStream.java)
at ExTest.readMyFile(ExTest.java:19)
at ExTest.main(ExTest.java:7)
...
will not pass with my script, but is valid!
Then, if I run my script passing a log file with stacktraces for example, my script will failed, because it loops line by line.
I have the correct pattern and I need to validade the logger date format, but I don't have wrong date format pattern to skip lines.
I don't know how I can solve this problem. Does somebody can help me?
Thanks
You need to anchor your search for the date to the start of the line (otherwise the date could appear anywhere in the line - not just at the beginning).
The following snippet will loop over all lines that do not begin with a valid date. You still have to determine if the lines constitute errors or not.
DATEFMT='^[0-9]{4}-[0-9]{2}-[0-9]{2} [012][0-9]:[0-9]{2}:[0-9]{2},[0-9]{3}Z'
egrep -v ${DATEFMT} /path/to/log | while read LINE; do
echo ${LINE} # did not begin with date.
done
So just (silently) discard a single stack trace. In somewhat verbose bash:
STATE=idle
while read -r line; do
case $STATE in
idle)
if [[ $line =~ ^java\..*Exception ]]; then
STATE=readingexception
else
processLine "$line"
fi
;;
readingexception)
if ! [[ $line =~ ^' '*'at ' ]]; then
STATE=idle
processLine "$line"
fi
;;
*)
echo "Urk! internal error [$STATE]" >&2
exit 1
;;
esac
done <logfile
This relies on processLine not continuing on error, else you will need to track a tad more state to avoid two consecutive stack traces.
This makes 2 assumptions.
lines that begin with whitespace are continuations of previous lines. we're matching a leading space, or a leading tab.
lines that have non-whitespace characters starting at ^ are new log lines.
If a line matching #2 doesn't match the date format, we have an error, so print the error, and include the line number.
count=0
processLine() {
count=$(( count + 1 ))
line="$#"
result=$( echo $line | egrep '^[0-9]{4}-[0-9]{2}-[0-9]{2} [012][0-9]:[0-9]{2}:[0-9]{2},[0-9]{3}Z' -a -c )
if (( $result == 0 )); then
# if result = 0, then my line did not start with the proper date.
# if the line starts with whitespace, then it may be a continuation
# of a multi-line log entry (like a java stacktrace)
continues=$( echo $line | egrep "^ |^ " -a -c )
if (( $continues == 0 )); then
# if we got here, then the line did not start with a proper date,
# AND the line did not start with white space. This is a bad line.
echo "The line is not with correct date format: "
echo "$count: $line"
exit 1
fi
fi
}
Create a condition to check if the line starts with a date. If not, skip that line as it is part of a multi-line log.
processLine(){
# get all args
line="$#"
result=`echo $line | egrep "[0-9]{4}-[0-9]{2}-[0-9]{2} [012][0-9]:[0-9]{2}:[0-9]{2},[0-9]{3}Z" -a -c`
if [ "$result" == "0" ]; then
echo "Log entry is multi-lined - continuing."
fi
}