I use this BASH script to check if file.txt has between 1 and 10 lines:
if [[ `wc -l file.txt | awk '{ print $1 }'` -le "10" && `wc -l file.txt | awk '{ print $1}'` -ge "2" ]]
echo "It has between 1 and 10 lines."
fi
This code is too verbose. If I make a change to one part, it is easy to forget to make a change to the repeated part.
Is there a way to simplify the syntax?
One option would be to do the whole thing using awk:
awk 'END{if(1<=NR&&NR<=10) print "It has between 1 and 10 lines."}' file.txt
As pointed out in the comments (thanks rici), you might want to prevent awk from processing the rest of your file once it has read 10 lines:
awk 'NR>10{exit}END{if(1<=NR&&NR<=10) print "It has between 1 and 10 lines."}' file.txt
The END block is still processed if exit is called, so it is still necessary to have both checks in the if.
Alternatively, you could store the result of wc -l to a variable in bash:
lines=$(wc -l < file.txt)
(( 1 <= lines && lines <= 10)) && echo "It has between 1 and 10 lines."
Note that redirecting the file into wc means that you just get the number without the filename.
Get the line count, then check it against the range bounds:
lc=$(wc -l < file.txt)
if (( 1 <= lc && lc <= 10 )); then
echo "It has between 1 and 10 lines"
fi
Related
I have a shell script that gives me a txt file containing certain numbers.
For instance, " 48 347 345 221 1029 3943 1245 7899 " .
It only contains one line.
I want to trigger another shell script if one of those numbers exceeds 500.
How can and compare the numbers and run the shell script?
Thanks in advance
cat text.txt | if [ awk '{print $1}' -ge 500 ] then command.sh fi
Using awk you can try this:
awk '{for(i=1;i<=NF;i++)if($i > 500)exit 1}' text.txt || ./command.sh
command.sh will be executed if the awk command exits with a non-zero code because of the || operator.
Throw the file contents into an array -
$: declare -a lst=( $(<text.txt) )
Then run a quick loop. If you want to run the command for each hit,
$: for n in "${lst[#]}"
do (( n > 500 )) && echo "command.sh # $n > 500"
done
command.sh # 1029 > 500
command.sh # 3943 > 500
command.sh # 1245 > 500
command.sh # 7899 > 500
If you just want a quick and dirty version,
$: for n in $(<text.txt); do (( n > 500 )) && command.sh; done
But I recommend you take the time to do the horribly complicated 2 steps, lol
If you want to run it just once if any number is over 500,
$: for n in "${lst[#]}"
do (( n > 500 )) && { echo "command.sh # $n > 500"; break; }
done
command.sh # 1029 > 500
And don't use cat like that. Try to never use cat like that.
If you *REALLY needed the file as input to an if, then do this:
if condition
then action
fi < text.txt
You could use a simple for loop like this:
for n in $(cat test.txt)
do
if [ $n -gt 500 ]; then
something.sh
fi
done
You can simply :
[[ "48 347 345 221 500" =~ ([ ][5-9][0-9][0-9]|[ ][1-9][0-9]{3,}) ]] && echo "Hi"
Hi
[[ "48 347 345 221" =~ ([ ][5-9][0-9][0-9]|[ ][1-9][0-9]{3,}) ]] && echo "Hi"
Hope this helps!
When you don't want to convert the sring to numbers, it will become nasty.
Look for a number with at least 4 digits (first not 0), or a number with 3 digits, first 5 or higher will give
# incorrect, includes 500
grep -Eq "([1-9][0-9]|[5-9])[0-9][0-9]" text.txt && command.sh
This will also perform command.sh with number 500.
# incorrect, will fail for 5000 and 1500
grep -E "([1-9][0-9]|[5-9])[0-9][0-9]" text.txt | grep -qv 500 && command.sh
Fixing it becomes too complex. awk seems to be the most natural way.
An artificial way is converted the input to lines with one number and add an extra line with the border:
(echo "500 border"; tr ' ' '\n' < text.txt) |
sort -n |
tail -1 |
grep -qv "border" && command.sh
I have a txt file like
Peugeot:406:1999:Silver:1
Ford:Fiesta:1995:Red:2
Peugeot:206:2000:Black:1
Ford:Fiesta:1995:Red:2
I am looking for a command That counts the number of red Ford Fiesta cars.
The last number in each line is the amount of that particular car.
The command I am looking for CANNOT use the -c option of grep.
so this command should just output the number 4.
Any help would be welcome, thank you.
A simple bit of awk would do the trick:
awk -F: '$1=="Ford" && $4=="Red" { c+=$5 } END { print c }' file
Output:
4
Explanation:
The -F: switch means that the input field separator is a colon, so the car manufacturer is $1 (the 1st field), the model is $2, etc.
If the 1st field is "Ford" and the 4th field is "Red", then add the value of the 5th (last) field to the variable c. Once the whole file has been processed, print out the value of c.
For a native bash solution:
c=0
while IFS=":" read -ra col; do
[[ ${col[0]} == Ford ]] && [[ ${col[3]} == Red ]] && (( c += col[4] ))
done < file && echo $c
Effectively applies the same logic as the awk one above, without any additional dependencies.
Methods:
1.) use some scripting language for counting, like awk or perl and such. Awk solution already posted, here is an perl solution.
perl -F: -lane '$s+=$F[4] if m/Ford:.*:Red/}{print $s' < carfile
#or
perl -F: -lane '$s+=$F[4] if ($F[0]=~m/Ford/ && $F[3]=~/Red/)}{print $s' < carfile
both examples prints
4
2.) The second method is based on shell-pipelining
filter out the right rows
extract the column with the count
sum the numbers
e.g some examples:
grep 'Ford:.*:Red:' carfile | cut -d: -f5 | paste -sd+ | bc
the grep filter out the right rows
the cut get the last column
the paste creates an line like 2+2 what can be counted by
the bc for counting
Another example:
sed -n 's/\(Ford:.*:Red\):\(.*\)/\2/p' carfile | paste -sd+ | bc
the sed filter and extract
another example - different way of counting
(echo 0 ; sed -n 's/\(Ford:.*:Red\):\(.*\)/\2+/p' carfile ;echo p )| dc
numbers are counted by RPN calculator called dc, e.g. it works like 0 2 + - first comes the values and as the last the operation.
the first echo puts into the stack 0
the sed creates a stream of numbers like 2+ 2+
the last echo p prints the stack
exists many other possibilies how count a strem of numbers.
e.g counting by bash
while read -r num
do
sum=$(( $sum + $num ))
done < <(sed -n 's/\(Ford:.*:Red\):\(.*\)/\2/p' carfile)
and pure bash:
while IFS=: read -r maker model year color count
do
if [[ "$maker" == "Ford" && "$color" == "Red" ]]
then
(( sum += $count ))
fi
done < carfile
echo $sum
I'm trying to replace numbers from -20 to 30 using sed, but it adds "v" character. What's wrong?
For example: SINR=-18, output must be "c", but output is "vc".
I tryed to delete 1st character, but it returns 1 instead of j.
SINR=`curl -s http://10.0.0.1/status | awk '/3GPP.SINR=/ {print $0}' | awk -F "3GPP.SINR=" '{print $2}'` # returns number
echo $SINR | sed "s/-20/a/;s/-19/b/;s/-18/c/;s/-17/d/;s/-16/e/;s/-15/f/;s/-14/g/;s/-13/h/;s/-12/i/;s/-11/j/;s/-10/k/;s/-9/l/;s/-8/m/;s/-7/n/;s/-6/o/;s/-5/p/;s/-4/q/;s/-3/r/;s/-2/s/;s/-1/t/;s/0/u/;s/1/v/;s/2/w/;s/3/x/;s/4/y/;s/5/z/;s/6/A/;s/7/B/;s/8/C/;s/9/D/;s/10/E/;s/11/F/;s/12/G/;s/13/H/;s/14/I/;s/15/J/;s/16/K/;s/17/L/;s/18/M/;s/19/N/;s/20/O/;s/21/P/;s/22/Q/;s/23/R/;s/24/S/;s/25/T/;s/26/U/;s/27/V/;s/28/W/;s/29/X/;s/30/Y/"
This way would be more elegant and less error-prone:
echo $SINR | awk 'BEGIN { chars="abcdefg" } { print substr(chars, $1 + 21, 1) }'
Of course, chars should contain all the letters you need for the mapping. That is, all the way until ...VWXY as in your example, I just wrote until g to keep it short and sweet.
With this solution your problem disappears.
You don't really need sed or awk if you have bash like you say you do. You can use arrays, which is maybe even less error-prone ;-)
map=({a..z} {A..Z}) # Create map of your characters
SINR=-18 # Set your SINR number to something
SINR=$(($SINR+20)) # Add an offset to get to right place
result=${map[$SINR]} # Lookup your result
echo $result # Print it
c
If you have a mapping process, you're surely better off building a switch statement, a couple of if's, or even using bash associative arrays (bash >= 4.0). For example, you could tackle your problem with the following snippet:
function mapper() {
if [[ $1 -ge -20 && $1 -le 5 ]]; then
printf \\$(printf '%03o' $(( $1 + 117 )) )
elif [[ $1 -ge 6 && $1 -le 30 ]]; then
printf \\$(printf '%03o' $(( $1 + 59 )) )
else
echo ""; return 1
fi
return 0
}
And use like below:
$ mapper -20
a
$ mapper 5
z
$ mapper 6
A
$ mapper 30
Y
$ mapper $SINR
c
echo "${SINR}" | sed 's/-20/a/;t;s/-19/b/;t;s/-18/c/;t;s/-17/d/;t;s/-16/e/;t;s/-15/f/;t;s/-14/g/;t;s/-13/h/;t;s/-12/i/;t;s/-11/j/;t;s/-10/k/;t;s/-9/l/;t;s/-8/m/;t;s/-7/n/;t;s/-6/o/;t;s/-5/p/;t;s/-4/q/;t;s/-3/r/;t;s/-2/s/;t;s/-1/t/;t;s/0/u/;t;s/1/v/;t;s/2/w/;t;s/3/x/;t;s/4/y/;t;s/5/z/;t;s/6/A/;t;s/7/B/;t;s/8/C/;t;s/9/D/;t;s/10/E/;t;s/11/F/;t;s/12/G/;t;s/13/H/;t;s/14/I/;t;s/15/J/;t;s/16/K/;t;s/17/L/;t;s/18/M/;t;s/19/N/;t;s/20/O/;t;s/21/P/;t;s/22/Q/;t;s/23/R/;t;s/24/S/;t;s/25/T/;t;s/26/U/;t;s/27/V/;t;s/28/W/;t;s/29/X/;t;s/30/Y/'
Use the t after s// to accelerate a bit.
vc is normaly not occuring if SINR is just a number like specified
This is my code:
nb_lignes=`wc -l $1 | cut -d " " -f1`
for i in $(seq $nb_lignes)
do
m=`head $1 -n $i | tail -1`
//command
done
Please how can i change it to get Get 20% of lines in File randomly to apply "command" on each line ?
20% or 40% or 60 % (it's a parameter)
Thank you.
This will randomly get 20% of the lines in the file:
awk -v p=20 'BEGIN {srand()} rand() <= p/100' filename
So something like this for the whole solution (assuming bash):
#!/bin/bash
filename="$1"
pct="${2:-20}" # specify percentage
while read line; do
: # some command with "$line"
done < <(awk -v p="$pct" 'BEGIN {srand()} rand() <= p/100' "$filename")
If you're using a shell without command substitution (the <(...) bit), you can do this - but the body of the loop won't be able to have any side effects in the outer script (e.g. any variables it sets won't be set anymore once the loop completes):
#!/bin/sh
filename="$1"
pct="${2:-20}" # specify percentage
awk -v p="$pct" 'BEGIN {srand()} rand() <= p/100' "$filename" |
while read line; do
: # some command with "$line"
done
Try this:
file=$1
nb_lignes=$(wc -l $file | cut -d " " -f1)
num_lines_to_get=$((20*${nb_lignes}/100))
for (( i=0; i < $num_lines_to_get; i++))
do
line=$(head -$((${RANDOM} % $nb_lignes)) $file | tail -1)
echo "$line"
done
Note that ${RANDOM} only generates numbers less than 32768 so this approach won't work for large files.
If you have shuf installed, you can use the following to get a random line instead of using $RANDOM.
line=$(shuf -n 1 $file)
you can do it with awk.see below:
awk -v b=20 '{a[NR]=$0}END{val=((b/100)*NR)+1;for(i=1;i<val;i++)print a[i]}' all.log
the above command prints 20% of all the lines starting from begining of the file.
you just have to change the value of b on command line to get the required % of lines.
tested below:
> cat temp
1
2
3
4
5
6
7
8
9
10
> awk -v b=10 '{a[NR]=$0}END{val=((b/100)*NR)+1;for(i=1;i<val;i++)print a[i]}' temp
1
> awk -v b=20 '{a[NR]=$0}END{val=((b/100)*NR)+1;for(i=1;i<val;i++)print a[i]}' temp
1
2
>
shuf will produce the file in a randomized order; if you know how many lines you want, you can give that to the -n parameter. No need to get them one at a time. So:
shuf -n $(( $(wc -l < $FILE) * $PCT / 100 )) "$file" |
while read line; do
# do something with $line
done
shuf comes standard with GNU/Linux distros afaik.
I have the following data in a Tab delimited file:
_ DATA _
Col1 Col2 Col3 Col4 Col5
blah1 blah2 blah3 4 someotherText
blahA blahZ blahJ 2 someotherText1
blahB blahT blahT 7 someotherText2
blahC blahQ blahL 10 someotherText3
I want to make sure that the data in 4th column of this file is always an integer. I know how to do this in perl
Read each line, Store value of 4th column in a variable
check if that variable is an integer
if above is true, continue the loop
else break out of the loop with message saying file data not correct
But how would I do this in a shell script using standard linux/unix filter? My guess would be to use grep, but I am not sure how?
cut -f4 data | LANG=C grep -q '[^0-9]' && echo invalid
LANG=C for speed
-q to quit at first error in possible long file
If you need to strip the first line then use tail -n+2 or you could get hacky and use:
cut -f4 data | LANG=C sed -n '1b;/[^0-9]/{s/.*/invalid/p;q}'
awk is the tool most naturally suited for parsing by columns:
awk '{if ($4 !~ /^[0-9]+$/) { print "Error! Column 4 is not an integer:"; print $0; exit 1}}' data.txt
As you get more complex with your error detection, you'll probably want to put the awk script in a file and invoke it with awk -f verify.awk data.txt.
Edit: in the form you'd put into verify.awk:
{
if ($4 !~/^[0-9]+$/) {
print "Error! Column 4 is not an integer:"
print $0
exit 1
}
}
Note that I've made awk exit with a non-zero code, so that you can easily check it in your calling script with something like this in bash:
if awk -f verify.awk data.txt; then
# action for success
else
# action for failure
fi
You could use grep, but it doesn't inherently recognize columns. You'd be stuck writing patterns to match the columns.
awk is what you need.
I can't upvote yet, but I would upvote Jefromi's answer if I could.
Sometimes you need it BASH only, because tr, cut & awk behave differently on Linux/Solaris/Aix/BSD/etc:
while read a b c d e ; do [[ "$d" =~ ^[0-9] ]] || echo "$a: $d not a numer" ; done < data
Edited....
#!/bin/bash
isdigit ()
{
[ $# -eq 1 ] || return 0
case $1 in
*[!0-9]*|"") return 0;;
*) return 1;;
esac
}
while read line
do
col=($line)
digit=${col[3]}
if isdigit "$digit"
then
echo "err, no digit $digit"
else
echo "hey, we got a digit $digit"
fi
done
Use this in a script foo.sh and run it like ./foo.sh < data.txt
See tldp.org for more info
Pure Bash:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer"; fi; ((linenum++)); done < data.txt
To stop at the first error, add a break:
linenum=1; while read line; do field=($line); if ((linenum>1)); then [[ ! ${field[3]} =~ ^[[:digit:]]+$ ]] && echo "FAIL: line number: ${linenum}, value: '${field[3]}' is not an integer" && break; fi; ((linenum++)); done < data.txt
cut -f 4 filename
will return the fourth field of each line to stdout.
Hopefully that's a good start, because it's been a long time since I had to do any major shell scripting.
Mind, this may well not be the most efficient compared to iterating through the file with something like perl.
tail +2 x.x | sort -n -k 4 | head -1 | cut -f 4 | egrep "^[0-9]+$"
if [ "$?" == "0" ]
then
echo "file is ok";
fi
tail +2 gives you all but the first line (since your sample has a header)
sort -n -k 4 sorts the file numerically on the 4th column, letters will rise to the top.
head -1 gives you the first line of the file
cut -f 4 gives you the 4th column, of the first line
egrep "^[0-9]+$" checks if the value is a number (integers in this case).
If egrep finds nothing, $? is 1, otherwise it's 0.
There's also:
if [ `tail +2 x.x | wc -l` == `tail +2 x.x | cut -f 4 | egrep "^[0-9]+$" | wc -l` ] then
echo "file is ok";
fi
This will be faster, requiring two simple scans through the file, but it's not a single pipeline.
#OP, use awk
awk '$4+0<=0{print "not ok";exit}' file