I want to count the number of nodes in a graphviz file in a Makefile to use it to start a process for each node.
When I run
grep -- -\> graph.gv | while read line; do for w in $line; do echo $w; done; done | grep [Aa-Zz] | sort | uniq | wc -l
in the shell, it prints the number of nodes as expected.
However, when I use it in my Makefile
NODES := $(shell grep -- -\> graph.gv | while read line; do for w in $line; do echo $w; done; done | grep [Aa-Zz] | sort | uniq | wc -l)
${NODES} is always 0.
You'll need to escape the $ sign. Say:
NODES := $(shell grep -- -\> graph.gv | while read line; do for w in $$line; do echo $$w; done; done | grep [Aa-Zz] | sort | uniq | wc -l)
Related
So I have an expression that I want to extract some lines from a text and count them. I can grep them as follows:
$ cat medsCounts_totals.csv | grep -E 'NumMeds": 0' | wc -l
Which is fine. Now I want to loop over with the string ...
$ for i in {0..10}; do expr="NumMeds\": $i"; echo $expr; done
However, when I try to use $expr
for i in {0..10}; do expr="NumMeds:\" $i"; cat medsCounts_totals.csv | grep -E "$expr" | wc -l ; done
I get nothing. How do I solve this problem in an elegant manner?
there is a typo in
for i in {0..10}; do expr="NumMeds:\" $i"; cat medsCounts_totals.csv | grep -E "$expr" | wc -l ; done
it should be
"NumMeds\": $i"
my task is to create a script that displays the frequency of random characters in the file. The output should display frequency of a to z (case insensitive) in percent.
I created the script below and I just wonder if there is a way how to make it simplier?
#!/bin/bash
echo Hello, please tell me in which file shall I count the letters:
read file
TOTAL=$( grep -o [[:alpha:]] $file | wc -l )
A=$( grep -io a $file | wc -l )
B=$( grep -io b $file | wc -l )
C=$( grep -io c $file | wc -l )
D=$( grep -io d $file | wc -l )
E=$( grep -io e $file | wc -l )
F=$( grep -io f $file | wc -l )
G=$( grep -io g $file | wc -l )
H=$( grep -io h $file | wc -l )
I=$( grep -io i $file | wc -l )
J=$( grep -io j $file | wc -l )
K=$( grep -io k $file | wc -l )
L=$( grep -io l $file | wc -l )
M=$( grep -io m $file | wc -l )
N=$( grep -io n $file | wc -l )
O=$( grep -io o $file | wc -l )
P=$( grep -io p $file | wc -l )
Q=$( grep -io q $file | wc -l )
R=$( grep -io R $file | wc -l )
S=$( grep -io s $file | wc -l )
T=$( grep -io t $file | wc -l )
U=$( grep -io u $file | wc -l )
V=$( grep -io v $file | wc -l )
W=$( grep -io w $file | wc -l )
X=$( grep -io x $file | wc -l )
Y=$( grep -io y $file | wc -l )
Z=$( grep -io z $file | wc -l )
echo Frequency of 'a': $(($A*100/$TOTAL))%
echo Frequency of 'b': $(($B*100/$TOTAL))%
echo Frequency of 'c': $(($C*100/$TOTAL))%
echo Frequency of 'd': $(($D*100/$TOTAL))%
echo Frequency of 'e': $(($E*100/$TOTAL))%
echo Frequency of 'f': $(($F*100/$TOTAL))%
echo Frequency of 'g': $(($G*100/$TOTAL))%
echo Frequency of 'h': $(($H*100/$TOTAL))%
echo Frequency of 'i': $(($I*100/$TOTAL))%
echo Frequency of 'j': $(($J*100/$TOTAL))%
echo Frequency of 'k': $(($K*100/$TOTAL))%
echo Frequency of 'l': $(($L*100/$TOTAL))%
echo Frequency of 'm': $(($M*100/$TOTAL))%
echo Frequency of 'n': $(($N*100/$TOTAL))%
echo Frequency of 'o': $(($O*100/$TOTAL))%
echo Frequency of 'p': $(($P*100/$TOTAL))%
echo Frequency of 'q': $(($Q*100/$TOTAL))%
echo Frequency of 'r': $(($R*100/$TOTAL))%
echo Frequency of 's': $(($S*100/$TOTAL))%
echo Frequency of 't': $(($T*100/$TOTAL))%
echo Frequency of 'u': $(($U*100/$TOTAL))%
echo Frequency of 'v': $(($V*100/$TOTAL))%
echo Frequency of 'w': $(($W*100/$TOTAL))%
echo Frequency of 'x': $(($X*100/$TOTAL))%
echo Frequency of 'y': $(($Y*100/$TOTAL))%
echo Frequency of 'z': $(($Z*100/$TOTAL))%
I considered using for loop as in below script which replace the first part of above script...but then I got stuck as I do not know if there is any way to work with those outputs further?
#!/bin/bash
echo File:
read file
TOTAL=$( grep -o [[:alpha:]] $file | wc -l )
for letter in {a..z}
do echo grep -io $letter $file | wc -l
done
I also want to ask if there is any way how to have output of my script with two decimal places?
This is my first script so please be merciful :) But I will be grateful for any feedback or advice how to get better.
You were almost there! Here's a solution with 2 variants, depending on the output you want and if you want to use bc.
#!/bin/bash
echo File:
read file
TOTAL=$( grep -o "[[:alpha:]]" "$file" | wc -l )
for letter in {a..z}
do
count=$(grep -io $letter "$file" | wc -l)
echo "Frequency of $letter : $(bc <<< "scale=2; $count*100/$TOTAL")%" # Variant with floats, requires bc
echo "Frequency of $letter : $(($count*100/$TOTAL))%" # Variant with integers
done
You can use the awk command inside your bash script
awk -vFS="" 'BEGIN{OFMT="%.2f"} {for(i=1;i<=NF;i++){ if($i~/[a-zA-Z]/) { w[tolower($i)]++} sum++} }END{for(i in w) print i,(100*w[i]/sum),"%"}'
Short and direct, basically I want to use the value of $command on a variable, instead using it inside the while loop as a command itself. So:
This Works, but I think it's ugly:
#!/bin/bash
IFS=$'\n'
lsof=`which lsof`
whoami=`whoami`
while true ; do
execution_array=($(${lsof} -iTCP -P 2> /dev/null | grep ':' | grep ${whoami} | awk '{print $9}' | cut -f2 -d'>' | sort | uniq ))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS
This doesn't work ( no output happens ), but i think is less ugly:
#!/bin/bash
IFS=$'\n'
lsof=`which lsof`
whoami=`whoami`
command="${lsof} -iTCP -P 2> /dev/null | grep ':' | grep ${whoami} | awk '{print $9}' | cut -f2 -d'>' | sort | uniq"
while true ; do
execution_array=($(command))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS
This solved my problem:
#!/bin/bash
IFS=$'\n'
lsof=$(which lsof)
list_connections() {
${lsof} -iTCP -P 2> /dev/null | grep ':' | grep $(whoami) | awk '{print $9}' | cut -f2 -d'>' | sort | uniq
}
while true ; do
execution_array=($(list_connections))
for i in ${execution_array[*]}; do
echo $i
done
sleep 1
done
unset IFS
i have a question about how to store the output into variable and then later pipeline into another command
var=$(ps -auxc | grep -vE '^USER' )
#get top CPU
echo $var | sort -nr -k3 | head -1
#get top memory
echo $var | sort -nr -k4 | head -1
Make sure to use quotes in assignment and while accessing variable:
var="$(ps -auxc | grep -vE '^USER')"
#get top CPU
sort -nr -k3 <<< "$var" | head -1
#get top memory
sort -nr -k4 <<< "$var" | head -1
I'm not sure if this would always work:
IFS= read -rd '' var < <(ps -auxc | grep -vE '^USER') ## -d '' may be -d $'\0'
echo -n "$var" | sort -nr -k3 | head -1
However using readarray could:
readarray -t var < <(ps -auxc | grep -vE '^USER')
printf '%s\n' "${var[#]}" | sort -nr -k4 | head -1
I want to tee and get the results from multiple shell commands connected in the pipeline. I made a simple example to explain the point. Suppose I wanna count the numbers of 'a', 'b' and 'c'.
echo "abcaabbcabc" | tee >(tr -dc 'a' | wc -m) >(tr -dc 'b' | wc -m) >(tr -dc 'c' | wc -m) > /dev/null
Then I tried to assign the result from each count to a shell variable, but they all end up empty.
echo "abcaabbcabc" | tee >(A=$(tr -dc 'a' | wc -m)) >(B=$(tr -dc 'b' | wc -m)) >(C=$(tr -dc 'c' | wc -m)) > /dev/null && echo $A $B $C
What is the right way to do it?
Use files. They are the single most reliable solution. Any of the commands may need different time to run. There is no easy way to synchronize command redirections. Then most reliable way is to use a separate "entity" to collect all the data:
tmpa=$(mktemp) tmpb=$(mktemp) tmpc=$(mktemp)
trap 'rm "$tmpa" "$tmpb" "$tmpc"' EXIT
echo "abcaabbcabc" |
tee >(tr -dc 'a' | wc -m > "$tmpa") >(tr -dc 'b' | wc -m > "$tmpb") |
tr -dc 'c' | wc -m > "$tmpc"
A=$(<"$tmpa")
B=$(<"$tmpb")
C=$(<"$tmpc")
rm "$tmpa" "$tmpb" "$tmpc"
trap '' EXIT
Second way:
You can prepend the data from each stream with a custom prefix. Then sort all lines (basically, buffer them) on the prefix and then read them. The example script will generate only a single number from each process substitution, so it's easy to do:
read -r A B C < <(
echo "abcaabbcabc" |
tee >(
tr -dc 'a' | wc -m | sed 's/^/A /'
) >(
tr -dc 'b' | wc -m | sed 's/^/B /'
) >(
tr -dc 'c' | wc -m | sed 's/^/C /'
) >/dev/null |
sort |
cut -d' ' -f2 |
paste -sd' '
)
echo A="$A" B="$B" C="$C"
Using temporary files with flock to synchronize the output of child processes could look like this:
tmpa=$(mktemp) tmpb=$(mktemp) tmpc=$(mktemp)
trap 'rm "$tmpa" "$tmpb" "$tmpc"' EXIT
echo "abcaabbcabc" |
(
flock 3
flock 4
flock 5
tee >(
tr -dc 'a' | wc -m |
{ sleep 0.1; cat; } > "$tmpa"
# unblock main thread
flock -u 3
) >(
tr -dc 'b' | wc -m |
{ sleep 0.2; cat; } > "$tmpb"
# unblock main thread
flock -u 4
) >(
tr -dc 'c' | wc -m |
{ sleep 0.3; cat; } > "$tmpc"
# unblock main thread
flock -u 5
) >/dev/null
# wait for subprocesses to finish
# need to re-open the files to block on them
(
flock 3
flock 4
flock 5
) 3<"$tmpa" 4<"$tmpb" 5<"$tmpc"
) 3<"$tmpa" 4<"$tmpb" 5<"$tmpc"
A=$(<"$tmpa")
B=$(<"$tmpb")
C=$(<"$tmpc")
declare -p A B C
You can use this featured letter frequency analysis
#!/usr/bin/env bash
declare -A letter_frequency
while read -r v k; do
letter_frequency[$k]="$v"
done < <(
grep -o '[[:alnum:]]' <<<"abcaabbcabc" |
sort |
uniq -c
)
for k in "${!letter_frequency[#]}"; do
printf '%c = %d\n' "$k" "${letter_frequency[$k]}"
done
Output:
c = 3
b = 4
a = 4
Or to only assign $A, $B and $C as in your example:
#!/usr/bin/env bash
{
read -r A _
read -r B _
read -r C _
}< <(
grep -o '[[:alnum:]]' <<<"abcaabbcabc" |
sort |
uniq -c
)
printf 'a=%d\nb=%d\nc=%d\n' "$A" "$B" "$C"
grep -o '[[:alnum:]]': split each alphanumeric character on its own line
sort: sort lines of characters
uniq -c: count each instance and output count and character for each
< <( command group; ): the output of this command group is for stdin of the command group before
If you need to count occurrence of non-printable characters, newlines, spaces, tabs, you have to make all these commands output and deal with null delimited lists. It can sure be done with the GNU versions of these tools. I let it to you as an exercise.
Solution to the count arbitrary characters except null:
As demonstrated, works also with Unicode.
#!/usr/bin/env bash
declare -A character_frequency
declare -i v
while read -d '' -r -N 8 v && read -r -d '' -N 1 k; do
character_frequency[$k]="$v"
done < <(
grep --only-matching --null-data . <<<$'a¹bc✓ ✓\n\t\t\u263A☺ ☺ aabbcabc' |
head --bytes -2 | # trim the newline added by grep
sort --zero-terminated | # sort null delimited list
uniq --count --zero-terminated # count occurences of char (null delim)
)
for k in "${!character_frequency[#]}"; do
printf '%q = %d\n' "$k" "${character_frequency[$k]}"
done
Output:
$'\n' = 1
$'\t' = 2
☺ = 3
\ = 7
✓ = 2
¹ = 1
c = 3
b = 4
a = 4