I have the following script which sends the results of an iwlist scan via OSC:
#!/bin/bash
NUM_BANKS=20
while [[ "$input" != "\e" ]] ; do
networks=$(iwlist wlan0 scanning | awk 'BEGIN{ FS="[:=]"; OFS = " " }
/ESSID/{
#gsub(/ /,"\\ ",$2)
#gsub(/\"/,"",$2)
essid[c++]=$2
}
/Address/{
gsub(/.*Address: /,"")
address[a++]=$0
}
/Encryption key/{ encryption[d++]=$2 }
/Quality/{
gsub(/ dBm /,"")
signal[b++]=$3
}
END {
for( c in essid ) { print "/wlan_scan ",essid[c],signal[c],encryption[c] }
}'
)
read -t 0.1 input
echo "$networks" | while read network; do
set $network
hash=` echo "$2" | md5sum | awk '{ print $1 }'| tr '[:lower:]' '[:upper:]'`
bank=`echo "ibase=16;obase=A; $hash%$NUM_BANKS " | bc`
echo "$1$bank $2 $3 $4"
echo "$1$bank $2 $3 $4" | sendOSC -h localhost 9997
done
#echo "$networks" | sendOSC -h localhost 9997
done
An example of the output from this is '/wlan_scan13 BTHomehub757 -85 On', which is then sent via the sendOSC program.
I basically need to replace the iwlist scan data with the results of this tshark scan:
sudo tshark -I -i en1 -T fields -e wlan.sa_resolved -e wlan_mgt.ssid -e radiotap.dbm_antsignal type mgt subtype probe
which similarly outputs two strings and an int, outputting a result like:
'Hewlett-_91:fa:xx EE-BrightBox-mjmxxx -78'.
So eventually I want the script to give me an output in this instance of
'/wlan13 Hewlett-_91:fa:xx EE-BrightBox-mjmxxx -78'.
Both scans constantly generate results in this format at about the same rate, updating as new wifi routers are detected, and these are sent out as soon as they arrive over the sendOSC program.
This is probably a pretty simple edit for an experienced coder, but I've been trying to work this out for days and I figured I should ask for help!
If someone could clarify what needs to stay and what needs to go here I'd really appreciate it.
Many thanks.
Do you really want to replace commands? The sane approach would seem to be to add an option to the script to specify which piece of code to run, and include them both.
# TODO: replace with proper option parsing
case $1 in
--tshark) command=tshark_networks; shift;;
*) command=iwlist_networks;;
esac
tshark_networks () {
sudo tshark -I -i en1 -T fields \
-e wlan.sa_resolved \
-e wlan_mgt.ssid \
-e radiotap.dbm_antsignal type mgt subtype probe
}
iwlist_networks () {
iwlist wlan0 scanning | awk .... long Awk script here ....
}
while [[ "$input" != "\e" ]] ; do
networks=$($command)
read -t 0.1 input
echo "$networks" | while read network; do
: the rest as before, except fix your indentation
This also has the nice side effect that the hideous iwlist command is encapsulated in its own function, outside of the main loop.
... Well, in fact, I might refactor the main loop to
while true; do
$command |
while read a b c d; do
hash=$(echo "$b" | md5sum | awk '{ print toupper($1) }')
bank=$(echo "ibase=16;obase=A; $hash%$NUM_BANKS " | bc)
echo "$a$bank $b $c $d"
echo "$a$bank $b $c $d" | sendOSC -h localhost 9997
done
read -t 0.1 input
case $input in '\e') break;; esac
done
Related
I have a file that has multiple blocks of lines like so
line1
line1
-----
machine:chrome
purpose:
language:v2
request:v3
additional: v4
os:v4
-----
machine:firefox
purpose:
language:v2
request:v6
os:v4
-----
machine:helper
purpose:
language:v2
request:v8
os:v4
-----
another line
The lines don't necessarily have the same elements but they all start with machine and end with os.
I can only use shell commands so what I want to do is parse the line starting with machine in each block starting with machine and ending in os and use the parsed result is a command whose result is to be inserted in request.
so parse each line that has machine in it and use that value to run a different shell command with its result and then populate request with that result. As a challenge I was wondering if this could be done using only sed and awk.
My expected output for the above would be:
line1
line1
-----
machine:chrome
purpose:
language:v2
request:[output of result of ps -ef chrome | awk '{print $2}']
additional: v4
os:v4
-----
machine:firefox
purpose:
language:v2
request:[output of result of ps -ef firefox | awk '{print $2}']
os:v4
-----
machine:helper
purpose:
language:v2
request:[output of result of ps -ef helper | awk '{print $2}']
os:v4
-----
another line
Update:
Trying to do this in sed alone I got the following:
gsed -r '/machine/,/os/ {/machine/ {s/.*?:\s*([^\s]+)/ps -ef | grep \1\n/e}; /request/ {s/.*?:\s*([^\s]+)//}}' filename
Which does not work but It runs the ps -ef | grep [machinename] and stores it in the buffer. Now I'd like to know if I can use the buffer value in the request substitution and if so how?
A
Edit: Because of changed requirements I am updating the script. The folowing produces the required output:
#!/bin/bash
function processlines() {
local line machine request
# Skips first three lines, but it is not really necessary.
#for k in `seq 1 3`; do read line; echo "$line"; done
while true; do
read -r line || return 0
if echo "$line" | grep '^machine:' >> /dev/null; then
machine="$(echo "$line" | cut -d ':' -f 2)"
echo "$line"
elif echo "$line" | grep '^request:' >> /dev/null; then
request="$(echo YOUR COMMAND HERE "$machine")"
echo "request:$request"
else
echo "$line"
fi
done
}
processlines < test.txt
Note: This works as long as the fields appear in the order shown by you. If "request" appears before "machine" or if one of both is missing in the file, the script would break. Please let me know if this can be the case.
Old answer:
You don't need sed or awk for that. It's doable almost by pure bash + tail/cut:
cat test.txt | tail -n +4 | while read machineline; do
[[ "$machineline" == "another line" ]] && break
read purposeline
read languageline
read requestline
read osline
read separatorline
machine="$(echo $machineline | cut -d ':' -f 2)"
purpose="$(echo $purposeline | cut -d ':' -f 2)"
language="$(echo $languageline | cut -d ':' -f 2)"
request="$(echo $requestline | cut -d ':' -f 2)"
os="$(echo $osline | cut -d ':' -f 2)"
separator="$(echo $separatorline | cut -d ':' -f 2)"
# Here do anything with the variables...
echo "machine is '$machine'" \
"purpose is '$purpose'" \
"language is '$language'" \
"request is '$request'" \
"os is '$os'" \
"separator is '$separator'"
done
And if you need the "machine" value only, then it is way easier:
cat test.txt | grep '^machine:' | cut -d ':' -f 2 | while read machinevalue; do
# call your other command here...
echo "machine value is '$machinevalue'"
done
A word of caution: If your values contain the character ":" this script would break and then you would have to use sed 's/^machine://g' instead of cut -d ':' -f 2.
A possible optimization would be to use bash for extracting the parts of the string but I am too lazy for that and unless I need the performance, I prefer using shell commands because I remember them more easily.
Regarding I was wondering if this could be done using only sed and awk - no, it can't because the task requires a shell to call ps so any sed or awk script would need to spawn a subshell to call ps, they can't call it on their own. So if you tried to do that then in terms of calls you'd end up with something like shell { awk { system { subshell { ps } } } } (which clearly isn't only using awk anyway) instead of simply shell { ps }.
Using md5sum (a very common application for this technique) for the example instead of ps -ef which would produce different output on everyone's different machines, you can tweak it to use ps -ef later, you COULD do the following (but don't, see the 2nd script below for a better approach):
$ cat tst.sh
#!/usr/bin/env bash
infile="$1"
while IFS= read -r line; do
if [[ "$line" =~ ^([^:]+):(.*) ]]; then
tag="${BASH_REMATCH[1]}"
val="${BASH_REMATCH[2]}"
case "$tag" in
machine )
mac="$val"
;;
request )
val="$(printf '%s' "$mac" | md5sum | cut -d' ' -f1)"
;;
esac
line="${tag}:${val}"
fi
printf '%s\n' "$line"
done < "$infile"
$ ./tst.sh file
line1
line1
-----
machine:chrome
purpose:
language:v2
request:554838a8451ac36cb977e719e9d6623c
additional: v4
os:v4
-----
machine:firefox
purpose:
language:v2
request:d6a5c9544eca9b5ce2266d1c34a93222
os:v4
-----
machine:helper
purpose:
language:v2
request:fde5d67bfb6dc4b598291cc2ce35ee4a
os:v4
-----
another line
While the above would work it'd be very inefficient (see why-is-using-a-shell-loop-to-process-text-considered-bad-practice) since it's looping through every line of input using shell so the following is how I'd really approach a task like this as it's far more efficient since it only has shell loop through each of the machine: lines from the input (which is unavoidable and is a far smaller number of iterations than if it had to read every input line) and the rest is done with a single call to sed to generate the input for the shell loop and a single call to awk to produce the output:
$ cat tst.sh
#!/usr/bin/env bash
infile="$1"
sed -n 's/^machine://p' "$infile" |
while IFS= read -r mac; do
printf '%s\n%s\n' "$mac" "$(printf '%s' "$mac" | md5sum | cut -d' ' -f1)"
done |
awk '
NR==FNR {
if ( NR % 2 ) {
mac = $0
}
else {
map[mac] = $0
}
next
}
{
tag = val = $0
sub(/:.*/,"",tag)
sub(/[^:]*:/,"",val)
}
tag == "machine" { mac = val }
tag == "request" { $0 = tag ":" map[mac] }
{ print }
' - "$infile"
$ ./tst.sh file
line1
line1
-----
machine:chrome
purpose:
language:v2
request:554838a8451ac36cb977e719e9d6623c
additional: v4
os:v4
-----
machine:firefox
purpose:
language:v2
request:d6a5c9544eca9b5ce2266d1c34a93222
os:v4
-----
machine:helper
purpose:
language:v2
request:fde5d67bfb6dc4b598291cc2ce35ee4a
os:v4
-----
another line
Here's what each of the above steps does:
Get just the machine: lines and remove the machine: part so we can have shell loop through just the parts it needs to call some command (e.g. ps -ef or md5sum) on:
$ sed -n 's/^machine://p' "$infile"
chrome
firefox
helper
Loop through each of those lines producing a mapping from that word to the output of the shell command you need to run on it (we generate the mapping in pairs of lines so the subsequent awk can parse it robustly even if the machine name from the input contained :s):
$ sed -n 's/^machine://p' "$infile" |
while IFS= read -r mac; do
printf '%s\n%s\n' "$mac" "$(printf '%s' "$mac" | md5sum | cut -d' ' -f1)"
done
chrome
554838a8451ac36cb977e719e9d6623c
firefox
d6a5c9544eca9b5ce2266d1c34a93222
helper
fde5d67bfb6dc4b598291cc2ce35ee4a
Pass that mapping to awk which separates it into the part before the : (which I'm calling a tag) and the part after it (which I'm calling a value) and stores the mapping in an array:
NR==FNR {
if ( NR % 2 ) {
mac = $0
}
else {
map[mac] = $0
}
next
}
It now reads the input file again, then using that array to modify the request: lines before printing each line (populating tag and val this way instead of setting FS to : and using $1 and $2 so we can again handle any input that contains :s in other locations):
{
tag = val = $0
sub(/:.*/,"",tag)
sub(/[^:]*:/,"",val)
}
tag == "machine" { mac = val }
tag == "request" { $0 = tag ":" map[mac] }
{ print }
The above assumes the shell command output is a single line each time it's called.
In pure bash:
#!/bin/bash
while IFS= read -r line; do
if [[ $line = machine:* ]]; then mach=${line#*:}
elif [[ $line = os:* ]]; then mach=""; fi
if [[ $line = request:* && $mach ]]; then
printf 'request:'
your_command "$mach"
else
printf '%s\n' "$line"
fi
done < file
If the output of your command doesn't end with a newline character, then place an echo after your command.
I've written the bash script (searchuser) which should display all the users who are executing a specific program or a script (at least a bash script). But when searching for scripts fails because the command the SO is executing is something like bash scriptname.
This script acts parsing the ps command output, it search for all the occurrences of the specified program name, extracts the user and the program name, verifies if the program name is that we're searching for and if it's it displays the relevant information (in this case the user name and the program name, might be better to output also the PID, but that is quite simple). The verification is accomplished to reject all lines containing program names which contain the name of the program but they're not the program we are searching for; if we're searching gedit we don't desire to find sgedit or gedits.
Other issues I've are:
I would like to avoid the use of a tmp file.
I would like to be not tied to GNU extensions.
The script has to be executed as:
root# searchuser programname <invio>
The script searchuser is the following:
#!/bin/bash
i=0
search=$1
tmp=`mktemp`
ps -aux | tr -s ' ' | grep "$search" > $tmp
while read fileline
do
user=`echo "$fileline" | cut -f1 -d' '`
prg=`echo "$fileline" | cut -f11 -d' '`
prg=`basename "$prg"`
if [ "$prg" = "$search" ]; then
echo "$user - $prg"
i=`expr $i + 1`
fi
done < $tmp
if [ $i = 0 ]; then
echo "No users are executing $search"
fi
rm $tmp
exit $i
Have you suggestion about to solve these issues?
One approach might looks like such:
IFS=$'\n' read -r -d '' -a pids < <(pgrep -x -- "$1"; printf '\0')
if (( ! ${#pids[#]} )); then
echo "No users are executing $1"
fi
for pid in "${pids[#]}"; do
# build a more accurate command line than the one ps emits
args=( )
while IFS= read -r -d '' arg; do
args+=( "$arg" )
done </proc/"$pid"/cmdline
(( ${#args[#]} )) || continue # exited while we were running
printf -v cmdline_str '%q ' "${args[#]}"
user=$(stat --format=%U /proc/"$pid") || continue # exited while we were running
printf '%q - %s\n' "$user" "${cmdline_str% }"
done
Unlike the output from ps, which doesn't distinguish between ./command "some argument" and ./command "some" "argument", this will emit output which correctly shows the arguments run by each user, with quoting which will re-run the given command correctly.
What about:
ps -e -o user,comm | egrep "^[^ ]+ +$1$" | cut -d' ' -f1 | sort -u
* Addendum *
This statement:
ps -e -o user,pid,comm | egrep "^\s*\S+\s+\S+\s*$1$" | while read a b; do echo $a; done | sort | uniq -c
or this one:
ps -e -o user,pid,comm | egrep "^\s*\S+\s+\S+\s*sleep$" | xargs -L1 echo | cut -d ' ' -f1 | sort | uniq -c
shows the number of process instances by user.
I have following output of linux command brctl show. I would like to extract all the associated interfaces (as given in the interface column) and store it in some array. Can somebody suggest a way to achieve this?
root#XXXX:~# brctl show
bridge name bridge id STP enabled interfaces
br-lan 7fff.00c0ca7e0288 no eth0
wlan1_1
wlan1_1.sta1
I am missing the exact formatting for some reason. But the output of brctl show looks pretty much like above with the exception of a few spacings.
So I would like to store eth0,wlan1_1, w;an1_1.sta1 in some array if possible.
Thanks
You can use cut and tail. You may have to adjust the number:
brctl show | tail -n +2 | cut -c 36-
If the tab characters are converted to spaces in your output, you will need:
brctl show | tail -n +2 | cut -c 46-
Bash script solution
Note: I tested with a datafile of your data, but it should work with the brctl show output as well:
#!/bin/bash
declare -i cnt=0
declare -a ifaces
while read -r line || [ -n "$line" ]; do
[ $cnt -eq 0 ] && { ((cnt++)); continue; }
ifaces+=( "${line//* /}" )
done <<<$(brctl show)
printf "%s\n" ${ifaces[#]}
tested with done <"$1" as substitute:
$ bash brctl.sh dat/brctl.txt
eth0
wlan1_1
wlan1_1.sta1
Shell script solution, more compatible and with less failure possibilities:
#!/bin/sh
BridgeMembers ()
{
local TheBridge="$1"
local CurrentLine=""
local CurrentMac=""
local CurrentNIC=""
IFS="$(printf "\n\b")" ; for CurrentLine in $(brctl showmacs "$TheBridge" | tail -n +2) ; do unset IFS
IsLocal="$(OneWord () { echo $3; }; OneWord $CurrentLine)"
if [ "$IsLocal" = "yes" ] ; then
CurrentMac="$(OneWord () { echo $2; }; OneWord $CurrentLine)"
CurrentNic="$(env LANG=en ifconfig | grep -e $CurrentMac)"
CurrentNic="$(OneWord () { echo $1; }; OneWord $CurrentNic)"
echo "$CurrentNic"
fi
done
}
BridgeMembers "$1"
Note that with "brctl show" you didn't get same column widths all times.
I am developing a script on a Solaris 10 SPARC machine to calculate how many patches got installed successfully during a patch delivery. I would like to display to the user:
(X) of 33 patches were successfully installed
I would like my script to output dynamically replacing the "X" so the user knows there is activity occurring; sort of like a counter. I am able to show counts, but only on a new line. How can I make the brackets update dynamically as the script performs its checks? Don't worry about the "pass/fail" ... I am mainly concerned with making my output update in the bracket.
for x in `cat ${PATCHLIST}`
do
if ( showrev -p $x | grep $x > /dev/null 2>&1 ); then
touch /tmp/patchcheck/* | echo "pass" >> /tmp/patchcheck/$x
wc /tmp/patchcheck/* | tail -1 | awk '{print $1}'
else
touch /tmp/patchcheck/* | echo "fail" >> /tmp/patchcheck/$x
wc /tmp/patchcheck/* | tail -1 | awk '{print $1}'
fi
done
The usual way to do that is to emit a \r carriage return (CR) at some point and to omit the \n newline or line feed (LF) at the end of the line. Since you're using awk, you can try:
awk '{printf "\r%s", $1} END {print ""}'
For most lines, it outputs a carriage return and the data in field 1 (without a newline at the end). At the end of the input, it prints an empty string followed by a newline.
One other possibility is that you should place the awk script outside your for loop:
for x in `cat ${PATCHLIST}`
do
if ( showrev -p $x | grep $x > /dev/null 2>&1 ); then
touch /tmp/patchcheck/* | echo "pass" >> /tmp/patchcheck/$x
wc /tmp/patchcheck/* | tail -1
else
touch /tmp/patchcheck/* | echo "fail" >> /tmp/patchcheck/$x
wc /tmp/patchcheck/* | tail -1
fi
done | awk '{ printf "\r%s", $1} END { print "" }'
I'm not sure but I think you can apply similar streamlining to the rest of the repetitious code in the script:
for x in `cat ${PATCHLIST}`
do
if showrev -p $x | grep -s $x
then echo "pass"
else echo "fail"
fi >> /tmp/patchcheck/$x
wc /tmp/patchcheck/* | tail -1
done | awk '{ printf "\r%s", $1} END { print "" }'
This eliminates the touch (which doesn't seem to do much), and especially not when the empty output of touch is piped to echo which ignores its standard input. It eliminates the sub-shell in the if line; it uses the -s option of grep to keep it quiet.
I'm still a bit dubious about the wc line. I think you're looking to count the number of files, in effect, since each file should contain one line (pass or fail), unless you listed some patch twice in the file identified by ${PATCHLIST}. In which case, I'd probably use:
for x in `cat ${PATCHLIST}`
do
if showrev -p $x | grep -s $x
then echo "pass"
else echo "fail"
fi >> /tmp/patchcheck/$x
ls /tmp/patchcheck | wc -l
done | awk '{ printf "\r%s", $1} END { print "" }'
This lists the files in /tmp/patchcheck and counts the number of lines output. It means you could simply print $0 in the awk script since $0 and $1 are the same. To the extent efficiency matters (not a lot), this is more efficient because ls only scans a directory, rather than having wc open each file. But it is more particularly a more accurate description of what you are trying to do. If you later want to count the passes, you can use:
for x in `cat ${PATCHLIST}`
do
if showrev -p $x | grep -s $x
then echo "pass"
else echo "fail"
fi >> /tmp/patchcheck/$x
grep '^pass$' /tmp/patchcheck/* | wc -l
done | awk '{ printf "\r%s", $1} END { print "" }'
Of course, this goes back to reading each file, but you're getting more refined information out of it now (and that's the penalty for the more refined information).
Here is how I got my patch installation script working the way I wanted:
while read pkgline
do
patchadd -d ${pkgline} >> /var/log/patch_install.log 2>&1
# Create audit file for progress indicator
for x in ${pkgline}
do
if ( showrev -p ${x} | grep -i ${x} > /dev/null 2>&1 ); then
echo "${x}" >> /tmp/pass
else
echo "${x}" >> /tmp/fail
fi
done
# Progress indicator
for y in `wc -l /tmp/pass | awk '{print $1}'`
do
printf "\r${y} out of `wc -l /patchdir/master | awk '{print $1}'` packages installed for `hostname`. Last patch installed: (${pkgline})"
done
done < /patchdir/master
new to bash scripting so just wondering if i am doing this code right at all. im trying to search /etc/passwd and then grep and print users.
usage ()
{
echo "usage: ./file.sk user"
}
# test if we have two arguments on the command line
if [ $# != 1 ]
then
usage
exit
fi
if [[ $# < 0 ]];then
usage
exit
fi
# Search for user
fullname=`grep $1 /etc/passwd | cut -f 5 -d :`
firstname=`grep $1 /etc/passwd | cut -f 5 -d : | cut -f 1 -d " "`
#check if there. if name is founf: print msg and line entry
not sure as how to this or if im doing this right...
am i doing this right?
grep $1 /etc/passwd | while IFS=: read -r username passwd uid gid info home shell
do
echo $username: $info
done
This might work for you:
fullname=$(awk -F: '/'$1'/{print $5}' /etc/passwd)
firstname=${fullname/ *}
You're on the right track.
But I think the 2nd if [[ $# < 0 ]] .... fi block doesn't get you much. Your first test case gets the situation right, 'This script requires 1 argument or quits'.
Also, I don't see what you need firstname for, so a basic test is
case "${fullname:--1}" in
-[1] ) printf "No userID found for input=$1\n" ; exit 1 ;;
* )
# assume it is OK
# do what every you want after this case block
;;
esac
You can of course, duplicate this using "${firstname}" if you really need the check.
OR as an equivalent if ... fi is
if [[ "${fullname}" == "" ]] ; then
printf "No userID found for input=$1\n" ; exit 1
fi
note to be more efficient, you can parse ${fullname} to get firstname without all the calls to grep etc, i.e.
firstname=${fullname%% *}
Let me know if you need for me to explain :--1} and %% *} variable modifiers.
I hope this helps.
Instead of this:
fullname=`grep $1 /etc/passwd | cut -f 5 -d :`
firstname=`grep $1 /etc/passwd | cut -f 5 -d : | cut -f 1 -d " "`
Try this:
fullname=$(cut -f5 -d: /etc/passwd | grep "$1")
if [[ $? -ne 0 ]]; then
# not found, do something
fi
firstname=${fullname%% *} # remove the space and everything after
Note that I changed my answer to cut before grep so that it doesn't get false positives if some other field matches the full name you are searching for.
You can simply by reading your input to an array and then printing out your desired fields, something like this -
grep $1 /etc/passwd | while IFS=: read -a arry; do
echo ${arry[0]}:${arry[4]};
done
Test:
jaypal:~/Temp] echo "root:*:0:0:System Administrator:/var/root:/bin/sh" |
while IFS=: read -a arry; do
echo ${arry[0]}:${arry[4]};
done
root:System Administrator