I am attempting to grab a specific field from a curl request. It works perfectly fine via the shell, but as soon as I try to assign a variable it stops working.
This is the stand-alone command.
curl --silent 'http://alerts.weather.gov/cap/wwaatmget.php?x=MNC131&y=0' | grep -E ' <title>' | awk -F '[<>]' '{print $3}'
This is the small bit from the script. It echos a blank variable.
weatheralert=$(curl --silent 'http://alerts.weather.gov/cap/wwaatmget.php?x=MNC131&y=0' | grep -E ' <title>' | awk -F '[<>]' '{print $3}')
echo "Current Weather Alert: $weatheralert"
Note that there is a tab character exists before <title> tag not 3 or 4 whitespace. So copy paste a tab character or use grep -P '\t<title>' command.
$ weatheralert=$(curl --silent 'http://alerts.weather.gov/cap/wwaatmget.php?x=MNC131&y=0' | grep -P '\t<title>'|awk -F '[<>]' '{print $3}')
$ echo "Current Weather Alert: $weatheralert"
Current Weather Alert: There are no active watches, warnings or advisories
Related
I want to preface that I am a newbie that picked up shell scripting 2 weeks ago.
Hey guys I need help with something, hope someone can point me in the right direction. I have a script that works when I run it from the command line but every time I run it with a crontab, the output is a few empty files. Does anyone know why?
That's the code down there
#!/bin/bash
#Provide an IP address as an argument to use nmap
#make sure to add the full range with (0-225 or 0/24) at the end
IPADDRESS=$(hostname -I | awk '{print $1}')
network-scan(){
if [ $1 ]
then
sudo nmap -sn $1
else
sudo nmap -sn 192.168.1.0-255
fi
}
#Scan the whole network and only prints the IP addresses minus your own
#Sends the IP addresses to a file
network-scan | grep -i 'Nmap scan report' | \
sed 's/\ /\n/g'|sed 's/(//g'|sed 's/)//g' | \
grep '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' | grep -v ${IPADDRESS} > ip_addresses
#Scan the whole network and only prints the MAC addresses
#Sends the MAC addresses to a file
network-scan | grep -i 'MAC Address:' | \
awk '{print $3}' > mac_addresses
#Put the IP and MAC addresses in the same file
paste ip_addresses mac_addresses | \
column -s $'\t' -t > "scan_$(date +%d-%m-%Y_%H:%M:%S)"
#Notify that a file with the IP and MAC addresses has been created on the Desktop
echo "A file containing the results of the scan has been created on the Desktop"
exit 0
You are using
network-scan | grep
without passing any parameter.
Hence network-scan function always using
sudo nmap -sn 192.168.1.0-255
when you run it from command line are you passing any parameter ?
echo $IPADDRESS inside the script when executing at cron and at command line for debugging.
network-scan | grep -i 'Nmap scan report' | \
sed 's/\ /\n/g'|sed 's/(//g'|sed 's/)//g' | \
grep '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' | grep -v ${IPADDRESS}
Since you are obtaining empty output, validate each command and append(test) each OR operators to know where it is removing required output.
I want to see what countries are trying to access my VPS. I have installed a tool called "goiplookup", which was forked from another effort called "geoiplookup". If I type this at the command line:
goiplookup 8.8.8.8
It returns this:
US, United States
So I figured out how to get a list of IPs that are trying to access my server by using this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}'
Which gives a long list of IPs like this:
1.1.1.1
2.2.2.2
3.3.3.3
I cannot figure out how to get this list of IPs to be processed by the "goiplookup" tool. I tried this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | goiplookup
but that did not work. I also tried with no luck:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | xargs -0 goiplookup
Try this:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort | uniq | xargs -n 1 goiplookup
I added | sort | uniq to ensure each IP only appears once
and xargs -n 1 so that each found IP is processes by goiplookup
I would put it into a file and make a small utility to parse it:
sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort -u > ./file.txt
cat ./file.txt | while read -r line; do
temp$(echo $line)
goiplookup $temp
done
This will read through the file one line at a time and execute the goiplookup with each IP.
sudo grep disconnect /var/log/auth.log | awk '!/COMMAND/ && !seen[$0]++ {system("geoiplookup \""$9"\""}
Note that geoiplookup only allows one IP per invocation.
The whole thing can be done in awk, but using grep allows the rest to be run unprivileged.
Consider whether grep -w (match whole word) is appropriate, and in awk you can do a similar thing with !/(^|[^[:alnum:]_])COMMAND($|[^[:alnum:]_])/.
I just made a shell script, which works.
#!/bin/bash
readarray -t array < <(sudo grep "disconnect" /var/log/auth.log | grep -v COMMAND | awk '{print $9}' | sort | uniq)
for ip in "${array[#]}"
do
:
country=$(/usr/local/bin/goiplookup -c $ip)
echo "$ip $country"
done
I'm working on a script, that should find certain disks and add hostname to them.
I'm using this for 40 servers with a for loop in bash
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i findmnt -o SIZE,TARGET -n -l |
grep '1.8T\|1.6T\|1.7T' |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=$HOSTNAME '{print HOSTNAME ":" $0}'; done |
tee sorted.log
can you help out with the quoting here? It looks like awk gets piped (hostname) from localhost, not the remote server.
Everything after the first pipe is running locally, not on the remote server.
Try quoting the entire pipeline to have it run on the remote server:
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i "findmnt -o SIZE,TARGET -n -l |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=\$HOSTNAME '{print HOSTNAME \":\" \$0}'" ;
done | tee sorted.log
This is a shorter version of your stuff:
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '/M/{print HOSTNAME ":" $2}'
Applied to the above:
for i in myservers{1..40}
do ssh user#$i bash -c '
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '"'"'/M/{print HOSTNAME ":" $2}'"'"' '
done |
tee sorted.log
see: How to escape the single quote character in an ssh / remote bash command?
I trying to add text (predefined) between a sorted output and saved to a new file.
I'm using a curl command to gather my info.
$ curl --user XXX:1234!## "http://......"
Then using grep to find IP addresses and sorting so they only appear once.
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
I need to add <my_text_predefined> ([0-9]{1,3}[\.]){3}[0-9]{1,3} <my_text_predefined> between the regex ip address and then saved to a new file.
The script below only get my the ip address
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
123.12.0.12
123.56.98.76
$ curl --user some_user:password "http://...." | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u | sed 's/.*/<prefix> -s & <suffix>/'
So if we need print some text for each IP ... try xargs
for i in {1..100}; do echo $i; done | xargs -n1 echo "Values are:"
if based on IP you would need to take decision put in a loop
for file $(curl ...) do ...
and check $file or do something with it ...
#!/bin/bash
vm1_MAC=`virsh -c qemu:///system domiflist instance-00000009 -e | grep virbr0 | awk '{print $5}'`
vm2_MAC=`virsh -c qemu:///system domiflist instance-0000000d -e | grep -i virbr0 | awk -e '{print $5}'`
vm1_IP=`arp -e | grep $vm1_MAC | awk '{print $1}'`
vm2_IP=`arp -e | grep $vm2_MAC | awk '{print $1}'`
echo "VM1 IP Address: $vm1_IP"
echo "VM2 IP Address: $vm2_IP"
The shell script was meant to display the IP addresses of my two openstack instances but I am receiving grep command option error:
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
Usage: grep [OPTION]... PATTERN [FILE]...
Try 'grep --help' for more information.
VM1 IP Address:
VM2 IP Address:
Is there anyone here who can assist me as I am not a bash script expert, just need to do this to get some tasks done. Thank you
This message typically happens when you try to grep for a string which starts with a dash.
The immediate workaround is to use grep -e "$variable" but really, you want to avoid the useless use of grep here.
#!/bin/bash
vm1_MAC=$(virsh -c qemu:///system domiflist instance-00000009 -e | awk "/virbr0/"'{print $5}')
vm2_MAC=$(virsh -c qemu:///system domiflist instance-0000000d -e | awk -e 'tolower($0) ~ /vibr0/ {print $5}')
vm1_IP=$(arp -e | awk -v mac="$vm1_MAC" '$0 ~ mac {print $1}')
vm2_IP=$(arp -e | awk -v mac="$vm2_MAC" '$0 ~ mac {print $1}')
Incidentally, this also demonstrates three different ways to pass a regex to Awk. Notice as well how we prefer the modern $(command) substitution over the dinosaur `backtick` syntax.