I have a file full of hostnames - one in each line.
Now I'd like to check if these hostnames exist (and eventually delete them from the file if not).
I already fail at the first part:
#!/bin/bash
while read host; do
ping -c1 "$host"
done <hosts
Only gives me
ping: unknown host google.com
(put google.com in the file for testing)
I also tried removing the quotes - no effect.
However when running the command from a terminal that's what I get:
$ ping -c1 "google.com"
PING google.com (173.194.112.100) 56(84) bytes of data.
...
What's the issue here?
Most likely your hosts file is in DOS line endings format (CR-LF line endings), so read fills variable with google.com\r.
Simplest way would be to convert file to UNIX line endings with dos2unix hosts.
I was trying to do the same thing and I ended up with this script which you may find nice and useful. Still working on the part to try and recognize the "ping: unknown host hostname.co.hk" but that's okay.
#!/bin/bash
cat list | while read domains; do
reply=$(ping -c 1 $domains| grep -E '(from)' | wc -l)
if [ $reply == 1 ]; then
DOMAIN=$(ping -c 1 $domains |grep 64 |awk '{print$4}')
IP=$(ping -c 1 $domains | grep -E -o "([0-9]{1,3}[\.]){3}[0-9]{1,3}" | head -1)
echo "$domains -> $DOMAIN -> $IP" >> ping.log;echo "$domains -> $DOMAIN -> $IP"
else
echo "-----------------ping failed, verify domain name"
fi
done
cat ping.log
Related
When I run the following command in a terminal it works, but not from a script:
eval $(printf "ssh foo -f -N "; \
for port in $(cat ~/bar.json | grep '_port' | grep -o '[0-9]\+'); do \
printf "-L $port:127.0.0.1:$port ";\
done)
The error I get tells me that printf usage is wrong, as if the -L argument within quotes would've been an argument to printf itself.
I was wondering why that is the case. Am I missing something obvious?
__
Context (in case my issue is an XY problem): I want to start and connect to a jupyter kernel running on a remote computer. To do so I wrote a small script that
sends a command per ssh for the remote to start the kernel
copies via scp a configuration file that I can use to connect to the kernel from my local computer
reads the configuration file and opens appropriate ssh tunnels between local and remote
For those not familiar with jupyter, a configuration file (bar.json) looks more or less like the following:
{
"shell_port": 35932,
"iopub_port": 37145,
"stdin_port": 42704,
"control_port": 39329,
"hb_port": 39253,
"ip": "127.0.0.1",
"key": "4cd3e12f-321bcb113c204eca3a0723d9",
"transport": "tcp",
"signature_scheme": "hmac-sha256",
"kernel_name": ""
}
And so, in my command above, the printf statement creates an ssh command with all the 5 -L port forwarding for my local computer to connect to the remote, and eval should run that command. Here's the full script:
#!/usr/bin/env bash
# Tell remote to start a jupyter kernel.
ssh foo -t 'python -m ipykernel_launcher -f ~/bar.json' &
# Wait a bit for the remote kernel to launch and write conf. file
sleep 5
# Copy the conf. file from remote to local.
scp foo:~/bar.json ~/bar.json
# Parse the conf. file and open ssh tunnels.
eval $(printf "ssh foo -f -N "; \
for port in $(cat ~/bar.json | grep '_port' | grep -o '[0-9]\+'); do \
printf "-L $port:127.0.0.1:$port ";\
done)
Finally, jupyter console --existing ~/foo.json connects to remote.
As #that other guy says, bash's printf builtin barfs on printf "-L ...". It thinks you're passing it a -L option. You can fix it by adding --:
printf -- "-L $port:127.0.0.1:$port "
Let's make that:
printf -- '-L %s:127.0.0.1:%s ' "$port" "$port"
But since we're here, we can do a lot better. First, let's not process JSON with basic shell tools. We don't want to rely on it being formatting a certain way. We can use jq, a lightweight and flexible command-line JSON processor.
$ jq -r 'to_entries | map(select(.key | test(".*_port"))) | .[].value' bar.json
35932
37145
42704
39329
39253
Here we use to_entries to convert each field to a key-value pair. Then we select entries where the .key matches the regex .*_port. Finally we extract the corresponding .values.
We can get rid of eval by constructing the ssh command in an array. It's always good to avoid eval when possible.
#!/bin/bash
readarray -t ports < <(jq -r 'to_entries | map(select(.key | test(".*_port"))) | .[].value' bar.json)
ssh=(ssh foo -f -N)
for port in "${ports[#]}"; do ssh+=(-L "$port:127.0.0.1:$port"); done
"${ssh[#]}"
Hope this time it's not a duplicate. I didn't find anything.
My code:
#!/bin/bash
FILE=/home/user/srv.txt
TICKET=task
while read LINE; do
ssh -nT $LINE << 'EOF'
touch info.txt
hostname >> info.txt
ifconfig | grep inet | awk '$3 ~ "cast" {print $2}' >> info.txt
grep -i ^server /etc/zabbix/zabbix_agentd.conf >> info.txt
echo "- Done -" >> info.txt
EOF
ssh -nT $LINE "cat info.txt" >> $TICKET.txt
done < $FILE #End
My issue:
if I only use ssh $LINE it will only ssh to the host on the first line and also display an error Pseudo-terminal will not be allocated because stdin is not a terminal.
using ssh -T , fix the error message above and it will create the file info.txt
using ssh -nT , fix the error where ssh only read the first line but I get an error message cat: info.txt: No such file or directory. If I ssh to the hosts, I can confirm that there is no info.txt file in my home folder. and with ssh -T, I have this file in my home folder.
I tried with the option -t, also HERE, EOF without ' ... ' but no luck
Do I miss something?
Thanks for your help,
Iswaren
You have two problems.
If you invoke ssh without -n it may consume the $FILE input (it drains its stdin)
If you invoke ssh with -n it won't read its stdin, so none of the commands will be executed
However, the first ssh has had its input redirected to come from a heredoc, so it does not need -n.
As stated in the comments, the second ssh call is not needed. Rather than piping into info.txt and then copying that into a local file, just output to the local file directly:
while read LINE; do
ssh -T $LINE >>$TICKET.txt <<'EOF'
hostname
ifconfig | grep inet | awk '$3 ~ "cast" {print $2}'
grep -i ^server /etc/zabbix/zabbix_agentd.conf
echo "- Done -"
EOF
done <$FILE
Here is my script in which I use local variable inside a remote machine using heredoc. But the loop under the heredoc takes the first variable value only. The loop runs fine inside the heredoc but with the same values.
#!/bin/bash
prod_web=($(cat /tmp/webip.txt));
new_prod_app_private_ip=($(cat /tmp/ip.txt));
no_n=($(cat /tmp/serial.txt));
ssh -t -o StrictHostKeyChecking=no ubuntu#${prod_web[0]} -p 2345 -v << EOF
set -xv
for (( x = 0; x < '${#no_n[#]}'; x++ ))
do
sudo su
echo '${no_n[x]}'
echo '${new_prod_app_private_ip[x]}'
curl -fIkSs https://'${new_prod_app_private_ip[x]}':9002 | head -n 1
done
EOF
So, my ip.txt file contains values like:
10.0.1.0
10.0.2.0
10.0.3.0
My serial.txt file:
9
10
11
So, my loop runs for only the first IP (present in /tmp/ip.txt) in the remote machine, three times. I want to run it for all the three IPs. My remote ip is present in the file /tmp/webip.txt.
Got stuck for a long time, any help is appreciated. Is there any other solution that I can go with?
There are 2 environments. On your local machine and on the remote machine. You need to think how to transfer data/variables/state/objects/handles between these machines.
If you set something on your local machine (ie. prod_web=($(cat /tmp/webip.txt));) and then just ssh to remote host (ie. ssh user#host 'echo "${prod_web[#]}"'), the variable will not be visible/exported to the remote machine. You can:
scp the files {ip,serial}.txt and execute the whole script on the remote machine, then cleanup , ie. remove the {ip,serial}.txt files from the remote machine
pass the files {ip,serial}.txt somehow merged/joined/pasted to the stdin of the ssh and then read up stdin on the remove machine
create all the commands to run on your local machine and then pass pre-prepared commands to remote machine, like ssh .... "$(for ...; do; echo curl ...; done)"
I would go with the second option, as I like passing everything using pipes and don't like to cleanup after me - removing temporary files in case of error can be a mess.
My script would probably look like this:
#!/bin/bash
set -euo pipefail
read -r host _ <webip.txt
paste serial.txt ip.txt | ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu#"$host" '#!/bin/bash
set -euo pipefail
while read -r no_n ip; do
for ((i = 0; i < no_n; ++i)); do
printf "%s\n" "$no_n"
printf "%s\n" "$ip"
curl -fIkSs https://"$ip":9002 | head -n 1
done
done
'
As the remote script would become larger and less qouting friendly, I would save it into another remote_scripts.sh and execute ssh ... -m remote_scripts.sh.
I don't get what you are trying to do with that sudo su, which 100% does not do what you want.
If the no_n magic number is the number of times to execute that curl and you have xargs and you don't really care about errors, you can just do a magic and confusing oneliner:
#!/bin/bash
set -euo pipefail
read -r host _ <webip.txt
paste serial.txt ip.txt | ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu#"$host" 'xargs -n2 -- sh -c "seq 0 \"\$1\" | xargs -n1 -- sh -c \"curl -fIkSs https://\\\"\\\$1\\\":9002 | head -n 1\" -- \"\$2\"" --'
Preparing all the command to run maybe actually more readable and may save some nasty qouting to resolve. But this really depends on how big serial.txt and ip.txt are and how big are the commands to be executed on the remote machine, as you want to minimize the number of bytes transferred between machines.
Here the commands to run are constructed on local machine (ie. "$(...)" is passed to ssh) and executed on remote machine:
# semi-readable script, not as fast and no xargs
ssh -t -o StrictHostKeyChecking=no -p 2345 -v ubuntu#"$host" "$(paste serial.txt ip.txt | while read -r serial ip; do
seq 0 "$serial" | while read -r _; do
echo "curl -fIkSs \"https://$ip:9002\" | head -n 1"
done
done)"
HERE-doc does not expand shell commands, so:
$ cat <<EOF
> echo 1
> EOF
echo 1
but you can use command substitution $( ... ):
$ cat <<EOF
> $(echo 1)
> EOF
1
#!/bin/bash
ip route add 10.105.8.100 via 192.168.1.100
date
cat /home/xxx/Documents/list.txt | while read output
do
ping="ping -c 3 -w 3 -q 'output'"
if $ping | grep -E "min/avg/max/mdev" > /dev/null; then
echo 'connection is ok'
else
echo "router $output is down"
then
cat /home/xxx/Documents/roots.txt | while read outputs
do
cd /home/xxx/Documents/routers
php rebootRouter.php "outputs" admin admin
done
fi
done
The other documents are:
lists.txt
10.105.8.100
roots.txt
192.168.1.100
when i run the script, the result is a reboot of the router am trying to ping. It doesn't ping.
Is there a problem with the bash script.??
If your files only contain a single line, there's no need for the while-loop, just use read:
read -r router_addr < /home/xxx/Documents/list.txt
# the grep is unnecessary, the return-code of the ping will be non-zero if the host is down
if ping -c 3 -w 3 -q "$router_addr" &> /dev/null; then
echo "connection to $router_addr is ok"
else
echo "router $router_addr is down"
read -r outputs < /home/xxx/Documents/roots.txt
cd /home/xxx/Documents/routers
php rebootRouter.php "$outputs" admin admin
fi
If your files contain multiple lines, you should redirect the file from the right-side of the while-loop:
while read -r output; do
...
done < /foo/bar/baz
Also make sure your files contain a newline at the end, or use the following pattern in your while-loops:
while read -r output || [[ -n $output ]]; do
...
done < /foo/bar/baz
where || [[ -n $output ]] is true even if the file doesn't end in a newline.
Note that the way you're checking for your routers status is somewhat brittle as even a single missed ping will force it to reboot (for example the checking computer returns from a sleep-state just as the script is running, the ping fails as the network is still down but the admin script succeeds as the network just comes up at that time).
I have a text file with a lists of IP addresses called address.txt which contains the following
172.26.26.1 wlan01
172.26.27.65 wlan02
172.26.28.180 wlan03
I need to write a bash script that reads the only IP addresses, ping them and output to a another text file to something like this:
172.26.26.1 IS UP
172.26.27.65 IS DOWN
172.26.28.180 IS DOWN
I am fairly new to bash scripting so I am not sure where to start with this. Any help would be much appreciated.
In Linux this would work:
awk '{print $1}' < address.txt | while read ip; do ping -c1 $ip >/dev/null 2>&1 && echo $ip IS UP || echo $ip IS DOWN; done
I don't have a cygwin now to test, but it should work there too.
Explanation:
With awk we get the first column from the input file and pipe it into a loop
We send a single ping to $ip, and redirect standard output and standard error to /dev/null so it doesn't pollute our output
If ping is successful, the command after && is executed: echo $ip IS UP
If ping fails, the command after || is executed: echo $ip IS DOWN
Somewhat more readable, expanded format, to put in a script:
#!/bin/sh
awk '{print $1}' < address.txt | while read ip; do
if ping -c1 $ip >/dev/null 2>&1; then
echo $ip IS UP
else
echo $ip IS DOWN
fi
done