I am using this script to check a list of ips I own to see if they are on the spam block list.
auto.sh:
while read ip ; do
./blacklist.sh $ip
done < block.txt
blacklist.sh is the above linked script.
block.txt lists each of my ips one line at a time (I have several /22).
A typical output of a blocked ip scan looks like this:
Warning: PTR lookup failed
b.barracudacentral.org : 127.0.0.2
bb.barracudacentral.org : 127.0.0.2
black.junkemailfilter.com : 127.0.0.2
cbl.abuseat.org : 127.0.0.2
cidr.bl.mcafee.com : 127.0.0.4
dnsbl.justspam.org : 127.0.0.2
hostkarma.junkemailfilter.com : 127.0.0.2
----------------------------------------------------------
Results for <my ip>
Tested: 117
Passed: 110
Invalid: 0
Blacklisted: 7
----------------------------------------------------------
what I want to do is have the script spit out output to a file when the text above doesn't say "Blacklisted: 0".
I am not sure how to approach this, will this work?
sudo ./auto.sh "conditions where Blacklisted: is > 0" >> 12.txt
Thanks for any help
Put the output in a temporary file and
then check its content:
./auto.sh > 12_temp.txt
grep -q 'Blacklisted:[ \t]*0$' 12_temp.txt || cat 12_temp.txt >> 12.txt
rm -f 12_temp.txt
Related
I am running a for loop around 3000 volumes within a ssh connection on a storage Server where this runs in a loop one by one, whereas i want the command vol show-footprint "$vols" -fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1 to run parallel over multiple volumes at a time, lets say run it at 10 volumes in a go.
Here myTotalVol contains all 3k volume names like below:
myvol001
myvol002
myvol003
myvol004
myvol005
Below is the small for loop which is working.
for vols in $(cat myTotalVol);
do
echo -n "$vols " ;\
ssh storageServer01 "row 0; set -unit MB; \
vol show-footprint "$vols" -fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1"; \
done
Please suggest if I can run the mentioned command over multiple volumes at a time which are kept in myTotalVol file.
Edit:
As asked by Mark Setchell in the comment section, hence below is just a view how its working ...
$ ssh store01
Last login time: 6/30/2022 10:49:41
store01::> row 0;set -unit MB
(rows)
store01::> vol show-footprint myvol001 -fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1
vserver volume volume-blocks-footprint volume-blocks-footprint-bin0 volume-blocks-footprint-bin1
-------- -------------------- ----------------------- ---------------------------- ----------------------------
myvol001 myvol00198703MB 48272MB 51988MB
as you see the command vol show-footprint myvol001 -fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1 here, i have to run this command over 3000 Volumes like i have myvol001 in this command so, myvol001 will go into the variable like i am using into the for loop and there i am using "$vols" which are referring to 3k vols from a file.
I'm not sure exactly what you are driving at, but you should be able to make a compound statement that generates the commands you want and then send that via ssh like this:
{ printf "row 0; set -unit MB;\n"
while read -r vol ; do
printf "vol show-footprint ${vol} -fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1\n"
done < myTotalVol } | ssh store01
You can try it out and see what it produces if you put a comment character before the | like this:
{ ...
...
done < myTotalVol } # | ssh store01
Or you can do it with awk:
awk 'BEGIN{print "row 0; set -unit MB"} {print "vol show-footprint", $1, "-fields volume-blocks-footprint,volume-blocks-footprint-bin0,volume-blocks-footprint-bin1"}' myTotalVol | ssh store01
Again, put # in front of | ssh store01 in order to see and check the output without sending it to ssh.
I have a simple bash script as follows that is part of a docker image.
test.sh,
#!/bin/bash
set -e
logit() {
log_date=`date +"%F %T"`
echo "[$log_date][INFO] $1"
}
waitForServerToStart() {
while true; do
logit "Testing .... 1"
netstat -anpt
logit "Testing .... 2"
netstat -anpt | grep tcp
logit "Testing .... 3"
sleep 5
logit "Testing .... 4"
done
}
waitForServerToStart
run.sh,
#!/bin/sh
/test.sh &
# Run forever
while true; do sleep 5; done
Dockerfile,
FROM openjdk:8u191-jre-alpine3.9
COPY files/run.sh /
COPY files/test.sh /
CMD ["/run.sh"]
If I run this container I only get the following output which leads me to believe somehow grep and "pipe" seem to get blocked.
[2019-03-06 11:10:45][INFO] Testing .... 1
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 172.17.0.2:58278 xxx.xxx.xx.xx:443 FIN_WAIT2 -
[2019-03-06 11:10:45][INFO] Testing .... 2
Can someone please shed some light around this ?
It works fine If I comment out netstat -anpt | grep tcp. I would then see the subsequent log lines and it would also continue in the loop.
[2019-03-06 11:25:36][INFO] Testing .... 3
[2019-03-06 11:25:41][INFO] Testing .... 4
This one has me puzzled! But I have a solution for you:
Use awk instead of grep
In test.sh use this instead:
netstat -anpt | awk /tcp/
So that the file looks like this:
#!/bin/bash
set -e
logit() {
log_date=`date +"%F %T"`
echo "[$log_date][INFO] $1"
}
waitForServerToStart() {
while true; do
logit "Testing .... 1"
netstat -anpt
logit "Testing .... 2"
netstat -anpt | awk /tcp/
logit "Testing .... 3"
sleep 5
logit "Testing .... 4"
done
}
waitForServerToStart
For a reason that I cannot explain - grep will not return when reading from the pipe when invoked from the script. I created your container locally, ran it and entered it - and the command netstat -anpt | grep tcp runs just fine and exits. If you replace it with netstat -anpt | cat in your test.sh script, then it will also pass just fine.
I looked all over the place for the someone with an identical issue with grep in a container from the distro you are using, the version etc. - but came up empty handed.
I believe that it may have to do with grep waiting for a EOF character that never lands - but I am not sure.
I'm writing a script that will check/open ports/protocols in the event any are blocked. What I have so far is below. The port/protocol names look strange to me. I would have expected IP addresses, but I've never done this before. Would the host be IP address of the DSLAM? Also, can I run nc without specifying host if it's the current machine? Otherwise, does this script do what is needed?
#!/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
echo -e "############################nnnPresent ports opened on this machine are
$(iptables -nL INPUT | grep ACCEPT | grep dpt)
nCompleted listing...nnn#########################"
#these look funny to me
PORTS=( 123 161 69 "UDP" 80 443 22 8443 8080 23 25 3307 "TCP" "HTTPS" "SNMP" "SFTP" "TFTP")
#modified ip's for public sharing
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
for PORT in "${PORTS[#]}"
do
#see which ones need opening...0 is pass (open), 1 fail, 5 timeout; need host still
#alternatively try nmap
nc -z -v -w5 ${HOST} ${PORT}
#if it's not open, then open it
if [ "$?" ne 0 ]; then #shellcheck err this line: Couldn't parse this test expression.
iptables -A INPUT -m tcp -p tcp --dport "$PORT" -j ACCEPT &&
{ service iptables save;
service iptables restart;
echo -e "Ports opened through iptables are n$(iptables -nL INPUT | grep ACCEPT | grep dpt)"; }
else
echo "Port $PORT already open"
fi
done
done
I've been referring to test if port is open, and also open port.
These lines seem odd, OP edit #6 adds an outer for loop which assigns the same value to $HOST on each go-round:
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
< stuff ... >
done
Assuming running < stuff ... > four times is not necessary, then
the seven lines above, as written, would be equivalent to:
HOST="10.x.x.x"
< stuff ... >
(Fixed.) Remove the commas from this line:
PORTS=( 123, 161, 69, UDP, 80, 443, 22, 8443, 8080, 23, 25,
3307, TCP, HTTPS, SNMP, SFTP, TFTP)
bash does not use commas to define arrays, and if commas are used
they become chars in the the array data. Example, given the array
exactly as it is above:
echo ${PORTS[0]}
Outputs:
123,
I want to create a script that gathers information about the ec2 instance ( id, ip, os, users mb other if needed ), but i need help with getting info about running system - i think it easy to get OS info from /etc/os-release ? And the second question about yaml - is it possible parse output to data.txt as yaml ?
Please help me add OS info to data.txt :)
#!/bin/bash
URL="http://169.254.169.254/latest/meta-data/"
which curl > /dev/null 2>&1
if [ $? == 0 ]; then
get_cmd="curl -s"
else
get_cmd="wget -q -O -"
fi
get () {
$get_cmd $URL/$1
}
data_items=(instance-id
local-ipv4
public-ipv4
)
yaml=""
for meta_thing in ${data_items[*]}; do
data=$(get $meta_thing)
entry=$(printf "%-30s%s" "$meta_thing:" "$data\n")
yaml="$yaml$entry"
done
echo -e "$yaml" > data.txt
maybe add
lsb_release -a >> data.txt
uname -a >> data.txt
To the end of the script
I have the following txt document:
deviceIDs.txt
UDID0=LGH811dec0bfd6
UDID1=41006289e4b2a179
UDID2=d9a7aa45
PORT0=4567
PORT1=4568
PORT2=4569
BOOTPORT0=5556
BOOTPORT1=5557
BOOTPORT2=5558
I want to be able to write the following bash script:
#!/bin/bash
source /path/deviceIDs.txt
for ((i=0;i<=2;i++))
do
echo $UDID$i
echo $PORT$i
echo $BOOTPORT$i
done
however this doesn't work. Presumable because the dollar sign is being used twice in one expression. I've tried formatting the variables using {} too but still no success.
You can use bash's form of indirection:
#!/bin/bash
source deviceIDs.txt
for ((i=0;i<=2;i++))
do
for x in UDID PORT BOOTPORT
do
y=$x$i
echo ${!y}
done
done
The key here is that ${!y} returns the value of the variable that is named by y.
This produces the output:
LGH811dec0bfd6
4567
5556
41006289e4b2a179
4568
5557
d9a7aa45
4569
5558
Alternative
If we are allowed to change the data file format, consider defining your variables as arrays this way:
$ cat IDarrays.txt
UDID=(LGH811dec0bfd6 41006289e4b2a179 d9a7aa45)
PORT=(4567 4568 4569)
BOOTPORT=(5556 5557 5558)
With that, the script can be written:
#!/bin/bash
source IDarrays.txt
for ((i=0;i<=2;i++))
do
echo ${UDID[$i]}
echo ${PORT[$i]}
echo ${BOOTPORT[$i]}
done
I didn't knew John1024's solution with !y.
If you can't modify deviceIDs.txt because it is generated by someone out of your control, you could use arrays like this too:
#!/bin/bash
source /path/deviceIDs.txt
u=($UDID{0..2});p=($PORT{0..2});b=($BOOTPORT{0..2})
for i in {0..2}; do echo -e ${u[i]}"\n"${p[i]}"\n"${b[i]}"\n"; done
LGH811dec0bfd6
4567
5556
41006289e4b2a179
4568
5557
d9a7aa45
4569
5558