insert hosts in check via for loop - bash

following problem:
we have a file called "file.conf"
192.168.30.1|192.168.30.1|os
_gateway|192.168.30.2|Linux 2.6.18 - 2.6.22
...
first is the hostname second is the ipv4
now we have a script where he should automaticaly insert the hosts and ip's via automated user from checkMK
#!/bin/bash
FILE=filename
source $FILE
for i in ${FILE}
do
HOSTNAME=$(cat $i | cut -d '|' -f1)
IP=$(cat $i | cut -d '|' -f2)
curl "http://checkmkadress/check_mk/host&user" -d 'request={"hostname":"'"$HOSTNAME"'","folder":"ansible","attributes":{"ipaddress":"'"$IP"'","site":"sitename","tag_agent":"cmk-agent"}}'
done
but if we do it like that we get the following error cause he try's to put in every host in host and every ip in ip without going through all lines
{"result": "Check_MK exception: Failed to parse JSON request: '{\"hostname\":\"allhostnames":{\"ipaddress\":all_ips\",\"site\":\"sitename\",\"tag_agent\":\"cmk-agent\"}}': Invalid control character at: line 1 column 26 (char 25)", "result_code": 1}
how can we make the curl script go through each line to get host and ip individually

Using what you already done, try in this way.
FILE="filename"
source $FILE
while read line
do
HOSTNAME=$(echo $line | cut -d '|' -f1)
IP=$(echo $line | cut -d '|' -f2)
#your curl command here
done <$FILE
OR, which I prefer
while IFS='|' read -r host ip description
do
#your curl command here
echo "$host : $ip : $description"
done <$FILE

Related

need help in grep (BASH)

I try to make a tool to get list of proxies and You have downloaded Index for one of the free proxy sites I use this :
wget http://free-proxy.cz/en/proxylist/country/all/https/ping/all
and outputs something like that :
<script type="text/javascript">document.write(Base64.decode("MTg1LjExNC4xMzcuMTQ="))</script></td><td style=""><span class="fport" style=''>12195</span></td><td><small>HTTPS</small></td><td class="left"><div style="padding-left:2px"><img src="/flags/blank.gif" class="flag flag-ua" alt="Ukraine" /> Ukraine</div></td><td class="small"><small></small></td><td class="small"><small></small></td><td class="small"><small>High anonymity</small></td><td> <i class="icon-black icon-question-sign"></i></td><td><small>2.4%</small><div class="progress"><div class="fill" style="width:4%;background-color:red;"></div></div></td><td><div style="padding-left:5px"><small>649 ms</small> <div class="progress"><div class="fill" style="width:94%;background-color:#A5DA74;;"></div></div></div></td><td><small>8 hours ago</small></td></tr><tr><td style="text-align:center" class="left"><script type="text/javascript">document.write(Base64.decode("MTYxLjk3LjEzOC4yMzg="))</script></td><td style=""><span class="fport" style=''>3128</span></td><td>
As you can see the IP is encrypted in base64 and port is normal
I try to grep base64 codes first and this is work ↓
echo (outputs) | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2
and I try this code to get ports ↓
echo (output) | grep -Eo "(class=\"fport\" style=''>[0-9]{1,9})" | cut -d '>' -f2
how can I mix it and make it like that
(base64 code):(port)
and after that I wanna decrypt the base64 code and make it look like :
IP:PORT
1st step
base64 is not an encryption, but an encoding. If you are working on
Linux or other Unix-variants, the command base64, base64-encoder/decoder,
will be pre-installed. If not, it will be easily installed with your
OS-dependent package manager.
Then please try to execute:
base64 -d <<< "MTg1LjExNC4xMzcuMTQ="
It will output:
185.114.137.14
2nd step
Then we can combine the base64 decoder with your command pipeline.
The problem is base64-coding ignores newlines and we need to process
the result of the pipeline line by line. Assuming the variable $output
holds the output of the wget command, please try:
while IFS= read -r line; do
base64 -d <<< "$line"
echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)
It will print something like:
185.114.137.14
161.97.138.238
The <(command) notation is a process substitution and the output of
echo .. grep .. cut pipeline is fed to the while loop via stdin
and the while loop processes the base64-encoded strings line by line.
3rd step
Now we want to merge the IPs and PORTs in a format of IP:PORT.
We can make use of paste command. The final script will be:
paste -d ":" <(
while IFS= read -r line; do
base64 -d <<< "$line"
echo
done < <(echo "$output" | grep -Eo '("[A-Za-z0-9]{12,30}[=]{0,2}")' | cut -d '"' -f2)
) \
<(echo "$output" | grep -Eo "(class=\"fport\" style=''>[0-9]{1,9})" | cut -d '>' -f2)
Output:
185.114.137.14:12195
161.97.138.238:3128
The paste command takes filenames as arguments. Here we make use
of process substitution again in a manner as: paste <(command1) <(command2)
which saves to create temporary files.

Whois search for public IP on a csv file using bash script

I have a daily report in the csv file format of a list of public IP address and I need to to fill in the Hostname for the public IP. The Hostname can be an OrgId or netname.
I need to do a bash script to automate the whois search instead of searching manually one by one and filling it up on the csv file.
Example: This is an excerpt of a long list of Public IP address
Port,Type,S_Host,S_IP,Port,D_Host,D_IP,Port
2,tcp,N/A,8.8.8.8,2,N/A,47.246.57.232,8
3,tcp,N/A,47.246.57.232,2,N/A,217.17.81.9,3
I need to do a whois search on the IPs in column 4 and 7 then, fill in the Hostname inside field 3 and 6.
Desired output:
Port,Type,S_Host,S_IP,Port,D_Host,D_IP,Port
2,tcp,Google,8.8.8.8,2,Alibaba,47.246.57.232,8
3,tcp,Alibaba,47.246.57.232,2,MVTV,217.17.81.9,3
A very simple approach could be to read the list of IP addresses (i.e. pubIP.lst) and write it out into a new file but with resolved hostnames (i.e. hosts.lst).
#!/bin/bash
resolveHostname() {
# You may change or extend this function to your needs
dig -x "$1" +short
}
# Make sure there is no file with resolved hostnames
rm hosts.lst
while read LINE; # by line from a list
do
# Each Comma Separated Value (CSV) into a variable
PORT=$(echo "${LINE}" | cut -d "," -f 1)
TYPE=$(echo "${LINE}" | cut -d "," -f 2)
# SRC_HOST=$(echo "${LINE}" | cut -d "," -f 3)
SRC_IP=$(echo "${LINE}" | cut -d "," -f 4)
SRC_PORT=$(echo "${LINE}" | cut -d "," -f 5)
# DEST_HOST=$(echo "${LINE}" | cut -d "," -f 6)
DEST_IP=$(echo "${LINE}" | cut -d "," -f 7)
DEST_PORT=$(echo "${LINE}" | cut -d "," -f 8)
# And write it out the columns into a new file
# but for Col 3,6 with hostnames instead of IP
echo "${PORT},${TYPE},$(resolveHostname ${SRC_IP}),${SRC_IP},${SRC_PORT},$(resolveHostname ${DEST_IP}),${DEST_IP},${DEST_PORT}" >> hosts.lst
done < pubIP.lst
Thanks to
Passing parameters to a Bash function

process every line from command output in bash

From every line of nmap network scan output I want to store the hosts and their IPs in variables (for further use additionaly the "Host is up"-string):
The to be processed output from nmap looks like:
Nmap scan report for samplehostname.mynetwork (192.168.1.45)
Host is up (0.00047s latency).
thats my script so far:
#!/bin/bash
while IFS='' read -r line
do
host=$(grep report|cut -f5 -d' ')
ip=$(grep report|sed 's/^.*(//;s/)$//')
printf "Host:$host - IP:$ip"
done < <(nmap -sP 192.168.1.1/24)
The output makes something I do not understand. It puts the "Host:" at the very beginning, and then it puts "IP:" at the very end, while it completely omits the output of $ip.
The generated output of my script is:
Host:samplehostname1.mynetwork
samplehostname2.mynetwork
samplehostname3.mynetwork
samplehostname4.mynetwork
samplehostname5.mynetwork - IP:
In separate, the extraction of $host and $ip basically works (although there might a better solution for sure). I can either printf $host or $ip alone.
What's wrong with my script? Thanks!
Your two grep commands are reading from standard input, which they inherit from the loop, so they also read from nmap. read gets one line, the first grep consumes the rest, and the second grep exits immediately because standard input is closed. I suspect you meant to grep the contents of $line:
while IFS='' read -r line
do
host=$(grep report <<< "$line" |cut -f5 -d' ')
ip=$(grep report <<< "$line" |sed 's/^.*(//;s/)$//')
printf "Host:$host - IP:$ip"
done < <(nmap -sP 192.168.1.1/24)
However, this is inefficient and unnecessary. You can use bash's built-in regular expression support to extract the fields you want.
regex='Nmap scan report for (.*) \((.*)\)'
while IFS='' read -r line
do
[[ $line =~ $regex ]] || continue
host=${BASH_REMATCH[1]}
ip=${BASH_REMATCH[2]}
printf "Host:%s - IP:%s\n" "$host" "$ip"
done < <(nmap -sP 192.168.1.1/24)
Try this:
#!/bin/bash
while IFS='' read -r line
do
if [[ $(echo $line | grep report) ]];then
host=$(echo $line | cut -f5 -d' ')
ip=$(echo $line | sed 's/^.*(//;s/)$//')
echo "Host:$host - IP:$ip"
fi
done < <(nmap -sP it-50)
Output:
Host:it-50 - IP:10.0.0.10
I added an if clause to skip unwanted lines.

DD-WRT Bash script at startup issue

Hey all I have the following BASH script running at startup on my WRT1900ac linksys:
USER="admin"
PASS="passhere"
PROTOCOL="http"
ROUTER_IP="192.168.1.1"
# Port to connect to which will provide the JSON data.
PORT=9898
while [ 1 ]
do
# Grab connected device MAC addresses through router status page.
MACS=$(curl -s --user $USER:$PASS $PROTOCOL://$ROUTER_IP/Status_Wireless.live.asp)
# clear temp JSON file
echo > temp.log
# Get hostname and IP (just in case there is no hostname).
for MAC in $(echo $MACS | grep -oE "wl_mac::[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}:[a-z0-9]{2}" | cut -c 9-);
do
grep 0x /proc/net/arp | awk '{print $1 " " $4}' | while IFS= read -r line
do
IP=$(echo $line | cut -d' ' -f1)
MACTEMP=$(echo $line | cut -d' ' -f2)
HOST=$(arp -a | grep $IP | cut -d' ' -f1)
# if no hostname exists, just use IP.
if [ "$HOST" == "" ]
then
HOST=$IP
fi
if [ "$MAC" == "$MACTEMP" ]
then
JSON="{'hostname' : '$HOST', 'mac_address' : '$MAC'}"
echo $JSON >> temp.log
fi
done
done
# Provide the JSON formatted output on $PORT of router.
# This allows one connection before closing the port (connect, receive data, close).
# Port will reopen every 5 minutes with new data as setup in a cron job.
echo -e "HTTP/1.1 200 OK\n\n $(cat temp.log)" | nc -l -p$PORT >/dev/null
# Wait for 10 seconds and do it all over.
sleep 10
done
And for some reason when I reboot the router and then try to visit http://192.168.1.1:9898 it just shows a blank page even though I have my android cell phone connected via wifi to the router and the router shows the MAC address on the status page.
What should be on that page is all the wireless MAC address that are currently connected to the router and displaying them out in JSON form.
Any BASH guru's here that can help spot the problem?
I think it should be
echo -e "HTTP/1.1 200 OK\n\n $(cat temp.log)" | nc -l -p$PORT 0.0.0.0 >/dev/null

bash script pulling variables from .txt, keeps giving syntax error while trying to use mount command

Ive been trying to get this to work for the last week and cannot figure out why this is not working. I get mixed results typing directly into the terminal, but keep getting syntax error messages when running from the .sh. using ubuntu 11.10
It looks like part of the mount command gets pushed to the next line not allowing it to complete properly.. I have no idea why this is happening or how to prevent it from going to the second line.
i have several lines defined as follows in mounts.txt, that gets read from mount-drives.sh below
I have called it to run using sudo so it shouldnt be a permissions issue.
Thanks for taking a look, let me know if additional info is needed.
mounts.txt
mountname,//server/share$,username,password,
mount-drives.sh ---origional, updated below
#!/bin/bash
while read LINE;
do
# split lines up using , to separate variables
name=$(echo $LINE | cut -d ',' -f 1)
path=$(echo $LINE | cut -d ',' -f 2)
user=$(echo $LINE | cut -d ',' -f 3)
pass=$(echo $LINE | cut -d ',' -f 4)
echo $name
echo $path
echo $user
echo $pass
location="/mnt/test/$name/"
if [ ! -d $location ]
then
mkdir $location
fi
otherstuff="-o rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777,username=$user,password=$pass"
mount -t cifs $otherstuff $path $location
done < "/path/to/mounts.txt";
mount-drives.sh ---updated
#!/bin/bash
while read LINE
do
name=$(echo $LINE | cut -d ',' -f 1)
path=$(echo $LINE | cut -d ',' -f 2)
user=$(echo $LINE | cut -d ',' -f 3)
pass=$(echo $LINE | cut -d ',' -f 4)
empty=$(echo $LINE | cut -d ',' -f 5)
location="/mount/test/$name/"
if [ ! -d $location ]
then
mkdir $location
fi
mounting="mount -t cifs $path $location -o username=$user,password=$pass,rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777"
$mounting
echo $mounting >> test.txt
done < "/var/www/MediaCenter/mounts.txt"
Stab in the dark (after reading the comments). The "$pass" is picking up a newline because the mounts.txt was created in windows and has windows line endings. Try changing the echo $pass line to:
echo ---${pass}---
and see if it all shows up correctly.
There's a lot here that could stand improvement. Consider the following -- far more compact, far more correct -- approach:
while IFS=, read -u 3 -r name path user pass empty _; do
mkdir -p "$location"
cmd=( mount \
-t cifs \
-o "rw,uid=1000,gid=1000,file_mode=0777,dir_mode=0777,username=$user,password=$pass" \
"$path" "$location" \
)
printf -v cmd_str '%q ' "${cmd[#]}" # generate a string corresponding with the command
echo "$cmd_str" >>test.txt # append that string to our output file
"${cmd[#]}" # run the command in the array
done 3<mounts.txt
Unlike the original, this will work correctly even if your path or location values contain whitespace.

Resources