Cron job creates empty files - bash

I want to preface that I am a newbie that picked up shell scripting 2 weeks ago.
Hey guys I need help with something, hope someone can point me in the right direction. I have a script that works when I run it from the command line but every time I run it with a crontab, the output is a few empty files. Does anyone know why?
That's the code down there
#!/bin/bash
#Provide an IP address as an argument to use nmap
#make sure to add the full range with (0-225 or 0/24) at the end
IPADDRESS=$(hostname -I | awk '{print $1}')
network-scan(){
if [ $1 ]
then
sudo nmap -sn $1
else
sudo nmap -sn 192.168.1.0-255
fi
}
#Scan the whole network and only prints the IP addresses minus your own
#Sends the IP addresses to a file
network-scan | grep -i 'Nmap scan report' | \
sed 's/\ /\n/g'|sed 's/(//g'|sed 's/)//g' | \
grep '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' | grep -v ${IPADDRESS} > ip_addresses
#Scan the whole network and only prints the MAC addresses
#Sends the MAC addresses to a file
network-scan | grep -i 'MAC Address:' | \
awk '{print $3}' > mac_addresses
#Put the IP and MAC addresses in the same file
paste ip_addresses mac_addresses | \
column -s $'\t' -t > "scan_$(date +%d-%m-%Y_%H:%M:%S)"
#Notify that a file with the IP and MAC addresses has been created on the Desktop
echo "A file containing the results of the scan has been created on the Desktop"
exit 0

You are using
network-scan | grep
without passing any parameter.
Hence network-scan function always using
sudo nmap -sn 192.168.1.0-255
when you run it from command line are you passing any parameter ?
echo $IPADDRESS inside the script when executing at cron and at command line for debugging.
network-scan | grep -i 'Nmap scan report' | \
sed 's/\ /\n/g'|sed 's/(//g'|sed 's/)//g' | \
grep '[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*' | grep -v ${IPADDRESS}
Since you are obtaining empty output, validate each command and append(test) each OR operators to know where it is removing required output.

Related

ssh remote command execution quoting and piping awk

I'm working on a script, that should find certain disks and add hostname to them.
I'm using this for 40 servers with a for loop in bash
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i findmnt -o SIZE,TARGET -n -l |
grep '1.8T\|1.6T\|1.7T' |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=$HOSTNAME '{print HOSTNAME ":" $0}'; done |
tee sorted.log
can you help out with the quoting here? It looks like awk gets piped (hostname) from localhost, not the remote server.
Everything after the first pipe is running locally, not on the remote server.
Try quoting the entire pipeline to have it run on the remote server:
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i "findmnt -o SIZE,TARGET -n -l |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=\$HOSTNAME '{print HOSTNAME \":\" \$0}'" ;
done | tee sorted.log
This is a shorter version of your stuff:
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '/M/{print HOSTNAME ":" $2}'
Applied to the above:
for i in myservers{1..40}
do ssh user#$i bash -c '
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '"'"'/M/{print HOSTNAME ":" $2}'"'"' '
done |
tee sorted.log
see: How to escape the single quote character in an ssh / remote bash command?

Sorted ouput, needs to have text inserted between string

I trying to add text (predefined) between a sorted output and saved to a new file.
I'm using a curl command to gather my info.
$ curl --user XXX:1234!## "http://......"
Then using grep to find IP addresses and sorting so they only appear once.
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
I need to add <my_text_predefined> ([0-9]{1,3}[\.]){3}[0-9]{1,3} <my_text_predefined> between the regex ip address and then saved to a new file.
The script below only get my the ip address
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
123.12.0.12
123.56.98.76
$ curl --user some_user:password "http://...." | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u | sed 's/.*/<prefix> -s & <suffix>/'
So if we need print some text for each IP ... try xargs
for i in {1..100}; do echo $i; done | xargs -n1 echo "Values are:"
if based on IP you would need to take decision put in a loop
for file $(curl ...) do ...
and check $file or do something with it ...

Shell script to make directories and subdirectories with variable names

I'm trying to create script to be run by cron to create multiple folders with subfolders.
DATE=`date +%Y-%m-%d`
IP_ADDR=`ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p'`
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
If i run this script manually everything is created as expected. When script is ran by cron subdirectory $IP_ADDR is not created and there is no errors.
I suspect that /sbin is not part of the PATH for the environment that the cron job runs under. You should specify the full path for the ifconfig command:
IP_ADDR=$(/sbin/ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p')
It's also better practice (in general) to use $() for command substitution.
Try to use debug mode :
set -x
DATE=`date +%Y-%m-%d`
IP_ADDR=`ifconfig | grep -v '127.0.0.1' | sed -n 's/.*inet addr:\([0-9.]\+\)\s.*/\1/p'`
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
set +x
Then, redirect the output of your cron to a file and have a look, you should find useful information in it.
You are not far off, but there are several ordering caveats that could cause problems. Many systems have different formats for the ifconfig output line. Some with inet xxx.xxx.xxx.xxx, others with inet addr:xxx.xxx.xxx.xxx. (those are the two most common). You may also need to handle the case where there are multiple wired inet interfaces (2+ NICs in the box). However, if you have only 1 NIC, you could try the following to handle the common ifconfig formats:
DATE=`date +%Y-%m-%d`
IP_ADDR=$(ifconfig |
grep -v '127.0.0.1' |
grep -E 'inet[ ](addr:)*[0-9]{1,3}([.][0-9]{1,3}){3}' |
sed -e 's/^.*inet \(addr:\)*//' -e 's/ .*$//')
/bin/mkdir -p /mnt/db-backup/12/$DATE/$IP_ADDR/
or with IP_ADDR written as one line:
IP_ADDR=$(ifconfig | grep -v '127.0.0.1' | grep -E 'inet[ ](addr:)*[0-9]{1,3}([.][0-9]{1,3}){3}' | sed -e 's/^.*inet \(addr:\)*//' -e 's/ .*$//')

curl complex usage with pattern

I'm trying to get 2 files using curl based on some pattern but that doesn't seem to work:
Files:
SystemOut_15.04.01_21.12.36.log
SystemOut_15.04.01_15.54.05.log
curl -f -k -u "login:password" https://myserver/cgi-bin/logviewer/index.cgi?getlogfile=SystemOut_15.04.01_21.12.36.log'&'server=qwerty123.com'&'numlines=100000000'&'appenv=MBL%20-%20PROD'&'directory=/app/WAS/was85/profiles/node/logs/mbl-server1
I know there is -A key but it doesn't work since my file is inside the link.
How can I extract those 2 files using a pattern?
Did that myself. One curl gets the list of logs on the webpage. Another downloads those files.
The code looks like:
for file in $(curl -f -k -u "user:pwd" https://selfservice.pwj.com/cgi-bin/logviewer/index.cgi?listdirectory=/app/smx_client_mob/data/log'&'appenv=MBL%20-%20PROD'&'server=xshembl04pap.she.pwj.com | grep href | sed 's/.*href="//' | sed 's/".*//' | sed 's/javascript:getLog//g' | sed "s/['();]//g" | grep -i 'service' | grep '^[a-zA-Z].*'); do
curl -o $file -f -k -u "user:pwd" https://selfservice.pwj.com/cgi-bin/logviewer/index.cgi?getlogfile="$file"'&'server=xshembl04pap.she.pwj.com'&'numlines=100000000'&'appenv=MBL%20-%20PROD'&'directory=/app/smx_client_mob/data/log; done

tcpdump: Output only source and destination addresses

Problem description:
I want to print only the source and destination address from a tcpdump[1].
Have one working solution, but believe it could be improved a lot. An example that captures 5 packets, just as an example of what I'm looking for:
tcpdump -i eth1 -n -c 5 ip | \
cut -d" " -f3,5 | \
sed -e 's/^\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\)\..* \([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\).*$/\1 > \2/'
Question:
Can this be done in any easier way? Performance is also an issue here.
[1] A part of a test if the snort home_net is correctly defined, or if we see traffic not defined in the home_net.
Solution:
Ok, thanks to everyone who have replied to this one. There have been two concerns related to the answers, one is the compatibility across different linux-versions and the second one is speed.
Here is the results on the speed test I did. First the grep-version:
time tcpdump -l -r test.dmp -n ip 2>/dev/null | grep -P -o '([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+).*? > ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)' | grep -P -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' | xargs -n 2 echo >/dev/null
real 0m5.625s
user 0m0.513s
sys 0m4.305s
Then the sed-version:
time tcpdump -n -r test.dmp ip | sed -une 's/^.* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..*$/\1 > \3/p' >/dev/null
reading from file test.dmp, link-type EN10MB (Ethernet)
real 0m0.491s
user 0m0.496s
sys 0m0.020s
And the fastest one, the awk-version:
time tcpdump -l -r test.dmp -n ip | awk '{ print gensub(/(.*)\..*/,"\\1","g",$3), $4, gensub(/(.*)\..*/,"\\1","g",$5) }' >/dev/null
reading from file test.dmp, link-type EN10MB (Ethernet)
real 0m0.093s
user 0m0.111s
sys 0m0.013s
Unfortunately I have not been able to test how compatible they are, but the awk needs gnu awk to work due to the gensub function. Anyway, all three solutions works on the two platforms I have tested them on. :)
Here's one way using GNU awk:
tcpdump -i eth1 -n -c 5 ip | awk '{ print gensub(/(.*)\..*/,"\\1","g",$3), $4, gensub(/(.*)\..*/,"\\1","g",$5) }'
Try this:
tcpdump -i eth1 -n -c 5 ip 2>/dev/null | sed -r 's/.* ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+).* > ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+).*/\1 > \2/'
If running from a .sh script, remember to escape \1 & \2 as required.
Warning You have to use unbuffered ou line-buffered output to monitor the output of another command like tcpdump.
But you command seem correct.
To simplify, you could:
tcpdump -i eth1 -n -c 5 ip |
sed -une 's/^.* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..*$/\1 > \3/p'
Notice the u switch usefull without -c 5 at tcpdump
tcpdump -ni eth1 ip |
sed -une 's/^.* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..* \(\([0-9]\{1,3\}\.\?\)\{4\}\)\..*$/\1 > \3/p'
& here is a grep only solution:
tcpdump -l -i eth1 -n -c 5 ip 2>/dev/null | grep -P -o '([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+).*? > ([0-9]+\.[0-9]+\.[0-9]+\.[0-9]+)' | grep -P -o '[0-9]+\.[0-9]+\.[0-9]+\.[0-9]+' | xargs -n 2 echo
Note -l, in case you don't want to limit the number of packets using -c.

Resources