I'm having trouble removing the output from the netstat command when I check if a current port is being used. I dont need the output of the command but rather just the error code.
Running netstat -anp | grep 1521 &>/dev/null results in
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
being shown as the output. I have also tried:
2>/dev/null| cut -d' ' -f45-50 | sed 's/[^0-9]*//g'
which just results in a bunch of empty lines after the Not all processes... output but still displays that in the output.
ended up solving my own problem.
Use the following
netstat -taepn 2>/dev/null | grep -Po "\b(\d|\.)+:1521\b" 1>/dev/null
This is specific for my situation when I just want the error code and no output
The accepted answer doesn't work for me, CentOS6.5 32bit here. It outputs nothing no matter what state the port is.
I found my solution:
netstat -antp 2>/dev/null | grep [port-num]
This will omit the first two lines of netstat if it is executed by regular user.
The warning two lines needed to be omitted:
(Not all processes could be identified, non-owned process info
will not be shown, you would have to be root to see it all.)
and works for two situations:
the port is "LISTEN" state
the port is "TIME_WAIT" state.
Instead of
netstat -anp
you may check using
sudo netstat -anp
This worked for me.
Related
For example,
sudo service --status-all | grep -oE 'php[0-9]+.[0-9]+'
generates the following output.
[ ? ] hwclock.sh
[ ? ] networking
php7.0
php7.3
My goal is to extract the version of another software package and put it into a configuration script so that the script won't break if that package gets upgraded or downgraded. If my understanding of regular expressions and the piping operator is correct, the first two lines shouldn't even show up in the output.
Can anyone explain to me why this is happening?
Redirecting the output to stderr eliminated the unwanted lines. I also piped the output to tail -1 to get the last line only--the one with the latest version.
sudo service --status-all 2>/dev/null | grep -oE 'php[0-9]+.[0-9]+' | tail -1
I am trying to find when and how the cache is flushed, planned to use the command redis-cli monitor | grep -iE "del|flush" > redis_log.txt for that, but for some reason the file is empty. If i use the command without > redis_log.txt part - it shows a correct output in the terminal, if i use redis-cli monitor > redis_log.txt command - it also saves an actual output to the file, but together it fails, only an empty file is created. Has anybody met a similar issue before?
As mentioned in the comments, the issue you notice certainly comes from the I/O buffering applied to the grep command, especially when its standard output is not attached to a terminal, but redirected to a file or so.
To be more precise, see e.g. this nice blog article which concludes with this wrap-up:
Here’s how buffering is usually set up:
STDIN is always buffered.
STDERR is never buffered.
if STDOUT is a terminal, line buffering will be automatically selected. Otherwise, block buffering (probably 4096 bytes) will be used.
[…] these 3 points explain all “weird” behaviors.
General solution
To tweak the I/O streams buffering of a program, a very handy program provided by coreutils is stdbuf.
So for your use case:
you may want to replace grep -iE "del|flush" with:
stdbuf -o0 grep -iE "del|flush" to completely disable STDOUT buffering;
or if you'd like some trade-off and just have STDOUT line-buffering,
you may want to replace grep -iE "del|flush" with:
either stdbuf -oL grep -iE "del|flush",
or grep --line-buffered -iE "del|flush".
Wrap-up
Finally as suggested by #jetchisel, you'll probably want to redirect STDERR as well to your log file in order not to miss some errors messages… Hence, for example:
redis-cli monitor | stdbuf -o0 grep -iE "del|flush" > redis_log.txt 2>&1
I am running ufw in "open" mode just to collect stats to see if there are any attempts to access the server. UFW is running in "medium" logging so I can see all access to the server. When I check ufw.log, I need to run through the whole list manually.
I currently use:
grep 'IN=eth0' uwf.log
But this still leaves too many records for me to check manually
What I really need is:
Grep must only look for lines that contain IN=eth0 ( This part is
easy)
Grep must IGNORE lines with SRC=0.0.0.0 (These are dhcp broadcasts)
Grep must IGNORE lines with SRC=10.0.1.15 (10.0.X.X is my Nagios checking ftp service)
Can someone please help,
Thank you.
I would use awk:
awk '/IN=eth0/ && !/SRC=0\.0\.0\.0/ && !/SRC=10\.0\.1\.15/' uwf.log
Since awk supports boolean operations, multiple conditions can be expressed in a pretty simple way.
grep -v : This will make grep to exclude lines which matches the pattern.
grep -E : This will make grep to accept regular expressions (in this case multiple possibilities - which is separated by pipe).
grep "IN=eth0" uwf.log |grep -Ev 'SRC=0\.0\.0\.0|SRC=10\.0\.1\.15'
With GNU sed:
sed -rn '/IN=eth0/{/(SRC=0\.0\.0\.0|SRC=10\.0\.1\.15)/!p}' uwf.log
I need to resolve a host name to an IP address in a shell script. The code must work at least in Cygwin, Ubuntu and OpenWrt(busybox).
It can be assumed that each host will have only one IP address.
Example:
input
google.com
output
216.58.209.46
EDIT:
nslookup may seem like a good solution, but its output is quite unpredictable and difficult to filter. Here is the result command on my computer (Cygwin):
>nslookup google.com
Unauthorized answer:
Serwer: UnKnown
Address: fdc9:d7b9:6c62::1
Name: google.com
Addresses: 2a00:1450:401b:800::200e
216.58.209.78
I've no experience with OpenWRT or Busybox but the following one-liner will should work with a base installation of Cygwin or Ubuntu:
ipaddress=$(LC_ALL=C nslookup $host 2>/dev/null | sed -nr '/Name/,+1s|Address(es)?: *||p')
The above works with both the Ubuntu and Windows version of nslookup. However, it only works when the DNS server replies with one IP (v4 or v6) address; if more than one address is returned the first one will be used.
Explanation
LC_ALL=C nslookup sets the LC_ALL environment variable when running the nslookup command so that the command ignores the current system locale and print its output in the command’s default language (English).
The 2>/dev/null avoids having warnings from the Windows version of nslookup about non-authoritative servers being printed.
The sed command looks for the line containing Name and then prints the following line after stripping the phrase Addresses: when there's more than one IP (v4 or 6) address -- or Address: when only one address is returned by the name server.
The -n option means lines aren't printed unless there's a p commandwhile the-r` option means extended regular expressions are used (GNU sed is the default for Cygwin and Ubuntu).
If you want something available out-of-the-box on almost any modern UNIX, use Python:
pylookup() {
python -c 'import socket, sys; print socket.gethostbyname(sys.argv[1])' "$#" 2>/dev/null
}
address=$(pylookup google.com)
With respect to special-purpose tools, dig is far easier to work with than nslookup, and its short mode emits only literal answers -- in this case, IP addresses. To take only the first address, if more than one is found:
# this is a bash-specific idiom
read -r address < <(dig +short google.com | grep -E '^[0-9.]+$')
If you need to work with POSIX sh, or broken versions of bash (such as Git Bash, built with mingw, where process substitution doesn't work), then you might instead use:
address=$(dig +short google.com | grep -E '^[0-9.]+$' | head -n 1)
dig is available for cygwin in the bind-utils package; as bind is most widely used DNS server on UNIX, bind-utils (built from the same codebase) is available for almost all Unix-family operating systems as well.
Here's my variation that steals from earlier answers:
nslookup blueboard 2> /dev/null | awk '/Address/{a=$3}END{print a}'
This depends on nslookup returning matching lines that look like:
Address 1: 192.168.1.100 blueboard
...and only returns the last address.
Caveats: this doesn't handle non-matching hostnames at all.
TL;DR; Option 2 is my preferred choice for IPv4 address. Adjust the regex to get IPv6 and/or awk to get both. There is a slight edit to option 2 suggested use given in EDIT
Well a terribly late answer here, but I think I'll share my solution here, esp. because the accepted answer didn't work for me on openWRT(no python with minimal setup) and the other answer errors out "no address found after comma".
Option 1 (gives the last address from last entry sent by nameserver):
nslookup example.com 2>/dev/null | tail -2 | tail -1 | awk '{print $3}'
Pretty simple and straight forward and doesn't really need an explanation of piped commands.
Although, in my tests this always gave IPv4 address (because IPv4 was always last line, at least in my tests.) However, I read about the unexpected behavior of nslookup. So, I had to find a way to make sure I get IPv4 even if the order was reversed - thanks regex
Option 2 (makes sure you get IPv4):
nslookup example.com 2>/dev/null | sed 's/[^0-9. ]//g' | tail -n 1 | awk -F " " '{print $2}'
Explanation:
nslookup example.com 2>/dev/null - look up given host and ignore STDERR (2>/dev/null)
sed 's/[^0-9. ]//g' - regex to get IPv4 (numbers and dots, read about 's' command here)
tail -n 1 - get last 1 line (alt, tail -1)
awk -F " " '{print $2} - Captures and prints the second part of line using " " as a field separator
EDIT: A slight modification based on a comment to make it actually more generalized:
nslookup example.com 2>/dev/null | printf "%s" "$(sed 's/[^0-9. ]//g')" | tail -n 1 | printf "%s" "$(awk -F " " '{print $1}')"
In the above edit, I'm using printf command substitution to take care of any unwanted trailing newlines.
I'm trying to program a little "dirty" website filter - e.g. an user wants to visit an erotic website (based on domain name)
So basically, I got something like
#!/bin/bash
sudo tshark -i any tcp port 80 or tcp port 443 -V | grep "Host.*keyword"
It works great but now I need to do some actions after I find something (iptables and DROPing packets...). The problem I got is that tcp dumping is still running. If I had a complete file with data, the thing I'm trying to reach is easy to solve.
In pseudocoude, I'd like to have something like:
if (tshark and grep found something)
iptables - drop packets
sleep 600 # a punishment for an user
iptables accept packets I was dropping
else
still look for a match in the tcp dump that's still running
Thanks for your help.
Maybe you could try something like the following:
tshark OPTIONS 2>&1 | grep --line-buffered PATTERN | while read line; do
# actions for when the pattern is found, the matched input is in $line
break
done
The 2>&1 is important so that when PATTERN is matched and the while loop terminates, tshark has nowhere to write to and terminates because of the broken pipe.
If you want to keep tshark running and analyze future output, just remove the break. This way, the while loop never terminates and it keeps reading the filtered output from tshark.