I have a remote machine that I can get a shell into but can't do port forwarding, so everything has to go over stdin/stdout through the shell process. There's an HTTP server on the remote machine that I would like to connect to using a browser on my local machine.
What I've tried is the following:
mkfifo /tmp/socattunnel
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | remote_shell -- socat - TCP:localhost:6006 2>/dev/null >/tmp/socattunnel
The idea is to run one socat instance on my local machine, listening on port 12345, and one instance running on the remote machine connects to the existing HTTP server running on port 6006. The local instance reads from a named pipe and writes to the remote shell, which writes back to the pipe, completing the cycle.
With this setup, I can do curl http://localhost:12345 on my local machine and it works fine, but if I try to run it again, I get curl: (52) Empty reply from server. Here's the socat log:
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | remote_shell -- socat - TCP:localhost:6006 2>/dev/null >/tmp/socattunnel
2019/04/25 14:43:57 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:01 socat[46138] N accepting connection from LEN=16 AF=2 127.0.0.1:57506 on LEN=16 AF=2 127.0.0.1:12345
2019/04/25 14:44:01 socat[46138] N forked off child process 46141
2019/04/25 14:44:01 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:01 socat[46141] N reading from and writing to stdio
2019/04/25 14:44:01 socat[46141] N starting data transfer loop with FDs [6,6] and [0,1]
2019/04/25 14:44:02 socat[46141] N socket 1 (fd 6) is at EOF
2019/04/25 14:44:02 socat[46141] N exiting with status 0
2019/04/25 14:44:02 socat[46138] N childdied(): handling signal 20
2019/04/25 14:44:05 socat[46138] N accepting connection from LEN=16 AF=2 127.0.0.1:57507 on LEN=16 AF=2 127.0.0.1:12345
2019/04/25 14:44:05 socat[46138] N forked off child process 46173
2019/04/25 14:44:05 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:05 socat[46173] N reading from and writing to stdio
2019/04/25 14:44:05 socat[46173] N starting data transfer loop with FDs [6,6] and [0,1]
2019/04/25 14:44:05 socat[46173] E write(1, 0x7fde27004e00, 74): Broken pipe
2019/04/25 14:44:05 socat[46173] N exit(1)
2019/04/25 14:44:05 socat[46138] N childdied(): handling signal 20
In particular: E write(1, 0x7fde27004e00, 74): Broken pipe in the second child.
I then tried to reproduce the problem without going over the remote shell, by using tee:
python3 -m http.server 6006 &
mkfifo /tmp/socattunnel
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | tee /dev/stderr | socat - TCP:localhost:6006 >/tmp/socattunnel
And this also runs into the same problem. Is there an alternative approach for creating a bidirectional HTTP tunnel over a file/pipe?
Related
I have a set of commands that I am attempting to run in a script. To be exact, the lines are
rm tmp_pipe
mkfifo tmp_pipe
python listen_pipe.py &
while [ true ]; do nc -l -w30 7036 >>tmp_pipe; done &
listen_pipe.py is simply
if __name__ == "__main__":
f = open("tmp_pipe")
vals = " "
while "END" not in vals:
vals = f.readline()
if len(vals) > 0:
print(vals)
else:
f = open("tmp_pipe")
If I run the commands in the order shown I get my desired output, which is a connection to an ESP device that streams motion data. The connection resets after 30 seconds if the ESP device leaves the network range or if the device is turned off. The python script continues to read from the pipe and does not terminate when the tcp connection is reset. However, if I run this code inside a script file nc fails to connect and the device remains in an unconnected state indefinitely. The script is just
#!/bin/bash
rm tmp_pipe
mkfifo tmp_pipe
python listen_pipe.py &
while [ true ]; do nc -l -w30 7036 >>tmp_pipe; done &
This is being run on Ubuntu 16.04. Any suggestions are greatly welcomed, I have been fighting with this code all day. Thanks,
Ian
I'm writing a script that will check/open ports/protocols in the event any are blocked. What I have so far is below. The port/protocol names look strange to me. I would have expected IP addresses, but I've never done this before. Would the host be IP address of the DSLAM? Also, can I run nc without specifying host if it's the current machine? Otherwise, does this script do what is needed?
#!/bin/bash
PATH=/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin
echo -e "############################nnnPresent ports opened on this machine are
$(iptables -nL INPUT | grep ACCEPT | grep dpt)
nCompleted listing...nnn#########################"
#these look funny to me
PORTS=( 123 161 69 "UDP" 80 443 22 8443 8080 23 25 3307 "TCP" "HTTPS" "SNMP" "SFTP" "TFTP")
#modified ip's for public sharing
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
for PORT in "${PORTS[#]}"
do
#see which ones need opening...0 is pass (open), 1 fail, 5 timeout; need host still
#alternatively try nmap
nc -z -v -w5 ${HOST} ${PORT}
#if it's not open, then open it
if [ "$?" ne 0 ]; then #shellcheck err this line: Couldn't parse this test expression.
iptables -A INPUT -m tcp -p tcp --dport "$PORT" -j ACCEPT &&
{ service iptables save;
service iptables restart;
echo -e "Ports opened through iptables are n$(iptables -nL INPUT | grep ACCEPT | grep dpt)"; }
else
echo "Port $PORT already open"
fi
done
done
I've been referring to test if port is open, and also open port.
These lines seem odd, OP edit #6 adds an outer for loop which assigns the same value to $HOST on each go-round:
HOSTS=( "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x" "10.x.x.x")
for HOST in "${HOSTS[#]}"
do
< stuff ... >
done
Assuming running < stuff ... > four times is not necessary, then
the seven lines above, as written, would be equivalent to:
HOST="10.x.x.x"
< stuff ... >
(Fixed.) Remove the commas from this line:
PORTS=( 123, 161, 69, UDP, 80, 443, 22, 8443, 8080, 23, 25,
3307, TCP, HTTPS, SNMP, SFTP, TFTP)
bash does not use commas to define arrays, and if commas are used
they become chars in the the array data. Example, given the array
exactly as it is above:
echo ${PORTS[0]}
Outputs:
123,
I am trying to parse rsyslog logs. For this i am sending all my logs to socat which is then sending them to Unix Domain Socket. That socket is created via perl script which is listening on that socket to parse logs.
My bash script to which rsyslog is sending all log is
if [ ! `pidof -x log_parser.pl` ]
then
./log_parser.pl & 1>&1
fi
if [ -S /tmp/sock ]
then
/usr/bin/socat -t0 -T0 - UNIX-CONNECT:/tmp/sock 2>> /var/log/socat.log
fi
/tmp/sock is created using perl script log_parser.pl which is
use IO::Socket::UNIX;
sub socket_create {
$socket_path = '/tmp/sock';
unlink($socket_path);
$listner = IO::Socket::UNIX->new(
Type => SOCK_STREAM,
Local => $socket_path,
Listen => SOMAXCONN,
Blocking => 0,
)
or die("Can't create server socket: $!\n");
$socket = $listner->accept()
or die("Can't accept connection: $!\n");
}
socket_create();
while(1) {
chomp($line=<$socket>);
print "$line\n";
}
There is this error i am getting from socat which is
2015/02/24 11:58:01 socat[4608] E connect(3, AF=1 "/tmp/sock", 11): Connection refused
I am no champion in sockets so i am not able to understand what is this. Please help. Thanks in advance.
The main issue is that when i kill my perl script then bash script is suppose to call it again and start it.
What actually happening is that sript is started but socat is not started instead it give this error and never start.
I can duplicate your error if I don't run your perl program before trying to use socat. Here is what works for me:
1) my_prog.pl:
use strict;
use warnings;
use 5.016;
use Data::Dumper;
use IO::Socket::UNIX;
my $socket_path = '/tmp/sock';
unlink $socket_path;
my $socket = IO::Socket::UNIX->new(
Local => $socket_path,
Type => SOCK_STREAM,
Listen => SOMAXCONN,
) or die "Couldn't create socket: $!";
say "Connected to $socket_path...";
my $CONN = $socket->accept()
or die "Whoops! Failed to open a connection: $!";
{
local $/ = undef; #local -> restore previous value when the enclosing scope, delimited by the braces, is exited.
#Setting $/ to undef puts file reads in 'slurp mode' => whole file is considered one line.
my $file = <$CONN>; #Read one line.
print $file;
}`
2) $ perl my_prog.pl
3) socat -u -v GOPEN:./data.txt UNIX-CONNECT:/tmp/sock
The -u and -v options aren't necessary:
-u Uses unidirectional mode. The first address is only used for
reading, and the second address is only used for writing (exam-
ple).
-v Writes the transferred data not only to their target streams,
but also to stderr. The output format is text with some conver-
sions for readability, and prefixed with "> " or "< " indicating
flow directions.
4) You can also do it like this:
cat data.txt | socat STDIN UNIX-CONNECT:/tmp/sock
Pipe stdout of cat command to socat, then list STDIN as one of socat's files.
Response to comment:
This bash script works for me:
#!/usr/bin/env bash
echo 'bash script'
../pperl_programs/my_prog.pl &
sleep 1s
socat GOPEN:./data.txt UNIX-CONNECT:/tmp/sock
It looks like the perl script doesn't have enough time to setup the socket before socat tries to transfer data.
I am trying to write a webserver in bash using socat. I am having trouble serving image requests. I'm opening the socat listening connection like so:
socat -T 30 -d -d TCP-L:$LISTENIP,reuseaddr,fork,crlf SYSTEM:"$0 \"docroot=$DOCROOT\""
I'm serving the image with the following, where $1 is the docroot and $2 is the image file name.
function serve_png {
if [ -e $1$2 ]
then
SIZE=`stat -c '%s' $1$2`
echo -ne "HTTP/1.1 200 OK\nContent-type: image/png\nContent-length: $SIZE\n\n"
cat $1$2
else
echo -ne "HTTP/1.1 404 Not Found\nContent-type: text/html\n\n404 - Not found\n"
fi
}
The image fails to display in firefox due to it "containing errors." I'm getting the following output at console.
2014/01/25 08:00:41 socat[11551] N listening on AF=2 0.0.0.0:8080
2014/01/25 08:00:45 socat[11551] N accepting connection from AF=2 $MYIP:55765 on AF=2 $SERVERIP:8080
2014/01/25 08:00:45 socat[11552] N forking off child, using socket for reading and writing
2014/01/25 08:00:45 socat[11551] N forked off child process 11552
2014/01/25 08:00:45 socat[11551] N listening on AF=2 0.0.0.0:8080
2014/01/25 08:00:45 socat[11552] N forked off child process 11553
2014/01/25 08:00:45 socat[11552] N forked off child process 11553
2014/01/25 08:00:45 socat[11552] N starting data transfer loop with FDs [4,4] and [3,3]
2014/01/25 08:00:45 socat[11552] W read(3, 0x8e2e388, 8192): Connection reset by peer
2014/01/25 08:00:45 socat[11552] N socket 2 to socket 1 is in error
2014/01/25 08:00:45 socat[11552] N socket 2 (fd 3) is at EOF
2014/01/25 08:00:45 socat[11552] N socket 1 (fd 4) is at EOF
2014/01/25 08:00:45 socat[11552] N socket 2 (fd 3) is at EOF
2014/01/25 08:00:45 socat[11552] N exiting with status 0
I have seen similar scripts using netcat, but I'm unable to get it working using socat. I'd like to keep using socat as it has the ability to fork and handle multiple connections. Any insights would be appreciated.
Steffen Ullrich had it with omitting the crlf flag from the socat command. It was causing Cariage Returns/Line Feeds to be inserted into the stream automatically by socat (hence the file corruption). After omitting that option, everything worked as expected.
Hadoop datanode cannot communicate with KDC but the kinit works.
sudo kinit -k -t /etc/hadoop/conf/hdfs.keytab hdfs/symbio5.us-west-1.compute.internal#US-WEST-1.COMPUTE.INTERNAL
And the UDP port also is OK.
nc symbio5.us-west-1.compute.internal -v -z -u 88
Connection to symbio5.us-west-1.compute.internal 88 port [udp/kerberos] succeeded!
But in hadoop kerberos debug output, it always said "Receive timed out".
KrbAsReq calling createMessage
KrbAsReq in createMessage
KrbKdcReq send: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000, number of retries =3, #bytes=216
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =1, #bytes=216
SocketTimeOutException with attempt: 1
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =2, #bytes=216
SocketTimeOutException with attempt: 2
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =3, #bytes=216
SocketTimeOutException with attempt: 3
KrbKdcReq send: error trying symbio5.us-west-1.compute.internal java.net.SocketTimeoutException: Receive timed out
But on symbio5 itself, Which has the Namenode and KDC and a datanode, it is ok.
KrbAsReq calling createMessage
KrbAsReq in createMessage
KrbKdcReq send: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000, number of retries =3, #bytes=217
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =1, #bytes=217
KrbKdcReq send: #bytes read=776
KrbKdcReq send: #bytes read=776
KdcAccessibility: remove symbio5.us-west-1.compute.internal
Can anybody give me some helps? Thanks very much.
Finally I fixed this problem, just make the Hadoop to use TCP port to communicate with Kerberos, not UDP.
sudo vim /etc/krb5.conf
...
[libdefaults]
...
udp_preference_limit =1
...