Hadoop datanode cannot communicate with KDC but the kinit works.
sudo kinit -k -t /etc/hadoop/conf/hdfs.keytab hdfs/symbio5.us-west-1.compute.internal#US-WEST-1.COMPUTE.INTERNAL
And the UDP port also is OK.
nc symbio5.us-west-1.compute.internal -v -z -u 88
Connection to symbio5.us-west-1.compute.internal 88 port [udp/kerberos] succeeded!
But in hadoop kerberos debug output, it always said "Receive timed out".
KrbAsReq calling createMessage
KrbAsReq in createMessage
KrbKdcReq send: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000, number of retries =3, #bytes=216
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =1, #bytes=216
SocketTimeOutException with attempt: 1
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =2, #bytes=216
SocketTimeOutException with attempt: 2
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =3, #bytes=216
SocketTimeOutException with attempt: 3
KrbKdcReq send: error trying symbio5.us-west-1.compute.internal java.net.SocketTimeoutException: Receive timed out
But on symbio5 itself, Which has the Namenode and KDC and a datanode, it is ok.
KrbAsReq calling createMessage
KrbAsReq in createMessage
KrbKdcReq send: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000, number of retries =3, #bytes=217
KDCCommunication: kdc=symbio5.us-west-1.compute.internal UDP:88, timeout=30000,Attempt =1, #bytes=217
KrbKdcReq send: #bytes read=776
KrbKdcReq send: #bytes read=776
KdcAccessibility: remove symbio5.us-west-1.compute.internal
Can anybody give me some helps? Thanks very much.
Finally I fixed this problem, just make the Hadoop to use TCP port to communicate with Kerberos, not UDP.
sudo vim /etc/krb5.conf
...
[libdefaults]
...
udp_preference_limit =1
...
Related
I have a remote machine that I can get a shell into but can't do port forwarding, so everything has to go over stdin/stdout through the shell process. There's an HTTP server on the remote machine that I would like to connect to using a browser on my local machine.
What I've tried is the following:
mkfifo /tmp/socattunnel
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | remote_shell -- socat - TCP:localhost:6006 2>/dev/null >/tmp/socattunnel
The idea is to run one socat instance on my local machine, listening on port 12345, and one instance running on the remote machine connects to the existing HTTP server running on port 6006. The local instance reads from a named pipe and writes to the remote shell, which writes back to the pipe, completing the cycle.
With this setup, I can do curl http://localhost:12345 on my local machine and it works fine, but if I try to run it again, I get curl: (52) Empty reply from server. Here's the socat log:
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | remote_shell -- socat - TCP:localhost:6006 2>/dev/null >/tmp/socattunnel
2019/04/25 14:43:57 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:01 socat[46138] N accepting connection from LEN=16 AF=2 127.0.0.1:57506 on LEN=16 AF=2 127.0.0.1:12345
2019/04/25 14:44:01 socat[46138] N forked off child process 46141
2019/04/25 14:44:01 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:01 socat[46141] N reading from and writing to stdio
2019/04/25 14:44:01 socat[46141] N starting data transfer loop with FDs [6,6] and [0,1]
2019/04/25 14:44:02 socat[46141] N socket 1 (fd 6) is at EOF
2019/04/25 14:44:02 socat[46141] N exiting with status 0
2019/04/25 14:44:02 socat[46138] N childdied(): handling signal 20
2019/04/25 14:44:05 socat[46138] N accepting connection from LEN=16 AF=2 127.0.0.1:57507 on LEN=16 AF=2 127.0.0.1:12345
2019/04/25 14:44:05 socat[46138] N forked off child process 46173
2019/04/25 14:44:05 socat[46138] N listening on LEN=16 AF=2 0.0.0.0:12345
2019/04/25 14:44:05 socat[46173] N reading from and writing to stdio
2019/04/25 14:44:05 socat[46173] N starting data transfer loop with FDs [6,6] and [0,1]
2019/04/25 14:44:05 socat[46173] E write(1, 0x7fde27004e00, 74): Broken pipe
2019/04/25 14:44:05 socat[46173] N exit(1)
2019/04/25 14:44:05 socat[46138] N childdied(): handling signal 20
In particular: E write(1, 0x7fde27004e00, 74): Broken pipe in the second child.
I then tried to reproduce the problem without going over the remote shell, by using tee:
python3 -m http.server 6006 &
mkfifo /tmp/socattunnel
socat -d -d TCP-LISTEN:12345,crlf,reuseaddr,fork - </tmp/socattunnel | tee /dev/stderr | socat - TCP:localhost:6006 >/tmp/socattunnel
And this also runs into the same problem. Is there an alternative approach for creating a bidirectional HTTP tunnel over a file/pipe?
I have a ruby script that uploads a file via FTP.
I tested this with a LINUX server and the upload was working well.
But when I changed the upload destination to the production server which is a Windows Server 2012 R2 running IIS, I got the following error message:
A Connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond. - connect<2> <Errno::ETIMEDOUT>
I do however find the connection in the FTPlog:
2015-01-28 10:48:55 12.34.56.78 - 87.65.43.21 21 ControlChannelOpened - - 0 0 0 0 0
2015-01-28 10:48:55 12.34.56.78 - 87.65.43.21 21 USER .... 331 0 0 23 17 0
2015-01-28 10:48:55 12.34.56.78 WIN-0CFF8VSL25E\.... 87.65.43.21 21 PASS *** 230 0 0 21 15 94
2015-01-28 10:48:55 12.34.56.78 WIN-0CFF8VSL25E\.... 87.65.43.21 21 TYPE I 200 0 0 20 8 0
2015-01-28 10:48:55 12.34.56.78 WIN-0CFF8VSL25E\.... 87.65.43.21 21 PASV - 227 0 0 50 6 0
2015-01-28 10:49:16 - WIN-0CFF8VSL25E\.... 87.65.43.21 49994 DataChannelClosed - - 0 0 0 0 0
2015-01-28 10:49:16 12.34.56.78 WIN-0CFF8VSL25E\.... 87.65.43.21 21 ControlChannelClosed - - 0 0 141 46 21469
I am doing almost the same task with a batch script today, using windows native ftp cli, and this works fine.
Here is the ruby code:
require "net/ftp"
require "rubygems"
require "zip"
puts "Copying data"
FileUtils.cp("...", "...")
folder = "C:\\..."
input_filenames = ['...']
zipfile_name = "C:\\...\\....zip"
puts "Compressing data"
Zip::File.open(zipfile_name, Zip::File::CREATE) do |zipfile|
input_filenames.each do |filename|
zipfile.add(filename, folder + '\\' + filename)
end
end
progress = 0.0;
file_size = File.size(zipfile_name)
NET::FTP.debug_mode = true
puts "Uploading data:"
ftp = Net::FTP.new('12.34.56.78','xxx','xxx')
ftp.debug_mode = true
ftp.read_timeout = 10000
ftp.passive = true
ftp.putbinaryfile('C:\\...\\....zip', '....zip') do |data|
progress = progress.to_f+data.length.to_f
print "\rProgress: " + ((progress/file_size)*100).round(2).to_s + "%"
end
ftp.close()
puts "\nUpload completed!"
I am assuming there is no problem in the script, since it was working fine with a different server. What I am assuming it is, is a difference in how IIS' FTP server handles/responds to connections vs how vsftpd handles them.
Any suggestions to how to fix this?
I set passive to false and that solved my problem.
I am trying to write a webserver in bash using socat. I am having trouble serving image requests. I'm opening the socat listening connection like so:
socat -T 30 -d -d TCP-L:$LISTENIP,reuseaddr,fork,crlf SYSTEM:"$0 \"docroot=$DOCROOT\""
I'm serving the image with the following, where $1 is the docroot and $2 is the image file name.
function serve_png {
if [ -e $1$2 ]
then
SIZE=`stat -c '%s' $1$2`
echo -ne "HTTP/1.1 200 OK\nContent-type: image/png\nContent-length: $SIZE\n\n"
cat $1$2
else
echo -ne "HTTP/1.1 404 Not Found\nContent-type: text/html\n\n404 - Not found\n"
fi
}
The image fails to display in firefox due to it "containing errors." I'm getting the following output at console.
2014/01/25 08:00:41 socat[11551] N listening on AF=2 0.0.0.0:8080
2014/01/25 08:00:45 socat[11551] N accepting connection from AF=2 $MYIP:55765 on AF=2 $SERVERIP:8080
2014/01/25 08:00:45 socat[11552] N forking off child, using socket for reading and writing
2014/01/25 08:00:45 socat[11551] N forked off child process 11552
2014/01/25 08:00:45 socat[11551] N listening on AF=2 0.0.0.0:8080
2014/01/25 08:00:45 socat[11552] N forked off child process 11553
2014/01/25 08:00:45 socat[11552] N forked off child process 11553
2014/01/25 08:00:45 socat[11552] N starting data transfer loop with FDs [4,4] and [3,3]
2014/01/25 08:00:45 socat[11552] W read(3, 0x8e2e388, 8192): Connection reset by peer
2014/01/25 08:00:45 socat[11552] N socket 2 to socket 1 is in error
2014/01/25 08:00:45 socat[11552] N socket 2 (fd 3) is at EOF
2014/01/25 08:00:45 socat[11552] N socket 1 (fd 4) is at EOF
2014/01/25 08:00:45 socat[11552] N socket 2 (fd 3) is at EOF
2014/01/25 08:00:45 socat[11552] N exiting with status 0
I have seen similar scripts using netcat, but I'm unable to get it working using socat. I'd like to keep using socat as it has the ability to fork and handle multiple connections. Any insights would be appreciated.
Steffen Ullrich had it with omitting the crlf flag from the socat command. It was causing Cariage Returns/Line Feeds to be inserted into the stream automatically by socat (hence the file corruption). After omitting that option, everything worked as expected.
guys,i have a problem on using xinetd,the error message is 'xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)'
the system operation is : CentOS release 6.2;
i installed the xinetd by the command 'sudo yum install xinetd'
i edited the /etc/services, add my port 6033 for my service named 'mysqlchk'
the service 'mysqlchk' in /etc/xinetd.d/mysqlchk is
service mysqlchk
{
disable = no
flags = REUSE
socket_type = stream
port = 6033
wait = no
user = fulu
server = /home/fulu/download/mysqlchk_status2.sh
log_on_failure += USERID
}
the shell file /home/fulu/download/mysqlchk_status2.sh content is
echo 'test'
6.i can run the command /home/fulu/download/mysqlchk_status2.sh straightly and get the result 'test'
when i telnet 127.0.0.1 6033,i get the output
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
Connection closed by foreign host.
then i tail the log file /var/log/messages,it shows
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: START: mysqlchk pid=20126 from=127.0.0.1
Apr 22 22:01:47 AY1304111122016 xinetd[20126]: execv( /home/fulu/download/mysqlchk_status2.sh ) failed: Exec format error (errno = 8)
Apr 22 22:01:47 AY1304111122016 xinetd[20001]: EXIT: mysqlchk status=0 pid=20126 duration=0(sec)
i don't know why,can anybody help me ?
I'm sorry, after questioning it i suddenly found the answer. If you want the shell to be run in other program you need add '#!/bin/echo' at the first line of the shell file (of course the echo can be changed)
I'm trying to configure my pureftpd behind the firewall to act as a passive ftp/TLS server.
Acting machines:
Server: 192.168.3.220 (internal network, default route to the router at 192.168.3.1)
Configuration: pureftpd with PassivePorts 64000 64300, MasqueradeAddress ww.xx.yy.zz (this one is configured on router)
Router: internal: 192.168.3.1, DNAT rule (PREROUTING chain) ww.xx.yy.zz tcp/21,64000:64300 NATed to address 192.168.3.220, FORWARD chain accepting these packets both directions.
Client1: external server with fixed public IP
Client2: NATed machine somewhere - on 192.168.5.x network
Scenario1:
- Client1: connect OK, login OK, command 'ls':
gets OK, after PASV:
---> PASV
GNUTLS: REC[0x28ecce0]: Sending Packet[9] Application Data(23) with length: 6
GNUTLS: REC[0x28ecce0]: Sent Packet[10] Application Data(23) with length: 37
GNUTLS: ASSERT: gnutls_buffers.c:322
GNUTLS: ASSERT: gnutls_buffers.c:322
GNUTLS: REC[0x28ecce0]: Expected Packet[9] Application Data(23) with length: 65536
GNUTLS: REC[0x28ecce0]: Received Packet[9] Application Data(23) with length: 64
GNUTLS: REC[0x28ecce0]: Decrypted Packet[9] Application Data(23) with length: 31
<--- 200 Protection set to Private
---> LIST
---> ABOR
Interesting thing: 227 from server, which I see in paranoid log from pureftpd, I don't see on the client - only the 200 Protection set to Private
...waits cca 30sec and reconnects using ACTIVE(!!) mode -> ls
Scenario2:
- using Client2 (sorry for czech locales):
---> USER xxxxxx
<--- 331 Password required for xxxxxx
---> PASS XXXX
<--- 230 User xxxxxx logged in
---> PWD
<--- 230 Ls oi a:2013-01-03 21:19:00
---> PBSZ 0
<--- 257 "/" is the current directory
---> PROT P
<--- 200 PBSZ 0 successful
---> PASV
<--- 200 Protection set to Private
---> LIST
---> ABOR
---- Přerušený datový socket bude uzavřen (means closing data socket)
---- Řídicí socket bude uzavřen (means closing control socket)
---- Pasivní režim bude vypnut (means Passive will be turned off)
---- dns cache hit
---- Navazuje se spojení na ftp1.xxxxxxxxx.cz (ww.xx.yy.zz) port 21
<--- 220 ww.xx.yy.zz FTP server ready
...
---> USER xxxxxx
<--- 331 Password required for xxxxxx
---> PASS XXXX
<--- 230 User xxxxxx logged in
---> PWD
<--- 230 Ls oi a:2013-01-03 21:19:22
---> PBSZ 0
<--- 257 "/" is the current directory
---> PROT P
<--- 200 PBSZ 0 successful
---> PORT 192,168,5,xx,185,136
<--- 200 Protection set to Private
---> LIST
<--- 500 Illegal PORT command
---- Closing data socket
---> QUIT
ls: Nepřekonatelná chyba: 500 Illegal PORT command
<--- 425 Unable to build data connection: Connection refused
iptables on the NAT machine don't increase my accounting counters on ports 64000:64300, so I expect there's no passive connection made at all.
So... the real problem was the second 230 reply:
---> PWD
<--- 230 Ls oi a:2013-01-03 21:19:22
This is a known issue of the PureFTPd 1.3.3a (default debian squeeze)
The solution was to compile PureFTPd from wheezy (1.3.4a-2), now everything works fine.
Thank you all, who tried to figure out what's going on. Tldv