I am trying to execute following code multiple times from one machine/ip
#!/bin/bash
for i in `seq 1 10`;
do
ssh 11.156.112.111 pwd
done
# I know that the script is useless, it's just for demonstration.
I can't execute the script in more than two terminal at the same time, when I try to run the script in the third terminal, it freezes and I can't login but I can login from another machine/ip.
my configs
# .ssh/config
Host *
ServerAliveInterval 10
ServerAliveCountMax 10
# sshd_config
UsePAM no
MaxSessions 100
MaxStartups 100
Here is the output of verbose mode, it freezes on the last line:
OpenSSH_7.2p2, OpenSSL 1.0.2f 28 Jan 2016
debug1: Reading configuration data /home/neelix/.ssh/config
debug1: /home/neelix/.ssh/config line 1: Applying options for *
debug1: /home/neelix/.ssh/config line 11: Applying options for 11.156.112.111
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolving "11.156.112.111" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 11.156.112.111 [11.156.112.111] port 22.
debug1: Connection established.
debug1: identity file /home/neelix/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /home/neelix/.ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2
After many support tickets with my hosting provider, I found out that they had IDS(Intrusion Detection System) and it detected me as someone who is doing a brute force attack on port 22 and changing my port fixed the problem.
For future readers the comments above are quite helpful to narrow down the problem, specially answers to this question. I hope it saves someone else a couple of days!
Related
I'm trying to implement SSH server which acts as SCP sink (without calling external scp command) in GO.
I've ended up in partially working code:
https://gist.github.com/Seitanas/ad02158e8d5d2acedd9e7973ae44c77c.
The problem is that when I try to scp file to this service, local scp client copies data correctly but returns 1.
I think something happens at channel close. Maybe I'm missing something inside SCP protocol?
I'm guessing that ssh returns -1 and then scp exits with 1
Testing results:
echo "contents"> test.txt
scp -v -P2222 test.txt foo#127.0.0.1:/
Executing: program /usr/bin/ssh host 127.0.0.1, user foo, command scp -v -t /
...
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: password
debug1: Next authentication method: password
foo#127.0.0.1's password:
debug1: Authentication succeeded (password).
Authenticated to 127.0.0.1 ([127.0.0.1]:2222).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: Sending environment.
debug1: Sending env LC_TERMINAL_VERSION = 3.4.12
debug1: Sending env LC_CTYPE = en_US.UTF-8
debug1: Sending env LC_TERMINAL = iTerm2
debug1: Sending command: scp -v -t /
Sending file modes: C0644 9 test.txt
test.txt 0% 0 0.0KB/s --:-- ETAdebug1: channel 0: free: client-session, nchannels 1
test.txt 100% 9 21.4KB/s 00:00
debug1: fd 0 clearing O_NONBLOCK
Transferred: sent 2032, received 1116 bytes, in 0.0 seconds
Bytes per second: sent 1245098.0, received 683823.5
debug1: Exit status -1
echo $?
1
Output from go code:
go run test.go
2022/02/15 17:17:21 Command: C
2022/02/15 17:17:21 File info: 0644 9 test.txt
2022/02/15 17:17:21 File size: 9
2022/02/15 17:17:21 Content: contents
Thank you for any ideas.
Right, I've found what the problem is.
I've forgot that I'm simulating SCP binary ant server side, so i need to simulate its exit code also.
So I've added additional line at the end:
channel.SendRequest("exit-status", false, []byte{0, 0, 0, 0})
This will send exit 0 status via channel.
Without it SCP at client side was waiting for remote transfer to finish, but socket was closing right after transfer without exit code.
Now everything works as expected:
debug1: Sending command: scp -v -t /
Sending file modes: C0644 9 test.txt
test.txt 0% 0 0.0KB/s --:-- ETAdebug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
test.txt 100% 9 14.1KB/s 00:00
debug1: fd 0 clearing O_NONBLOCK
Transferred: sent 2152, received 1172 bytes, in 0.0 seconds
Bytes per second: sent 1391979.3, received 758085.4
debug1: Exit status 0
I'm trying to set sftpuser restricted by ChrootDirectory on AIX7.1. But it's not working and shown "connection to localhost closed by remote host"
Now, I configurated in /etc/ssh/sshd_config
Match User testuser
ChrootDirectory /Share
ForceCommand internal-sftp
AllowTcpForwarding no
X11Forwarding no
The /Share owner by root user and permission is 755 but it's still not working and show debug as below.
sftp -vvv testuser#localhost
OpenSSH_8.1p1, OpenSSL 1.0.2u 20 Dec 2019
debug1: Authentication succeeded (password).
Authenticated to localhost ([127.0.0.1]:22).
debug2: fd 5 setting O_NONBLOCK
debug3: fd 6 is O_NONBLOCK
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Requesting no-more-sessions#openssh.com
debug3: send packet: type 80
debug1: fd 0 clearing O_NONBLOCK
debug3: fd 1 is not O_NONBLOCK
Connection to localhost closed by remote host.
Transferred: sent 2256, received 2228 bytes, in 0.0 seconds
Bytes per second: sent 270661.7, received 267302.4
debug1: Exit status -1
Connection closed
How can I solve it? Thanks in advance for your help.
I have created a bash script on linux which I can invoke remotely via ssh. For some reason, the shell script produces the expected output only when I run the command directly on the box. However, if I run the exact same command with the exact same input (from remote box via ssh), no output is received.
Interestingly enough, I created a "usage" string which prints out when you pass in no arguments and if I invoke the script via ssh without arguments, I see the usage string. This tells me that the ssh client can receive output from the remote shell script.
I've tried invoking the command from another linux box on the same subnet. I've tried invoking the command from Windows (cygwin/open ssh). In both of these cases, the behavior is the same.
I'm completely stumped. How can I get the output of a shell script invoked via ssh.
In case you're wondering what's in the shell script, it just produces a list of files modified in the last 20 minutes, then invokes a python program to process those files:
if [ $# -lt 1 ]
then
echo "Usage: $0 <cluster>"
echo "-----------------------"
echo "Example: $0 mycluster"
exit
fi
clusterName=$1
find /mylogs/abc/123/mydir -mmin -20 -name "mylog*.log" > /home/myuser/file_names.txt
python filterDatesInFiles.py | sed -e "s/^/${HOSTNAME},\/mylogs\/abc\/123\/mydir\/^*~%$/"
in case you're interested in the python script, here it is:
#!/usr/bin/env python
import datetime, sys, time
# Example log line:
# [2/20/18 10:03:27:187 GMT-07:00] 0000001c SystemOut O ....
f = open("/home/myuser/file_names.txt")
lines = f.readlines()
f.close()
for line in lines:
line = line.strip()
if len(line) == 0:
continue
f = open(line)
fileLines = f.readlines()
f.close()
for fileLine in fileLines:
fileLineStripped = fileLine.strip()
if len(fileLine) == 0:
continue
if not fileLine.startswith("["):
continue
fields = fileLine.split(" ")
dateTime = fields[0].split("[")[1] + " " + fields[1]
dtObj = datetime.datetime.strptime(dateTime, "%m/%d/%y %H:%M:%S:%f")
currentTime = datetime.datetime.now()
d1_ts = time.mktime(dtObj.timetuple())
d2_ts = time.mktime(currentTime.timetuple())
minutesAgo = int(d2_ts-d1_ts) / 60
if minutesAgo <= 15:
print fileLineStripped
However, I know this script works, because, as I said, I can run this script directly on the target box and it produces the correct output.
Here is how I'm invoking it via ssh:
ssh myuser#mylinuxserver '/home/myuser/logScraper.sh mycluster'
Note - when I run that ssh command with "-v", here is the output I get:
$ ssh -v myuser#myhost.mydomain.com '/home/myuser/logScraper.sh mycluster'
OpenSSH_6.1p1, OpenSSL 1.0.1c 10 May 2012
debug1: Connecting to myhost.mydomain.com [123.45.67.61] port 22.
debug1: Connection established.
debug1: identity file /home/myremoteuser/.ssh/id_rsa type 1
debug1: identity file /home/myremoteuser/.ssh/id_rsa-cert type -1
debug1: identity file /home/myremoteuser/.ssh/id_dsa type -1
debug1: identity file /home/myremoteuser/.ssh/id_dsa-cert type -1
debug1: identity file /home/myremoteuser/.ssh/id_ecdsa type -1
debug1: identity file /home/myremoteuser/.ssh/id_ecdsa-cert type -1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: match: OpenSSH_7.4 pat OpenSSH*
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.1
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-sha1 none
debug1: kex: client->server aes128-ctr hmac-sha1 none
debug1: sending SSH2_MSG_KEX_ECDH_INIT
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ECDSA 93:a2:73:80:08:d5:cc:ad:78:c0:df:48:68:cb:b0:f4
debug1: Host 'myhost.mydomain.com' is known and matches the ECDSA host key.
debug1: Found key in /home/myremoteuser/.ssh/known_hosts:40
debug1: ssh_ecdsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /home/myremoteuser/.ssh/id_rsa
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic,password
debug1: Trying private key: /home/myremoteuser/.ssh/id_dsa
debug1: Trying private key: /home/myremoteuser/.ssh/id_ecdsa
debug1: Next authentication method: password
myuser#myhost.mydomain.com's password:
debug1: Authentication succeeded (password).
Authenticated to myhost.mydomain.com ([123.45.67.61]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: client_input_global_request: rtype hostkeys-00#openssh.com want_reply 0
debug1: Sending command: /home/myuser/logScraper.sh mycluster
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2080, received 2392 bytes, in 0.5 seconds
Bytes per second: sent 4000.0, received 4600.0
debug1: Exit status 0
Any clues why I cannot get the output when running via ssh?
I doubt that I'll get an answer here as AIX is very rare thing but I should try at least.
The background
We have the program. The program uses golang.org/x/crypto/ssh library to connect to the remote services and do some things. The program is part of the large service and widely tested by end-users. It works without issues (at least related to connection) not only with all Linux-based clients (include quite old things like Ubuntu 12.02) but also with the clients on FreeBSD, OpenBSD, NetBSD, MacOSX, Solaris SPARC, HP-UX and other *nixes. So looks like it wasn't tested only on the Samsung refrigerators. And yesterday I was sure that it will be able to connect to the refrigerator and do what is needed without any issues. But that was yesterday...
The problem
Today we decided to add AIX support to our program. And we partly failed.
The problem description is simple: after pty request program stops working. I mean I can do ssh.RequestPty it executes without any issues but when I'm trying to execute commands after the app just hangs. Without errors, without nothing. Just hangs.
When it works?
It works in PuTTY/KiTTY so I'm able to connect to the remote host.
If I remove requestPty - everything works. But we need pty for the sudo.
It works without issues if I request session.Shell even with pty requested. So if I write kind of interactive shell, it works perfectly.
What have I tried so far
I tried to debug so far as I could. The last command that executes is ch.sendMessage(msg) from ssh/channel.go. I mean it writes packet and that's all. No data returned from the remote host.
For the tests, I used 3 versions of AIX - 5.3, 6.1 and 7.1. No difference.
OpenSSH versions are different:
5.3 - OpenSSH_5.2p1, OpenSSL 0.9.8k 25 Mar 2009
6.1 & 7.1 - OpenSSH_6.0p1, OpenSSL 1.0.1e 11 Feb 2013
All machines are running in LPARs but I doubt this is related to the issue.
I have no idea what is wrong. And I even can't say if this is common AIX issue or only our test machine. Here is the sample program that should write IT WORKS if it works
package main
import (
"golang.org/x/crypto/ssh"
)
func main() {
server := "127.0.0.1:22"
user := "root"
p := "password"
config := &ssh.ClientConfig{
User: user,
Auth: []ssh.AuthMethod{ssh.Password(p)},
}
conn, err := ssh.Dial("tcp", server, config)
if err != nil {
panic(err.Error())
}
defer conn.Close()
session, err := conn.NewSession()
if err != nil {
panic(err.Error())
}
defer session.Close()
// Comment below and everything works
modes := ssh.TerminalModes{
ssh.ECHO: 0,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
}
if err := session.RequestPty("xterm", 80, 40, modes); err != nil {
panic(err.Error())
}
// Comment above and everything works
session.Run("echo 1")
println("IT WORKS")
}
If you have AIX somewhere around and can run this code against it I'd appreciate your feedback.
If you have any ideas (even crazy) why it may fail and where else I can look, don't be shy.
Update (2017-03-02):
By suggestion from #LorinczyZsigmond I launched sshd in debug mode. Results are a bit strange.
Here is part of Debian 9.0 OpenSSH_6.0p1 Debian-4+deb7u3, OpenSSL 1.0.1t 3 May 2016 log after sample program execution:
debug1: session_input_channel_req: session 0 req pty-req
debug1: Allocating pty.
debug1: session_pty_req: session 0 alloc /dev/pts/1
debug1: SELinux support disabled
debug1: server_input_channel_req: channel 0 request exec reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req exec
debug2: fd 3 setting TCP_NODELAY
debug3: packet_set_tos: set IP_TOS 0x10
debug1: Setting controlling tty using TIOCSCTTY.
debug2: channel 0: rfd 10 isatty
debug2: fd 10 setting O_NONBLOCK
debug3: fd 8 is O_NONBLOCK
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
It works as expected.
Now the same block from AIX 7.1 OpenSSH_6.0p1, OpenSSL 1.0.1e 11 Feb 2013 log:
debug1: session_input_channel_req: session 0 req pty-req
debug1: Allocating pty.
debug1: session_pty_req: session 0 alloc /dev/pts/42
debug1: server_input_channel_req: channel 0 request exec reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req exec
debug1: Values: options.num_allow_users: 0
debug1: RLOGIN VALUE :1
debug1: audit run command euid 0 user root command 'whoami'
setsid: Operation not permitted.
After setsid: Operation not permitted. it does nothing until I kill it with Ctrl+C. When I kill it it returns:
debug2: fd 4 setting TCP_NODELAY
debug3: packet_set_tos: set IP_TOS 0x10
debug2: channel 0: rfd 10 isatty
debug2: fd 10 setting O_NONBLOCK
debug3: fd 8 is O_NONBLOCK
debug2: notify_done: reading
Exiting on signal 2
debug1: do_cleanup
debug1: session_pty_cleanup: session 0 release /dev/pts/42
debug1: audit session close euid 0 user root tty name /dev/pts/42
debug1: audit event euid 0 user root event 12 (SSH_connabndn)
debug1: Return Val-1 for auditproc:0
And sends the result of whoami back to the client. This looks like a bug in SSH server, but is this possible for the 2 different versions?
Another interesting fact is when I run sshd with truss (kind of strace for AIX) the output looks like this:
debug1: session_input_channel_req: session 0 req pty-req
debug1: Allocating pty.
debug1: session_pty_req: session 0 alloc /dev/pts/42
debug1: server_input_channel_req: channel 0 request exec reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req exec
debug1: Values: options.num_allow_users: 0
debug1: RLOGIN VALUE :1
debug1: audit run command euid 0 user root command 'whoami'
debug2: fd 4 setting TCP_NODELAY
debug3: packet_set_tos: set IP_TOS 0x10
debug2: channel 0: rfd 10 isatty
debug2: fd 10 setting O_NONBLOCK
debug3: fd 8 is O_NONBLOCK
setsid: Operation not permitted.
debug2: channel 0: rcvd eof
debug2: channel 0: output open -> drain
debug2: channel 0: obuf empty
debug2: channel 0: close_write
debug2: channel 0: output drain -> closed
But truss output is a bit more strange than strace one (at least for someone who don't use *nix trace tools on daily basis) so I don't understand what is going on in the logs. If there is someone more skilled with this stuff here is the part of the trace data http://pastebin.com/YdzQwbt2 from debug1: RLOGIN VALUE :1.
Also, in the logs, I found that ssh.Shell() works because it doesn't request pty. It starts an interactive session (or something like that). But in my case, the interactive session is not an option.
better late than never
IBM said it was a bug in openssh - race condition while PTY allocation
https://www-01.ibm.com/support/docview.wss?uid=isg1IV82042
fixed in package openssh.base.server:7.5.102.1500
it strange that bug only occurs in aix, never in linux. nevertheless, problem is solved in my case
I had similar problem with "Allocating pty" and then exiting from ssh session. Here is log of my sshd debug:
sshd drops connection with error :3004-010 Failed setting terminal ownership and mode.
debug1: Allocating pty.
debug1: session_pty_req: session 0 alloc /dev/pts/2
debug1: Ignoring unsupported tty mode opcode 13 (0xd)
debug1: Ignoring unsupported tty mode opcode 18 (0x12)
debug1: server_input_channel_req: channel 0 request env reply 0
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req env
debug2: Ignoring env request LANG: disallowed name
debug1: server_input_channel_req: channel 0 request shell reply 1
debug1: session_by_channel: session 0 channel 0
debug1: session_input_channel_req: session 0 req shell
debug1: Values: options.num_allow_users: 0
debug1: RLOGIN VALUE :1
setsid: Operation not permitted.
The OS is AIX 7.1 (7100-04-03-1642)
The goal of my environment is to authenticate user on AIX through remote ldap user over ssh (ldap server actually is novell eDirectory).
So, I had similar issue with user authentication.
I fixed login over ssh as in eDirectory Schema (rfc2703), added following object extensions to the user:
posixAccount
posixGroup
shadowAccount
uamPosixUser (as I am not sure is it necessary this object)
I just want to note that on OS AIX following user isn't local, not exist in /etc/passwd and /etc/group.
V.Davidov
I am using shell script scp command to move one file from one server to another using ssh.
My command looks like this:
scp -P 23 -i $TEST_SSHKEY $DESTINATION_PATH/$FILETOCOPY $USER_ID#$SERVER_BOX:
File is copying but scp is returning return code as 1.
Why it is happening and what is the solution for this?
G'day,
Try enabling verbose mode (-v) to see more details of what's going on rather than just looking at a return code of 1 which means "something bad happened".
I have this same issue. The problem is the server on the remote end not sending an exit code. Compare these transaction excerpts.
The first causes $? = 1:
debug1: Sending command: scp -v -f /cfg/running-config
Sink: C0644 3398 running-config
running-config 100% 3398 3.3KB/s 00:00
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Connection to xxxxx closed by remote host.
Transferred: sent 2576, received 5216 bytes, in 1.9 seconds
Bytes per second: sent 1346.9, received 2727.3
debug1: Exit status -1
The second causes $?=0:
debug1: Sending command: scp -v -f /cfg/running-config
Sink: C0644 3940 running-config
running-config 100% 3940 3.9KB/s 00:00
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: channel 0: free: client-session, nchannels 1
debug1: fd 0 clearing O_NONBLOCK
debug1: fd 1 clearing O_NONBLOCK
Connection to xxxxx closed by remote host.
Transferred: sent 2624, received 5984 bytes, in 1.3 seconds
Bytes per second: sent 2026.8, received 4622.2
These are two HP switches on different firmware versions. The first one is very old; the second one is new and correctly returns exit-status reply 0 to the client.