ssh exec a simple command cost a few seconds - performance

I find it costs one more seconds that ssh exec a simple command, does it normal? if not, how to speed up it?
[root#ops-test-vm-154:~]# time ssh root#10.17.1.155 'echo "hello,world!"'
hello,world!
real 0m1.805s
user 0m0.009s
sys 0m0.005s
there is low latency between vm-154 and vm-155
[root#ops-test-vm-154:~]# ping 10.17.1.155
PING 10.17.1.155 (10.17.1.155) 56(84) bytes of data.
64 bytes from 10.17.1.155: icmp_seq=1 ttl=64 time=0.142 ms
64 bytes from 10.17.1.155: icmp_seq=2 ttl=64 time=0.136 ms
64 bytes from 10.17.1.155: icmp_seq=3 ttl=64 time=0.129 ms
64 bytes from 10.17.1.155: icmp_seq=4 ttl=64 time=0.110 ms
^C
--- 10.17.1.155 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 4421ms
rtt min/avg/max/mdev = 0.110/0.128/0.142/0.014 ms
BTW: I need check service status real time by executing a script in vm-155, so vm-154 execute command ssh vm-155 status.sh every second. But even a simple command echo helloworld cost one more second. So the solution is terrible. I hope speed up it, or may be a better solution.
Best Wishes!
There is vm-155 /etc/ssh/sshd_config, I add UseDNS no and execute service sshd restart, but still need one more second to echo hello,world!
Protocol 2
SyslogFacility AUTHPRIV
PasswordAuthentication yes
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL LANGUAGE
AcceptEnv XMODIFIERS
X11Forwarding yes
UseDNS no
Subsystem sftp /usr/libexec/openssh/sftp-server

One thing that you could try is to run SSH in verbose mode and see at which stage it wastes the most time.
ssh -vvv root#10.17.1.155 'echo "hello,world!"'
And then based on your findings adopt your ssh config file to exclude slow cipher suites and other CPU intensive things. Some tips about that here.
However, you will not be able to achieve close to real-time performance over ssh if you establish a new connection every time. You could put your script/command into a loop and set seep value to 1s.
ssh root#10.17.1.155 'while true; do echo "Hello, world!"; sleep 1s; done'
But I would use something that is designed for such application like SNMP protocol. Here is an example configuration:
https://www.incredigeek.com/home/snmp-and-shell-script/

One source of delays during the SSH connection process is DNS lookups by the server. When a client connects to the server, the server can optionally look up the IP address of the client to get its hostname. Depending on a variety of issues, the query may take anywhere from a fraction of a second to ten seconds or more to complete.
The most widely deployed SSH server is OpenSSH. The OpenSSH sshd server has a setting named UseDNS which controls whether it performs DNS queries on incoming connections or not:
UseDNS
Specifies whether sshd(8) should look up the remote host name, and to check that the resolved host name for the remote IP address maps back to the very same IP address.
You should check that UseDNS is set to "no" on the server which you're connecting to.

Related

Multiple pings from a client to a server

I have 2 Virtual Machines. One is a Windows(say client). The other one is a CentOS(say server).
My Windows (Client) IP is 1.1.1.1
My CentOS (Server) has an app running and is listening on port 12345. IP address of the CentOS VM is 2.2.2.2.
I want to generate multiple pings from the windows VM from specific IP/ports to CentOS VM specific IP/ports.
i.e generate multiple pings from a certain port + source IP of 1.1.1.1 to destination IP of 2.2.2.2 + destination port number 12345.
I am looking for something like the foll:
**** ping DIP D.Port SIP S.Port -count 1000 ****
Please note: I need to run this ping from my windows CMD.
Is there a way I can do this from my windows CMD line?
check your system docs, on mine it is:
Mac_3.2.57$ping -S 10.0.0.148 -c 100000 1.1.1.1
PING 1.1.1.1 (1.1.1.1) from 10.0.0.148: 56 data bytes
64 bytes from 1.1.1.1: icmp_seq=0 ttl=59 time=25.521 ms
64 bytes from 1.1.1.1: icmp_seq=1 ttl=59 time=18.837 ms
64 bytes from 1.1.1.1: icmp_seq=2 ttl=59 time=16.605 ms

ssh client to show server-supported algorithms

In order to check that all the servers across a fleet aren't supporting deprecated algorithms, I'm (programmatically) doing this:
telnet localhost 22
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
SSH-2.0-OpenSSH_8.0p1 Ubuntu-6build1
SSH-2.0-Censor-SSH2
4&m����&F �V��curve25519-sha256,curve25519-sha256#libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1Arsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256,ssh-ed25519lchacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.comlchacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.com�umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1�umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1none,zlib#openssh.comnone,zlib#openssh.comSSH-2.0-Censor-SSH2
Connection closed by foreign host.
Which is supposed to be a list of supported algorithms for the various phases of setting up a connection. (kex, host key, etc). Every time I run, I get a different piece of odd data at the start - always a different length.
There's an nmap plugin - ssh2-enum-algos - which returns the data in it's complete form, but I don't want to run nmap; I have a go program which opens the port, and sends the query, but it gets the same as telnet. What am I missing, and how do I fix it?
For comparison, here's the top few lines from the output of nmap script:
$ nmap --script ssh2-enum-algos super
Starting Nmap 7.80 ( https://nmap.org ) at 2019-12-27 22:15 GMT
Nmap scan report for super (192.168.50.1)
Host is up (0.0051s latency).
rDNS record for 192.168.50.1: supermaster
Not shown: 998 closed ports
PORT STATE SERVICE
22/tcp open ssh
| ssh2-enum-algos:
| kex_algorithms: (12)
| curve25519-sha256
| curve25519-sha256#libssh.org
| ecdh-sha2-nistp256
| ecdh-sha2-nistp384
| ecdh-sha2-nistp521
Opening a tcp connection to port 22, (in golang, with net.Dial) then accepting and sending connection strings leaves us able to Read() from the Reader for the connection. Thence the data is in a standard format described by the RFC. From this, I can list the algorithms supported in each phase of an ssh connection. This is very useful for measuring what is being offered, rather than what the appears to be configured (it's easy to configure sshd to use a different config file).
It's a useful thing to be able to do from a security POV.
Tested on every version of ssh I can find from 1.x on a very old solaris or AIX box, to RHEL 8.1.
In some cases you can specify an algorithm to use, and if you specify one that is not supported the server will reply with a list of supported algorithms.
For example, to check for supported key exchange algorithms you can use:
ssh 127.0.0.1 -oKexAlgorithms=diffie-hellman-group1-sha1
diffie-hellman-group1-sha1 is insecure and should be missing from most modern servers. The server will probably respond with something like:
Unable to negotiate with 127.0.0.1 port 22: no matching key exchange method found. Their offer: curve25519-sha256,curve25519-sha256#libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
Exit 255
Typing: "ssh -Q cipher | cipher-auth | mac | kex | key"
will give you a list of the algorithms supported by your client
Typing: "man ssh"
will let you see what options you can specify with the -o argument, including Cipher, MACs, and KexAlgorithms

How to make script in bash aware that a server is still busy installing/configuring and wait for reboot?

The issue / dilemma
I am currently busy creating a script to kickstart servers (with CentOS 6.x and CentOS 7.x) remotely. So far the script is working, but hangs on one minor thing. Well actually it does not hang, but it does not give detailed information about what is happening. In other words, I am not getting the correct information back in bash about the job being finished correctly.
I have tried various things, however it's hanging with the following message (which is being repeated endlessly):
servername is still installing and configuring packages...
PING 100.125.150.175 (100.125.150.175) 56(84) bytes of data.
64 bytes from 100.125.150.175: icmp_seq=1 ttl=63 time=0.152 ms
64 bytes from 100.125.150.175: icmp_seq=2 ttl=63 time=0.157 ms
64 bytes from 100.125.150.175: icmp_seq=3 ttl=63 time=0.157 ms
64 bytes from 100.125.150.175: icmp_seq=4 ttl=63 time=0.143 ms
64 bytes from 100.125.150.175: icmp_seq=5 ttl=63 time=0.182 ms
--- 100.125.150.175 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 120025ms
rtt min/avg/max/mdev = 0.143/0.158/0.182/0.015 ms
servername is still installing and configuring packages...
PING 100.125.150.175 (100.125.150.175) 56(84) bytes of data.
64 bytes from 100.125.150.175: icmp_seq=1 ttl=63 time=0.153 ms
64 bytes from 100.125.150.175: icmp_seq=2 ttl=63 time=0.132 ms
64 bytes from 100.125.150.175: icmp_seq=3 ttl=63 time=0.142 ms
etc....
So for some reason it does not contine to the next line of code or does the next action. Since it's only feedback to me (or another user), it's not a majorissue. But it would be nice to get this functional and providing (detailed) information back about the current progress or what the script/server is actually doing at the moment. This is not the case for the above (last) piece of code unfortunately.
This is the current code snippet I have (yes, it's a mess):
while true;
do
#ping -c3 -i3 $HWNODEIP > /dev/null
#ping -c5 -i30 $HWNODEIP > /dev/null
ping -c5 -i30 $HWNODEIP
if [ $? -eq 1 ] || [ $? -eq 2 ] || [ $? -eq 68 ]
then
echo -e " "
echo -e "Kickstart part II also done. $HOSTNAME will be rebooted one more time."
sleep 5
######return 0
echo -e " "
printf "%s" "Waiting for $HOSTNAME to come back online: "
while ! ping -c 1 -n -w 30 $HWNODEIP &> /dev/null
do
printf "%c" "."
#sleep 10
done
echo -e " "
echo -e "Reboot is done and $HOSTNAME is back online. Performing final check. Please wait..."
sleep 10
echo -e " "
sudo /usr/local/collectHWdata.pl $HWNODEIP
ssh root#$HWNODEIP "while ! test -e /root/kickstart-DONE; do sleep 3; done; echo KICKSTART IS DONE\!"
echo -e " "
exit
else
echo -e " "
echo -e "$HOSTNAME is still installing and configuring packages..."
fi
done
Sidenote: I removed > /dev/null #5 for debugging (not that it helped)
I am guessing I am using things incorrectly and I am by no means a experienced scripter; I can only do minor stuff, but ofcourse I am doing my best. I have been fooling around with this since last week and still no result on this part.
What am I trying to achieve?
The server is rebooted after the selected CentOS version, creating partitions and setting up the network. This all works. The above snippet is after that reboot. Now it will install packages I selected, configure various things (like Nagios) and install/compile certain PERL modules. And a few other minor things.
This is done correctly in the background. I wanted to make the script (the above piece of code) that the server is still busy with installing things and such. Since I lack the knowledge to do that, I decided for a different approach; check if the server is online (in other words that it's still installing). As long as the server is online, it's still installing/configuring things obviously. After that is done, the server will reboot once more to perform the final 2 commands (as seen in my snippet). However (here is the problem) it never does those commands, though the kickstart is completely done.
So I am guessing I am doing something wrong and even might messed up things (or got confused by doing so). Maybe someone has an idea, solution or a completely different approach to tackle and fix this problem (or at least I hope so).
Other things I have tried so far? Well I tried a various of ping commands and I also tried nc (netcat) but also without a good result. I every single time hit a brick wall with the last 2 commands and it keeps pinging instead of showing that the kickstart was done... I think I have spend several hours (since last week) on this already without getting anywhere.
So I am hoping someone can take a look at this and tell me what I am doing wrong and maybe there is a better approach (other than pinging a server) to see if it's still busy. Maybe a (remote) check on yum, perl or a service, so that the script knows it's still busy.
Sorry for the long post, but I know when I provide as much information as possible including code examples and results, this is more "appreciated". So I am hoping I provided adequate information. If not, let me know. I will try to add as much information as I can. As always I am always willing to learn or change my approach.
Thank you already for reading my post!
As noted in the comments under the question:
The server may already be rebooted by the time ping -c5 -i30 $HWNODEIP finishes. The command sends 5 packets (-c flag), waiting 30 seconds between each packet (-i interval flag). So thats's 5*30 = 150 seconds, which is a bit more than 2 minutes. A server could reboot just fine within 2 minutes, especially if there's SSD in use. So try lowering the total time it would take this command to complete.
[ $? -eq 68 ] is probably unnecessary. $HWNODEIP is just ip address, and exit code 68 is for domain name not being resolved, which doesn't apply to IP addresses.
The if statement could be simplified to
if ! ping -c5 -i30 "$HWNODEIP"
These are minor suggestions,probably not bulletproof. As confirmed by OP in the comments, lowering interval helps. There's other small improvements that could be done (like quoting variables), but that's outside the scope of the question, so I'll leave it for now.

How can I specify which protocol to use (IPv4 or IPv6) when pinging a website (bash)?

I currently have a shell script which simply takes a URL as an argument and then sends a ping request to it as follows:
ping -c 5 $1
It is required of me to ping to the site using IPv4 and IPv6 where possible, I will then compare results. I have read the man page of ping and cannot see a flag which specifies which protocol to use, I was expecting it to accept a flag -4 for IPv4 and -6 for IPv6 but this does not seem to be the case.
I came across the DNS lookup utility dig which looks promising but have not managed to implement it in my code. My script must take a URL as an argument and no other arguments. I hope this is clear and thanks for your help.
Use ping and ping6 that are available in most distributions.
/tmp $ dig google.com A google.com AAAA +short
172.217.4.174
2607:f8b0:4007:801::200e
/tmp $ ping -c 2 172.217.4.174
PING 172.217.4.174 (172.217.4.174): 56 data bytes
64 bytes from 172.217.4.174: icmp_seq=0 ttl=53 time=35.619 ms
64 bytes from 172.217.4.174: icmp_seq=1 ttl=53 time=34.220 ms
/tmp $ ping6 -c 2 2607:f8b0:4007:801::200e
PING6(56=40+8+8 bytes) 2602:306:b826:68a0:f40e:abca:efdb:71f --> 2607:f8b0:4007:801::200e
16 bytes from 2607:f8b0:4007:801::200e, icmp_seq=0 hlim=55 time=77.735 ms
16 bytes from 2607:f8b0:4007:801::200e, icmp_seq=1 hlim=55 time=81.518 ms

Windows 7 ping general failure

I'm trying to understand the behaviour of ping command. Trying to experiment on a windows 7 PC.
On the command prompt, I issued the following command:
ping <some hostname> -l 4096
The output I get is
Pinging <some hostname> [xx.xx.xxx.xx] with 4096 bytes of data:
General failure.
General failure.
General failure.
General failure.
Ping statistics for xx.xx.xxx.xx:
Packets: Sent = 4, Received = 0, Lost = 4 (100% loss),
However, ping <same hostname> -l 32 works just fine.
So my question is why is the server behaving differently for different packet sizes? Is it related to thwart? Or is that my local ping program is configured by default in such a way so as to not sent bigger packets?
Note that -l flag lets you specify the ping req's buffer size.
Your ping packet is probably larger than the local media's MTU, and it's on a network type where fragmentation isn't allowed. Ethernet IPv6 would be one such configuration.

Resources