My code is supposed to SSH to a remote-host (let’s say Routers) and run multiple commands on the remote-host and return the outputs.
The code attached is simplified and has three parts:
Main function: Reads list of commands and then by using the ExecCommands function dials/ssh to a remote-host to execute the commands.
ExecCommands function takes the remote-host IP, list of commands and SSH ClientConfig that is used for SSH. Then it dials to the IP and run the commands one-by-one. At the end, returns the output of all commands in only one string
InsecureClientConfig function that actually doesn’t do much except creating a SSH ClientConfig which is used for ExecCommands function
This program works well when I just want to apply some commands or config and save the wholes result. I mean ExecCommands takes the bunch of commands, push all of them to the remote-host and returns (or saves) the whole output of applied commands in one string as output.
Problem:
I cannot process the output of each command individually. For example, assume that I apply CMD1, CMD2, CMD3, … to the remote-host#1 by using ExecCommands function. Since it gives me back the whole output in one string, it is hard to find which output belongs to which CMD
Goal:
Modify or re-design ExecCommands function to the way that it provides separate output for each command it applies. It means if for remote-host#1 it applies 10 commands, I should have 10 separate strings as output.
Conditions/Restrictions:
I can not create any extra session for commands and must apply all commands in the first SSH session I created, i.e. cannot create multiple Sessions and use Run, Shell, Output, Start function in SSH package
No re-authentication is allowed. For example, I have only a single one-time-password that can be used for all remote-hosts.
Remote hosts don't support "echo" like commands similar to what you have in Linux
The remote-hosts dont’s support any type of APIs
Points:
Main focus is the function ExecCommands. I put a simplified version of the whole code to give an idea
I am using stdout, err := session.StdoutPipe() to run multiple commands which means -as pipe - it's Reader only is possible to be read when the job is done.
An option is to use Session.Stdout and Session.Stdin inside of the for loop in ExecCommands function. Tried but was not successful.
Code:
package main
import (
"errors"
"fmt"
"io/ioutil"
"log"
"time"
"golang.org/x/crypto/ssh"
)
func main() {
// List of the commands should be sent to the devices
listCMDs := []string{
"set cli op-command-xml-output on",
"test routing fib-lookup virtual-router default ip 1.1.1.1",
"test routing fib-lookup virtual-router default ip 2.2.2.2",
"show interface ethernet1/1",
"show interface ethernet1/2",
"test security-policy-match protocol 6 source 1.1.1.1 destination 2.2.2.2 destination-port 443 from ZONE1 to ZONE2",
"test security-policy-match protocol 6 source 10.0.0.1 destination 10.0.2.1 destination-port 443 from ZONE1 to ZONE2",
"exit",
}
sshconfig := InsecureClientConfig("admin", "admin")
s, err := ExecCommands("192.168.1.250", listCMDs, sshconfig)
fmt.Println(s, err)
}
// ExecCommands ...
func ExecCommands(ipAddr string, commands []string, sshconfig *ssh.ClientConfig) (string, error) {
// Gets IP, credentials and config/commands, SSH Config (Timeout, Ciphers, ...) and returns
// output of the device as "string" and an error. If error == nil, means program was able to SSH with no issue
// Creating outerr as Output Error.
outerr := errors.New("nil")
outerr = nil
// Creating Output as String
var outputStr string
// Dial to the remote-host
client, err := ssh.Dial("tcp", ipAddr+":22", sshconfig)
if err != nil {
log.Fatal(err)
}
defer client.Close()
// Create sesssion
session, err := client.NewSession()
if err != nil {
log.Fatal(err)
}
defer session.Close()
// StdinPipee() returns a pipe that will be connected to the remote command's standard input when the command starts.
// StdoutPipe() returns a pipe that will be connected to the remote command's standard output when the command starts.
stdin, err := session.StdinPipe()
if err != nil {
log.Fatal(err)
}
stdout, err := session.StdoutPipe()
if err != nil {
log.Fatal(err)
}
// Start remote shell
err = session.Shell()
if err != nil {
log.Fatal(err)
}
// Send the commands to the remotehost one by one.
for _, cmd := range commands {
_, err := stdin.Write([]byte(cmd + "\n"))
if err != nil {
log.Fatal(err)
}
}
// Wait for session to finish
err = session.Wait()
if err != nil {
log.Fatal(err)
}
strByte, _ := ioutil.ReadAll(stdout)
outputStr = string(strByte)
return outputStr, outerr
}
// InsecureClientConfig ...
func InsecureClientConfig(userStr, passStr string) *ssh.ClientConfig {
SSHconfig := &ssh.ClientConfig{
User: userStr,
Timeout: 5 * time.Second,
Auth: []ssh.AuthMethod{ssh.Password(passStr)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Config: ssh.Config{
Ciphers: []string{"aes128-ctr", "aes192-ctr", "aes256-ctr", "aes128-cbc", "aes192-cbc",
"aes256-cbc", "3des-cbc", "des-cbc"},
KeyExchanges: []string{"diffie-hellman-group1-sha1",
"diffie-hellman-group-exchange-sha1",
"diffie-hellman-group14-sha1"},
},
}
return SSHconfig
}
This works properly:
package main
import (
"bufio"
"errors"
"fmt"
"log"
"time"
"golang.org/x/crypto/ssh"
)
func main() {
// List of the commands should be sent to the devices
listCMDs := []string{
"set cli op-command-xml-output on\n",
"test routing fib-lookup virtual-router default ip 1.1.1.1\n",
"test routing fib-lookup virtual-router default ip 2.2.2.2\n",
"show interface ethernet1/1\n",
"show interface ethernet1/2\n",
"test security-policy-match protocol 6 source 1.1.1.1 destination 2.2.2.2 destination-port 443 from ZONE1 to ZONE2\n",
"test security-policy-match protocol 6 source 10.0.0.1 destination 10.0.2.1 destination-port 443 from ZONE1 to ZONE2\n",
"exit",
}
sshconfig := InsecureClientConfig("admin", "Ghazanfar1!")
s, _ := ExecCommands("192.168.1.249", listCMDs, sshconfig)
for _, item := range s {
fmt.Println(item)
fmt.Println("-------------------------------")
}
}
// ExecCommands ...
func ExecCommands(ipAddr string, commands []string, sshconfig *ssh.ClientConfig) ([]string, error) {
// Gets IP, credentials and config/commands, SSH Config (Timeout, Ciphers, ...) and returns
// output of the device as "string" and an error. If error == nil, means program was able to SSH with no issue
// Creating outerr as Output Error.
outerr := errors.New("nil")
outerr = nil
// Creating Output as String
var outputStr []string
var strTmp string
// Dial to the remote-host
client, err := ssh.Dial("tcp", ipAddr+":22", sshconfig)
if err != nil {
log.Fatal(err)
}
defer client.Close()
// Create sesssion
session, err := client.NewSession()
if err != nil {
log.Fatal(err)
}
defer session.Close()
// StdinPipee() returns a pipe that will be connected to the remote command's standard input when the command starts.
// StdoutPipe() returns a pipe that will be connected to the remote command's standard output when the command starts.
stdin, err := session.StdinPipe()
if err != nil {
log.Fatal(err)
}
stdout, err := session.StdoutPipe()
if err != nil {
log.Fatal(err)
}
// Start remote shell
err = session.Shell()
if err != nil {
log.Fatal(err)
}
stdinLines := make(chan string)
go func() {
scanner := bufio.NewScanner(stdout)
for scanner.Scan() {
stdinLines <- scanner.Text()
}
if err := scanner.Err(); err != nil {
log.Printf("scanner failed: %v", err)
}
close(stdinLines)
}()
// Send the commands to the remotehost one by one.
for i, cmd := range commands {
_, err := stdin.Write([]byte(cmd + "\n"))
if err != nil {
log.Fatal(err)
}
if i == len(commands)-1 {
_ = stdin.Close() // send eof
}
// wait for command to complete
// we'll assume the moment we've gone 1 secs w/o any output that our command is done
timer := time.NewTimer(0)
InputLoop:
for {
timer.Reset(time.Second)
select {
case line, ok := <-stdinLines:
if !ok {
log.Println("Finished processing")
break InputLoop
}
strTmp += line
strTmp += "\n"
case <-timer.C:
break InputLoop
}
}
outputStr = append(outputStr, strTmp)
//log.Printf("Finished processing %v\n", cmd)
strTmp = ""
}
// Wait for session to finish
err = session.Wait()
if err != nil {
log.Fatal(err)
}
return outputStr, outerr
}
// InsecureClientConfig ...
func InsecureClientConfig(userStr, passStr string) *ssh.ClientConfig {
SSHconfig := &ssh.ClientConfig{
User: userStr,
Timeout: 5 * time.Second,
Auth: []ssh.AuthMethod{ssh.Password(passStr)},
HostKeyCallback: ssh.InsecureIgnoreHostKey(),
Config: ssh.Config{
Ciphers: []string{"aes128-ctr", "aes192-ctr", "aes256-ctr", "aes128-cbc", "aes192-cbc",
"aes256-cbc", "3des-cbc", "des-cbc"},
KeyExchanges: []string{"diffie-hellman-group1-sha1",
"diffie-hellman-group-exchange-sha1",
"diffie-hellman-group14-sha1"},
},
}
return SSHconfig
}
Since you have limited number of commands to run on special hardwares and you know the pattern of the each command's output, you may use strings.Split or regexp to split the output.
And if you do not have echo command, but know any command with fast response with unique output pattern, then you may replace it with echo command in the following example (number 2).
Since a session only accepts one call to Run, Start, Shell, Output, or CombinedOutput, and you do not want to start a new session per command:
The key is to use a strings.Builder and empty it using sb.Reset() befor sending the command, and using io.Copy to copy concurrently the session's stdout into strings.Builder (assuming you do not need session's stderr):
sb := new(strings.Builder)
go io.Copy(sb, stdout)
This works if you know how much to wait for each command (tested):
sb := new(strings.Builder)
go io.Copy(sb, stdout)
commands := []string{"uname -a", "sleep 1", "pwd", "whoami", "exit"}
wait := []time.Duration{10, 1200, 20, 10, 10} // * time.Millisecond
ans := []string{}
time.Sleep(10 * time.Millisecond) // wait for the ssh greetings
// Send the commands to the remotehost one by one.
for i, cmd := range commands {
sb.Reset()
fmt.Println("*** command:\t", cmd)
_, err := stdin.Write([]byte(cmd + "\n"))
if err != nil {
log.Fatal(err)
}
time.Sleep(wait[i] * time.Millisecond) // wait for the command to finish
s := sb.String()
fmt.Println("*** response:\t", s)
ans = append(ans, s)
}
Using string delimiter and strings.Split (Note: You may replace echo with any fast command with known output pattern):
sb := new(strings.Builder)
go io.Copy(sb, stdout)
commands := []string{"uname -a", "sleep 1", "pwd", "whoami"}
delim := "********--------========12345678"
for _, cmd := range commands {
_, err = stdin.Write([]byte("echo " + delim + "\n"))
if err != nil {
log.Fatal(err)
}
_, err := stdin.Write([]byte(cmd + "\n"))
if err != nil {
log.Fatal(err)
}
}
_, err = stdin.Write([]byte("exit\n"))
if err != nil {
log.Fatal(err)
}
err = session.Wait() // Wait for session to exit
if err != nil {
log.Fatal(err)
}
ans := strings.Split(sb.String(), delim)
ans = ans[1:] // remove ssh greetings
Check this out: https://github.com/yahoo/vssh
You can set sessions to how many commands you need to run concurrently then send each command to remote host through run method and get the result individually!
I'm using goexpect for connection to multiple wi-fi access points.
For some of them I need to use telnet, and SSH for others.
So, I need fastest way to check if telnet is open for some IP.
Code now looks like
e, _, err := expect.Spawn(fmt.Sprintf("telnet %s", ip), -1)
res, _, err := e.Expect(userRE, timeout) // we expect user prompt
if err != nil {
// if timeout, pass control to code block which handle SSH connection
}
I suppose there is a better and faster way to tell if telnet is open.
Any suggestions?
This worked for me after getting the github.com/reiver/go-telnet package:
package main
import (
"fmt"
"github.com/reiver/go-telnet"
)
func main() {
address := "127.0.0.1:8080"
_, err := telnet.DialTo(address)
if err != nil {
fmt.Println(err)
fmt.Println("Telnet closed")
} else {
fmt.Println("Telnet open")
}
}
I'm trying to execute a command that asks for several inputs for example if you try to copy a file from local device to the remote device we use scp test.txt user#domain:~/ then it asks us for the password. What I want is I want to write a go code where I provide the password in the code itself for example pass:='Secret Password'. Similarly, I have CLI command where it asks us for several things such as IP, name, etc so I need to write a code where I just declare all the values in the code itself and when I run the code it doesn't ask anything just take all the inputs from code and run CLI command in case of copying file to remote it should not ask me for password when I run my go binary it should directly copy my file to remote decide.
func main() {
cmd := exec.Command("scp", "text.txt", "user#domain:~/")
stdin, err := cmd.StdinPipe()
if err = cmd.Start(); err != nil {
log.Fatalf("failed to start command: %s", err)
}
io.WriteString(stdin, "password\n")
if err = cmd.Wait(); err != nil {
log.Fatalf("command failed: %s", err)
}
}
If I use this code it is stuck on user#domain's password:
And no file is copied to the remote device.
Solution 1
You can bypass this with printf command
cmd := "printf 'John Doe\nNew York\n35' | myInteractiveCmd"
out, err := exec.Command("bash", "-c", cmd).Output()
Solution 2
You can use io.Pipe(). Pipe creates a synchronous in-memory pipe and you can write your answers into io.Writer and your cmd will read from io.Reader.
r, w := io.Pipe()
cmd := exec.Command("myInteractiveCmd")
cmd.Stdin = r
go func() {
fmt.Fprintf(w, "John Doe\n")
fmt.Fprintf(w, "New York\n")
fmt.Fprintf(w, "35\n")
w.Close()
}()
cmd.Start()
cmd.Wait()
Testing info
To test this I wrote cmd which asks for name, city, age and writes the result in file.
reader := bufio.NewReader(os.Stdin)
fmt.Print("Name: ")
name, _ := reader.ReadString('\n')
name = strings.Trim(name, "\n")
...
One way to go about this is to use command-line flags:
package main
import (
"flag"
"fmt"
"math"
)
func main() {
var (
name = flag.String("name", "John", "Enter your name.")
ip = flag.Int("ip", 12345, "What is your ip?")
)
flag.Parse()
fmt.Println("name:", *name)
fmt.Println("ip:", *ip)
}
Now you can run the program with name and ip flags:
go run main.go -name="some random name" -ip=12345678910`
some random name
ip: 12345678910
This channel is a good resource—he used to work for the Go team and made tons of videos on developing command-line programs in the language. Good luck!
I come across this question when trying to run the linux make menuconfig through golang os/exec.
To accomplish what you are trying to achieve try to set the cmd.Stdin to os.Stdin. Here is a working example:
package main
import (
"fmt"
"os"
"os/exec"
)
type cmdWithEnv struct {
pwd string
command string
cmdArgs []string
envs []string
}
func runCommand(s cmdWithEnv) error {
cmd := exec.Command(s.command, s.cmdArgs...)
if len(s.pwd) != 0 {
cmd.Dir = s.pwd
}
env := os.Environ()
env = append(env, s.envs...)
cmd.Env = env
fmt.Printf("%v\n", cmd)
cmd.Stdout = os.Stdout
cmd.Stderr = os.Stderr
cmd.Stdin = os.Stdin // setting this allowed me to interact with ncurses interface from `make menuconfig`
err := cmd.Start()
if err != nil {
return err
}
if err := cmd.Wait(); err != nil {
return err
}
return nil
}
func buildPackage() {
makeKernelConfig := cmdWithEnv{
pwd: "linux",
command: "make",
cmdArgs: []string{"-j12", "menuconfig"},
envs: []string{"CROSS_COMPILE=ccache arm-linux-gnueabihf-", "ARCH=arm"},
}
runCommand(makeKernelConfig)
}
func main() {
buildPackage()
}
I need a way to read input from console twice (1 -from cat outupt, 2 - user inputs password), like this:
cat ./test_data/test.txt | app account import
My current code skips password input:
reader := bufio.NewReader(os.Stdin)
raw, err := ioutil.ReadAll(reader)
if err != nil {
return cli.NewExitError(err.Error(), 1)
}
wlt, err := wallet.Deserialize(string(raw))
if err != nil {
return cli.NewExitError(err.Error(), 1)
}
fmt.Print("Enter password: ")
pass := ""
fmt.Fscanln(reader, &pass)
Also tried to read password with Scanln - doesn't works.
Note:
cat (and piping at all) can't be used with user input, as shell redirects inputs totally.
So the most simple solutions are:
to pass filename as argument
redirect manually app account import < ./test.txt
Read file and password separately (and don't show password), try this:
package main
import (
"fmt"
"io/ioutil"
"log"
"github.com/howeyc/gopass"
)
func main() {
b, err := ioutil.ReadFile("./test_data/test.txt") // just pass the file name
if err != nil {
fmt.Print(err)
}
fmt.Println(string(b))
fmt.Print("Password: ")
pass, err := gopass.GetPasswd()
if err != nil {
log.Fatalln(err)
}
fmt.Println(string(pass))
}
and go get github.com/howeyc/gopass first.
This question is about golang.org/x/crypto/ssh package and maybe pseudo-terminal behaviour.
The code
Here is the demo code. You can run it on your local machine just change credentials to access SSH.
package main
import (
"bufio"
"fmt"
"golang.org/x/crypto/ssh"
"io"
)
func main() {
var pipe io.Reader
whichPipe := "error" // error or out
address := "192.168.1.62:22"
username := "username"
password := "password"
sshConfig := &ssh.ClientConfig{
User: username,
Auth: []ssh.AuthMethod{ssh.Password(password)},
}
connection, err := ssh.Dial("tcp", address, sshConfig)
if err != nil {
panic(err)
}
session, err := connection.NewSession()
if err != nil {
panic(err)
}
modes := ssh.TerminalModes{
ssh.ECHO: 0,
ssh.ECHOCTL: 0,
ssh.TTY_OP_ISPEED: 14400,
ssh.TTY_OP_OSPEED: 14400,
}
if err := session.RequestPty("xterm", 80, 0, modes); err != nil {
session.Close()
panic(err)
}
switch whichPipe {
case "error":
pipe, _ = session.StderrPipe()
case "out":
pipe, _ = session.StdoutPipe()
}
err = session.Run("whoami23")
scanner := bufio.NewScanner(pipe)
for scanner.Scan() {
fmt.Println(scanner.Text())
}
}
Actual result
Empty line
Expected result
bash: whoami23: command not found
Current "solution"
To get expected result you have two options:
Change whichPipe value to out. Yes, all errors going to stdout in case if you use tty.
Remove session.RequestPty. But in my case, I need to run sudo commands which require tty (servers are out of my control so I can't disable this requirement).
I use third way. I check err from err = session.Run("whoami23") and if it's not nil I mark content of session.StdoutPipe() as STDERR one.
But this method has limits. For example, if I run something like sudo sh -c 'uname -r; whoami23;' the whole result will be marked as error while uname -r returns output to STDOUT.
The question
While the behaviour looks logical to me (all that SSH client sees from pty is output without differentiations) I'm still not sure if I may miss something and there is a trick that allows to split these outputs.