Grep into a file on remote host and scp to local - bash

I am trying to find all the datanodes in a remote host, write them into a .txt file and copy it back into my local machine.
I have used the following commands:
# Port forwarding to remote host and scp back to local.
ssh -f user#remoteHost -L 22222:remoteHost-1:22 -N &&
ssh -t -p 22222 user#localhost "consul members | grep data | awk '{ print $2 > /tmp/data_nodes }'" &&
scp -t -p 22222 user#localhost:/tmp/data_nodes.txt .
This doesn't work somehow. The data_nodes file is not getting created inside /tmp directory.
Any help will be greatly appreciated. Thanks.

If I understand correctly you want to run a remote command and get its output, then just do
ssh <opts> "cmd1 | cmd2 | cmd3" > /path/to/local.file

Related

bash script to access a file in a remote host three layers deep

So in the terminal I access the remote host through ssh -p then once I'm in i have to cd /directory1/directory2/. Then I want to find the latest directory which I do using ls -td -- */ | head -n 1 then using this I want to cd into that and tail -n 1 file1
All these commands work in the terminal but I want to automate it to where I can just type ./tailer.sh and have that be output.
Any ideas would be appreciated.
The shell script tailer.sh can look something like this
#!/bin/bash
ssh -p <PORT> <HOST_NAME> '( cd /directory1/directory2/ && LATEST_DIR=$(ls -td -- */ | head -n 1) && cd ${LATEST_DIR} && tail -n 1 file1 )'
Then give execute permissions to tailer.sh using chmod u+x tailer.sh
Run the script using ./tailer.sh

Turn off the return message from the executed command

I'm developing a bash script, I've used ssh command in my bash script to run some commands on a remote server and I need to get the result from the command which runs on the remote server. so I wrote this code:
db="$(ssh -t user#host 'mysql --user=username -ppassword -e \"SHOW DATABASES;\" | grep -Ev \"(Database|information_schema|performance_schema)\"' | grep -Ev \"(mysql)\")"
But each time which I run my bash script, I will get Connection to host closed. in first of the db result. this is a default message from ssh command.
Also, If I use > /dev/null 2>&1 end of my command the db variable would be empty.
How can I turn off the return message from the executed command?
Like this :
#!/bin/bash
db=$(
ssh -t user#host bash<<EOF
mysql --user=username -ppassword -e "SHOW DATABASES" |
grep -Ev "(Database|information_schema|performance_schema|mysql)" \
2> >(grep -v 'Connection to host closed')
EOF
)
or if Connection to host closed comes from STDOUT :
...
mysql --user=username -ppassword -e "SHOW DATABASES" |
grep -Ev "(Database|information_schema|performance_schema|mysql|Connection to host closed)"
...

Scape quotes on remote command

I'm try to pass a commadn on remote server.
Command work fine on local server, but when try pass on remote server trought ssh get error for bad scpaing
ls -t /root/mysql/*.sql | awk 'NR>2 {system(\"rm \"" $0 \"\"")}'
Full comnand
ssh root#host -p XXX "mysqldump --opt --all-databases > /root/mysql/$(date +%Y%m%d%H%M%S).sql;ls -t /root/mysql/*.sql | awk 'NR>2 {system(\"rm \"" $0 \"\"")}'"
Actually no need to use awk and avoid all that quotes escaping:
ls -t /root/mysql/*.sql | tail -n +1 | xargs rm
This is assuming your *.sql files don't have any whitespaces otherwise you should use stat command and sort the output using sort.

Shell script to grep logs on different host and write the grepped output to a file on Host 1

Shell script needs to
ssh to Host2 from Host1
cd /test/test1/log
grep logs.txt for string error
write the grepped output to a file
and move that file to Host1
This can be accomplished by specifying the -f option to ssh:
ssh user#host -f 'echo "this is a logfile">logfile.txt'
ssh user#host -f 'grep logfile logfile.txt' > locallogfile.txt
cat locallogfile.txt
An example using a different directory and cd changing directories to it:
ssh user#host -f 'mkdir -p foo/bar'
ssh user#host -f 'cd foo/bar ; echo "this is a logfile">logfile.txt'
ssh user#host -f 'cd foo/bar ; echo "this is a logfile">logfile.txt'
ssh user#host -f 'cd foo/bar ; grep logfile logfile.txt' > locallogfile.txt
cat locallogfile.txt

Waiting for input from script that is running remotely via ssh

There is a script I'm running that I can not install on the remote machine.
clear && printf '\e[3J'
read -p "Please enter device: " pattern
read -p "Enter date: (YYYY-MM-DD): " date
pfix=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 5 | head -n 1)
mkdir /home/user/logCollectRes/"${pfix}"
ssh xxx.xxx.xxx.xxx 'bash -s' < /usr/local/bin/SearchAdvanced.sh ${pattern} ${date} ${pfix}
In that script, I would like to be able to use read.
ls -g *"${pattern}"*
read -p "Select one of these? [y/n] " "found";
I've tried adding the -n on the read as well as the -t -t option on ssh. As you can see the script presents information that is only seen once the script starts, so I can't use the read on local machine.
EDIT: So lets say server B stores syslogs for 5K computers. The file names are given by using the internal IP of the device and the date at the end.
/var/log/remotes/192.168.1.500201505050736.gz
/var/log/remotes/192.168.1.500201505050936.gz
/var/log/remotes/192.168.1.500201505051136.gz
/var/log/remotes/192.168.1.600201505050836.gz
/var/log/remotes/192.168.1.600201505051036.gz
/var/log/remotes/192.168.1.600201505051236.gz
I'd like to be able to select the IP address from the main script, list all the files matching that IP address, and then select which I want to scp to my local machine.
After speaking with some coworkers I found the answer to be running two scripts: The first pulls the ls -g result and directs the answer to a variable on the local machine. I then print that output with the read option of selecting on of the files. The 2nd script will take that answer and scp the file from the remote machine
In the main script
ssh xxx.xxx.xxx.xxx 'bash -s' < /usr/local/bin/SearchAdvanced.sh ${pattern} ${date} > ${result}
then as a follow up
printf "${result}"
read -p "Select file: "

Resources