I have a script that logs in to a remote host to pull a directory listing to later present options to the user. It was all working perfectly, until some of the directories started having spaces in them. I have tried several syntaxes and googled the life out of this and I am now at the end of my tether. The original command was this:
SERVERDIRS=($(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -l --time-style="long-iso" $FROMFOLDER | egrep '^d' | awk '{print $8}'))
I first off changed this code to be able to read the spaces like this:
SERVERDIRS=($(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -l --time-style="long-iso" $FROMFOLDER | egrep '^d' | cut -d' ' -f8-))
However This resulted in each word being recognised as a variable. I have tried many ways to try to solve this, two of which were:
SERVERDIRS=($(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -d $FROMFOLDER* |rev| cut -d'/' -f1|rev|sed s/^/\"/g|sed s/$/\"/g))
SERVERDIRS=($(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -d $FROMFOLDER* |rev| cut -d'/' -f1|rev|sed 's/ /\\ /g'))
SERVERDIRS=(`sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -d $FROMFOLDER* |rev| cut -d'/' -f1|rev|sed 's/ /\\ /g'`)
How can I resolve these directories in to separate elements correctly?
If you're trying to read one array value per line instead of space-separated, then $() syntax won't help. Try readarray (Bash 4):
readarray SERVERDIRS < <(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -l --time-style="long-iso" $FROMFOLDER | egrep '^d' | cut -d' ' -f8-)
or assign IFS and read with -d, -r, and -a set:
IFS=$'\n' read -d '' -r -a SERVERDIRS < <(sshpass -p $PASS ssh -oStrictHostKeyChecking=no $USER#$SERVER ls -l --time-style="long-iso" $FROMFOLDER | egrep '^d' | cut -d' ' -f8-)
or, really, any other answer to this SO question.
If you're unfamiliar with <() syntax, it's known as process substitution and will allow your variable to be set in your current environment rather than the instantly-discarded subshell that a pipe would create.
Bear in mind that this process is a little dangerous; filenames can also contain newlines, so it's usually much preferred to use find ... -print0.
If you only need to list directories, try this
ls -d /usr/local/src/*/
or
ls -d /path/to/your/directory/*/
You can then loop through all directories
#!/bin/bash
aa=`ls -d /usr/local/src/*/`
for dir in "${aa}[#]"
do
echo "$dir"
done
This works if dir names contain spaces.
Related
I don't get why this doesn't work:
filesToInclude="$(ssh -t $host ls -t /var/log/*.LOG | sort | egrep -A6 "$LastBootUp" | tr '\n' '[:space:]' | tr -s [:space:] ' ')"
allALL="$( ssh $host grep -Ev "$excludeSearch" $filesToInclude )"
on another server, which is capable of ag this works totally fine.
if I copy the output of filesToInclude to $filesToInclude manually, it works.
that is the output:
grep: o such file or directory
bash: 0m/var/log/A-MINI_23311_H007164M49_220419_1906_XX.LOG: No such file or directory
I'm working on a script, that should find certain disks and add hostname to them.
I'm using this for 40 servers with a for loop in bash
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i findmnt -o SIZE,TARGET -n -l |
grep '1.8T\|1.6T\|1.7T' |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=$HOSTNAME '{print HOSTNAME ":" $0}'; done |
tee sorted.log
can you help out with the quoting here? It looks like awk gets piped (hostname) from localhost, not the remote server.
Everything after the first pipe is running locally, not on the remote server.
Try quoting the entire pipeline to have it run on the remote server:
#!/bin/bash
for i in myservers{1..40}
do ssh user#$i "findmnt -o SIZE,TARGET -n -l |
sed 's/^[ \t]*//' |
cut -d ' ' -f 2 |
awk -v HOSTNAME=\$HOSTNAME '{print HOSTNAME \":\" \$0}'" ;
done | tee sorted.log
This is a shorter version of your stuff:
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '/M/{print HOSTNAME ":" $2}'
Applied to the above:
for i in myservers{1..40}
do ssh user#$i bash -c '
findmnt -o SIZE,TARGET -n -l |
awk -v HOSTNAME=$HOSTNAME '"'"'/M/{print HOSTNAME ":" $2}'"'"' '
done |
tee sorted.log
see: How to escape the single quote character in an ssh / remote bash command?
I trying to add text (predefined) between a sorted output and saved to a new file.
I'm using a curl command to gather my info.
$ curl --user XXX:1234!## "http://......"
Then using grep to find IP addresses and sorting so they only appear once.
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
I need to add <my_text_predefined> ([0-9]{1,3}[\.]){3}[0-9]{1,3} <my_text_predefined> between the regex ip address and then saved to a new file.
The script below only get my the ip address
$ curl --user XXX:1234!## "http://......" | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u
123.12.0.12
123.56.98.76
$ curl --user some_user:password "http://...." | grep -E -o -m1 '([0-9]{1,3}[\.]){3}[0-9]{1,3}' | sort -u | sed 's/.*/<prefix> -s & <suffix>/'
So if we need print some text for each IP ... try xargs
for i in {1..100}; do echo $i; done | xargs -n1 echo "Values are:"
if based on IP you would need to take decision put in a loop
for file $(curl ...) do ...
and check $file or do something with it ...
Here's the command I'm currently running:
curl 'http://test.com/?id=12345' | grep -o -P '(?<=content="2;url=).*?(?=")'
The response from this command is a URL, like this:
$ curl 'http://test.com/?id=12345' | grep -o -P '(?<=content="2;url=).*?(?=")'
http://google.com
I want to use whatever that URL is to essentially do this:
curl 'http://test.com/?id=12345' | grep -o -P '(?<=content="2;url=).*?(?=")' | curl 'http://google.com'
Is there any simple way to do this all in one line?
Use xargs with a place holder for the output from stdin with the -I{} flag as below. The -r flag is to ensure the curl command is not invoked on a empty output from previous grep output.
curl 'http://test.com/?id=12345' | grep -o -P '(?<=content="2;url=).*?(?=")' | xargs -r -I{} curl {}
A small description about the flags, -I and -r from the GNU xargs man page,
-I replace-str
Replace occurrences of replace-str in the initial-arguments with
names read from standard input.
-r, --no-run-if-empty
If the standard input does not contain any nonblanks, do not run
the command. Normally, the command is run once even if there is
no input. This option is a GNU extension
(or) if you are looking for a bash approach without other tools,
curl 'http://test.com/?id=12345' | grep -o -P '(?<=content="2;url=).*?(?=")' | while read line; do [ ! -z "$line" ] && curl "$line"; done
I'm try to pass a commadn on remote server.
Command work fine on local server, but when try pass on remote server trought ssh get error for bad scpaing
ls -t /root/mysql/*.sql | awk 'NR>2 {system(\"rm \"" $0 \"\"")}'
Full comnand
ssh root#host -p XXX "mysqldump --opt --all-databases > /root/mysql/$(date +%Y%m%d%H%M%S).sql;ls -t /root/mysql/*.sql | awk 'NR>2 {system(\"rm \"" $0 \"\"")}'"
Actually no need to use awk and avoid all that quotes escaping:
ls -t /root/mysql/*.sql | tail -n +1 | xargs rm
This is assuming your *.sql files don't have any whitespaces otherwise you should use stat command and sort the output using sort.