I can the following command on a machine (let's call it machine A).
find /foo/bar -name "*" -type f -exec md5sum {} + | awk '{print $1}' | sort
This command lists the md5s of the files under /foo/bar.
However, when I append this command into an ssh command
ssh -i ~/.ssh/my_key my_user#123.123.123.123 'find /foo/bar -name "*" -type f -exec md5sum {} + | awk '{print $1}' | sort'
It generates the following error
awk: cmd. line:1: {print
awk: cmd. line:1: ^ unexpected newline or end of string
123.123.123.123 is the IP address of machine A. The ssh command is run from another machine.
Is it possible to append the command into the ssh command?
I've tried the following command, it doesn't work, either.
ssh -i ~/.ssh/my_key my_user#123.123.123.123 'find /foo/bar -name "*" -type f -exec md5sum {} + | awk \'{print $1}\' | sort'
The single quote after awk terminates the opening single quote before find.
In your case, there is no parameter expansion being done in your command. Hence the easiest would be to use double quotes for the ssh command.
ssh -i ~/.ssh/my_key my_user#123.123.123.123 "find /foo/bar -name '*' -type f -exec md5sum {} + | awk '{print $1}' | sort"
An alternative would be to replace each ' around the awk argument by '"'"', but this is awkward to read.
Single quotes cannot appear (even escaped) inside a single-quoted string. But they can appear (properly escaped) in an ANSI-quoted string.
ssh -i ~/.ssh/my_key my_user#123.123.123.123 \
$'find /foo/bar -name "*" -type f -exec md5sum {} + | awk \'{print $1}\' | sort'
Related
I have crafted this sed command which looks to be working fine, only it's being applied to all the files in my directory :
find . -type f -name '*.js' -not -path './node_modules/*' -exec sed -i .bak -E '
1i\
const env = require('\''env-var'\'');
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+) \|\| ([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'').default('\''\3'\'')\4/g
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'')\3/g
' {} \;
I wish to apply those transformations only to the files which match this grep command :
grep -r "process\.env\." --exclude-dir=node_modules
I tried using the pipe but I can't make the two working together. What's the right way to handle it?
EDIT: I tried this
➜ app-service git:(chore/adding-env-example) ✗ grep -r "process\.env\." --exclude-dir=node_modules | sed -i .bak -E '
1i\
const env = require('\''env-var'\'');
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+) \|\| ([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'').default('\''\3'\'')\4/g
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'')\3/g
' {} \;
sed: {}: No such file or directory
I want only the files containing process.env.SOMETHING to be edited.
Work with pipes. xargs comes handy:
find ... -print |
xargs -d '\n' grep -l 'regex' |
xargs -d '\n' sed 'stuff'
xargs: illegal option -- d
You can:
install GNU xargs
install GNU parallel
write a bash loop to read the files line by line, see https://mywiki.wooledge.org/BashFAQ/001
make sure your files do not have spaces or tabs or newlines in filnames and just remove -d '\n' option.
Suggesting to reverse order of commands. sed on filtered list of files.
Files filter is combination of grep filter on find filter: grep -l "process\.env\." $(find . -type f -name '*.js' -not -path './node_modules/*').
sed -i .bak -E '
1i\
const env = require('\''env-var'\'');
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+) \|\| ([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'').default('\''\3'\'')\4/g
s/(^|[^[:alnum:]_])process\.env\.([[:alnum:]_]+)($|[^[:alnum:]_])/\1env.get('\''\2'\'')\3/g
' $(grep -l "process\.env\." $(find . -type f -name '*.js' -not -path './node_modules/*'))
I have a command that will find all .png image files on my linux machine, but is there something I can add to it that will also copy each one over to a mapped drive / location I have set up?
Code to find .png is here.
find / -type f -exec file --mime-type {} \; | awk '{if ($NF == "image/png") print $0 }'
The mapped drive on the machine is as follows, as reported via df -h
/mnt/nas
If you need to check the MIME type, after your command, extract a file name part by using sed and execute cp by using xargs.
find / -type f -exec file --mime-type {} \; | awk '{if ($NF == "image/png") print $0 }' \
| sed 's/:.\+$//' | xargs -i cp -p {} /mnt/nas
If you can determine the file type by the extension .png, the command is below:
find / -name '*.png' -exec cp -p {} /mnt/nas \;
or
find / -name '*.png' | xargs -i cp -p {} /mnt/nas
Note that I checked the commands on GNU bash only.
It's a bit strange that you run file --mime-type to determine whether the file is a png. Couldn't you just look at the file extension like so?
find / -type f -iname '*.png'
find uses short-circuit evaluation. If one of its test (e.g. -iname, but also -exec) fails, then the subsequent tests will be ignored. Therefore you can use
find / -type f -iname '*.png' -exec cp {} /mnt/nas \;
For faster execution you might want to switch to -exec cp -t /mnt/nas {} + if your cp supports -t.
If you want to stick to file --mime-type, use
find / -type f -exec sh -c \
'file -b --mime-type "$0" | grep -qx image/png && cp "$0" /mnt/nas' {} \;
Both of these approaches correctly handle all filenames, even such with linebreaks in them.
You could adjust your original command by printing the filename instead of the whole line in awk. Then copy the file with xargs (you can also use the awk system subcommand)
find / -type f -exec file --mime-type {} \; | \
awk '{if ($NF == "image/png") print $1}' \
xargs -I% cp % /mnt/nas
Note that the print of the filename in awk is brittle because the awk default delimiter is space/tabs. And the filename may contain space.
I would specify the ': ' delimiter since that is which one used by file --mime-type (and also not need a if, awk has a straight way for that case)
find / -type f -exec file --mime-type {} \; | \
awk -F ': ' '$NF=="image/png" {print $1}' \
xargs -I% cp % /mnt/nas
If ":" is used in the filenames you could use any other separator such as #####
my command was this
ls -l|grep "\-[r,-][w,-]x*"|tr -s " " | cut -d" " -f9
but for the result I get all the files, not only the ones for which user has a right to execute ( the first x bit is set on).
I'm running linux ubuntu
You can use find with the -perm option:
find . -maxdepth 1 -type f -perm -u+x
OK -- if you MUST use grep:
ls -l | grep '^[^d]..[sx]' | awk '{ print $9 }'
Don't use grep. If you want to know if a file is executable, use test -x. To check all files in the current directory, use find or a for loop:
for f in *; do test -f "$f" -a -x "$f" && echo "$f"; done
or
find . -maxdepth 1 -type f -exec test -x {} \; -print
Use awk with match
ls -l|awk 'match($1,/^...x/) {print $9}'
match($1,/^...x/): match first field for the regular expression ^...x, ie search for owner permission ending in x.
I am trying to get the application version using the below command
#!/bin/sh
appVersion=$(ssh username#server find '/dir1/dir2/dir3' -type f -name "file.json" -exec grep "version" {} \;| awk -F ': ' '{print $2}' | sed 's/\"//g')
echo $appVersion
Unfortunately am getting the below exception
find: missing argument to `-exec'
Please help me to resolve this issue.
Run the below script, which will behave as expected.
When you run command over ssh, you should use single quote for the multiple commands
#!/bin/sh
appVersion=$(ssh username#server 'find '/dir1/dir2/dir3' -type f -name "file.json" -exec grep "version" {} \;| awk -F ': ' '{print $2}' | sed 's/\"//g'')
echo $appVersion
Refer this link for further details
The below example shows the way how I need the file search and output type which works well in local find.
> find /DBBACKMEUP/ -not -name "localhost*" -type f -name "*2012-10-26*" -exec du -b {} \; | awk '{print $2 "\t" $1}' | awk -F'/' '{print $NF}'
monitor_2012-10-26_22h00m.11.29.135.Friday.sql.gz 119601
test_2012-10-26_22h00m.10.135.Friday.sql.gz 530
status_2012-10-26_22h00m.1.29.135.Friday.sql.gz 944
But I need to print the same command on many servers. So I have planned to exec like this.
>ssh root#192.168.87.80 "find /DBBACKMEUP/ -not -name "localhost*" -type f -name "*2012-10-26*" -exec du -b {} \; | awk '{print $2 "\t" $1}' | awk -F'/' '{print $NF}'"
Ofcourse this gives be a blank output. Any way to parse such a search string in shell and get the output that I desire by ssh?
Thanks!!
Looks like your ssh command there has lots of quotes and double-quotes, which may be the root of your problem (no pun intended). I'd recommend that you create a shell script that will run the find command you desire, them place a copy of it on each server. After that, simply use ssh to execute that shell script instead of trying to pass in a complex command.
Edit:
I think I misunderstood; please correct me if I'm wrong. Are you looking for a way to create a loop that will run the command on a range of IP addresses? If so, here's a recommendation - create a shell script like this:
#!/bin/bash
for ((C=0; C<255; C++)) ; do
for ((D=0; D<255; D++)) ; do
IP="192.168.$C.$D"
ssh root#$IP "find /DBBACKMEUP/ -not -name "localhost*" -type f -name "*2012-10-26*" -exec du -b {} \; | awk '{print "\$"2 \"\\t\" "\$"1}' | awk -F'/' '{print "\$"NF}'"
done
done
Each server?? That must be 749 servers - Your option goes good for hardworkers.. my approach goes good for lazy goose ;) Just a trial did the click ;)
ssh root#192.168.47.203 "find /DBBACKMEUP/ -not -name "localhost*" -type f -name "*2012-10-26*" -exec du -b {} \; | awk '{print "\$"2 \"\\t\" "\$"1}' | awk -F'/' '{print "\$"NF}'"
Tel_Avaya_Log_2012-10-26_22h00m.105.23.Friday.sql.gz 2119
test_2012-10-26_22h00m.10.25.Friday.sql.gz 529
OBD_2012-10-26_22h00m.103.2.203.Friday.sql.gz 914