To find some file with grep and delete them with rm I tried following command -
$ ls | grep 2019 | xargs -i rm \"{}\"
That did not work. Got the following error message -
rm: cannot remove '"2019-05-10 00:00:00-TO-2019-05-10 23:59:59_PDT_disconnection_info.csv"': No such file or directory
Looks like xargs is taking quotes literally. So, tried echoing instead of passing directly -
ls | grep 2019 | xargs -i echo \"{}\" | xargs rm
This worked.
Why does not it work without echoing?
The proper quoting is done by xargs, there is no need to quote it again. Just:
... | xargs -i rm {}
Or better, because rm accepts multiple arguments, just do:
... | xargs rm
Why does not it work without echoing?
When not used with -i, -I, -d or similar, the xargs utility handles proper quoting in input with double or single quotes or escaping with a backslash. The quotes are removed by the second xargs and rm is passed unquoted string. From man xargs:
.... xargs reads
items from the standard input, delimited by blanks (which can be
protected with double or single quotes or a backslash)
Compare:
$ echo "\e\r\t\q\e" | xargs -t echo
echo ertqe
ertqe
Also see Why you shouldn't parse the output of ls(1).
Related
I'm trying to remove all .js and .js.map files from any sub-directory of src called __tests__.
$ find . -path './src/**' -name __tests__ | # find subdirectories
> sed -E 's/([^ ]+__tests__)/\1\/*.js \1\/*.js.map/g' | # for each subdirectory, concat *.js and *.js.map
> xargs rm # remove files
This fails with the following errors:
rm: cannot remove './src/game/__tests__/*.js': No such file or directory
rm: cannot remove './src/game/__tests__/*.js.map': No such file or directory
rm: cannot remove './src/helpers/__tests__/*.js': No such file or directory
rm: cannot remove './src/helpers/__tests__/*.js.map': No such file or directory
However, if I change my xargs rm to xargs echo rm, copy and paste the output, and run it, it works.
$ find . -path './src/**' -name __tests__ | sed -E 's/([^ ]+__tests__)/\1\/*.js \1\/*.js.map/g' |
> xargs echo rm # echo command to remove files
rm ./src/game/__tests__/*.js ./src/game/__tests__/*.js.map ./src/helpers/__tests__/*.js ./src/helpers/__tests__/*.js.map
$ rm ./src/game/__tests__/*.js ./src/game/__tests__/*.js.map ./src/helpers/__tests__/*.js ./src/helpers/__tests__/*.js.map
Wrapping the output of my echo in $(...) and prepending rm results in the same error as before.
$ rm $(find . -path './src/**' -name __tests__ | sed -E 's/([^ ]+__tests__)/\1\/*.js \1\/*.js.map/g' | xargs echo rm
rm: cannot remove './src/game/__tests__/*.js': No such file or directory
rm: cannot remove './src/game/__tests__/*.js.map': No such file or directory
rm: cannot remove './src/helpers/__tests__/*.js': No such file or directory
rm: cannot remove './src/helpers/__tests__/*.js.map': No such file or directory
What am I doing wrong?
I doubt it matters, but I'm using GitBash on Windows.
First, to explain the issue: In find | sed | xargs rm, the shell only sets up communication between those programs, but it doesn't actually process the results in any way. That's a problem here because *.js needs to be expanded by a shell to replace it with a list of filenames; rm treats every argument it's given as a literal name. (This is unlike Windows, where programs do their own command-line parsing and glob expansion).
Arguably, you don't need find here at all. Consider:
shopt -s globstar # enable ** as a recursion operator
rm ./src/**/__tests__/*.js{,.map} # delete *.js and *.js.map in any __tests__ directory under src
...or, if you do want to use find, let it do the work of coming up with a list of individual files matching *.js, instead of leaving that work to happen later:
find src -regextype posix-egrep -regex '.*/__tests__/[^/]*[.]js([.]map)?' -delete
You need to have your globs (*) expanded. File name expansion is performed by the shell on UNIX, not by rm or other programs. Try:
.... | xargs -d $'\n' sh -c 'IFS=; for f; do rm -- $f; done' sh
...to explain this:
The -d $'\n' ensures that xargs splits only on newlines (not spaces!), and also stops it from treating backslashes and quotes as special.
sh -c '...' sh runs ... as a script, with sh as $0, and subsequent arguments in $1, etc; for f; will thus iterate over those arguments.
Clearing IFS with IFS= prevents string-splitting from happening when $f is used unquoted, so only glob expansion happens.
Using the -- argument to rm ensures that it treats subsequent arguments as filenames, not options, even if they start with dashes.
That said, if you have really a lot of files for each pattern, you might run into an "argument list too long", even though you are using xargs.
Another caveat is that filenames containing newlines can potentially be split into multiple names (depending on the details of the version of find you're using). A way to solve this that will work with all POSIX-compliant versions of find might be:
find ./src -type d -name __tests__ -exec sh -c '
for d; do
rm -- "$d"/*.js{,.map}
done
' sh {} +
Saying that I have two files t1 and t2, they have the same content: abc.
Now I want to delete all files, who contains the string abc.
So I tried to execute the command: grep -rl abc . | rm but it doesn't work.
Then I add xargs: grep -rl abc . | xargs rm and it works.
I can't understand clearly what xargs did.
grep puts the output as stdout. But rm cannot process data from stdin (the pipe links both).
You want instead, that the output of grep is put as argument of rm. So xargs command "convert" stdin into arguments of xargs first argument, and it call the command (the first argument).
As alternative, you could do
rm `grep -rl abc .`
or
rm $(grep -rl abc .)
But xargs handles well also the case where there are too many arguments for a single call of the command. The above command will give you shell error (argument string too long).
rm doesn't read from standard input (except when prompting, like with -i) but takes its arguments on the command line. That's what xargs does for you: read things from standard input and give them to rm as arguments.
Example with echo:
$ (echo a; echo b; date) | xargs echo
a b tor 12 apr 2018 14:18:50 CEST
I just want to batch modify the suffix of the files,but it doesn't work!
The command line I used as below:
ls *html | xargs -I{} echo "\`echo {} | sed 's/html/css/g'\`"
However, when I just used ls *html,it shows:
file1.html file2.html file3.html file4.html file5.html
used ls *html | sed 's/html/css/g',it shows as I expected!
like this:
file1.css file2.css file3.css file4.css file5.css
I work on Mac OS. Could anyone give me some suggestions?
Thans in advance.
Because the backquotes are in double quotes, it gets executed immediately by the shell and not by xargs on each file.
The result is the same as
ls *html | xargs -I{} echo "{}"
However, if you use single quotes, you run into other troubles. You end up having to do something like this:
ls *html | xargs -I{} sh -c 'echo `echo {} | sed '\''s/html/css/g'\''`'
but it gets to be a mess, and we haven't even got to the actual renaming yet.
Using a loop is a bit nicer:
for file in *html; do
newname=${file%html}css
mv "$file" "$newname"
done
Using GNU Parallel:
ls *html | parallel echo before {} after {.}.css
I am trying to write a bashrc function to add a safety check to the find command when it is paired with "-exec rm". This script cuts off everything beginning with "-exec rm" and replaces it with a "-print0" which is then passed over to sed to re-add missing quotes if the user uses a quoted experssion.
I am running into a issue where the quotes I am adding via sed are not being passed to the line with the execution of the find command.
Rewriting the find command and re-adding quotes:
FIND_VAR=$(echo "$#" | sed "s/-exec rm.*/-print0/g" | sed 's/\*.* / "&" /g' | sed 's/ "/"/g')
Running the find command with the modifications:
FIND_LIST=$(/bin/find $FIND_VAR | sed 's|\./| \./|g')
What I would like to accomplish is if the user types the following:
find ./ -type f -name "*.txt" -exec rm -rf {} \;
The command is re-written via the bashrc to run as:
find ./ -type f -name "*.txt" -print0 | sed 's|\./| \./|g'
This generates a list of files each spaced out which is passed to a modified rm function:
./file1.txt ./file2.txt ./file3.txt ...
The full function is listed below for reference:
function find () {
if echo "$#" | grep -q '-exec rm' ; then
echo "Found rm command as part of find"
FIND_VAR=$(echo "$#" | sed "s/-exec rm.*/-print0/g" | sed 's/\*.* / "&" /g' | sed 's/ "/"/g')
echo "/bin/find $FIND_VAR"
FIND_LIST=$(/bin/find $FIND_VAR | sed 's|\./| \./|g')
echo "$FIND_LIST"
echo "rm$FIND_LIST"
else
/bin/find "$#"
fi
}
Don't put a command in a string. It will not work. See http://mywiki.wooledge.org/BashFAQ/050 for more details and discussion.
You have the command in an array. Use it.
Walk the array, find the arguments you want to remove and remove them then add the arguments you want to add to the array and run the new command.
I want to find a string in some files in a recursive directory and replace it with another string.
The string I want to find is 0x04030000&0xffff0000.
The string I want to replace is 0x80863a30.
I tried some examples but grep and sed cause some errors.
Like grep says 0xffff0000 command not found
Im Using Mac OS X 10.8.5
This was the command I tried and the error I received:
localhost:~ User$ grep -rl 0x04030000&0xffff0000 /Users/Niresh/Desktop/TTT | xargs sed -i 's/0x04030000&0xffff0000/0x80863a30/g'
[4] 1166
-bash: 0xffff0000: command not found
grep: warning: recursive search of stdin
[4]+ Stopped grep -rl 0x04030000
Tried This Command Too But Not Working
localhost:~ Niresh$ mkdir TTT
localhost:~ Niresh$ echo "TTT 0x04030000&0xffff0000 bar" > TTT/bar
localhost:~ Niresh$ find TTT -type f -exec sed -i'' -e 's/0x04030000&0xffff0000/0x80863a30/g' {} \;
localhost:~ Niresh$ cat TTT/bar
TTT 0x80863a30 bar
localhost:~ Niresh$ TTT 0x80863a30 bar
-bash: TTT: command not found
but I have Tried This Too sed -i'' -e 's/0x04030000&0xffff0000/0x80863a30/g' "Here Was My Path to The Text File"
But The Command Execute Successfully but The Text Remains Still
I Read The Man Page of sed
It Shows
Bugs Multibyte characters containing a byte with value 0x5C (ASCII `\') may be
incorrectly treated as line continuation characters in arguments to the
a'',c'' and i'' commands. Multibyte characters cannot be used as
delimiters with thes'' and ``y'' commands.
BSD May 10, 2005
Your & is being interpeted by your shell. To avoid this, you should quote your search:
grep -rl '0x04030000&0xffff0000' /Users/Niresh/Desktop/TTT | xargs sed -i 's/0x04030000&0xffff0000/0x80863a30/g'
FatalError has already identified the & problem in your command line.
Also, this is a bad way to handle files, due to the Parsing LS problem. Instead of relying on xargs to capture filenames as the output of grep, you should use find. For example:
# find /Users/Niresh/Desktop/TTT -type f \
-exec grep -q '0x04030000&0xffff0000' {} \; \
-exec sed -i'' -e 's/0x04030000&0xffff0000/0x80863a30/g' {} \;
(split to multiple lines for easier reading.)
The idea here is that find will be responsible for your filenames, and will run each of the tools to do (1) the analysis and (2) the modification. Note that the grep here is probably redundant, as sed will simply make no changes to files where it doesn't find the search string.
This saves you from issues where there are special characters in your filenames (spaces or newlines or backslashes) which would be misinterpreted by your pipe to xargs.
UPDATE (per comments):
Works for me:
ghoti#mac:~ 507$ mkdir foo
ghoti#mac:~ 508$ echo "foo 0x04030000&0xffff0000 bar" > foo/bar
ghoti#mac:~ 509$ find foo -type f -exec sed -i'' -e 's/0x04030000&0xffff0000/0x80863a30/g' {} \;
ghoti#mac:~ 510$ cat foo/bar
foo 0x80863a30 bar
ghoti#mac:~ 511$
If this doesn't work for you, please update your question with the results of your attempts.
You can use the following:
find /Users/Niresh/Desktop/TTT -type f -exec sed -i 's/0x04030000&0xffff0000/0x80863a30/g' {} \;
This should do the job.
grep -rl 0x04030000&0xffff0000 /Users/Niresh/Desktop/TTT | xargs sed -i 's/0x04030000\&0xffff0000/0x80863a30/g'
The & Was identified as a special character to fix it just to add a slash \ before the & ( \& ) solved the problem