I have a folder named test which has two files style.css and a hidden file named .DS_Store. My aim is to recursively replace all "changefrom.this" strings in all files under test to "to.this". So I came up with:
folder_root="test"
# change text in files
find $folder_root/ -type f -print0 | xargs -0 -n 1 sed -i -e 's/changefrom.this/to.this/g'
And while the strings do get replaced in the style.css file for instance, the execution outputs an error:
sed: RE error: illegal byte sequence
And I get some new files in the test folder: style.css-e and !2766!.DS_Store. Didn't expect that. What's going on here?
Can you try this simplified command with find -exec sed:
find "$folder_root/" -type f -exec sed -i 's/changefrom\.this/to.this/g' '{}' +
If changefrom.this is not the actual pattern you're using then let us know what that pattern is as it might be causing problems.
Try this:
find $folder_root/ -type f -print0 | LC_ALL=en_US.CP437 xargs -0 -n 1 sed -i -e 's/changefrom.this/to.this/g'
If it works, the problem is your terminal encoding doesn't match the file's encoding. CP437 doesn't have bad bytes so this fixes it for regexes in 7 bit ascii.
Related
I intend to find all the markdown files which contain the word desire using pipeline
In [37]: !find -E . -iregex ".*/[^/]+\.md" -print0 -exec grep -i "desire" "{}" \; | grep ".md"
grep: ./Coding/CNL.md: Is a directory
Binary file (standard input) matches
How to solve such a problem?
About the errors:
grep: ./Coding/CNL.md: Is a directory
means a directory was passed as argument to grep and grep can't process directories, adding -type f option, filters only files.
Binary file (standard input) matches
Means stadard input (because grep is used in a pipe there's no file name) is detected as a binary file, grep doesn't print ouput to avoid special characters or escape sequences to be send to terminal. This may be due to -print0 option which uses NUL character (or \0) as output delimiter.
It's not clear why are you using -print0 and -exec grep ..., this will mix file names and files' content.
I can't find posts that help with this exact problem:
On Mac Terminal I want to read a txt file (example.txt) containing file names such as:
20130815 144129 865 000000 0172 0780.bmp
20130815 144221 511 000003 1068 0408.bmp
....100 more
And I want to search for them in a certain folder/subfolders (example_folder). After each find, the file should be copied to a new folder x (new_destination).
Your help would be much appreciated!
Chers,
Mo
You could use a piped command with a combination of ls, grep, xargs and cp.
So basically you start with getting the list of files
ls
then you filter them with egrep -e, grep -e or whatever flavor of grep Mac uses for their terminal. If you want to find all files ending with text you can use the regex .txt$ (which means ends with '.txt')
ls | egrep -e "yourRegexExpression"
After that you get an input stream, but cp doesn't work with input streams and only takes a bunch of arguments, that's why we use xargs to convert it to arguments. The final step is to add the flag -t to the argument to signify that the next argument is the target directory.
ls | egrep -e "yourRegexExpression" | xargs cp -t DIRECTORY
I hope this helps!
Edit
Sorry I didn't read the question well enough, I updated to be match your problem. Here you can see that the egrep command compiles a rather large regex string with all the file names in this way (filename1|filename2|...|fileN). The $() evaluates the command inside and uses the tr to translate newLines to "|" for the regex.
ls | egrep -e "("$(cat yourtextfile.txt | tr "\n" "|")")" | xargs cp -t DIRECTORY
You could do something like:
$ for i in `cat example.txt`
find /search/path -type f -name "$i" -exec cp "{}" /new/path \;
This is how it works, for every line within example.txt:
for i in `cat example.txt`
it will try to find a file matching the line $i in the defined path:
find /search/path -type f -name "$i"
And if found it will copy it to the desired location:
-exec cp "{}" /new/path \;
I have written some code that checks all the folders in the file and matches a regex statement to them however, there is a subfolder in the directory that I want it to enter and also perform the regex on but whenever I run the code it gives me this error
sed: couldn't edit TestFolder: not a regular file
I've looked all over S.O and the Internet and can't find anything helpful
I've tried to use code I've found to fix my problem but it isn't helping so I apologise for the potentially hideous code, it's pulled from various sources
`pwd = "$PWD"
find $pwd = -print | xargs -0 sed -i "/10.0.0.10/d" !(test.sh)
My directory structure follows
Test
-one.txt
-two.txt
TestFolder
-three.txt
With GNU find, GNU xargs, and GNU sed:
find . -type f -not -name 'test.sh' -print0 | xargs -0 sed -i '/\<10\.0\.0\.10\>/d'
So after a lot of searching and trying to interpret others' questions and answers to my needs, I decided to ask for myself.
I'm trying to take a directory structure full of images and place all the images (regardless of extension) in a single folder. In addition to this, I want to be able to remove images matching certain filenames in the process. I have a find command working that outputs all the filepaths for me
find -type f -exec file -i -- {} + | grep -i image | sed 's/\:.*//'
but if I try to use that to copy files, I have trouble with the spaces in the filenames.
cp `find -type f -exec file -i -- {} + | grep -i image | sed 's/\:.*//'` out/
What am I doing wrong, and is there a better way to do this?
With the caveat that it won't work if files have newlines in their names:
find . -type f -exec file -i -- {} + |
awk -vFS=: -vOFS=: '$NF ~ /image/{NF--;printf "%s\0", $0}' |
xargs -0 cp -t out/
(Based on answer by Jonathan Leffler and subsequent comments discussion with him and #devnull.)
The find command works well if none of the file names contain any newlines. Within broad limits, the grep command works OK under the same circumstances. The sed command works fine as long as there are no colons in the file names. However, given that there are spaces in the names, the use of $(...) (command substitution, also indicated by back-ticks `...`) is a disaster. Unfortunately, xargs isn't readily a part of the solution; it splits on spaces by default. Because you have to run file and grep in the middle, you can't easily use the -print0 option to (GNU) find and the -0 option to (GNU) xargs.
In some respects, it is crude, but in many ways, it is easiest if you write an executable shell script that can be invoked by find:
#!/bin/bash
for file in "$#"
do
if file -i -- "$file" | grep -i -q "$file:.*image"
then cp "$file" out/
fi
done
This is a little painful in that it invokes file and grep separately for each name, but it is reliable. The file command is even safe if the file name contains a newline; the grep is probably not.
If that script is called 'copyimage.sh', then the find command becomes:
find . -type f -exec ./copyimage.sh {} +
And, given the way the grep command is written, the copyimage.sh file won't be copied, even though its name contains the magic word 'image'.
Pipe the results of your find command to
xargs -l --replace cp "{}" out/
Example of how this works for me on Ubuntu 10.04:
atomic#atomic-desktop:~/temp$ ls
img.png img space.png
atomic#atomic-desktop:~/temp$ mkdir out
atomic#atomic-desktop:~/temp$ find -type f -exec file -i \{\} \; | grep -i image | sed 's/\:.*//' | xargs -l --replace cp -v "{}" out/
`./img.png' -> `out/img.png'
`./img space.png' -> `out/img space.png'
atomic#atomic-desktop:~/temp$ ls out
img.png img space.png
atomic#atomic-desktop:~/temp$
Trying to answer Using Bash/Perl to modify files based on each file's name I ended in a point in which I don't know how to use find and sed all together.
Let's say there is a certain structure of files in which we want to change a line, appending the name of the file.
If it was a normal for loop we would do:
for file in dir/*
do
sed -i "s/text/text plus $file/g" $file
done
But let's say we want to use find to change files from all subdirectories. In this case, I would use...
find . -type f -exec sed -i "s/text/text plus {}/g" {} \;
^
it does not like this part
but these {} within sed are not accepted and I get the error
sed: -e expression #1, char 20: unknown option to `s'
I found some similar questions (1) but could not generalize it enough to make it understandable for me in this case.
I am sure you guys will come with a great solution for this. Thanks!
I really think the issue is that your files name contains a / that is why sed believes it start the options strings.
Replace / by # in you sed command would do the job.
I try that on Linux BASH and it work perfectly
find . -type f -exec sed -i -e "s#text#test plus {}#g" {} \;
find would return pathnames (relative or absolute) depending upon the path you specify.
This would conflict with the delimiter you've specified, i.e. /. Change the delimiter for sed and you should be good:
find . -type f -exec sed -i "s|text|text plus {}|g" {} \;
EDIT: For removing the leading ./ from the paths, you can try:
find . -type f -exec sh -c '$f={}; f=${f/.\//}; sed -i "s|text|text plus ${f}|g" {}' \;
I'm certain that better solutions might exist ...