Closed. This question is off-topic. It is not currently accepting answers.
Want to improve this question? Update the question so it's on-topic for Stack Overflow.
Closed 9 years ago.
Improve this question
I want to convert all files recursively which include a specific string in the filename. The specific string in the output filename should be replaced with another string.
I already got the first parts - but have no idea how to replace the string in the output filename.
Input files should include '-default-' in the filename which should be relaced by '-head-' in the output file.
Thanks for your help :)
find . -name '*-default-*' -print0 | xargs -0 -L 1 -I {} convert -compress JPEG -resize "238" {} {}
As tripleee notes, xargs doesn't support manipulating substituted filenames. However, a shell loop might do this and be a little easier to comprehend. Like this:
find . -name '*-default-*' |
while IFS='' read -r f; do
convert -compress JPEG -resize 238 "$f" "${f/-default-/-head-}"
done
Assigning empty string to IFS prevents loss of leading and trailing whitespace. The "-r" switch to "read" disables interpreting backslash as an escape character. If you want to avoid the bashism noted by tripleee, use sed.
xargs has no facility for substituting parts of the input filename, but that is easy enough to work around. Run a simple shell script from xargs and you get the full repertoire of the shell at your disposal.
find . -name '*-default-*' -print0 |
xargs -0 -L 1 -I {} bash -c 'f="{}";
convert -compress JPEG -resize 238 "$f" "${f/-default-/-head-}"'
The ${var/str/subst} syntax is a bashism; it will have to be somewhat more involved if you require POSIX compatible shell script.
If you have tricky file names which may contain e.g. double quotes, you might have to use a standalone script; something like this, perhaps:
#!/bin/bash
for f; do
convert -compress JPEG -resize 238 "$f" "${f/-default-/-head-}"'
done
Then you could just ... | xargs ./script
Related
I'm using xargs to try to echo the name of a file, followed by its contents. Here is the command
find output/ -type f | xargs -I {} sh -c "echo {} ; cat {}"
However for some reason, the second replace after cat is not being replaced. Only for some files, so some files do work correctly.
To be clear, I'm not looking for a command that lets me echo the name of a file followed by its contents, I'm trying to understand why this specific command does not work.
Turns out that the command was too long, so it was working with shorter file names and failing for longer ones. From man xargs
-I replstr
Execute utility for each input line, replacing one or more occurrences of replstr in up to replacements (or 5 if no -R flag is specified) arguments to utility with the
entire line of input. The resulting arguments, after replacement is done, will not be allowed to grow beyond 255 bytes; this is implemented by concatenating as much
of the argument containing replstr as possible, to the constructed arguments to utility, up to 255 bytes. The 255 byte limit does not apply to arguments to utility
which do not contain replstr, and furthermore, no replacement will be done on utility itself. Implies -x.
The root cause of the problem is pointed out in Carlos' answer, but without a solution.
After some googling, I couldn't find a way to lift up the 255 characters limit.
So a probable way to workaround it, is to use shell variable as a substitution.
Example:
find . | xargs -I% sh -c 'F="%";iconv -f gb2312 -t utf-8 "$F">"$F.out";mv "$F.out" "$F"'
Remember to use single quotes at the outermost sh -c parameter string, we don't want the $F inside to be replaced by our parent shell.
Is it files with white space in the name that create problems? Try adding \", like this:
find output/ -type f | xargs -I {} sh -c "echo \"{}\" ; cat \"{}\""
This worked for me using Bash.
It's a fairly simple question and I'm sure the gurus here can figure it out right away, however I don't seem to be able to make it work (probably some quotes issue.
I want to place all instances of:
`which cat`
With the following:
/bin/cat
I am running the following command:
for file in $(find . -iname 'PATCH*'); do sed 's/\`which cat\`/\'\/bin\/cat/g' $file; done
I believe I have escaped all characters that don't need to be treated as special ones, however it doesn't seem to do the trick.
Please help :)
It is generally not a good idea to iterate over the output of find since file names can contain the $IFS which would break the loop. Use the -exec option of find instead:
find -iname 'PATCH*' -exec sed -i 's#`which cat`#/bin/cat#g' {} \;
Use a different sed delimiter.
sed 's~`which cat`~/bin/cat~g' file
Example:
$ echo '`which cat`' | sed 's~`which cat`~/bin/cat~g'
/bin/cat
I'm trying to run a program on every file on a dir.
But there is spaces in the name of the file. For example, a file can be named «/my/good/path/MY - AWFUL, FILE.DOC»
And when I'm trying to send the path to my the other tool (a python script), I've got an error saying «MY» is not a existing file. :(
Here is my current bash code:
#!/usr/bin/bash
for file in $(find "/my/pash" -name "*.DOC")
do
newvar=`program "$file"`
done
So… where is my problem?
Thanks everyone :)
Some correct answers, but no explanations so far:
a for loop is intended to iterate over words not lines. The given (unquoted) string is subject to word splitting (which is what is troubling you) and filename expansion, and then you iterate over the resulting words. You could set IFS to contain only a newline. The safest way is to use find -print0 and xargs -0 as demonstrated by Vytenis's answer
find -name "*.DOC" -print0 | xargs -r -0 -n1 program
#!/usr/bin/bash
find "/my/pash" -name "*.DOC" | while read file; do
newvar="$(program "$file")"
done
Note that this only fixes the case where a space or tab is in the file name. If you have a newline in the file name, it gets a little more complicated.
That is because the for loop will take every word inside the result of the find as an element to iterate over. for will see it as:
for file in {/my/good/path/MY, -, AWFUL, FILE.DOC}
echo "$file"
done
And will print:
/my/good/path/MY
-
AWFUL,
FILE.DOC
One solution to this problem is to use the xargs program to pass the result of the find as your python program argument:
find "/my/pash" -name "*.DOC" -print0 | xargs -0 -i program "{}"
the loop treats blanks as delimiter, so try this one:
find "/my/pash" -name "*.DOC" | while read file; do
newvar=`program "$file"`
done
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
So, my program is meant to cycle through a directory and its subdirectories, and when it finds a file that is larger than 100K, asks the user if they want to remove it.
I am using find to cycle through the directories. This is my code:
for file in $(find /home/* -print0 | xargs -0)
I have also tried
for file in $(find /home/* -exec process)
Etc, etc. Pretty much everything on the first five pages of Google.
Just to re-iterate, the problem is that find separates filenames with spaces in them. (i.e. "/home/Old data" would become "/home/Old" and "data"
Anyway, are there any better alternatives that I could be using?
Use a while loop to prevent word splitting:
while read -r file; do
# do something here
echo "${file}" # Remember to quote the variable
done < <(find /path -type f -size +200)
This is a problem with the shell, not with find. devnull has suggested a general fix, but for your specific problem, you can do:
find /path -type f -size +100k -exec ls -lh {} \; -exec rm -i {} \;
For files over 100k, this will list its attributes and ask the user whether to delete.
The process substitution $(...) isn't quoted, so all the filenames come out on one line, separated by spaces. There's no way then to tell which spaces are in filenames and which are separators. But if you quote it, "$(...)", then all the output comes out as a single multi-line string. So doing it this way, passing the command output to a for loop, doesn't work. Instead, use the xargs to do the work.
find /home/* -print0 | xargs -0 -i{} ls -l {}
This works exactly like your find|xargs except that the xargs is given a command to execute. In your case it will be a shell script in a file, e.g. mayberemove.sh. Normally xargs appends as many input lines onto the end of the command as it can, but the -i tells it to run the command with the input line in place of the (arbitrary) string {}. In this case it only uses one input line at a time. Because xargs isn't passing the argument through a shell, but instead runs the command using some variety of exec, there's no need for any more quoting. The -0 means that the arguments are delimited by a null byte, as output by find -print0, to avoid problems with whitespace.
Instead of ls -l, you will want something like:
find /home/* -print0 | xargs -0 -i{} ./mayberemove.sh
where mayberemove.sh is careful to quote its arguments, because it is passing them using the shell:
#!/bin/sh
if ....
then
rm "$1"
fi
This question already has answers here:
linux bash, camel case string to separate by dash
(9 answers)
Closed 6 years ago.
I am in the process of merging efforts with another developer. I am using UpperCamelCasing, but we decided to follow Google's HTML style guide in using lower case and separating words with hyphens. This decision requires me to rename quite some files on my filesystem. I first though this to be easy since I often use bash for renaming large collections of files. Unfortunately renaming on the Casing style appeared to be a bit more complicating and I did not manage to find an approach.
Can I convert files from one naming convention to another with Bash?
Try using rename command with -f option to rename files with desired substitutions.
rename -f 's/([a-z])([A-Z])/$1-$2/g; y/A-Z/a-z/' <list_of_files>
If you also want to extract <list_of_files> with some pattern, let's say extension .ext, you need to combine find with above command using xargs
find -type f -name "*.ext" -print0 | xargs -0 rename -f 's/([a-z])([A-Z])/$1-$2/g; y/A-Z/a-z/'
For example if you want to rename all files in pwd
$ ls
dash-case
lowerCamelCase
UpperCamelCase
$ rename -f 's/([a-z])([A-Z])/$1-$2/g; y/A-Z/a-z/' *
$ ls
dash-case
lower-camel-case
upper-camel-case
Try this:
for FILE in *; do NEWFILE=$((sed -re 's/\B([A-Z])/-\1/g' | tr [:upper:] [:lower:]) <<< "$FILE"); if [ "$NEWFILE" != "$FILE" ]; then echo mv \""$FILE"\" \""$NEWFILE"\"; fi; done
This should give you a list of "mv" statements on standard output. Double-check that they look right, then just add | bash to the end of the statement to run them all.
How does it work?
for FILE in *; do
NEWFILE=$((sed -re 's/\B([A-Z])/-\1/g' | tr [:upper:] [:lower:]) <<< "$FILE")
if [ "$NEWFILE" != "$FILE" ]; then
echo mv \""$FILE"\" \""$NEWFILE"\"
fi
done
The for FILE in * loops across all files in the current directory, acknowledging that there are a wide variety of ways to loop through all files. The sed statement matches only uppercase letters that, according to \B, aren't on a word boundary (i.e. at the beginning of the string). Because of this selective match, it makes the most sense to switch everything to lowercase in a separate call to tr. Finally, the condition ensures that you only see the filenames that change, and the trick of using echo ensures that you don't make changes to your filesystem without seeing them first.
I ran into a similar question and based on one answer there I came to the following solution. It is not a full Bash solution, since it relies on perl, but since it does the trick I am sharing it.
ls |for file in `xargs`; do mv $file `echo $file | perl -ne 'print lc(join("-", split(/(?=[A-Z])/)))'`; done