Assume I have files named "*.data.done". Now I want to rename them (recursively) back to "*.data" then ones which contains "pattern"
So here we go:
grep -l -R -F "pattern" --include '*.data.done' * | xargs -I{} mv {} ${{}::-5}
Well, this stripping of '.done' is not working (bash 4.3.11):
bash: ${{}::-5}: bad substitution
How can I do this most easiest way?
Placeholder {} cannot be used in BASH's string manipulations inside ${...}.
You can use:
grep -lRF "pattern" --include '*.data.done' . |
xargs -I{} bash -c 'f="{}"; mv "$f" "${f/.done}"'
However if you want to avoid spawning subshell for each file then use a for loop:
while IFS= read -d '' -r f; do
mv "$f" "${f/.done}"
done < <(grep -lRF "pattern" --include '*.data.done' --null .)
Related
This line gets absolute path, i used output to pass it to rsync, but rsync wants relative path
find /www-data/ -type f -exec sh -c 'if ! lsof `readlink -f {}` > /dev/null; then echo `realpath {}`; fi' \; | tr '\n' '\0'
No idea how to feed realpath --relative-to from above output
Full code:
cd /www-data
find ./ -type f -exec sh -c 'if ! lsof `readlink -f {}` > /dev/null; then echo `realpath {}`; fi' \; | tr '\n' '\0' | rsync -avz --from0 --files-from=- ./ /data/map/uploads/ --dry-run
Using tr '\n' '\000' is fundamentally broken. The reason you want to push in null-terminated strings is to disambiguate between newlines which are part of a file name, and those which aren't; but if you are replacing all newlines, you are not disambiguating anything. Perhaps see also https://mywiki.wooledge.org/BashFAQ/020
Somewhat similarly, echo `command` is just a useless use of echo, unless you specifically want the shell to squish whitespace and expand wildcards in the output from command.
If I'm allowed to guess slightly at what you are actually trying to ask here, try
find ./ -type f -exec sh -c 'for f; do
lsof "$(readlink -f "$f")" > /dev/null ||
printf "%s\0" "$(realpath --relative-to /var/www-data "$f")"
done' _ {} + |
rsync -avz --from0 --files-from=- ./ /data/mapis/clientuploads/ --dry-run
The crucial change is really to have find pass in the file names as arguments to sh -c '...' rather than try to replace {} smack dab in the middle of a string which may or may not require quoting.
Using -exec ... {} + with a + at the end should improve efficiency somewhat, at the very minor cost of adding a for loop to the embedded sh script.
I'm attempting to recursively remove a leading underscore from some scss files. Here's what I have.
find . -name '*.scss' -print0 | xargs -0 -n1 bash -c 'mv "$0" `echo $0 | sed -e 's:^_*::'`'
When I'm in a specific directory this works perfectly:
for FILE in *.scss; do mv $FILE `echo $FILE | sed -e 's:^_*::'`; done
What am I doing wrong in the find?
As the starting point is ., all paths that find prints start with a dot. Thus ^_* doesn't match anything and sed returns its input unchanged.
I wouldn't bother with sed or xargs though.
The script below works with any find and sh that isn't terribly broken, and properly handles filenames with underscores in the middle as well.
find . -name '_*.scss' -exec sh -c '
for fp; do # pathname
fn=${fp##*/} # filename
fn=${fn#"${fn%%[!_]*}"} # filename w/o leading underscore(s)
echo mv "$fp" "${fp%/*}/$fn"
done' sh {} +
A less portable but shorter and much cleaner alternative in bash looks like:
shopt -s globstar extglob nullglob
for fp in ./**/_*.scss; do
echo mv "$fp" "${fp%/*}/${fp##*/+(_)}"
done
Drop echo if the output looks good.
Look at the syntax highlighting of the sed command. Single quotes can't be nested. Easiest fix: switch to double quotes.
find . -name '*.scss' -print0 | xargs -0 -n1 bash -c 'mv "$0" `echo $0 | sed -e "s:^_*::"`'
I would recommend leaving $0 as the program name and using $1 for the first argument. That way if bash prints an error message it'll prefix it with bash:.
find . -name '*.scss' -print0 | xargs -0 -n1 bash -c 'mv "$1" `echo "$1" | sed -e "s:^_*::"`' bash
You can also simplify this with find -exec.
find . -name '*.scss' -exec bash -c 'mv "$1" `echo "$1" | sed -e "s:^_*::"`' bash {} ';'
You could also let bash do the substitution with its ${var#prefix} prefix removal syntax:
find . -name '*.scss' -exec bash -c 'mv "$1" "${1#_}"' bash {} ';'
You don't need find at all (unless you are trying to do this with an ancient version of bash, such as what macOS ships):
shopt -s globstar extglob
for f in **/*.scss; do
mv -- "$f" "${f##*(_)}"
done
${f##...} expands f, minus the longest prefix matching .... The extended pattern *(_) matches 0 or more _, analogous to the regular expression _*.
I use this command line to find all the SVGs (thousands) in a directory and convert them to PNGs using Inkscape. Works great. Here is my issue. It outputs the PNGs in the same directory. I would like to change the target directory.
for i in `find /home/wyatt/test/svgsDIR -name "*.svg"`; do inkscape $i --export-background-opacity=0 --export-png=`echo $i | sed -e 's/svg$/png/'` -w 700 ; done
It appears $i is the file_path + file_name, and sed does a search/replace on the file extension. How do I search/replace my file_path? Or is there a better way to define a different target path within this command line?
Any help is much appreciated.
Would you please try:
destdir="DIR" # replace with your desired directory name
mkdir -p "$destdir"
find /home/wyatt/test/svgsDIR -name "*.svg" -print0 | while IFS= read -r -d "" i; do
destfile="$destdir/$(basename -s .svg "$i").png"
inkscape "$i" --export-background-opacity=0 --export-png="$destfile" -w 700
done
or
destdir="DIR"
mkdir -p "$destdir"
for i in /home/wyatt/test/svgsDIR/*.svg; do
destfile="$destdir/$(basename -s .svg "$i").png"
inkscape "$i" --export-background-opacity=0 --export-png="$destfile" -w 700
done
This may be off-topic but it is not recommended to use a for loop relying on the word-splitting especially when dealing with the filenames. Please consider the filenames and the pathnames may contain whitespace, newline, tab or other special characters.
Or with a one-liners (split for readability)
find /home/wyatt/test/svgsDIR -name "*.svg" |
xargs -I{} sh -c 'inkscape "{}" --export-background-opacity=0 --export-png='$destdir'/$(basename {} .svg).png -w 700'
Might work with find built-in exec:
find /home/wyatt/test/svgsDIR -name "*.svg" -exec sh -c 'inkscape "{}" --export-background-opacity=0 --export-png='$destdir'/$(basename {} .svg).png -w 700' \;
Or by passing target-dir as arguments, to simplify quoting.
find /home/wyatt/test/svgsDIR -name "*.svg" -exec sh -c 'inkscape "$1" --export-background-opacity=0 --export-png="$2/$(basename $1 .svg).png" -w 700' '{}' "$targetdir" \;
I have managed to do this separately using
grep -r "zone 19" path
mkdir zone19
find . -name "ListOfFilesfromGrep" -exec mv -i {} zone19 \;
I just don't know how to combine the two, that is, how to input the list of files I get from grep into the find command.
You should use grep from within find:
find /path/to/dir -type f -exec grep -q "zone 19" {} \; -exec mv -i {} zone19 \;
You could try
grep -lr "zone 19" path | while read in ; do mv -i "$in" zone19; done
-l prints the filenames with matched string; while ... done move the files one by one.
Using GNU versions of the standard tools:
grep -l will give you the filenames.
mv -t will move to a given directory.
xargs -r will invoke a command using arguments from stdin, but only if there's at least one.
Combine them like this:
grep -l -r -e 'zone 19' path | xargs -r mv -i -t 'zone19'
Or (if your filenames might contain newlines etc):
grep -lZr -e 'zone 19' path | xargs -0r mv -it 'zone19'
You can pipe the result from grep and use xargs:
grep -lr "zone 19" path | xargs <command>
<command> will be applied on each result of grep. Note thta -o flag tells grep to show only matching parts.
Below is the command to move all files containing string "Hello" to folder zone19.
grep Hello * |cut -f1 -d":"|sort -u|xargs -I {} mv {} zone19
I have to run some commands on all files in a directory starting from the smallest file in size to the largest file. The script is running fine with 'ls' command and its sorting ability but certain posts mentioned to avoid using 'ls' in case there are filenames with spaces. Please guide me about any alternative way for this?
Thanks
for file in `ls -Sr directory/*.txt`; do
....commands....
done
Read your output line by line and use -1:
while IFS= read -r file; do
...
done < <(exec ls -1 -Sr directory/*.txt)
Another:
readarray -t files < <(exec ls -1 -Sr directory/*.txt)
for file in "${files[#]}"; do
...
done
That is, if you're really using bash.
For Zsh:`
while IFS= read -u 4 -r file; do
echo "$file"
done 4< =(exec ls -1 -Sr directory/*.txt)
try this out :-
for file in `find directory/ -iname "*.txt" -type f | xargs du | sort -n | awk '{print $2}'`; do
....commands....
done
sh$ xargs -d '\n' -L1 echo < <(ls -Sr directory/*.txt)
directory/a.txt
directory/a b.txt
Of course, replace echo by your actual command.
using find would be a better option. Assign the output of find command to a array and iterate through the array as below.(This will handle if there are spaces in the filename)
OLDIFS=$IFS
IFS=$'\n'
files=$(find directory -type f -name \*.txt -exec du {} \; |sort -n | cut -d$'\t' -f2-);
for file in ${files}; do
....commands....
done
IFS=$OLDIFS
If you want to search non recursively add -maxdepth 1
OLDIFS=$IFS
IFS=$'\n'
files=$(find directory -type f -maxdepth 1 -name \*.txt -exec du {} \; |sort -n | cut -d$'\t' -f2-);
for file in ${files}; do
....commands....
done
IFS=$OLDIFS
As long as you're quoting everything, I can't think of a problem with iterating over an output of ls -Sr that has file names with spaces.
When using command substitution, and whenever you reference the variable "$file" inside the loop, you should use double quotes.
for file in "$(ls -Sr directory/*.txt)"; do
....commands....
done
But if you want to avoid ls, you could do something like this:
while read -r _ file; do
....commands....
done < <(du directory/*.txt|sort -n)