Loop with variable in bash script - bash

I deploy my website using a post-receive git hook. Within that hook I use the YUI compressor to minify js and css files:
export TEMP=/var/www/example.com
GIT_WORK_TREE=/var/www/example.com git checkout master -f
#Minify mit YUI
(cd $TEMP/css && min style.css && rm style.css && mv style.min.css style.css)
(cd $TEMP/addons/css && min bootstrap.css && rm bootstrap.css && mv bootstrap.min.css
(cd $TEMP/js && min script.js && rm script.js && mv script.min.js script.js)
(cd $TEMP/addons/js && min startup.js && rm startup.js && mv startup.min.js startup.js)
Now I would like to not specify the exact files, but to search for all js and css files through all folders in $TEMP and repeat the minify procedure each time.
Could any one help me with the right loop and search-syntax for this case?
Thanks!

Just a guess here, but what about constructs like this?
find $TEMP -name \*.css -exec sh -c 'f="{}"; min "$f" && mv "${f%.css}.min.css" "$f"' \;
The idea is that the find command finds all your CSS files, then executes the min and mv commands. No need to rm, as the mv will overwrite.
You can probably figure out the equivalent line for your JavaScript files. :-)
Note that I haven't tested this, as I don't use min, but this isn't a question about minifying or YUI, it's a question about how to execute a command on multiple files in a directory tree.
UPDATE:
You can skip some files by putting extra logic into the find conditions:
find $TEMP -name \*.js -and -not -name \*.min.js -exec ...

In a single command line using bash terminal:
TEMP="/var/www/example.com"; pwd="$PWD"; IFS=$'\n'; for f in `find $TEMP -name "*.js" -or -name "*.css"`; do dir=`dirname $f`; cd "$dir"; min "$f"; ext="${f##*.}"; f2=`echo $f | sed "s/$ext$/min\.$ext/g"`; rm -f "$f"; mv "$f2" "$f"; done; cd "$pwd"
or if you place it in a sh file, then:
#!/bin/bash
TEMP="$1";
IFS=$'\n';
for f in `find $TEMP -name "*.js" -or -name "*.css"`; do
dir=`dirname $f`
cd "$dir"
min "$f"
ext="${f##*.}"
f2=`echo $f | sed "s/$ext$/min\.$ext/g"`
rm -f "$f"
mv "$f2" "$f"
done
and call the script with the path as an argument or hardcode it directly into the file.
The above algorithm takes care of both js and css files. It moves into the directory where the file is located and create the temporary file in that very same directory, as your original command does. It also should work even if you have spaces in file or directory names or if files contain the substring .js or .css within their names and not only in the extension.

Related

Rename files in several subdirectories

I want to rename a file present in several subdirectories using bash script.
my files are in folders:
./FolderA/ABCD/ABCD_Something.ctl
./FolderA/EFGH/EFGH_Something.ctl
./FolderA/WXYZ/WXYZ_Something.ctl
I want to rename all of the .ctl file with the same name (name.ctl).
I tried several command using mv or rename but didnt work.
Working from FolderA:
find . -name '*.ctl' -exec rename *.ctl name.ctl '{}' \;
or
for f in ./*/*.ctl; do mv "$f" "${f/*.ctl/name .ctl}"; done
or
for f in $(find . -type f -name '*.ctl'); do mv $f $(echo "$f" | sed 's/*.ctl/name.ctl/'); done
Can you help me using bash?
thanks
You can do this with one line with:
find . -name *.ctl -exec sh -c 'mv "$1" `dirname "$1"`/name.ctl' x {} \;
The x just allows the filename to be positional character 1 rather than 0 which (in my opinion) wrong to use as a parameter.
Try this:
find . -name '*.ctl' | while read f; do
dn=$(dirname "${f}")
# remove the echo after you sanity check the output
echo mv "${f}" "${dn}/name.ctl"
done
find should get all the files you want, dirname will get just the directory name, and mv will perform the rename. You can remove the quotes if you're sure that you'll never have spaces in the names.

Iterating over directory and replacing '+' from a file name to '_' in bash

I have a directory named as assets, which in further has a set of directories.
user_images
|-content_images
|-original
|-cropped
|-resize
|-gallery_images
|-slider_images
|-logo
These folders can have folders like original, cropped, resize. And these folders further will have images. These images are named something like this – 14562345+Image.jpeg. I need to replace all the images/files that have + to _.
for f in ls -a;
do
if [[ $f == ​+​ ]]
then
cp "$f" "${f//+/_}"
fi
done
I was able to do this in the current directory. But I need to iterate this to other many other directories. How can I do that?
You can use this loop using find in a process substitution:
cd user_images
while IFS= read -r -d '' f; do
echo "$f"
mv "$f" "${f//+/_}"
done < <(find . -name '*+*' -type f -print0)
With find -exec:
find user_images -type f \
-exec bash -c '[[ $0 == *+* ]] && mv "$0" "${0//+/_}"' {} \;
Notice that this uses mv and not cp as the question states "rename", but simply replace by cp if you want to keep the original files.
The bash -c is required to be able to manipulate the file names, otherwise we could use {} directly in the -exec action.
The following will run recursively and will rename all files replacing + with a _ :
find . -name '*+*' -type f -execdir bash -c 'for f; do mv "$f" "${f//+/_}"' _ {} +
Notice the use of -execdir :
Like -exec, but the specified command is run from the subdirectory containing the matched file, which is not normally the directory in which you started find -- Quoted from man find.
Which will protect us in case of directory names matching the pattern *+* which you do not want to rename.

Rename file names in current directory and all subdirectories

I have 4 files with the following names in different directories and subdirectories
tag0.txt, tag1.txt, tag2.txt and tag3.txt
and wish to rename them as tag0a.txt, tag1a.txt ,tag2a.txt and tag3a.txt in all directories and subdirectories.
Could anyone help me out using a shell script?
Cheers
$ shopt -s globstar
$ rename -n 's/\.txt$/a\.txt/' **/*.txt
foo/bar/tag2.txt renamed as foo/bar/tag2a.txt
foo/tag1.txt renamed as foo/tag1a.txt
tag0.txt renamed as tag0a.txt
Remove -n to rename after checking the result - It is the "dry run" option.
This can of course be done with find:
find . -name 'tag?.txt' -type f -exec bash -c 'mv "$1" ${1%.*}a.${1##*.}' -- {} \;
Here is a posix shell script (checked with dash):
visitDir() {
local file
for file in "$1"/*; do
if [ -d "$file" ]; then
visitDir "$file";
else
if [ -f "$file" ] && echo "$file"|grep -q '^.*/tag[0-3]\.txt$'; then
newfile=$(echo $file | sed 's/\.txt/a.txt/')
echo mv "$file" "$newfile"
fi
fi
done
}
visitDir .
If you can use bashisms, just replace the inner IF with:
if [[ -f "$file" && "$file" =~ ^.*/tag[0-3]\.txt$ ]]; then
echo mv "$file" "${file/.txt/a.txt}"
fi
First check that the result is what you expected, then possibly remove the "echo" in front of the mv command.
Using the Perl script version of rename that may be on your system:
find . -name 'tag?.txt' -exec rename 's/\.txt$/a$&/' {} \;
Using the binary executable version of rename:
find . -name 'tag?.txt' -exec rename .txt a.txt {} \;
which changes the first occurrence of ".txt". Since the file names are constrained by the -name argument, that won't be a problem.
Is this good enough?
jcomeau#intrepid:/tmp$ find . -name tag?.txt
./a/tag0.txt
./b/tagb.txt
./c/tag1.txt
./c/d/tag3.txt
jcomeau#intrepid:/tmp$ for txtfile in $(find . -name 'tag?.txt'); do \
mv $txtfile ${txtfile%%.txt}a.txt; done
jcomeau#intrepid:/tmp$ find . -name tag*.txt
./a/tag0a.txt
./b/tagba.txt
./c/d/tag3a.txt
./c/tag1a.txt
Don't actually put the backslash into the command, and if you do, expect a '>' prompt on the next line. I didn't put that into the output to avoid confusion, but I didn't want anybody to have to scroll either.

Bash rename extension recursive

I know there are a lot of things like this around, but either they don't work recursively or they are huge.
This is what I got:
find . -name "*.so" -exec mv {} `echo {} | sed s/.so/.dylib/` \;
When I just run the find part it gives me a list of files. When I run the sed part it replaces any .so with .dylib. When I run them together they don't work.
I replaced mv with echo to see what happened:
./AI/Interfaces/C/0.1/libAIInterface.so ./AI/Interfaces/C/0.1/libAIInterface.so
Nothing is replaced at all!
What is wrong?
This will do everything correctly:
find -L . -type f -name "*.so" -print0 | while IFS= read -r -d '' FNAME; do
mv -- "$FNAME" "${FNAME%.so}.dylib"
done
By correctly, we mean:
1) It will rename just the file extension (due to use of ${FNAME%.so}.dylib). All the other solutions using ${X/.so/.dylib} are incorrect as they wrongly rename the first occurrence of .so in the filename (e.g. x.so.so is renamed to x.dylib.so, or worse, ./libraries/libTemp.so-1.9.3/libTemp.so is renamed to ./libraries/libTemp.dylib-1.9.3/libTemp.so - an error).
2) It will handle spaces and any other special characters in filenames (except double quotes).
3) It will not change directories or other special files.
4) It will follow symbolic links into subdirectories and links to target files and rename the target file, not the link itself (the default behaviour of find is to process the symbolic link itself, not the file pointed to by the link).
for X in `find . -name "*.so"`
do
mv $X ${X/.so/.dylib}
done
A bash script to rename file extensions generally
#/bin/bash
find -L . -type f -name '*.'$1 -print0 | while IFS= read -r -d '' file; do
echo "renaming $file to $(basename ${file%.$1}.$2)";
mv -- "$file" "${file%.$1}.$2";
done
Credits to aps2012.
Usage
Create a file e.g. called ext-rename (no extension, so you can run it like a command) in e.g. /usr/bin (make sure /usr/bin is added to your $PATH)
run ext-rename [ext1] [ext2] anywhere in terminal, where [ext1] is renaming from and [ext2] is renaming to. An example use would be: ext-rename so dylib, which will rename any file with extension .so to same name but with extension .dylib.
What is wrong is that
echo {} | sed s/.so/.dylib/
is only executed once, before the find is launched, sed is given {} on its input, which doesn't match /.so/ and is left unchanged, so your resulting command line is
find . -name "*.so" -exec mv {} {}
if you have Bash 4
#!/bin/bash
shopt -s globstar
shopt -s nullglob
for file in /path/**/*.so
do
echo mv "$file" "${file/%.so}.dylib"
done
He needs recursion:
#!/bin/bash
function walk_tree {
local directory="$1"
local i
for i in "$directory"/*;
do
if [ "$i" = . -o "$i" = .. ]; then
continue
elif [ -d "$i" ]; then
walk_tree "$i"
elif [ "${i##*.}" = "so" ]; then
echo mv $i ${i%.*}.dylib
else
continue
fi
done
}
walk_tree "."

How do I copy directory structure containing placeholders

I have the situation, where a template directory - containing files and links (!) - needs to be copied recursively to a destination directory, preserving all attributes. The template directory contains any number of placeholders (__NOTATION__), that need to be renamed to certain values.
For example template looks like this:
./template/__PLACEHOLDER__/name/__PLACEHOLDER__/prog/prefix___FILENAME___blah.txt
Destination becomes like this:
./destination/project1/name/project1/prog/prefix_customer_blah.txt
What I tried so far is this:
# first create dest directory structure
while read line; do
dest="$(echo "$line" | sed -e 's#__PLACEHOLDER__#project1#g' -e 's#__FILENAME__#customer#g' -e 's#template#destination#')"
if ! [ -d "$dest" ]; then
mkdir -p "$dest"
fi
done < <(find ./template -type d)
# now copy files
while read line; do
dest="$(echo "$line" | sed -e 's#__PLACEHOLDER__#project1#g' -e 's#__FILENAME__#customer#g' -e 's#template#destination#')"
cp -a "$line" "$dest"
done < <(find ./template -type f)
However, I realized that if I want to take care about permissions and links, this is going to be endless and very complicated. Is there a better way to replace __PLACEHOLDER__ with "value", maybe using cp, find or rsync?
I suspect that your script will already do what you want, if only you replace
find ./template -type f
with
find ./template ! -type d
Otherwise, the obvious solution is to use cp -a to make an "archive" copy of the template, complete with all links, permissions, etc, and then rename the placeholders in the copy.
cp -a ./template ./destination
while read path; do
dir=`dirname "$path"`
file=`basename "$path"`
mv -v "$path" "$dir/${file//__PLACEHOLDER__/project1}"
done < <(`find ./destination -depth -name '*__PLACEHOLDER__*'`)
Note that you'll want to use -depth or else renaming files inside renamed directories will break.
If it's very important to you that the directory tree is created with the names already changed (i.e. you must never see placeholders in the destination), then I'd recommend simply using an intermediate location.
First copy with rsync, preserving all the properties and links etc.
Then change the placeholder strings in the destination filenames:
#!/bin/bash
TEMPL="$PWD/template" # somewhere else
DEST="$PWD/dest" # wherever it is
mkdir "$DEST"
(cd "$TEMPL"; rsync -Hra . "$DEST") #
MyRen=$(mktemp)
trap "rm -f $MyRen" 0 1 2 3 13 15
cat >$MyRen <<'EOF'
#!/bin/bash
fn="$1"
newfn="$(echo "$fn" | sed -e 's#__PLACEHOLDER__#project1#g' -e s#__FILENAME__#customer#g' -e 's#template#destination#')"
test "$fn" != "$newfn" && mv "$fn" "$newfn"
EOF
chmod +x $MyRen
find "$DEST" -depth -execdir $MyRen {} \;

Resources