I have a Makefile, I'm trying to create a rule that will build RPMs if they don't already exist (based on looking at particular directory) otherwise just copy those RPMs to some other directory.
Essentially what I'd like to do is:
if somedir/build does NOT has RPMs:
somedir/build.sh
cp -R somedir/build/* destination
I've been toying with using $(shell if [[ -d somedir/build ]] ....) but not having much luck.
This is working:
someproject_rpms:
#if [[ -d somedir/build ]] ; then \
count=$$(find somedir/build -name "*.rpm" -type f | wc -l); \
if [[ $$count -eq 0 ]]; then \
$$(cd somedir && ./build.sh); \
fi \
else \
$$(cd somedir && ./build.sh); \
fi; \
find somedir/build -name "*.rpm" -type f -exec cp {} somedestination \;
Not the prettiest, but it works.
You should use so-called "sentinel" files. These are empty files, that are used only for their timestamp, and the timestamp signifies the completion of some action, typically creating a bunch of other files, whose names we may or may not know. The two files named rpms are the sentinel files, and here is your solution Makefile:
destination/rpms: somedir/build/rpms Makefile
find $(<D) -name *.rpm -exec cp {} $(#D) \;
touch $#
somedir/build/rpms: $(PACKEES) Makefile
somedir/build.sh
touch $#
PACKEES right now is empty, but you should set it to be all the files you want to have packed into any of your rpms, in other words, all the files that build.sh reads. If you insist on using one build.sh script to build all your rpms at once (which is not a good idea), the above is the best you are going to get :)
Related
I need to copy all files of a specific type to a folder, which I'm doing like this:
find ./ -name '*.gql' -exec cp -prv '{}' '/path/to/dir/' ';'
But if there are two files with a identical name, although located in different subfolders, some files would be overwritten.
Is it possible to keep all files, which are copied? Maybe renaming the copied file or is it possible to keep the folder structure in the target directory?
c.f. https://ss64.com/osx/cp.html : Historic versions of the cp utility had a -r option...its use is strongly discouraged, as it does not correctly copy special files, symbolic links, or fifo's. You can use -n to prevent overwrites, but more complex logic will likely require custom code.
dest=/path/to/dir
while IFS= read -r -d '' filename # read null-delimited list from find
do basename="${filename##*/}"
if [[ -e "$dest/$basename" ]]
then ctr=0
while [[ -e "$dest/$basename.$((++ctr))" ]]; do : ; done # find available name
mv "$dest/$basename" "$dest/$basename.$ctr"
fi
cp -pv "$filename" "$dest/$basename"
done < <( find ./ -name '*.gql' -type f -print0 ) # null-delimit
I deploy my website using a post-receive git hook. Within that hook I use the YUI compressor to minify js and css files:
export TEMP=/var/www/example.com
GIT_WORK_TREE=/var/www/example.com git checkout master -f
#Minify mit YUI
(cd $TEMP/css && min style.css && rm style.css && mv style.min.css style.css)
(cd $TEMP/addons/css && min bootstrap.css && rm bootstrap.css && mv bootstrap.min.css
(cd $TEMP/js && min script.js && rm script.js && mv script.min.js script.js)
(cd $TEMP/addons/js && min startup.js && rm startup.js && mv startup.min.js startup.js)
Now I would like to not specify the exact files, but to search for all js and css files through all folders in $TEMP and repeat the minify procedure each time.
Could any one help me with the right loop and search-syntax for this case?
Thanks!
Just a guess here, but what about constructs like this?
find $TEMP -name \*.css -exec sh -c 'f="{}"; min "$f" && mv "${f%.css}.min.css" "$f"' \;
The idea is that the find command finds all your CSS files, then executes the min and mv commands. No need to rm, as the mv will overwrite.
You can probably figure out the equivalent line for your JavaScript files. :-)
Note that I haven't tested this, as I don't use min, but this isn't a question about minifying or YUI, it's a question about how to execute a command on multiple files in a directory tree.
UPDATE:
You can skip some files by putting extra logic into the find conditions:
find $TEMP -name \*.js -and -not -name \*.min.js -exec ...
In a single command line using bash terminal:
TEMP="/var/www/example.com"; pwd="$PWD"; IFS=$'\n'; for f in `find $TEMP -name "*.js" -or -name "*.css"`; do dir=`dirname $f`; cd "$dir"; min "$f"; ext="${f##*.}"; f2=`echo $f | sed "s/$ext$/min\.$ext/g"`; rm -f "$f"; mv "$f2" "$f"; done; cd "$pwd"
or if you place it in a sh file, then:
#!/bin/bash
TEMP="$1";
IFS=$'\n';
for f in `find $TEMP -name "*.js" -or -name "*.css"`; do
dir=`dirname $f`
cd "$dir"
min "$f"
ext="${f##*.}"
f2=`echo $f | sed "s/$ext$/min\.$ext/g"`
rm -f "$f"
mv "$f2" "$f"
done
and call the script with the path as an argument or hardcode it directly into the file.
The above algorithm takes care of both js and css files. It moves into the directory where the file is located and create the temporary file in that very same directory, as your original command does. It also should work even if you have spaces in file or directory names or if files contain the substring .js or .css within their names and not only in the extension.
Can't seem to crack this one.
I have a bash script to search a folder and exclude certain file types.
list=`find . -type f ! \( -name "*data.php" -o -name "*.log" -o -iname "._*" -o -path "*patch" \)`
I want to exclude files which start with dot-dash ._ but the above just refuses to work.
Here's some more of the script, but I am still getting files copied with start with ._
O/S is CentOS 5.3
list=`find . -type f ! \( -name "*data.php" -o -name "*.log" -o -iname "._*" -o -path "*patch" \)`
for a in $list; do
if [ ! -f "$OLDFOL$a" ]; then
cp --preserve=all --parents $a $UPGFOL
continue
fi
diff $a "$OLDFOL$a" > /dev/null
if [[ "$?" == "1" ]]; then
# exists & different so copy
cp --preserve=all --parents $a $UPGFOL
fi
done
First -- don't do it that way.
files="`find ...`"
splits names on whitespace, meaning that Some File becomes two files, Some and File. Even splitting on newlines is unsafe, as valid UNIX filenames can contain $'\n' (any character other than / and null is valid in a UNIX filename). Instead...
getfiles() {
find . -type f '!' '(' \
-name '*data.php' -o \
-name '*.log' -o \
-iname "._*" -o \
-path "*patch" ')' \
-print0
}
while IFS= read -r -d '' file; do
if [[ ! -e $orig_dir/$file ]] ; then
cp --preserve=all --parents "$file" "$dest_dir"
continue
fi
if ! cmp -q "$file" "$orig_dir/$file" ; then
cp --preserve=all --parents "$file" "$dest_dir"
fi
done < <(getfiles)
The above does a number of things right:
It is safe against filenames containing spaces or newlines.
It uses cmp -q, not diff. cmp exits immediately when a change is made, rather than needing to calculate the delta between two files, and is thus far faster.
Read BashFAQ #1, UsingFind, and BashPitfalls #1 to understand some of the differences between this and the original.
Also -- I've validated that this correctly excludes filenames which start with ._ -- but the original version did too. Perhaps what you really want is to exclude filenames matching *._* rather than ._*?
I'd like to do a Makefile that runs either with gnumake or makepp that packs all the files under given directiories:
DIRS:=$(shell find . -mindepth 2 -maxdepth 2 -not -name mp3 -not -name ".*" -type d)
PACKAGES = $(DIRS:%=%.npk)
all: packages
packages: $(PACKAGES)
%.npk: %/*
npack c $# #^
.PHONY: all packages
the problem is that there's no such thing as %/* in the dependencies.
I need the targets (X.npk) to depend on every file in directory X, but I don't know what the files are when I write the Makefile, 'cause they're generated later.
An example:
./dirA/x
./dirA/y
./dirB/e
./dirB/f
I'd like to create ./dirA.npk (depending on x,y), ./dirB.npk (e,f)
There's nothing I know about the dirs or the files in advance except that the find used in the 1st line finds all the dirs.
Try using the wildcard directive:
DEPS := $(foreach dir, $(DIRS), $(wildcard $(dir)/*))
%.npk: $(DEPS)
npack c $# $^
EDIT:
The above is just an example of using wildcard and makes each .npk file dependent on the files in all of the other folders. Your usage would be slightly different.
I think there may be an easier way to go about this. Why are you wanting to have a dependency on all of the files in the folder? Is it just to use the $^ operator? Or do you need to rebuild the .npk if any of the files changed?
One alternate (and possibly cleaner) solution would be to use the find utility in your recipe instead of $^ and use the .FORCE directive to always force the .npk file to be rebuilt. The downside is that .npk files may be rebuilt unnecessarily.
EDIT 2:
If there's not a way to do this cleanly with make commands, you can work around it by using .FORCE to ensure that the recipe is always run and move the "should I rebuild this file" check into the body of the recipe:
%.npk: .FORCE
check_for_rebuild.sh $# && npack c $# $^
where check_for_rebuild.sh is a shell script that does something like this:
#!/bin/bash
# Returns non-zero if the archive needs to be rebuilt
if [ -e $1 ]; then
folder_name=$(basename $1 .npk)
[ -z "$(find $folder_name -newer $1 -not -type d)" ] && return 0
fi
return 1
I don't really like that solution because it works around the problem instead of solving it directly, but it may be able to get you going in the meantime. If you are going to go that route, it's probably cleaner and easier to do everything in the shell script and either have the makefile simply invoke the script or get rid of the makefile entirely.
This is the solution I found:
it is based on the makedepend idea, with some "meta" scripting. Not very nice, but works.
PACKAGES :=
all: packages
-include Makefile.depend
packages: Makefile.depend $(PACKAGES)
depend: clean Makefile.depend
Makefile.depend:
#(PACKAGES= ; \
for DIR in `find . -mindepth 2 -maxdepth 2 -not -name mp3 -not -name ".*" -type d` ; \
do \
PACKAGE=`basename $${DIR}.npk` ; \
PACKAGES="$${PACKAGES} $${PACKAGE}" ; \
DEPS=`find $${DIR} -not -type d | sed -e 's#\([: ]\)#\\\\\1#' -e 's#^\./\(.*\)# \1#' | tr -d "\n"` ; \
SUBDIR=`echo $${DIR} | sed -e 's#^\./\([^/]\+\)/.*#\1#'` ; \
FILES=`echo \ $${DEPS} | sed -e "s# $${SUBDIR}/# #g"` ; \
echo "$${PACKAGE}:$${DEPS}" ; \
echo " #cd $${SUBDIR} ; \\" ; \
echo " npack c ../\$$# $${FILES} ; \\" ; \
echo ; \
done ; \
echo "PACKAGES = $${PACKAGES}" \
)>> Makefile.depend ; \
cleanall: clean
rm -f *.npk
clean:
#rm -f Makefile.depend
.PHONY: all packages depend clean
With makepp you can do this in 2 steps, via the :foreach rule modifier:
$(foreach).txt: $(foreach)/*: foreach */
&echo $(inputs) -o $(output)
This provides a rule for every subdirectory, which reexecutes whenever there is a change in the list of files therein.
I have the situation, where a template directory - containing files and links (!) - needs to be copied recursively to a destination directory, preserving all attributes. The template directory contains any number of placeholders (__NOTATION__), that need to be renamed to certain values.
For example template looks like this:
./template/__PLACEHOLDER__/name/__PLACEHOLDER__/prog/prefix___FILENAME___blah.txt
Destination becomes like this:
./destination/project1/name/project1/prog/prefix_customer_blah.txt
What I tried so far is this:
# first create dest directory structure
while read line; do
dest="$(echo "$line" | sed -e 's#__PLACEHOLDER__#project1#g' -e 's#__FILENAME__#customer#g' -e 's#template#destination#')"
if ! [ -d "$dest" ]; then
mkdir -p "$dest"
fi
done < <(find ./template -type d)
# now copy files
while read line; do
dest="$(echo "$line" | sed -e 's#__PLACEHOLDER__#project1#g' -e 's#__FILENAME__#customer#g' -e 's#template#destination#')"
cp -a "$line" "$dest"
done < <(find ./template -type f)
However, I realized that if I want to take care about permissions and links, this is going to be endless and very complicated. Is there a better way to replace __PLACEHOLDER__ with "value", maybe using cp, find or rsync?
I suspect that your script will already do what you want, if only you replace
find ./template -type f
with
find ./template ! -type d
Otherwise, the obvious solution is to use cp -a to make an "archive" copy of the template, complete with all links, permissions, etc, and then rename the placeholders in the copy.
cp -a ./template ./destination
while read path; do
dir=`dirname "$path"`
file=`basename "$path"`
mv -v "$path" "$dir/${file//__PLACEHOLDER__/project1}"
done < <(`find ./destination -depth -name '*__PLACEHOLDER__*'`)
Note that you'll want to use -depth or else renaming files inside renamed directories will break.
If it's very important to you that the directory tree is created with the names already changed (i.e. you must never see placeholders in the destination), then I'd recommend simply using an intermediate location.
First copy with rsync, preserving all the properties and links etc.
Then change the placeholder strings in the destination filenames:
#!/bin/bash
TEMPL="$PWD/template" # somewhere else
DEST="$PWD/dest" # wherever it is
mkdir "$DEST"
(cd "$TEMPL"; rsync -Hra . "$DEST") #
MyRen=$(mktemp)
trap "rm -f $MyRen" 0 1 2 3 13 15
cat >$MyRen <<'EOF'
#!/bin/bash
fn="$1"
newfn="$(echo "$fn" | sed -e 's#__PLACEHOLDER__#project1#g' -e s#__FILENAME__#customer#g' -e 's#template#destination#')"
test "$fn" != "$newfn" && mv "$fn" "$newfn"
EOF
chmod +x $MyRen
find "$DEST" -depth -execdir $MyRen {} \;