I have the following code from a make file, I know this creates bin folder in Home if that doesn't exist... but I couldn't understand what $$HOME/bin mean...
I googled and found $$ is to get the processid of the bash... but couldn't understand what $$HOME/bin mean... can someone please explain ?
.PHONY: home_bin
home_bin: ## Create home bin if not created
# if [[ ! -d "$$HOME/bin" ]]; then \
echo "Creating $$HOME/bin"; \
mkdir $$HOME/bin; \
echo "✔︎ $$HOME/bin created"; \
else \
echo "✔︎ $$HOME/bin already created"; \
fi
Thank you.
make itself performs expansion of $-prefixed characters; the $$ is expanded to a single literal $ to pass to the shell.
Consider a simple Makefile:
x=f
all:
xoo=3 && echo $xoo
which will output foo, because
make expands $x to the single character f.
make passes the string xoo=3 && echo foo to the shell for execution
Compare with
x=f
all:
xoo=3 && echo $$xoo
which outputs 3, because
make expands $$ to $
make passes the string xoo=3 && echo $xoo to the shell for execution
Related
Heres the code:
function rm {
cd ~/
if [[ -d ./Jam/projects/"$1" ]]; then
echo Removing $1 from projects...
rm -r ./Jam/projects/"$1"
elif [[ -d ./Jam/archive/"$1" ]]; then
echo Removing $1 from archives...
rm -r ./Jam/archive/"$1"
else
echo $1 does not exist \in ./Jam/projects/ or ./Jam/archive
exit
fi
echo Finished\!
}
When this is ran, $1 is "Hello World" (a Directory in i./Jam/archive/)
I get this output:
Removing HelloWorld from archives...
-r does not exist in ./Jam/projects/ or ./Jam/archive
Somehow, $1 is assigned to "-r".
I don't know how on earth this would happen. Any help is much appreciated.
Your function is called "rm" and inside your function "rm" you call rm -r thinking it's "normal rm", but it isn't - it's your function, which perfectly demonstrates the danger of calling your function a name that already has a well known meaning.
I got some code from here that works pretty well until I get "Argument list too long"
I am NOT a developer and pretty old too :) so if it is not much to ask please explain.
Is there a way the expand DIRCMD like eval does and pass each of the commands one at the time so eval does not break?
for (( ifl=0;ifl<$((NUMFIRSTLEVELDIRS));ifl++ )) { FLDIR="$(get_rand_dirname)"
FLCHILDREN="";
for (( ird=0;ird<$((DIRDEPTH-1));ird++ )) {
DIRCHILDREN=""; MOREDC=0;
for ((idc=0; idc<$((MINDIRCHILDREN+RANDOM%MAXDIRCHILDREN)); idc++)) {
CDIR="$(get_rand_dirname)" ;
# make sure comma is last, so brace expansion works even for 1 element? that can mess with expansion math, though
if [ "$DIRCHILDREN" == "" ]; then
DIRCHILDREN="\"$CDIR\"" ;
else
DIRCHILDREN="$DIRCHILDREN,\"$CDIR\"" ;
MOREDC=1 ;
fi
}
if [ "$MOREDC" == "1" ] ; then
if [ "$FLCHILDREN" == "" ]; then
FLCHILDREN="{$DIRCHILDREN}" ;
else
FLCHILDREN="$FLCHILDREN/{$DIRCHILDREN}" ;
fi
else
if [ "$FLCHILDREN" == "" ]; then
FLCHILDREN="$DIRCHILDREN" ;
else
FLCHILDREN="$FLCHILDREN/$DIRCHILDREN" ;
fi
fi
}
cd $OUTDIR
DIRCMD="mkdir -p $OUTDIR/\"$FLDIR\"/$FLCHILDREN"
eval "$DIRCMD"
echo "$DIRCMD"
}
I tried echo $DIRCMD but do not get the expanded list of commands
'echo mkdir -p /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
I had trouble following the code, but if I understood it correctly, you dynamically generate a mkdir -p command with a brace expansion:
'mkdir -p /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
Which then fails when you eval it due to your OS' maximum argument limit.
To get around that, you can instead generate a printf .. command since this is a Bash builtin and not subject to the argument limit, and feed its output to xargs:
dircmd='printf "%s\0" /mnt/nvme-test/rndpath/"r8oF"/{"rc","XXKR","p0H"}/{"5Dw0K","oT","rV","coU","uo"}/{"3m5m","uEdA","w4SJp","49"}'
eval "$dircmd" | xargs -0 mkdir -p
If your xargs doesn't support -0, you can instead use printf "%s\n" and xargs mkdir -p, though it won't behave as well if your generated names contain spaces and such.
If this is for benchmarking, you may additionally be interested to know that you can now use xargs -0 -n 1000 -P 8 mkdir -p to run 8 mkdirs in parallel, each creating 1000 dirs at a time.
So I'm trying to hack one of my makefiles to be simpler (simpler, as if, not defining a lot of rules how to transform subdirectory into .deb).
build-if-need-status-vars:
#if [ ! -f debs/1.deb ]; then \
$(eval STATUS_REBUILD=1) \
echo "component: file not found: 1"; exit;\
else \
if [ $(shell find sources/ -newer debs/1.deb 2>/dev/null | wc -l) -gt 0 ]; then \
$(eval STATUS_REBUILD=1) echo "component: newer files exists: 1"; exit;\
else \
$(eval STATUS_REBUILD=0) echo "component: no newer files: 0"; \
fi;\
fi
#echo "status $(STATUS_REBUILD)"
actual-target: build-if-need-status-vars
ifeq ($(STATUS_REBUILD), 1)
#echo first status: 1
else
#echo second status: 0
#echo different action
endif
all: actual-target
.PHONY: actual-target
Test with:
mkdir -p test/{sources,debs}; touch test/debs/1.deb; sleep 2; touch test/sources/1.src; (create makefile there and run)
Result:
component: file not found: 1
status 0
second status: 0
Regardless of what conditional block is executed, STATUS_REBUILD will always be 0 (last evaluated value), try it: touch test/debs/1.deb
So it seems that last $(eval) is always used.. How to avoid this behaviour and keep the correct assigned value (from first match in build-if-need-status-var)?
$(eval) is a make-level function. It is expanded in your recipe during recipe the recipe expansion stage.
The contents of a recipe are expanded in the second phase of makefile parsing (discussed briefly in the manual here).
I believe, but cannot say for sure (without testing), that recipes are not expanded until they are about to be run (but for the purposes here that doesn't change anything either way).
So your problem here is that all the $(eval) calls are expanded by the time make goes to run your shell script so you always see the last value in effect when the last line is run.
That all being said you don't actually need a make-level variable here. Your recipe is already only two shell executions.
You can simply include the last line in the same execution as the first (split) line and use a shell variable.
build-if-need-status-vars:
#if [ ! -f debs/1.deb ]; then \
STATUS_REBUILD=1; \
echo "component: file not found"; \
else \
if [ $(shell find sources/ -newer debs/1.deb 2>/dev/null | wc -l) -gt 0 ]; then \
STATUS_REBUILD=1; echo "component: newer files exists"; \
else \
STATUS_REBUILD=0; echo "component: no newer files"; \
fi;\
fi; \
echo "status $$STATUS_REBUILD"
Note that I needed to remove the exit pieces to make this work. If those are necessary in the real makefile (because this is a stripped down sample) then you can keep them by wrapping the if in a sub-shell and/or by rewriting the recipe.
Writing a script to optimize my images for the web. Having issues with filenames and directories with spaces in the names.
Heres what I have:
read -p "Enter full path from root (/) to your site... example /var/www/public_html: " path1
echo ""
#read -p "Enter in ImageMagick quality (default is 80) if unsure enter 80: " optjpg
#echo ""
#id="$(id -u optiimage)"
cmd="id -u optiimage"
eval $cmd
id=$(eval $cmd)
tmp1="${path1}/shell/optiimage/imagemagick"
tmp2="${path1}/shell/optiimage/imagemagick/jpg"
restore1="${path1}/shell/optiimage/restore"
restore2="${path1}/shell/optiimage/restore/imagemagick/jpg"
backup1="${path1}/shell/optiimage/backup"
backup2="${path1}/shell/optiimage/backup/imagemagick/jpg"
log1="${path1}/shell/optiimage/log/imagemagick/"
DATE="$(date +%a-%b-%y-%T)"
# Need user input for www path from root
##
## Make directories
##
############################################################################################################
mkdir -p ${tmp1}
mkdir -p ${tmp2}
mkdir -p ${restore1}
mkdir -p ${restore2}
mkdir -p ${backup1}
mkdir -p ${backup2}
mkdir -p ${log1}
mkdir -p ${path1}/build
echo "Processing JPG Files"
find $path1 -iname "*jpg" | \
#write out script to put on cron for image optimization
while read file;
do
# If not equal to optimage uid
# to check username id -u optimage
if [ -u "${id}" ]; then
filebase=`basename "$file" .jpg`
dirbase=`dirname "$file"`
echo "${dirbase}/${filebase}.jpg already optimized" >> ${log1}_optimized_$DATE.log
else
#simple log for size of image before optimization
ls -s $file >> ${log1}_before_$DATE.log
#Do the following if *.jpg found
filebase=`basename $file .jpg`
dirbase=`dirname $file`
echo "cp -p ${dirbase}/${filebase}.jpg ${tmp2}" >> ${path1}/build/backup_jpg.txt
echo "chown optiimage:www-data ${filebase}.jpg" >> ${path1}/build/restore_jpg.txt #${restore1}/imagemagick.sh
echo "cp -p ${filebase}.jpg ${dirbase}/${filebase}.jpg" >> ${path1}/build/restore_jpg.txt #${restore1}/imagemagick.sh
##
## ImageMagick
## Original Command:
## convert $file -quality 80 ${filebase}.new.jpg
##########################
echo "convert ${dirbase}/${filebase}.jpg -quality 80 ${tmp2}/${filebase}.jpg" >> ${path1}/build/imagemagick.txt
echo "mogrify -strip ${tmp2}/${filebase}.jpg" >> ${path1}/build/imagemagick.txt
echo "chown optiimage:www-data ${tmp2}/${filebase}.jpg" >> ${path1}/build/owner_jpg.txt
echo "rm ${dirbase}/${filebase}.jpg" >> ${path1}/build/remove_jpg.txt
echo "cp -p ${tmp2}/${filebase}.jpg ${dirbase}/" >> ${path1}/build/migrate_jpg.txt
simple log for size of image after optimization
ls -s $file >> ${log1}_after_$DATE.log
fi
done
I have edited this with suggestions some have given me. It didn't seem to work.
This works fine if I remove directories with spaces in the names otherwise it ends the name at the space and get errors directory doesn't exist.
You need to double-quote variable substitutions. This applies inside command substitutions as well as in the top-level lexical context. The only exception to this is assignment of a string variable from another string variable, e.g. str2=$str1;, although other types of variable assignments generally need quoting, such as assigning a string variable from an array slice, even if it only slices one element, e.g. str="${#:1:1}";.
Although unlikely to be a problem here, the read builtin strips leading and trailing whitespace if you provide one or more NAMEs; you can solve that by not providing any NAMEs at all, and just letting it store the whole line in the $REPLY variable by default.
You should always use the -r option of the read builtin, as that prevents its ill-advised default behavior of doing backslash interpolation/removal on the input data.
If you don't need any kind of interpolation in a string literal, prefer the '...' syntax to "...", as the former does not do any interpolation.
Prefer the [[ ... ]] expression evaluation form to the old-style [ ... ] form, as the former syntax is slightly more powerful.
Prefer the $(...) command substitution form to the old-style `...` form, as the former syntax has more favorable nesting properties (namely, no need to escape the nested command substitution delimiters).
find "$path1" -iname '*jpeg'| \
# write out script to put on cron for image optimization
while read -r; do
file=$REPLY;
# If not equal to optimage uid
# to check username id -u optimage
if [[ -u "$id" ]]; then
filebase=$(basename "$file" .jpeg);
dirbase=$(dirname "$file");
#MYBASENAME=$(basename "$1")
echo "${dirbase}/${filebase}.jpeg already optimized" >>"${log1}_optimized_$DATE.log";
fi;
done;
;
Quote your $file variable in every place where is used:
find $path1 -iname "*jpeg" | \
while read file;
do
if [ -u "${id}" ]; then
filebase=`basename "$file" .jpeg`
dirbase=`dirname "$file"`
fi
done
I want to create a multiline file with Make, having exact content:
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
How can I tell make to output a file with this exact content somewhere?
I tried this:
define SCRIPT_CONTENT
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
endef
export SCRIPT_CONTENT
bin/script:
#echo "$$SCRIPT_CONTENT" > bin/script
This paricular solution 1) wipes $ and first char after $-es and 2) is ugly because the definition should happen outside of the particular target where it's needed :(
I also tried this:
bin/script:
#echo '
#!/bin/bash
if [ "$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$( cat <<EOF
-Dmapred.job.tracker=$JT
EOF
)
${HADOOP_HOME}/bin/hadoop $1 $HADOOP_CONFIG_VARS ${*:2} 2>&1 | grep -v SLF4J
' > bin/script
This returns error when in make, works outside of make...
Any suggestion is very welcome!
Make wants any $ characters that should be reproduced literally to be escaped by inserting another $ in front of them.
More broadly, though, it seems like you're trying to use Make as a shell-script replacement. The more idomatic way to do this would be to put that content in a source file that you can copy to the destination, or to put it in a script that will write it into a specified destination. The Makefile then just has to invoke the copy command or the script.
With the help from this magnificent answer, I cooked up the following.
# From https://stackoverflow.com/a/8316519/874188
define \n
endef
define SCRIPT_CONTENT
#!/bin/bash
if [ "$$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi
export CONFIG_VARS=$$( cat <<EOF
-Dmapred.job.tracker=$$JT
EOF
)
$${HADOOP_HOME}/bin/hadoop $$1 $$HADOOP_CONFIG_VARS $${*:2} 2>&1 | grep -v SLF4J
endef
bin/script:
echo '$(subst $(\n),\n,$(SCRIPT_CONTENT))' >$#
When testing, I found that I needed to have a semicolon at the end of the echo line when it didn't have any redirection. I can speculate that there is a built-in echo which gets invoked when there are no shell metacharacters in the command line?
Also, notice that the definition cannot contain any single quotes, and that all dollar signs have to be doubled. Maybe one or the other of these restrictions could be removed; I was unsuccessful, but then I didn't spend too much time or effort.
You cannot do this in make. Beyond what Novelocrat says regarding $, there's the fact that make is line-oriented and does not have any ability to generate a command that contains a newline character. All newlines that appear unescaped (without a backslash before them) are parsed by make as ending that recipe line, and each recipe line is sent to a different invocation of the shell. If you want the entire command to be sent as a single string to the same shell, then you must escape the newlines.
However, make will remove all backslash/newline pairs before it runs the command.
The only possible way to do this completely within make is to generate the file one line at a time, like this:
bin/script:
#echo '#!/bin/bash' > $#
#echo 'if [ "$$JAVA_HOME" = "" ]; then echo "Please set JAVA_HOME"; exit 1; fi' >> $#
#echo 'export CONFIG_VARS=$$( cat <<EOF' >> $#
#echo ' -Dmapred.job.tracker=$$JT' >> $#
#echo 'EOF' >> $#
#echo ')' >> $#
#echo '$${HADOOP_HOME}/bin/hadoop $$1 $$HADOOP_CONFIG_VARS $${*:2} 2>&1 | grep -v SLF4J' >> $#
As Novelocrat says, the typical way this is done is to have the script file as a separate file and copy it where you want it, rather than generating it.