Below is my makefile
LATEST_GIT_COMMIT_HASH := $(shell git rev-parse HEAD | cut -c1-8)
docker:
#echo "\n" &&\
read -p "Please enter a version number (ex: 0001): " release_vesion &&\
echo $$release_vesion_${LATEST_GIT_COMMIT_HASH}
I try to concat the release_version from my input and the git version number then print them out.
But it seem not working.
How to concat the string with input value?
In bash you need to use ${var_name}concat_some_more_text, the braces seperate the variable from the rest of the concatented string. Eg:
var_name=bob
echo "1: $var_nameconcat_some_more_text"
echo "2: ${var_name}concat_some_more_text"
returns:
1:
2: bobconcat_some_more_text
So your code should look like:
LATEST_GIT_COMMIT_HASH := $(shell git rev-parse HEAD | cut -c1-8)
docker:
#echo "\n" &&\
read -p "Please enter a version number (ex: 0001): " release_vesion &&\
echo "release ver: $$release_vesion" ; \
echo "$${release_vesion}_$(LATEST_GIT_COMMIT_HASH)"
In make use $${...} for bash variable (you have correctly used the double $ but not the braces) and $(...) for make variables. You don't always need the braces for the bash variable but when concatenating the variable name to another string like you are then you do need them to separate them.
Related
I need to generate a random string and save in a variable, with a Makefile.
I wrote
install:
LOCALSTORAGE_ACCESS_TOKEN = echo $RANDOM | md5sum | head -c 20; echo;
echo $(LOCALSTORAGE_ACCESS_TOKEN)
When I launch it with make install, I get
LOCALSTORAGE_ACCESS_TOKEN = echo ANDOM | md5sum | head -c 20; echo;
/bin/sh: 1: LOCALSTORAGE_ACCESS_TOKEN: not found
d41d8cd98f00b204e980
echo
So
missing assigned value to the variable LOCALSTORAGE_ACCESS_TOKEN.
getting every time same result for RANDOM
I tried also
LOCALSTORAGE_ACCESS_TOKEN = echo $$RANDOM | md5sum | head -c 20; echo;
echo $(LOCALSTORAGE_ACCESS_TOKEN)
with same result
LOCALSTORAGE_ACCESS_TOKEN = echo $RANDOM | md5sum | head -c 20; echo;
/bin/sh: 1: LOCALSTORAGE_ACCESS_TOKEN: not found
d41d8cd98f00b204e980
echo
Each line of a recipe is a shell script, run by a different shell. Your first line is syntactically incorrect. You are trying to assign a make variable in a recipe, this is not how make works. Try:
install:
LOCALSTORAGE_ACCESS_TOKEN=$$(echo $$RANDOM | md5sum | head -c 20); \
echo $$LOCALSTORAGE_ACCESS_TOKEN
LOCALSTORAGE_ACCESS_TOKEN is now a shell variable, it is assigned using shell syntax in the first line of the recipe but is still defined in the second line because of the line continuation (the trailing \). Note the use of $$ instead of $ to escape the first expansion by make.
If you would like LOCALSTORAGE_ACCESS_TOKEN to be a make variable available in all lines of all recipes then simply assign it as a real make variable and outside any recipe :
LOCALSTORAGE_ACCESS_TOKEN := $(shell echo $$RANDOM | md5sum | head -c 20)
install:
echo $(LOCALSTORAGE_ACCESS_TOKEN)
This should work:
install:
$(eval LOCALSTORAGE_ACCESS_TOKEN := $(shell /bin/bash -c 'echo $$RANDOM' | md5sum | head -c 20; echo))
echo $(LOCALSTORAGE_ACCESS_TOKEN)
As you also noticed, you need $$RANDOM in order to mask the $ sign in the Makefile
You need to run the command in a bash, because $RANDOM may not work in all shells (did not work for me, so I had to use bash)
You need the $(shell ... so that the variable gets assigned the output of the command (instead of the command text itself)
You need $(eval ... because the code is inside a target, and you can not define variables in targets otherwise
I have the following command as a makefile command:
update-env:
echo "{ \"Variables\":${ENV_VALUE}}" > ./new_env.json && \
UPDATE_ENVVARS=$$(jq -s '.[0] as $$a |.[1] as $$b | $$a * $$b' old_env.json new_env.json | jq -c . ) && \
echo "${UPDATE_ENVVARS}"
the ${ENV_VALUE} is taken when
make update-env ENV_VALUE="{\\\"HOST_URL\\\": \\\"https:\\\/\\\/test.com\\\"}"
the file new_env.json is generated properly
when executing jq -s '.[0] as $a |.[1] as $b | $a * $b' old_env.json new_env.json | jq -c . it generates the appropriate compact json result desired.
When running everything in sequence (the assignment and echo for validation), I get an empty result.
My goal for the command is to merge two json output and assign it to the UPDATE_ENVVARS for it to be reused as an input for another command that will accept the json. Per testing, it came out empty when echo, when I execute the jq solo, the merge output is functional.
Only a minor bit of editing was needed:
update-env:
echo '{ "Variables":${ENV_VALUE}}' > ./new_env.json && \
UPDATE_ENVVARS=$$(jq -s '.[0] as $$a |.[1] as $$b | $$a * $$b' old_env.json new_env.json | jq -c . ) && \
echo "$${UPDATE_ENVVARS}"
Note:
We're using single quotes in the first-line echo -- the substitution is performed by make, not by the shell, so single quotes don't suppress it.
We're doubling up the $$ in the last line, so we're expanding the shell variable set in the second line of the recipe, not a make variable that nothing ever set at all.
See this working at https://replit.com/#CharlesDuffy2/RoyalIdolizedProfile
By contrast, if you want to assign to a make variable instead of a shell variable, this question is duplicative of Makefile command substitution problem, and the answer by Ignacio Vazquez-Abrams is appropriate.
So I want to export my products into my new website. I have an csv file with these data:
product id,image1,image2,image3,image4,image5
1,https://img.url/img1-1.png,https://img.url/img1-2.png,https://img.url/img1-3.png,https://img.url/img1-4.png,https://img.url/img1-5.png
2,https://img.url/img2-1.png,https://img.url/img2-2.png,https://img.url/img2-3.png,https://img.url/img2-4.png,https://img.url/img2-5.png
What I want to do is to make a script to read from that file, make directory named with product id, download images of the product and put them inside their own folder (folder 1 => image1-image5 of product id 1, folder 2 => image1-image5 of product id 2, and so on).
I can make a normal text file instead of using the excel format if it's easier to do. Thanks before.
Sorry I'm really new here. I haven't done the code yet because I'm clueless, but what I want to do is something like this:
for id in $product_id; do
mkdir $id && cd $id && curl -o $img1 $img2 $img3 $img4 $img5 && cd ..
done
Here is a quick and dirty attempt which should hopefully at least give you an idea of how to handle this.
#!/bin/bash
tr ',' ' ' <products.csv |
while read -r prod urls; do
mkdir -p "$prod"
# Potential bug: urls mustn't contain shell metacharacters
for url in $urls; do
wget -P "$prod" "$url"
done
done
You could equivalently do ( cd "$prod" && curl -O "$url" ) if you prefer curl; I generally do, though the availability of an option to set the output directory with wget is convenient.
If your CSV contains quotes around the fields or you need to handle URLs which contain shell metacharacters (irregular spaces, wildcards which happen to match files in the current directory, etc; but most prominently & which means to run a shell command in the background) perhaps try something like
while IFS=, read -r prod url1 url2 url3 url4 url5; do
mkdir -p "$prod"
wget -P "$prod" "$url1"
wget -P "$prod" "$url2"
: etc
done <products.csv
which (modulo the fixed quoting) is pretty close to your attempt.
Or perhaps switch to a less wacky input format, maybe generate it on the fly from the CSV with
awk -F , 'function trim (value) {
# Trim leading and trailing double quotes
sub(/^"/, "", value); sub(/"$/, "", value);
return value; }
{ prod=trim($1);
for(i=2; i<=NF; ++i) {
# print space-separated prod, url
print prod, trim($i) } }' products.csv |
while read -r prod url; do
mkdir -p "$prod"
wget -P "$prod" "$url"
done
which splits the CSV into repeated lines with the same product ID and one URL each, and any CSV quoting removed, then just loops over that instead. mkdir with the -p option helfully doesn't mind if the directory already exists.
If you followed the good advice that #Aaron gave you, this code can help you, as you seem to be new with bash I commented out the code for better comprehension.
#!/bin/bash
# your csv file
myFile=products.csv
# number of lines of file
nLines=$(wc -l $myFile | awk '{print $1}')
echo "Total Lines=$nLines"
# loop over the lines of file
for i in `seq 1 $nLines`;
do
# first column value
id=$(sed -n $(($i+1))p $myFile | awk -F ";" '{print $1}')
line=$(sed -n $(($i+1))p $myFile)
#create the folder if not exist
mkdir $id 2>/dev/null
# number of images in the line
nImgs=$(($(echo $line | awk -F ";" '{print NF-1}')-1))
# go to id folder
cd $id
#loop inside the line values
for j in `seq 2 $nImgs`;
do
# getting the image url to download it
img=$(echo $line | cut -d ";" -f $j)
echo "Downloading image $img**";echo
# downloading the image
wget $img
done
# go back path
cd ..
done
Trying it this way:
#!/bin/bash
myvals=`psql -d mydb -c "select id from table1 where 't'"`
ssh user1#host1.domain.tld "for i in $myvals; do echo \$i >> values; done"
As long as psql returns just one value, it works fine. But if its several values, I receive this response:
bash: -c: line 1: syntax error near unexpected token `2'
bash: -c: line 1: `2'
Also, I tried to:
myvals='1 2 3'
And then it works fine: the values 1 2 3 are appended to the "values" file on the remote host; no error mesages.
If I try another subshell command, such as myvals=ls /bin, errors reappear.
It's clear that $myvals is evaluated on the local host already but what makes the subshell results so different?
If It's Not Really An Array...
Iterating over a string as if it were an array is innately buggy. Don't do it. That said, to generate a safely-escaped (eval-safe) version of your value, use printf %q.
#!/bin/bash
myvals=`psql -d mydb -c "select id from table1 where 't'"`
printf -v myvals_q %q "$myvals"
ssh user1#host1.domain.tld \
"myvals=$myvals_q;"' for i in $myvals; do echo "$i"; done >>values'
If You Actually Had An Array
#!/bin/bash
readarray -t myvals < <(psql -d mydb -c "select id from table1 where 't'")
printf -v myvals_q '%q ' "${myvals[#]}"
ssh user1#host1.domain.tld \
"myvals=( $myvals_q );"' for i in "${myvals[#]}"; do echo "$i"; done >>values'
If You Don't Need To Store The Value Locally In The First Place
#!/bin/bash
ssh user1#host1.domain.tld \
'while read -r i; do echo "$i"; done >>values' \
< <(psql -d mydb -c "select id from table1 where 't'")
General Notes
Running echo "$i" >>values over and over in a loop is inefficient: Every time the line is run, it re-opens the values file. Instead, run the redirection >values over the whole loop; this truncates the file exactly once, at the loop's start, and appends all values generated therein.
Unquoted expansions are generally dangerous. For example, if foo='*', then $foo will be replaced with a list of files in the current directory, but "$foo" will emit the exact contents -- *. Similarly, tabs, whitespace runs, and various other contents can be unintentionally damaged by unquoted expansion, even when passing directly to echo.
You can switch quoting types in the same string -- thus, "$foo"'$foo' is one string, the first part of which is replaced with the value of the variable named foo, and the second component of which is the exact string $foo.
You can send the output as a file:
#!/bin/bash
psql -d mydb -c "select id from table1 where 't'" > /tmp/values
scp values user1#host1.domain.tld:/tmp/
or pipe it to the remote host:
psql -d mydb -c "select id from table1 where 't'" | \
ssh user1#host1.domain.tld 'while read line; do echo $line; done'
Is it possible to add spaces to the left of every output to stdout (and stderr if possible) when I run commands in a bash shell script?
I'd like to do something like:
#!/bin/bash
echo Installing: Something
echo " => installing prerequisite1"
## INSERT MAGICAL LEFT SPACES COMMAND HERE ##
apt-get install -q -y prerequisite
## ANOTHER MAGICAL CANCELLING LEFT SPACES COMMAND HERE ##
echo " => installing prerequisite2"
# ... the padding again ...
wget http://abc.com/lostzilla.tar.gz
tar vzxf lostzilla.tar.gz
cd lostzilla-1.01
./configure
make && make install
# ... end of padding ...
echo Done.
Any idea?
EDIT: Added quotes to the echo command, otherwise they won't be padded.
Yes, you can quote them for simple things:
echo ' => installing prerequisite1'
and pipe the output through sed for complex things:
tar vzxf lostzilla.tar.gz 2>&1 | sed 's/^/ /'
The 2>&1 puts stdout and stderr onto the stdout stream and the sed replaces every start-of-line marker with three spaces.
How well this will work on something like wget which does all sorts of cursor manipulations I'm not sure.
Example shown here:
pax> ls -1 p*
phase1.py
phase1.sh
phase2.py
phase2.sh
primes.c
primes.exe
primes.sh
primes.stat
pax> ls -1 p* | sed 's/^/ /'
phase1.py
phase1.sh
phase2.py
phase2.sh
primes.c
primes.exe
primes.sh
primes.stat
One trick I've used in the past is to ensure that the scripts themselves take care of the indentation:
#!/bin/bash
if [[ "${DONT_EVER_SET_THIS_VAR}" = "" ]] ; then
export DONT_EVER_SET_THIS_VAR=except_for_here
$0 | sed 's/^/ /'
exit
fi
ls -1 p*
This will re-run the script with indentation through sed if it's not already doing so. That way, you don't have to worry about changing all your output statements. A bit of a hack, I know, but I tend to just do what's necessary for quick-and-dirty shell scripts.
If you want to turn spacing on and off, use the following awk script:
#!/usr/bin/gawk -f
/^#SPACEON/ { spaces=1; }
/^#SPACEOFF/ { spaces=0; }
!/^#SPACE/ {
if(spaces) {
print " " $0;
} else {
print $0;
}
}
Note that there are slight problems with your bash scipt. Notably, the use of => in your echo statements will output the character = to the file "installing".
#!/bin/bash
echo Installing: Something
echo '=> installing prerequisite1'
echo '#SPACEON'
echo You would see apt-get install -q -y prerequisite
echo '#SPACEOFF'
echo '=> installing prerequisite2'
echo '#SPACEON'
echo You would see wget http://abc.com/lostzilla.tar.gz
echo You would see tar vzxf lostzilla.tar.gz
echo You would see cd lostzilla-1.01
echo You would see ./configure
echo You would see make \&\& make install
echo '#SPACEOFF'
echo Done.
Combining the two gives me:
$ ./do-stuff | ./magic-spacing
Installing: Something
=> installing prerequisite1
You would see apt-get install -q -y prerequisite
=> installing prerequisite2
You would see wget http://abc.com/lostzilla.tar.gz
You would see tar vzxf lostzilla.tar.gz
You would see cd lostzilla-1.01
You would see ./configure
You would see make && make install
Done.
Where do-stuff is your bash script and magic-spacing is my awk script above.
Depending on how the command writes to stdout, you can just indent with a simple awk script:
$ echo -e 'hello\nworld' | awk '{print " ",$0}'
hello
world
Quite un-magical you can use printf to do the following:
# space padding for single string
printf "%-4s%s\n" "" "=> installing prerequisite1"
# space padding for single command output
# use of subshell leaves original IFS intact
( IFS=$'\n'; printf " %s\n" $(command ls -ld * 2>&1) )
# note: output to stderr is unbuffered
( IFS=$'\n'; printf " %s\n" $(command ls -ld * 1>&2) )
It's also possible to group commands by enclosing them in curly braces and space-padd their output like so:
{
cmd1 1>&2
cmd2 1>&2
cmd3 1>&2
} 2>&1 | sed 's/.*/ &/'
It's possible to redirect stdout to stderr script/shell-wide using exec ...
(
exec 1>&2
command ls -ld *
) 2>&1 | sed 's/^/ /'
Use python pyp (The Pyed Piper):
ls -ld | pyp "' '+p"