I need to generate a random string and save in a variable, with a Makefile.
I wrote
install:
LOCALSTORAGE_ACCESS_TOKEN = echo $RANDOM | md5sum | head -c 20; echo;
echo $(LOCALSTORAGE_ACCESS_TOKEN)
When I launch it with make install, I get
LOCALSTORAGE_ACCESS_TOKEN = echo ANDOM | md5sum | head -c 20; echo;
/bin/sh: 1: LOCALSTORAGE_ACCESS_TOKEN: not found
d41d8cd98f00b204e980
echo
So
missing assigned value to the variable LOCALSTORAGE_ACCESS_TOKEN.
getting every time same result for RANDOM
I tried also
LOCALSTORAGE_ACCESS_TOKEN = echo $$RANDOM | md5sum | head -c 20; echo;
echo $(LOCALSTORAGE_ACCESS_TOKEN)
with same result
LOCALSTORAGE_ACCESS_TOKEN = echo $RANDOM | md5sum | head -c 20; echo;
/bin/sh: 1: LOCALSTORAGE_ACCESS_TOKEN: not found
d41d8cd98f00b204e980
echo
Each line of a recipe is a shell script, run by a different shell. Your first line is syntactically incorrect. You are trying to assign a make variable in a recipe, this is not how make works. Try:
install:
LOCALSTORAGE_ACCESS_TOKEN=$$(echo $$RANDOM | md5sum | head -c 20); \
echo $$LOCALSTORAGE_ACCESS_TOKEN
LOCALSTORAGE_ACCESS_TOKEN is now a shell variable, it is assigned using shell syntax in the first line of the recipe but is still defined in the second line because of the line continuation (the trailing \). Note the use of $$ instead of $ to escape the first expansion by make.
If you would like LOCALSTORAGE_ACCESS_TOKEN to be a make variable available in all lines of all recipes then simply assign it as a real make variable and outside any recipe :
LOCALSTORAGE_ACCESS_TOKEN := $(shell echo $$RANDOM | md5sum | head -c 20)
install:
echo $(LOCALSTORAGE_ACCESS_TOKEN)
This should work:
install:
$(eval LOCALSTORAGE_ACCESS_TOKEN := $(shell /bin/bash -c 'echo $$RANDOM' | md5sum | head -c 20; echo))
echo $(LOCALSTORAGE_ACCESS_TOKEN)
As you also noticed, you need $$RANDOM in order to mask the $ sign in the Makefile
You need to run the command in a bash, because $RANDOM may not work in all shells (did not work for me, so I had to use bash)
You need the $(shell ... so that the variable gets assigned the output of the command (instead of the command text itself)
You need $(eval ... because the code is inside a target, and you can not define variables in targets otherwise
Related
In Bash, you can see
set --help
-x Print commands and their arguments as they are executed.
Here's test code:
# make script
echo '
#!/bin/bash
set -x
n=$(echo "a" | wc -c)
for i in $(seq $n)
do
file=test_$i.txt
eval "ls -l | head -$i"> $file
rm $file
done
' > test.sh
# execute
chmod +x test.sh
./test.sh 2> stderr
# check
cat stderr
Output
+++ echo a
+++ wc -c
++ n=2
+++ seq 2
++ for i in $(seq $n)
++ file=test_1.txt
++ eval 'ls -l | head -1'
+++ ls -l
+++ head -1
++ rm test_1.txt
++ for i in $(seq $n)
++ file=test_2.txt
++ eval 'ls -l | head -2'
+++ ls -l
+++ head -2
++ rm test_2.txt
What is the meaning of the number of + signs at the beginning of each row in the file? It's kind of obvious, but I want to avoid misinterpreting.
In addition, can a single + sign appear there? If so, what is the meaning of it?
The number of + represents subshell nesting depth.
Note that the entire test.sh script is being run in a subshell because it doesn't begin with #!/bin/bash. This has to be on the first line of the script, but it's on the second line because you have a newline at the beginning of the echo argument that contains the script.
When a script is run this way, it's executed by the original shell in a subshell, approximately like
( source test.sh )
Change that to
echo '#!/bin/bash
set -x
n=$(echo "a" | wc -c)
for i in $(seq $n)
do
file=test_$i.txt
eval "ls -l | head -$i"> $file
rm $file
done
' > test.sh
and the top-level commands being run in the script will have a single +.
So for example the command
n=$(echo "a" | wc -c)
produces the output
++ echo a
++ wc -c
+ n=' 2'
echo a and wc -c are executed in the subshell created for the command substitution, so they get two +, while n=<result> is executed in the original shell with a single +.
From man bash:
-x
After expanding each simple command, for command, case command, select command, or arithmetic for command, display the expanded value of PS4, followed by the command and its expanded arguments or associated word list.
So what's PS4 here?
PS4
The value of this parameter is expanded as with PS1 and the value is printed before each command bash displays during an execution trace. The first character of the expanded value of PS4 is replicated multiple times, as necessary, to indicate multiple levels of indirection. The default is + .
The meaning of "indirection" is not further explained, as far as I can find...
I have the following command as a makefile command:
update-env:
echo "{ \"Variables\":${ENV_VALUE}}" > ./new_env.json && \
UPDATE_ENVVARS=$$(jq -s '.[0] as $$a |.[1] as $$b | $$a * $$b' old_env.json new_env.json | jq -c . ) && \
echo "${UPDATE_ENVVARS}"
the ${ENV_VALUE} is taken when
make update-env ENV_VALUE="{\\\"HOST_URL\\\": \\\"https:\\\/\\\/test.com\\\"}"
the file new_env.json is generated properly
when executing jq -s '.[0] as $a |.[1] as $b | $a * $b' old_env.json new_env.json | jq -c . it generates the appropriate compact json result desired.
When running everything in sequence (the assignment and echo for validation), I get an empty result.
My goal for the command is to merge two json output and assign it to the UPDATE_ENVVARS for it to be reused as an input for another command that will accept the json. Per testing, it came out empty when echo, when I execute the jq solo, the merge output is functional.
Only a minor bit of editing was needed:
update-env:
echo '{ "Variables":${ENV_VALUE}}' > ./new_env.json && \
UPDATE_ENVVARS=$$(jq -s '.[0] as $$a |.[1] as $$b | $$a * $$b' old_env.json new_env.json | jq -c . ) && \
echo "$${UPDATE_ENVVARS}"
Note:
We're using single quotes in the first-line echo -- the substitution is performed by make, not by the shell, so single quotes don't suppress it.
We're doubling up the $$ in the last line, so we're expanding the shell variable set in the second line of the recipe, not a make variable that nothing ever set at all.
See this working at https://replit.com/#CharlesDuffy2/RoyalIdolizedProfile
By contrast, if you want to assign to a make variable instead of a shell variable, this question is duplicative of Makefile command substitution problem, and the answer by Ignacio Vazquez-Abrams is appropriate.
Below is my makefile
LATEST_GIT_COMMIT_HASH := $(shell git rev-parse HEAD | cut -c1-8)
docker:
#echo "\n" &&\
read -p "Please enter a version number (ex: 0001): " release_vesion &&\
echo $$release_vesion_${LATEST_GIT_COMMIT_HASH}
I try to concat the release_version from my input and the git version number then print them out.
But it seem not working.
How to concat the string with input value?
In bash you need to use ${var_name}concat_some_more_text, the braces seperate the variable from the rest of the concatented string. Eg:
var_name=bob
echo "1: $var_nameconcat_some_more_text"
echo "2: ${var_name}concat_some_more_text"
returns:
1:
2: bobconcat_some_more_text
So your code should look like:
LATEST_GIT_COMMIT_HASH := $(shell git rev-parse HEAD | cut -c1-8)
docker:
#echo "\n" &&\
read -p "Please enter a version number (ex: 0001): " release_vesion &&\
echo "release ver: $$release_vesion" ; \
echo "$${release_vesion}_$(LATEST_GIT_COMMIT_HASH)"
In make use $${...} for bash variable (you have correctly used the double $ but not the braces) and $(...) for make variables. You don't always need the braces for the bash variable but when concatenating the variable name to another string like you are then you do need them to separate them.
I am looking for a way to set a variable in the statements passed to xargs. The value is to be manipulated in one of the commands. Using a file or another utility is an option but I am not sure why setting the bash variable in the sequence is always coming up as empty.
$ ls c*txt
codebase.txt consoleText.txt
$ ls c*txt | xargs -i bash -c "echo processing {}; v1={} && echo ${v1/txt/file}"
codebase.txt consoleText.txt
processing codebase.txt
processing consoleText.txt
The example above distills the question to the basics. I was expecting the behavior to be something like this but inline:
$ fname=codebase.txt; echo ${fname/txt/file}
codebase.file
Thank you.
This line is resolving ${v1/txt/file} to a value before the command is executed:
$ ls c*txt | xargs -i bash -c "echo processing {}; v1={} && echo ${v1/txt/file}"
And that means the bash -c doesn't even see ${v1/txt/file}
In this line the single quotes inhibit the variable substitution so echo processing {}; v1={} && echo ${v1/txt/file} is actually passed to bash -c as a parameter:
$ ls c*txt | xargs -i bash -c 'echo processing {}; v1={} && echo ${v1/txt/file}'
You could accomplish the same thing by escaping the dollar sign:
$ ls c*txt | xargs -i bash -c "echo processing {}; v1={} && echo \${v1/txt/file}"
I have a command line tool which receives two arguments:
TOOL arg1 -o arg2
I would like to invoke it with the same argument provided it for arg1 and arg2, and to make that easy for me, i thought i would do:
each <arg1_value> | TOOL $1 -o $1
but that doesn't work, $1 is not replaced, but is added once to the end of the commandline.
An explicit example, performing:
cp fileA fileA
returns an error fileA and fileA are identical (not copied)
While performing:
echo fileA | cp $1 $1
returns the following error:
usage: cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file target_file
cp [-R [-H | -L | -P]] [-fi | -n] [-apvX] source_file ... target_directory
any ideas?
If you want to use xargs, the [-I] option may help:
-I replace-str
Replace occurrences of replace-str in the initial-arguments with names read from standard input. Also, unquoted blanks do not terminate input items; instead the separa‐
tor is the newline character. Implies -x and -L 1.
Here is a simple example:
mkdir test && cd test && touch tmp
ls | xargs -I '{}' cp '{}' '{}'
Returns an Error cp: tmp and tmp are the same file
The xargs utility will duplicate its input stream to replace all placeholders in its argument if you use the -I flag:
$ echo hello | xargs -I XXX echo XXX XXX XXX
hello hello hello
The placeholder XXX (may be any string) is replaced with the entire line of input from the input stream to xargs, so if we give it two lines:
$ printf "hello\nworld\n" | xargs -I XXX echo XXX XXX XXX
hello hello hello
world world world
You may use this with your tool:
$ generate_args | xargs -I XXX TOOL XXX -o XXX
Where generate_args is a script, command or shell function that generates arguments for your tool.
The reason
each <arg1_value> | TOOL $1 -o $1
did not work, apart from each not being a command that I recognise, is that $1 expands to the first positional parameter of the current shell or function.
The following would have worked:
set - "arg1_value"
TOOL "$1" -o "$1"
because that sets the value of $1 before calling you tool.
You can re-run a shell to perform variable expansion, with sh -c. The -c takes an argument which is command to run in a shell, performing expansion. Next arguments of sh will be interpreted as $0, $1, and so on, to use in the -c. For example:
sh -c 'echo $1, i repeat: $1' foo bar baz will print execute echo $1, i repeat: $1 with $1 set to bar ($0 is set to foo and $2 to baz), finally printing bar, i repeat: bar
The $1,$2...$N are only visible to bash script to interpret arguments to those scripts and won't work the way you want them to. Piping redirects stdout to stdin and is not what you are looking for either.
If you just want a one-liner, use something like
ARG1=hello && tool $ARG1 $ARG1
Using GNU parallel to use STDIN four times, to print a multiplication table:
seq 5 | parallel 'echo {} \* {} = $(( {} * {} ))'
Output:
1 * 1 = 1
2 * 2 = 4
3 * 3 = 9
4 * 4 = 16
5 * 5 = 25
One could encapsulate the tool using awk:
$ echo arg1 arg2 | awk '{ system("echo TOOL " $1 " -o " $2) }'
TOOL arg1 -o arg2
Remove the echo within the system() call and TOOL should be executed in accordance with requirements:
echo arg1 arg2 | awk '{ system("TOOL " $1 " -o " $2) }'
Double up the data from a pipe, and feed it to a command two at a time, using sed and xargs:
seq 5 | sed p | xargs -L 2 echo
Output:
1 1
2 2
3 3
4 4
5 5