Why does "set VAR=value" not work in bash? [duplicate] - bash

This question already has an answer here:
Why does sourcing a script with "set var = value" break $#?
(1 answer)
Closed 3 years ago.
I am having problem setting a environment variable correctly.
These are first few lines of my deploy.sh:
if [[ -z ${PEM_PATH+x} ]]; then
printf "Please set the PEM_PATH environment variable\n"
exit 1
fi
This is my terminal output:
bash-3.2$ set PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem"
bash-3.2$ ls
Jenkinsfile bps-dashboard.iml mvnw node_modules package.json src webpack.config.js
README.md deploy.sh mvnw.cmd package-lock.json pom.xml target
bash-3.2$ set PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem"
bash-3.2$ echo $PEM_PATH
/Users/Elasticsearch/Desktop/ec2-poc.pem
bash-3.2$ sh deploy.sh
Please set the PEM_PATH environment variable
bash-3.2$
What am I doing wrong here ?

set does not set variables in bash (or other POSIX-family shells) -- it configures shell option flags, or changes the active argument list ($1, $2, and so on).
You can run any of the following to define PEM_PATH as an environment variable (without the export or the -x argument to declare or the use of set -a it would be a regular, non-exported shell variable):
export PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem" -- both defining and exporting the variable with a single command.
PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem"; export PEM_PATH -- first defining PEM_PATH as a regular shell variable, then promoting it to an environment variable
declare -x PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem" -- using the bash-only extension declare to define PEM_PATH as an exported variable.
set -a; PEM_PATH="/Users/Elasticsearch/Desktop/ec2-poc.pem"; set +a -- using set -a to make all variables exported to the environment by default; then setting the variable; then turning off that flag.

Related

Clean environment for bash shell

Is there a way to tell a bash script not to import any variables from the parent shell i.e. ignore exported variables. There is such capability in slurm --export=NONE so I wonder if there is an option I can put in the #!/bin/bash line to get similar behavior.
On linux the cleanest option I found was:
#!/usr/bin/env -S - bash
env
which for me prints:
PWD=/home/allan
SHLVL=1
_=/usr/bin/env
Another option is:
#!/usr/bin/env bash
[ -n "$HOME" ] && exec -c "$0"
env
Possible using $BASH_SOURCE[0] instead of $0 as the latter can be set by user. $BASH_SOURCE, however, is not always set. Hard-coding the script path would work but that's ugly.

Automatically conda activate when in directory and source/ export

Goal: Automatically execute bash commands if when in directory.
For example, if I enter a git project directory, I'd like bash to run the following for me:
conda activate
export VAR_NAME=foo
I attempted by appending to ~/.bashrc, but with no luck:
...
if [ -f "/home/me/PycharmProjects/project/" ]; then
conda activate project_venv
export KEY=foo
export SECRET=bar
fi
You can set PROMPT_COMMAND, see the bash docs. Bash executes its value (or if it's an array, each of its values) before every prompt, so you can do whatever you want when PWD changes.
You can add this function to your ~/.bashrc:
cd () {
command cd "$#" &&
if [[ $(pwd) = '/home/me/PycharmProjects/project' ]]; then
conda activate project_venv
export KEY=foo SECRET=bar
fi
}
Because you are exporting in a function you need to use declare -gx
declare --help will give you the best and most accurate reason why but it is because all function vars are technically local. The -g create a global exported var for a function is ignored if not in a function and the -x is what export is an alias for. export is just declare -x. You will also need to source your script files
So it will look like this
declare -gx KEY=foo
declare -gx SECRET=bar
cd () {
command cd "$#" &&
if [[ $(pwd) = '/home/me/PycharmProjects/project1' ]]; then
conda activate project1
source ~/miniconda3/etc/activate.d/env_vars.sh
elif [[ $(pwd) = '/home/me/PycharmProjects/project2' ]]; then
conda activate project2
else
source ~/miniconda3/etc/deactivate.d/env_vars.sh
fi
}
Full disclosure I'm not sure if the -x is completely necessary but I do it in case of sourcing a script.
Also storing in secrets in ~/.bashrc is a general no no as it leads to bad actors getting secrets. Ontop of slowing down your interactive shell loading times

How to export environment variables through Makefile [duplicate]

I am trying to set an Environment variable in a Makefile, so it can be used in another program running in the sam shell as make, but after make has run.
Update: This is not possible according to accepted answer with comments.
Steps:
run make test setting env: export TEST_ENV_ONE=OneString
run another program, that can read TEST_ENV_ONE
Tried this:
Not working:
test:
export TEST_ENV_ONE=OneString
$(shell export TEST_ENV_TWO=TwoString)
Afterwards this is empty:
echo $TEST_ENV_ONE
echo $TEST_ENV_TWO
Your export TEST_ENV_ONE=OneString above is running in a dedicated shell. The subsequent commands run in other shell instances. Therefore, they don't inherit the environment variable TEST_ENV_ONE.
You could use a top-level (i.e., not in a target's recipe) export directive in the makefile:
export env_var := MyEnvVariable
.PHONY: all
all:
echo env_var: $$env_var
This way, the variable env_var is exported to the shells that will execute the recipes.
If you run make with the makefile above:
$ make
echo env_var: $env_var
env_var: MyEnvVariable
As you can see from the output, the shell that run echo env_var: $env_var had the variable env_var in its environment.
If you want the environment variables to be exported to the shell from which you invoked make things are a bit difficult because, as explained by ネロク, you cannot directly export environment variables from a child process (make) to its parent process (the shell from which you invoke make). But if you accept to invoke make like this:
eval "$(make)"
then it is indeed possible: just echo export VARIABLE1=VALUE1; export VARIABLE2=VALUE2; ... in your recipe. Warning: you will also have to guarantee that nothing else gets echoed by make on the standard input. But you can use the standard error, instead. Example:
$ cat Makefile
export TEST_ENV_ONE := OneString
all:
#printf 'make variable TEST_ENV_ONE = %s\n' "$(TEST_ENV_ONE)" 1>&2
#printf 'in-recipe shell variable TEST_ENV_ONE = %s\n' "$$TEST_ENV_ONE" 1>&2
#printf 'export TEST_ENV_ONE="%s"\n' '$(TEST_ENV_ONE)'
$ unset TEST_ENV_ONE
$ printenv TEST_ENV_ONE
$ eval "$(make)"
make variable TEST_ENV_ONE = OneString
in-recipe shell variable TEST_ENV_ONE = OneString
$ printenv TEST_ENV_ONE
OneString
Note that make more or less considers environment variables as make variables. From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
So, unless the value of your variable is the result of complex computations by make itself, a much more natural way to obtain the same result would be to define the environment variable in the parent shell and to use it as is in the Makefile:
$ cat Makefile
all:
#printf 'make variable TEST_ENV_ONE = %s\n' "$(TEST_ENV_ONE)"
#printf 'in-recipe shell variable TEST_ENV_ONE = %s\n' "$$TEST_ENV_ONE"
$ export TEST_ENV_ONE=OneString
$ make
make variable TEST_ENV_ONE = OneString
in-recipe shell variable TEST_ENV_ONE = OneString
$ printenv TEST_ENV_ONE
OneString

how can I run a script at the bash command line such that its configuration is identical to its configuration when it is run by cron?

I want to write and test scripts that do not depend on any of the personal environment variables accessible to me at the command line. For instance, each time I open a new command line window on my macbook pro, $PATH is updated by having my personal bin directory prepended to it --- as well as several other directories, such as
/usr/sbin:/sbin:/Library/TeX/texbin:/Library/Apple/usr/bin
This occurs apparently because my $HOME/.profile is automatically sourced when a window opens up. In addition, a large number of other variables are defined at login because my personal .bashrc is sourced automatically.
Ultimately, I want to be able to run a script as a cron job --- using crontab -l. But if the script depends on my personal configuration, or if it tries to access a script that lives in my personal bin, it will halt with error.
So I write a script, test it at the command line, set up a cron job to run it; and when the time comes, it halts with error, because it turns out that the script depended on one of my personal environment variables.
Currently, then, the only way I know to test such a script is to set up a cron job --- say, five minutes from now---and then wait for the cron job to run. It's clumsy and slow.
Is there a direct and immediate way to run a script so that it does not know anything from $HOME/.profile?
The script I tested this with:
#!/usr/bin/perl
use strict; use warnings;
use Data::Dumper qw(Dumper);
$Data::Dumper::Sortkeys = 1;
print Dumper {%ENV};
The following command
> jaw20210419test.pl | wc
2532 3098 118462
produces many lines of output, as you see, because of all that I define in my .bashrc.
With the following,
> sudo jaw20210419test.pl | wc
Password:
20 59 831
>
the output is much less, but it still has an updated $PATH including my personal bin. I do not understand this, since the $PATH gets updated by $HOME/.profile, and that file sources my personal $HOME/u/kh/.bashrc.
If however I run this with a cron job, then it does not have an updated $PATH. The script says
$VAR1 = {
'HOME' => '/Users/kpr',
'LOGNAME' => 'kpr',
'PATH' => '/usr/bin:/bin',
'PWD' => '/Users/kpr',
'SHELL' => '/bin/sh',
'SHLVL' => '1',
'USER' => 'kpr',
'VERSIONER_PERL_VERSION' => '5.18',
'_' => '/Users/kpr/u/kh/bin/jaw20210419test.pl',
'__CF_USER_TEXT_ENCODING' => '0x1F5:0x0:0x0'
};
This is the configuration I want to test the script with. Note that it knows who I am --- $USER --- and where $HOME is.
By contrast, two other proposed solutions, from
How to start a shell without any user configuration?,
do not even know know where $HOME is:
> echo $cdbin # personal environment variable
/Users/kpr/u/kh/bin
> env -i perl $cdbin/jaw20210419test.pl
$VAR1 = {
'VERSIONER_PERL_VERSION' => '5.18',
'__CF_USER_TEXT_ENCODING' => '0x1F5:0x0:0x0'
};
> env --noprofile --norc perl $cdbin/jaw20210419test.pl
env: illegal option -- n
usage: env [-iv] [-P utilpath] [-S string] [-u name]
[name=value ...] [utility [argument ...]]
> echo $SHELL
/bin/bash
>
> env -i sh -c $cdbin/jaw20210419test.pl
$VAR1 = {
'PWD' => '/Users/kpr/u/sng/2021/FR-Wegelin-TO-stackoverflow',
'SHLVL' => '1',
'VERSIONER_PERL_VERSION' => '5.18',
'_' => '/Users/kpr/u/kh/bin/jaw20210419test.pl',
'__CF_USER_TEXT_ENCODING' => '0x1F5:0x0:0x0'
};
so I am still lost.
Of note, you can not technically do what you wish. Bash goes to great lengths to determine if you are running an interactive shell or a non-interactive shell. Running from cron will forever be non-interactive as is running from a terminal will forever be marked as interactive.
You want a number of entries not present in cron for your test shell, for example PS1 for a prompt. Also there are quite a few read only environment variables. If you try to unset them you get an error:
bash: unset: SHELLOPTS: cannot unset: readonly variable
You can get 95 percent there by this method:
% env -i bash --noprofile --norc
bash-3.2$ source be-cron.sh
bash-3.2$
The file be-cron.sh will "unset" most of the interactive environment variables:
#!/bin/bash
unset BASH
unset BASH_ENV
unset BASH_VERSION
unset COLORFGBG
unset COLORTERM
unset COLUMNS
unset COMMAND_MODE
unset DIRSTACK
unset DISPLAY
unset EDITOR
unset GROUP
unset GROUPS
unset HISTFILE
unset HISTFILESIZE
unset HISTSIZE
unset HOST
unset HOSTNAME
unset HOSTTYPE
unset IFS
unset ITERM_PROFILE
unset ITERM_SESSION_ID
unset LC_TERMINAL
unset LC_TERMINAL_VERSION
unset LINES
unset LaunchInstanceID
unset MACHTYPE
unset MAILCHECK
unset MANPATH
unset MANPATH_WITHOUT_PERLBREW
unset OPTERR
unset OPTIND
unset OSTYPE
unset PATH_WITHOUT_PERLBREW
unset PERLBREW_HOME
unset PERLBREW_PATH
unset PERLBREW_ROOT
unset PERLBREW_SHELLRC_VERSION
unset PIPESTATUS
unset PS2
unset PS4
unset SECURITYSESSIONID
unset SSH_AUTH_SOCK
unset TERM
unset TERM_PROGRAM
unset TERM_PROGRAM_VERSION
unset TERM_SESSION_ID
unset TMPDIR
unset VENDOR
unset XPC_FLAGS
unset XPC_SERVICE_NAME
unset __CFBundleIdentifier

Is it possible to set Environment variables in Makefile - to be used after

I am trying to set an Environment variable in a Makefile, so it can be used in another program running in the sam shell as make, but after make has run.
Update: This is not possible according to accepted answer with comments.
Steps:
run make test setting env: export TEST_ENV_ONE=OneString
run another program, that can read TEST_ENV_ONE
Tried this:
Not working:
test:
export TEST_ENV_ONE=OneString
$(shell export TEST_ENV_TWO=TwoString)
Afterwards this is empty:
echo $TEST_ENV_ONE
echo $TEST_ENV_TWO
Your export TEST_ENV_ONE=OneString above is running in a dedicated shell. The subsequent commands run in other shell instances. Therefore, they don't inherit the environment variable TEST_ENV_ONE.
You could use a top-level (i.e., not in a target's recipe) export directive in the makefile:
export env_var := MyEnvVariable
.PHONY: all
all:
echo env_var: $$env_var
This way, the variable env_var is exported to the shells that will execute the recipes.
If you run make with the makefile above:
$ make
echo env_var: $env_var
env_var: MyEnvVariable
As you can see from the output, the shell that run echo env_var: $env_var had the variable env_var in its environment.
If you want the environment variables to be exported to the shell from which you invoked make things are a bit difficult because, as explained by ネロク, you cannot directly export environment variables from a child process (make) to its parent process (the shell from which you invoke make). But if you accept to invoke make like this:
eval "$(make)"
then it is indeed possible: just echo export VARIABLE1=VALUE1; export VARIABLE2=VALUE2; ... in your recipe. Warning: you will also have to guarantee that nothing else gets echoed by make on the standard input. But you can use the standard error, instead. Example:
$ cat Makefile
export TEST_ENV_ONE := OneString
all:
#printf 'make variable TEST_ENV_ONE = %s\n' "$(TEST_ENV_ONE)" 1>&2
#printf 'in-recipe shell variable TEST_ENV_ONE = %s\n' "$$TEST_ENV_ONE" 1>&2
#printf 'export TEST_ENV_ONE="%s"\n' '$(TEST_ENV_ONE)'
$ unset TEST_ENV_ONE
$ printenv TEST_ENV_ONE
$ eval "$(make)"
make variable TEST_ENV_ONE = OneString
in-recipe shell variable TEST_ENV_ONE = OneString
$ printenv TEST_ENV_ONE
OneString
Note that make more or less considers environment variables as make variables. From GNU make manual:
Variables in make can come from the environment in which make is run.
Every environment variable that make sees when it starts up is
transformed into a make variable with the same name and value.
However, an explicit assignment in the makefile, or with a command
argument, overrides the environment. (If the ‘-e’ flag is specified,
then values from the environment override assignments in the makefile.
See Summary of Options. But this is not recommended practice.)
So, unless the value of your variable is the result of complex computations by make itself, a much more natural way to obtain the same result would be to define the environment variable in the parent shell and to use it as is in the Makefile:
$ cat Makefile
all:
#printf 'make variable TEST_ENV_ONE = %s\n' "$(TEST_ENV_ONE)"
#printf 'in-recipe shell variable TEST_ENV_ONE = %s\n' "$$TEST_ENV_ONE"
$ export TEST_ENV_ONE=OneString
$ make
make variable TEST_ENV_ONE = OneString
in-recipe shell variable TEST_ENV_ONE = OneString
$ printenv TEST_ENV_ONE
OneString

Resources