Set case sensitive Windows environment variables in Perl - windows

I am trying to set some Windows environment variables in Perl with case sensitivity in order to use them in a shell script.
However, I noticed that the environment variables are all in uppercase when I try to use them in lowercase in a shell script called from Perl with system command and run with Msys.
For example, the following script:
#!perl
system "echo echo TOTO=\$TOTO > toto.sh";
system "echo echo Titi=\$Titi >> toto.sh";
system "echo echo TITI=\$TITI >> toto.sh";
$ENV{'TOTO'}="0+0";
$ENV{'Titi'}="Not toto!";
system("sh toto.sh");
returns (while run in Msys) the following output:
TOTO=0+0
Titi=
TITI=Not toto!
Does anybody know whether this is possible in Windows (i.e. how to set the environment variable Titi and not TITI)?
Thanks

Windows environment variable identifiers are case-independent in the same way as Windows files.
Titi and TITI are both names for the same variable, and its value can be acccessed through either $ENV{TITI} or $ENV{Titi} in Perl. Likewise echo %TITI% and echo %Titi% on the command line will give the same result.
If you explain why it is that you need case-sensitive environment variable names then we may be able to help you.

The following should apply independently of the casing problem.
Let's take the first line:
system "echo echo TOTO=\$TOTO > toto.sh";
If you call
echo echo TOTO=\$TOTO
on your command line, the output will be:
echo TOTO=\$TOTO
which seems to be, what you want.
However, the line you are calling with system is first interpreted by perl, so your escaped \$ becomes a $.
The output then depends on the current value of $TOTO (before you set the environment variable in the script), which is probably empty.
You should change all those system calls to be single quoted:
system 'echo echo TOTO=\$TOTO > toto.sh';
If you do this, you will get the following output:
TOTO=0+0
Titi=
TITI=not TOTO
Titi is empty, because you didn't assign a value to it.

It appeared to be that it is not possible to do: Windows will only use the uppercase variables.
The way I did was to temporarily transform the script by replacing variables with their upper-case equivalent and switch them back to the previous state after using them.
In my case, I had to deal with the variables exported from the shell script Run_Session.sh. I have created a Perl script to perform the uppercase replacement and this is the result.
###############################
## SUBROUTINE uc_variables ##
#-----------------------------#
# This subroutine convert the environment variables exported
# by the top level script to upper case, and can revert the operation.
#
# Input:
# -----
# $mode: "ON" (default) or "OFF"
#
# Output:
# ------
# N/A
#-----------------------#
#########################
sub uc_variables {
my $mode=shift;
chdir $Bin;
if ($mode eq "OFF") {
foreach my $bak (glob "{*,*/*}.sh.bak") {
(my $orig = $bak)=~s/\.bak//;
move($bak,$orig);
}
} else {
#Check if back-up already exists
uc_variables("OFF") if (-e "Run_Session.sh.bak");
#List variables to update
my %uc_var;
open RUN_SESSION,"<Run_Session.sh" or die "Error while reading Run_Session.sh ($!)\n";
map { /export\s+(\w*[a-z]\w*)/ and ($uc_var{$1}=uc($1))=~s/Directory/DIR/i } <RUN_SESSION>;
close RUN_SESSION;
#Replace variables
foreach my $shell (glob "{*,*/*}.sh") {
#Back-up
copy($shell,"$shell.bak");
#Read data
open SHELL,"<$shell" or die "Error while reading $shell ($!)\n";
my $SHELL = join("", <SHELL>);
close SHELL;
#Replace
map { $SHELL=~s/$_/$uc_var{$_}/g } keys %uc_var;
#Print
open SHELL,">$shell" or die "Error while writing in $shell ($!)\n";
print SHELL $SHELL;
close SHELL;
}
}
} #end of uc_variables

Related

PowerShell flexibility in scope of environment variables via scripts

In bash, if I define a variable inside a script (say, set_var.sh), I can choose whether those definitions persist after "running" the script.
This will depend on how I "run" the script, the options being:
sh set_var.sh (variables do not persist)
./set_var.sh (variables do not persist, same as point 1)
source set_var.sh (variables persist)
This is irrespective from the variable being exported to the environment or not in set_var.sh.
Can the same as items 1 and 3 above be achieved with PowerShell 5.1 scripts, for both PS and $env variables?
For an illustration see (1) below.
EDIT:
As per answer by Mathias R. Jessen, the equivalent to item #3 above is "dot sourcing".
There is also an "intermediate" case (not identified above, perhaps there is also a way to get this in bash), where environment variables persist but PS variables don't.
My script:
# set_var.ps1
$env:TEST_VAR = 'test_var'
$TEST_VAR2 = 'test_var2'
What I checked:
> $env:TEST_VAR ; $TEST_VAR2 ;
> . .\set_var.ps1
> $env:TEST_VAR ; $TEST_VAR2 ;
test_var
test_var2
> Remove-Variable TEST_VAR2 ; Remove-Item env:TEST_VAR ;
> $env:TEST_VAR ; $TEST_VAR2 ;
> .\set_var.ps1
> $env:TEST_VAR ; $TEST_VAR2 ;
test_var
> Remove-Item env:TEST_VAR ;
> $env:TEST_VAR ; $TEST_VAR2 ;
> & .\set_var.ps1
> $env:TEST_VAR ; $TEST_VAR2 ;
test_var
(1) Example of persistent / non-persistent variables
I have script set_var.sh with the following contents:
#!/bin/bash
export TEST_VAR=test_var
TEST_VAR2=test_var2
Then the following commands prove my point:
$ echo $TEST_VAR ; echo $TEST_VAR2
$ sh set_var.sh ; echo $TEST_VAR ; echo $TEST_VAR2
$ source set_var.sh ; echo $TEST_VAR ; echo $TEST_VAR2
test_var
test_var2
$ echo $TEST_VAR ; echo $TEST_VAR2
test_var
test_var2
$ env | grep TEST_VAR
TEST_VAR=test_var
$ unset TEST_VAR ; unset TEST_VAR2
$ echo $TEST_VAR ; echo $TEST_VAR2
$ ./set_var.sh ; echo $TEST_VAR ; echo $TEST_VAR2
$ echo $TEST_VAR ; echo $TEST_VAR2
To complement Mathias' helpful answer:
Regarding your edit:
It is a script's direct invocation / invocation via & - which, unlike in sh / bash, runs in-process - preserves environment-variable changes, but not regular variables in the script.[1]
Using & with a script block ({ ... }) in lieu of a script file for a succinct demonstration:
PS> & { $foo = 'bar'; $env:foo = 'bar-env' }; "[$foo]", "[$env:foo]"
[] # regular variable in the script [block] scope went out of scope
[bar-env] # environment variable is visible to caller (the whole *process*)
As for the PowerShell equivalent of 1. (sh set_var.sh)
and 2. (./set_var.sh):
An explicit call to the PowerShell CLI (powershell.exe in Windows PowerShell, pwsh in PowerShell [Core], v6+): is needed to run a script in a child process.
Note: The following examples use pwsh, but the same applies to powershell.exe (whose CLI is a mostly identical to pwsh's, with only a few parameters having been added to the latter).
In the simplest case, use -File (which is the implied parameter in PowerShell [Core], but is required in Windows PowerShell):
pwsh -File set_var.ps1 # PowerShell v6+: same as: pwsh set_var.ps1
Note, however, that you'll only get textual output (strings) from the script this way rather than the rich objects that PowerShell normally supports.
The simplest way to receive objects instead - available from inside PowerShell only[2] - is to pass a script block in which the script is called instead, which automatically makes the CLI output XML-serialized representations of the script's output objects, and causes these representations to be automatically deserialized by the calling session:
pwsh { .\set_var.ps1 }
Note:
The .\ prefix is necessary, because PowerShell - by design, as a security feature - does not permit running scripts by mere file name from the current directory.
The XML-based serialization using the CLIXML format - which is also used for PowerShell's remoting and background jobs - has limits with respect to type fidelity; except for .NET primitive types and a handful well-known types, you get [pscustomobject]-based approximations of the original objects - see this answer.
[1] However, PowerShell does offer the ability to set variables across in-session scope boundaries, e.g. $global:foo = 'bar' to set a session-global variable or Set-Variable foo bar -Scope 1 to set a variable in the caller's (parent) scope (which when called from a script's top-level scope would also be the global scope).
[2] From outside of PowerShell, you can explicitly request output in CLIXML format (the XML-based object-serialization format), namely with the -of Xml CLI parameter, but you'll have to parse the XML yourself and reconstruct objects from it.
You want . - the dot-sourcing operator - which executes your script or command in the caller's scope, thereby persisting all variables after execution:
# set_vars.ps1
$TEST_VAR = 123
# script.ps1
. $PSScriptRoot/set_vars.ps1
echo $TEST_VAR # outputs 123
To avoid locally defined variables persisting after execution, either execute the script as-is, or by explicitly using the & invocation operator:
# script.ps1
& $PSScriptRoot/set_vars.ps1
echo $TEST_VAR
# outputs an empty string, assuming $TEST_VAR was not already defined in the callers scope

Exporting environment variables for local development and heroku deployment

I would like to setup some files for development, staging and production with environment variables, for example:
application_root/development.env
KEY1=value1
KEY2=value2
There would be similar files staging.env and production.env.
I am looking for a couple different bash scripts which would allow the loading of all these variables in either development or staging/production.
In local development I want to effectively run export KEY1=value1 for each line in the file.
For staging/production I will be deploying to Heroku and would like to effectively run heroku config:set -a herokuappname KEY1=value1 for each line in the staging or production.env files.
I know there are some gems designed for doing this but it seems like this might be pretty simple. I also like the flexibility of having the .env files as simple lists of keys and values and not specifically being tied to any language/framework. If I would have to change something about the way these variables need to be loaded it would be a matter of changing the script but not the .env files.
In the simplest form, you can load the key-value pairs into a bash array as follows:
IFS=$'\n' read -d '' -ra nameValuePairs < ./development.env
In Bash v4+, it's even simpler:
readarray -t nameValuePairs < ./development.env
You can then pass the resulting "${nameValuePairs[#]}" array to commands such as export or heroku config:set ...; e.g.:
export "${nameValuePairs[#]}"
Note, however, that the above only works as intended if the input *.env file meets all of the following criteria:
the keys are syntactically valid shell variable names and the lines have the form <key>=<value>, with no whitespace around =
the lines contain no quoting and no leading or trailing whitespace
there are no empty/blank lines or comment lines in the file.
the values are confined to a single line each.
A different approach is needed with files that do not adhere to this strict format; for instance, this related question deals with files that may contain quoted values.
Below is the source code for a bash script named load_env (the .sh suffix is generally not necessary and ambiguous):
You'd invoke it with the *.env file of interest, and it would perform the appropriate action (running heroku config:set … or export) based on the filename.
However, as stated, you must source the script (using source or its effective bash alias, .) in order to create environment variables (export) visible to the current shell.
To prevent obscure failures, the script complains if you pass a development.env file and have invoked the script without sourcing.
Examples:
./load_env ./staging.dev
. ./load_env ./development.dev # !! Note the need to source
load_env source code
#!/usr/bin/env bash
# Helper function that keeps its aux. variables localized.
# Note that the function itself remains defined after sourced invocation, however.
configOrExport() {
local envFile=$1 doConfig=0 doExport=0 appName
case "$(basename "$envFile" '.env')" in
staging)
doConfig=1
# Set the desired app name here.
appName=stagingapp
;;
production)
doConfig=1
# Set the desired app name here.
appName=productionapp
;;
development)
doExport=1
;;
*)
echo "ERROR: Invalid or missing *.env file name: $(basename "$envFile" '.env')" >&2; exit 2
esac
# Make sure the file exists and is readable.
[[ -r "$envFile" ]] || { echo "ERROR: *.env file not found or not readable: $envFile" >&2; exit 2; }
# If variables must be exported, make sure the script is being sourced.
[[ $doExport -eq 1 && $0 == "$BASH_SOURCE" ]] && { echo "ERROR: To define environment variables, you must *source* this script." >&2; exit 2; }
# Read all key-value pairs from the *.env file into an array.
# Note: This assumes that:
# - the keys are syntactically valid shell variable names
# - the lines contain no quoting and no leading or trailing whitespace
# - there are no empty/blank lines or comment lines in the file.
IFS=$'\n' read -d '' -ra nameValuePairs < "$envFile"
# Run configuration command.
(( doConfig )) && { heroku config:set -a "$appName" "${nameValuePairs[#]}" || exit; }
# Export variables (define as environment variables).
(( doExport )) && { export "${nameValuePairs[#]}" || exit; }
}
# Invoke the helper function.
configOrExport "$#"

How Can I access bash variables in tcl(expect) script

How Can I access bash variables in tcl(expect) script.
I have bash file say f1.sh which set some variables like
export var1=a1
export var2=a2
These variable I need to use in my expect script .
I tried using this in my script which does not work
system "./f1.sh"
puts "var1 is $::env(var1)"
puts "var2 is $::env(var2)"
But this does not seems to work.
I see that non of the variable from f1.sh are getting set as environment variable.
system "./f1.sh" << # Is this command in my script right ?
How I need to access these bash variables from tcl file.
I would say that this problem is rather general. First I met this problem, when I wanted to initialize Microsoft Visual Studio environment (which is done using .cmd script) in PoserShell. Later I've faced this problem with other scripting languages in any combinations (Bash, Tcl, Python etc.).
Solution provided by Hai Vu is good. It works well, if you know from the beginning, which variables you need. However, if you are going to use script for initialization of some environment it my contains dozens of variables (which you don't even need to know about, but which are needed for normal operation of the environment).
In general, the solution for the problem is following:
Execute script and at the end print ALL environment variables and capture the output.
Match lines of output for the pattern like "variable=value", where is what you want to get.
Set environment variables using facilities of your language.
I do not have ready made solution, but I guess, that something similar to this should work (note, that snippets below was not tested - they are aimed only to give an idea of the solution):
Execute script; print vars and capture the output (argument expanding - {*} - requires Tcl 8.5, here we can go without it, but I prefer to use it):
set bashCommand {bash -c 'myScriptName arg1 arg2 2>&1 >/dev/null && export -p'}
if [catch {*}${bashCommand} output] {
set errMsg "ERROR: Failed to run script."
append errMsg "\n" $output
error $errMsg
}
;# If we get here, output contains the output of "export -p" command
Parse the output of the command:
set vars [dict create]
foreach line [split $output "\n"] {
regex -- {^declare -x ([[:alpha:]_]*)=\"(.*)\"$} $line dummy var val
;# 3. Store var-val pair of set env var.
}
Store var-val pair or set env var. Here several approaches can be used:
3.1. Set Tcl variables and use them like this (depending on context):
set $var $val
or
variable $var $val
3.2. Set environment variable (actually, sub-case of 3.1):
global ::env
set ::env($var) $val
3.3 Set dict or array and use it within your application (or script) without modification of global environment:
set myEnv($var) val ;# set array
dict set myEnvDict $var $val ;# set dict
I'd like to repeat, that this is only the idea of the receipt. And more important, that as most of the modern scripting languages support regexes, this receipt can provide bridge between almost arbitrary pair of languages, but not only Bash<->Tcl
You can use a here-document, like this:
#!/bin/bash
process=ssh
expect <<EOF
spawn $process
...
EOF
Exported variables are only passed from a parent process to it's children, not the other way around. The script f1.sh (actually the bash instance that's running the script) gets it's own copies of var1 and var2 and it doesn't matter if it changes them, the changes are lost when it exits. For variable exporting to work, you would need to start the expect script from the bash script.
In f1.sh, printf what you want to return...
printf '%s\n%s\n' "$var1" "$var2"
...and read it with exec in Tcl:
lassign [split [exec ./f1.sh] \n] var1 var2
Perhaps I did not look hard enough, but I don't see any way to do this. When you execute the bash script, you create a different process. What happens in that process does not propagate back to the current process.
We can work-around this issue by doing the following (thanks to potrzebie for the idea):
Duplicate the bash script to a temp script
Append to the temp script some commands at the end to echo a marker, and a list of variables and their values
Execute the temp script and parse the output
The result is a list of alternating names and values. We use this list to set the environment variables for our process.
#!/usr/bin/env tclsh
package require fileutil
# Execute a bash script and extract some environment variables
proc getBashVar {bashScript varsList} {
# Duplicate the bash script to a temp script
set tempScriptName [fileutil::tempfile getBashVar]
file copy -force $bashScript $tempScriptName
# Append a marker to the end of the script. We need this marker to
# identify where in the output to begin extracting the variables.
# After that append the list of specified varibles and their values.
set f [open $tempScriptName a]
set marker "#XXX-MARKER"
puts $f "\necho \\$marker"
foreach var $varsList {
puts $f "echo $var \\\"$$var\\\" "
}
close $f
# Execute the temp script and parse the output
set scriptOutput [exec bash $tempScriptName]
append pattern $marker {\s*(.*)}
regexp $pattern $scriptOutput all vars
# Set the environment
array set ::env $vars
# Finally, delete the temp script to clean up
file delete $tempScriptName
}
# Test
getBashVar f1.sh {var1 var2}
puts "var1 = $::env(var1)"
puts "var2 = $::env(var2)"

Why doesn't this bit of code work? Setting variables and config file

I have recently just made this script:
if test -s $HOME/koolaid.txt ; then
Billz=$(grep / $HOME/koolaid.txt)
echo $Billz
else
Billz=$HOME/notkoolaid
echo $Billz
fi
if test -d $Billz ; then
echo "Ok"
else touch $Billz
fi
So basically, if the file $HOME/koolaid.txt file does NOT exist, then Billz will be set as $HOME/koolaid.txt. It then sucesfully creates the file.
However, if I do make the koolaid.txt then I get this
mkdir: cannot create directory : No such file or directory
Any help would be appreciated
Here is a difference between content of a variable and evaluated content...
if your variable contains a string $HOME/some - you need expand it to get /home/login/same
One dangerous method is eval.
bin=$(grep / ~/.rm.cfg)
eval rbin=${bin:-$HOME/deleted}
echo "==$rbin=="
Don't eval unless you're absolutely sure what you evaling...
Here are a couple things to fix:
Start your script with a "shebang," such as:
#!/bin/sh
This way the shell will know that you want to run this as a Bourne shell script.
Also, your conditional at the top of the script doesn't handle the case well in which .rm.cfg exists but doesn't contain a slash character anywhere in it. In that case the rbin variable never gets set.
Finally, try adding the line
ls ~
at the top so you can see how the shell is interpreting the tilde character; that might be the problem.

Export not working (from a function called to get its echo)

I have a code like this:
#!/usr/bin/env bash
test_this(){
export ABC="ABC"
echo "some output"
}
final_output="the otput is $(test_this)"
echo "$ABC"
Unfortunately the variable ABC is not being set.
I have to call test_this like that, since in my real program I give some arguments to it, it performs various complicated operations calling various other functions, which on the way export this or that (basing on those arguments), and at the end some output string is assembled to be returned. Calling it two times, once to get exports and once for the output string would be bad.
The question is: what can I do to have both the exports and the output string in place, but just by one call to such a function?
The answer that I am happy with (thank you paxdiablo):
#!/usr/bin/env bash
test_this(){
export ABC="ABC"
export A_VERY_OBSCURE_NAME="some output"
}
test_this
final_output="the otput is $A_VERY_OBSCURE_NAME"
echo "$ABC" #works!
unset A_VERY_OBSCURE_NAME
Yes, it is being set. Unfortunately it's being set in the sub-process that is created by $() to run the test_this function and has no effect on the parent process.
And calling it twice is probably the easiest way to do it, something like (using a "secret" parameter value to dictate behaviour if it needs to be different):
#!/usr/bin/env bash
test_this(){
export ABC="ABC"
if [[ "$1" != "super_sekrit_sauce" ]] ; then
echo "some output"
fi
}
final_output="the output is $(test_this)"
echo "1:$ABC:$final_output"
test_this super_sekrit_sauce
echo "2:$ABC:$final_output"
which outputs:
1::the output is some output
2:ABC:the output is some output
If you really only want to call it once, you could do something like:
#!/usr/bin/env bash
test_this(){
export ABC="ABC"
export OUTPUT="some output"
}
test_this
final_output="the output is ${OUTPUT}"
echo "1:$ABC:$final_output"
In other words, use the same method for extracting output as you did for the other information.

Resources