Zsh manual mentions that option -a means ALL_EXPORT,
ALL_EXPORT (-a, ksh: -a)
All parameters subsequently defined are automatically exported.
While export makes the variable available to sub-processes, the how can the same variable foo be local?
In local -a, the -a has the same meaning as it does for typeset:
-a
The names refer to array parameters. An array parameter
may be created this way, but it may not be assigned to in
the typeset statement. When displaying, both normal and
associative arrays are shown.
I think you might be confused on a number of fronts.
The ALL_EXPORT (-a) setting is for setopt, not local. To flag a variable for export with local, you use local -x.
And you're also confusing directions of propagation :-)
Defining a variable as local will prevent its lifetime from extending beyond the current function (outwards or upwards depending on how your mind thinks).
This does not affect the propagation of the variable to sub-processes run within the function (inwards or downwards).
For example, consider the following scripts qq.zsh:
function xyz {
local LOCVAR1
local -x LOCVAR2
LOCVAR1=123
LOCVAR2=456
GLOBVAR=789
zsh qq2.zsh
}
xyz
echo locvar1 is $LOCVAR1
echo locvar2 is $LOCVAR2
echo globvar is $GLOBVAR
and qq2.zsh:
echo subshell locvar1 is $LOCVAR1
echo subshell locvar2 is $LOCVAR2
When you run zsh qq.zsh, the output is:
subshell locvar1 is
subshell locvar2 is 456
locvar1 is
locvar2 is
globvar is 789
so you can see that neither local variable survives the return from the function. However, the auto-export of the local variables to a sub-process called within xyz is different. The one marked for export with local -x is available in the sub-shell, the other isn't.
Related
I read a lot of explanation of kinds of variable in bash shell script.
But I still confused because people uses mixed terminology for the same thing.
From my knowledge there are three kinds of variable even though I don't know exact official term for that.
I will define the name of them like belows for the clear communication.
Global Environment Variable
This is variable exported so that the whole script can access and child shell also can access
Global Variable
This is variable that can access in whole script scope but not all child shell.
Local Variable
This is variable that can acess only in a function scope. This variable defined with "local" keyword
Some people distinguish Global or Local by the point of that it is exported or not. They call the "2. Global Variable" as "Local Variable"
But some people distinguish Global or Local by the point of that it is function scope or the whole script scope.
What is exact official terminoloies for these three kinds of variables?
And what is the exact distinction for these?
For example,
$ cat vartest.sh
#!/bin/bash
var2="global_var2"
echo $var1 # (1)
echo $var2 # (2)
$ var1="local_var1"
$ source ./vartest.sh
local_var1
global_var2
I think var1 in (1) and var2 in (2) are totally same kind of variable. I can't find any difference between them.
Is it any different between (1) and (2) under the hood?
Continuing from above(or below) comments because comment doesn't have formatting function.
$ var1="local variable"
$ echo $SHLVL # echo shell level
1
$ bash # initiate sub-shell
$ echo $SHLVL
2 # this tells you are in sub-shell
$ echo $var1 # try to access local var from parent shell
$ exit # exit from sub-shell
$ echo $SHVL # confirm you are back to parent shell
1
$ export var1 # export 'var1' to environmental(global) variable
$ bash # initiate sub-shell again
$ echo $SHLVL # confirm you are in sub-shell
2
$ echo $var1 # now you can access this variable
local variable
$ exit
Here is another example on working with local (shell) variable in a script.
# Create a script as follow.
$ cat test.sh
#!/bin/bash
echo "Shell level is $SHLVL"
echo "var2 is $var2"
-----
$ bash test.sh # Run the script
Shell level is 2 # $SHLVL is accessible in all-level because it's environmental/global variable.
var2 is # var2 is empty because it's not created yet.
$ var2="passing var" bash test.sh
Shell level is 2
var2 is passing var # shell(local) variable is passed to the sub-shell/script
In a shell, global variable is environmental variable accessible in all-level. Local variable is only accessible in a current shell. And you can export local variable to global/environmental variable. I hope this clarifies a bit.
I think you are confused between shell environment and scripting.
Try following links.
https://tldp.org/LDP/Bash-Beginners-Guide/html/sect_03_02.html
https://tldp.org/LDP/abs/html/localvar.html
Hello I am trying to translate my .bashrc to fish format almost done, mostly is clear on the documentation but this part is giving me a headache.. is so my gnupg works with my yubikey ssh etc etc..
The fish version is latest 3.0 under Arch GNU/Linux
original on BASH:
# Set SSH to use gpg-agent
unset SSH_AGENT_PID
if [ "${gnupg_SSH_AUTH_SOCK_by:-0}" -ne $$ ]; then
export SSH_AUTH_SOCK="/run/user/$UID/gnupg/S.gpg-agent.ssh"
fi
echo "UPDATESTARTUPTTY" | gpg-connect-agent > /dev/null 2&>1
Mine half converted into fish:
set -e SSH_AGENT_PID
if [ "${gnupg_SSH_AUTH_SOCK_by:-0}" -ne $$ ]
set -x SSH_AUTH_SOCK="/run/user/$UID/gnupg/S.gpg-agent.ssh"
end
echo "UPDATESTARTUPTTY" | gpg-connect-agent > /dev/null 2>&1
so as you see above I have so far converted the stdin and stderror pine and the unset variable with set -e the error I am having is a bit more obscure to me:
~/.config/fish/config.fish (line 33): ${ is not a valid variable in fish.
if [ "${gnupg_SSH_AUTH_SOCK_by:-0}" -ne $$ ]
^
from sourcing file ~/.config/fish/config.fish
called during startup
Any help will be much appreciated,
BTW will be nice a migrate too :) are there any out there?
[edit] ok got this working thanks to the response below, now all my bash environment, profile, bashrc etc is translated to fish and using it solely as my shell 100%
You should not change your login shell until you have a much better understanding of fish syntax and behavior. For example, in fish the equivalent of $$ is %self or $fish_pid depending on which fish version you are using. You should always specify the version of the program you are having problems with.
Assuming you're using fish 2.x that would be written as
if not set -q gnupg_SSH_AUTH_SOCK_by
or test $gnupg_SSH_AUTH_SOCK_by -ne %self
set -gx SSH_AUTH_SOCK "/run/user/$UID/gnupg/S.gpg-agent.ssh"
end
Also, notice that there is no equal-sign between the var name and value in the set -x.
Since ${var:-value} expands to value if $var is empty, you can always replace it by writing your code out the long way:
begin
if test -n "$gnupg_SSH_AUTH_SOCK_by"
set result "$gnupg_SSH_AUTH_SOCK_by"
else
set result 0
end
if [ "$result" -ne %self ]
set -x SSH_AUTH_SOCK "/run/user/$UID/gnupg/S.gpg-agent.ssh"
end
set -e result
end
Note that I don't use (a) endorse, (b) condone the use of, or (c) fail to hold unwarranted prejudices against users of, fish. Thus, my advice is very much suspect, and it's likely that there are considerably better ways to do this.
I had a similar question, related to XDG_* variables.
var1="${XDG_CACHE_HOME:-$HOME/.cache}"/foo
var2="${XDG_CONFIG_HOME:-$HOME/.config}"/foo
var3="${XDG_DATA_HOME:-$HOME/.local/share}"/foo
some-command "$var1" "$var2" ...
What I found as the best alternative is to simply set univeral variables once for the defaults--
set -U XDG_CACHE_HOME ~/.cache
set -U XDG_CONFIG_HOME ~/.config
set -U XDG_DATA_HOME ~/.local/share
Then in fish config file(s) or scripts, simply use "$XDG_CONFIG_HOME"/.... The value of an exported environment variable will override the universal variable if set, otherwise the universal variable is there as a default/fallback. If the universal variable is used, it is not exported to child processes, while an exported environment variable is, which provides the full equivalent to bash|zsh parameter expansion.
I have a variable in a file called access.txt located at /home/ubuntu/pub/access.txt. The access.txt contents looks like this:
SFTP_VAR="JHGSYDDUIGUIGUIGUIG"
SQL_VAR="GUIIGGJHGBJHGJHGJH"
I have a script file on the same machine that is supposed to read and use this SFTP_VAR. I use the source statement to mention the location of access.txt. Here is the code on my script file.
#!/bin/bash
source /home/ubuntu/pub/access.txt
echo $SFTP_VAR
export SSHPASS=$SFTP_VAR
for f in /home/ubuntu/pub/sfmc/Upstream/Encrypted/* ;
do
echo put "$f"
done | sshpass -e sftp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-dss USERNAME#FTP_SERVER_IP:/Import
unset SSHPASS
When I run my script, I see the variable in the first echo. This means my script can see the file. But the export command immediately after the first echo does not seem to recognize my variable. This is when I replace the variable with its real value or:
echo 'JHGSYDDUIGUIGUIGUIG'
export SSHPASS='JHGSYDDUIGUIGUIGUIG'
The code works fine. What am I missing here and how can I read those variables in the code?
Edit: During further discussion in the comments it was found to be a problem with the quotes and not with sessions. Changing the double quotes to single quotes in the access.txt file solved it for OP.
I will leave my original answer below.
Your question is very poorly worded. I think I see your problem, but I am not sure.
It seems you might have some misunderstandings of how source and export work and how they affect your session. Lets test this:
First create a file with this content and give the path at line 6 of the script:
VAR1="value of VAR1"
VAR2="value of VAR2"
Then create this script called script.sh:
#!/bin/bash
echo "[The variables with the name VAR1 and VAR2 are undefined in out session]"
echo "VAR1: "$VAR1
echo "VAR2: "$VAR2
echo "[Source brings the variables of the file into our session, with them VAR1 and VAR2]"
source /home/X/access.txt
echo "[VAR1 and VAR2 of our session do now have the values specified in the file]"
echo "VAR1: "$VAR1
echo "VAR2: "$VAR2
echo "[A third undefined variable is used - empty]"
echo "VAR3: "$VAR3
echo "[The third variable gets the value of VAR1 and is exported]"
export VAR3=$VAR1
echo "[The third variale is now filled]"
echo "VAR3: "$VAR3
echo "[We unset the variable]"
unset VAR3
echo "[The variable is no longer set]"
echo "VAR3: "$VAR3
echo "[Set it again]"
export VAR3=$VAR1
echo "[Its back now]"
echo "VAR3: "$VAR3
I think it covers everything you want to do. Now on to testing:
First step:
Just use a normal call on itbash script.sh. Then look at the stdout. Everything should be there as is described by the echos.
Second step:
Now try echo $VAR3 from your shell where you previously called the script. It should not print anything. The variables did not hold their values after the script completed.
Third step:
Now call the script again with export script.sh. Output should look just the way it did in the first step.
Fourth step:
Now try echo $VAR3 again. It will now print the variable.
Why is that?
Bash is based on different sessions. Things you set after you started your session will not carry over to other open sessions and will be lost after you exit the session.
You can start new sessions by simply typing bash. But even this does not solve our problem.
When you call your script you use bash script.sh. This actually also starts a new session and then executes your script inside of this session. Not in the one you called it from.
After finishing it exits out of this session leaving behind the set variables. This is why you cant see them by simple using bash script.sh
In the third step we used source to call the script. This will get the variables into our session and thus we can print the values out.
But this still makes them only temporary as they will go away once we close the session where we sourced the script.
Put things in double quotes. Your example SFTP_VAR="JHGSYDDUIGUIGUIGUIG" should work fine, but try the password "*".
# Incorrect
SFTP_VAR="*"; export SSHPASS=$SFTP_VAR; echo $SSHPASS
# Correct
SFTP_VAR="*"; export SSHPASS="$SFTP_VAR"; echo "$SSHPASS"
Question,
I want to have a bash script that will have a global variable that can be incremented from other bash scripts.
Example:
I have a script like the following:
#! /bin/bash
export Counter=0
for SCRIPT in /Users/<user>/Desktop/*sh
do
$SCRIPT
done
echo $Counter
That script will call all the other bash scripts in a folder and those scripts will have something like the following:
if [ "$Output" = "$Check" ]
then
echo "OK"
((Counter++))
I want it to then increment the $Counter variable if it does equal "OK" and then pass that value back to the initial batch script so I can keep that counter number and have a total at the end.
Any idea on how to go about doing that?
Environment variables propagate in one direction only -- from parent to child. Thus, a child process cannot change the value of an environment variable set in their parent.
What you can do is use the filesystem:
export counter_file=$(mktemp "$HOME/.counter.XXXXXX")
for script in ~user/Desktop/*sh; do "$script"; done
...and, in the individual script:
counter_curr=$(< "$counter_file" )
(( ++counter_curr ))
printf '%s\n' "$counter_curr" >"$counter_file"
This isn't currently concurrency-safe, but your parent script as currently written will never call more than one child at a time.
An even easier approach, assuming that the value you're tracking remains relatively small, is to use the file's size as a proxy for the counter's value. To do this, incrementing the counter is as simple as this:
printf '\n' >>"$counter_file"
...and checking its value in O(1) time -- without needing to open the file and read its content -- is as simple as checking the file's size; with GNU stat:
counter=$(stat -f %z "$counter_file")
Note that locking may be required for this to be concurrency-safe if using a filesystem such as NFS which does not correctly implement O_APPEND; see Norman Gray's answer (to which this owes inspiration) for a working implementation.
You could source the other scripts, which means they're not running in a sub-process but "inline" in the calling script like this:
#! /bin/bash
export counter=0
for script in /Users/<user>/Desktop/*sh
do
source "$script"
done
echo $counter
But as pointed out in the comments i'd only advise to use this approach if you control the called scripts yourself. If they for example exit or have variables clashing with each other, bad things could happen.
As described, you can't do this, since there isn't anything which corresponds to a ‘global variable’ for shell scripts.
As the comment suggests, you'll have to use the filesystem to communicate between scripts.
One simple/crude way of doing what you describe would be to simply have each cooperating script append a line to a file, and the ‘global count’ is the size of this file:
#! /bin/sh -
echo ping >>/tmp/scriptcountfile
then wc -l /tmp/scriptcountfile is the number of times that's happened. Of course, there's a potential race condition there, so something like the following would sequence those accesses:
#! /bin/sh -
(
flock -n 9
echo 'do stuff...'
echo ping >>/tmp/stampfile
) 9>/tmp/lockfile
(the flock command is available on Linux, but isn't portable).
Of course, then you can start to do fancier things by having scripts send stuff through pipes and sockets, but that's going somewhat over the top.
Recently I wrote a script which sets an environment variable, take a look:
#!/bin/bash
echo "Pass a path:"
read path
echo $path
defaultPath=/home/$(whoami)/Desktop
if [ -n "$path" ]; then
export my_var=$path
else
echo "Path is empty! Exporting default path ..."
export my_var=$defaultPath
fi
echo "Exported path: $my_var"
It works just great but the problem is that my_var is available just locally, I mean in console window where I ran the script.
How to write a script which allow me to export global environment variable which can be seen everywhere?
Just run your shell script preceded by "." (dot space).
This causes the script to run the instructions in the original shell. Thus the variables still exist after the script finish
Ex:
cat setmyvar.sh
export myvar=exists
. ./setmyvar.sh
echo $myvar
exists
Each and every shell has its own environment. There's no Universal environment that will magically appear in all console windows. An environment variable created in one shell cannot be accessed in another shell.
It's even more restrictive. If one shell spawns a subshell, that subshell has access to the parent's environment variables, but if that subshell creates an environment variable, it's not accessible in the parent shell.
If all of your shells need access to the same set of variables, you can create a startup file that will set them for you. This is done in BASH via the $HOME/.bash_profile file (or through $HOME/.profile if $HOME/.bash_profile doesn't exist) or through $HOME/.bashrc. Other shells have their own set of startup files. One is used for logins, and one is used for shells spawned without logins (and, as with bash, a third for non-interactive shells). See the manpage to learn exactly what startup scripts are used and what order they're executed).
You can try using shared memory, but I believe that only works while processes are running, so even if you figured out a way to set a piece of shared memory, it would go away as soon as that command is finished. (I've rarely used shared memory except for named pipes). Otherwise, there's really no way to set an environment variable in one shell and have another shell automatically pick it up. You can try using named pipes or writing that environment variable to a file for other shells to pick it up.
Imagine the problems that could happen if someone could change the environment of one shell without my knowledge.
Actually I found an way to achieve this (which in my case was to use a bash script to set a number of security credentials)
I just call bash from inside the script and the spawned shell now has the export values
export API_USERNAME=abc
export API_PASSWORD=bbbb
bash
now calling the file using ~/.app-x-setup.sh will give me an interactive shell with those environment values setup
The following were extracted from 2nd paragraph from David W.'s answer: "If one shell spawns a subshell, that subshell has access to the parent's environment variables, but if that subshell creates an environment variable, it's not accessible in the parent shell."
In case a user need to let parent shell access your new environment variables, just issue the following command in parent shell:
source <your_subshell_script>
or using shortcut
. <your_subshell_script>
You got to add the variable in your .profile located in /home/$USER/.profile
Yo can do that with this command:
echo 'TEST="hi"' >> $HOME/.profile
Or by edit the file with emacs, for example.
If you want to set this variable for all users, you got to edit /etc/profile (root)
There is no global environment, really, in UNIX.
Each process has an environment, originally inherited from the parent, but it is local to the process after the initial creation.
You can only modify your own, unless you go digging around in the process using a debugger.
write it to a temporary file, lets say ~/.myglobalvar and read it from anywhere
echo "$myglobal" > ~/.myglobalvar
Environment variables are always "local" to process execution the export command allow to set environment variables for sub processes. You can look at .bashrc to set environment variables at the start of a bash shell. What you are trying to do seems not possible as a process cannot modify (or access ?) to environment variables of another process.
You can update the ~/.bashrc or ~/.bash_profile file which is used to initialize the environment.
Take a look at the loading behavior of your shell (explained in the manpage, usually referring to .XXXshrc or .profile). Some configuration files are loaded at login time of an interactive shell, some are loaded each time you run a shell. Placing your variable in the latter might result in the behavior you want, e.g. always having the variable set using that distinct shell (for example bash).
If you need to dynamically set and reference environment variables in shell scripts, there is a work around. Judge for yourself whether is worth doing, but here it is.
The strategy involves having a 'set' script which dynamically writes a 'load' script, which has code to set and export an environment variable. The 'load' script is then executed periodically by other scripts which need to reference the variable. BTW, the same strategy could be done by writing and reading a file instead of a variable.
Here's a quick example...
Set_Load_PROCESSING_SIGNAL.sh
#!/bin/bash
PROCESSING_SIGNAL_SCRIPT=./Load_PROCESSING_SIGNAL.sh
echo "#!/bin/bash" > $PROCESSING_SIGNAL_SCRIPT
echo "export PROCESSING_SIGNAL=$1" >> $PROCESSING_SIGNAL_SCRIPT
chmod ug+rwx $PROCESSING_SIGNAL_SCRIPT
Load_PROCESSING_SIGNAL.sh (this gets dynamically created when the above is run)
#!/bin/bash
export PROCESSING_SIGNAL=1
You can test this with
Test_PROCESSING_SIGNAL.sh
#!/bin/bash
PROCESSING_SIGNAL_SCRIPT=./Load_PROCESSING_SIGNAL.sh
N=1
LIM=100
while [ $N -le $LIM ]
do
# DO WHATEVER LOOP PROCESSING IS NEEDED
echo "N = $N"
sleep 5
N=$(( $N + 1 ))
# CHECK PROCESSING_SIGNAL
source $PROCESSING_SIGNAL_SCRIPT
if [[ $PROCESSING_SIGNAL -eq 0 ]]; then
# Write log info indicating that the signal to stop processing was detected
# Write out all relevent info
# Send an alert email of this too
# Then exit
echo "Detected PROCESSING_SIGNAL for all stop. Exiting..."
exit 1
fi
done
~/.bin/SOURCED/lazy script to save and load data as flat files for system.
[ ! -d ~/.megadata ] && mkdir ~/.megadata
function save_data {
[ -z "$1" -o -z "$2" ] && echo 'save_data [:id:] [:data:]' && return
local overwrite=${3-false}
[ "$overwrite" = 'true' ] && echo "$2" > ~/.megadata/$1 && return
[ ! -f ~/.megadata/$1 ] && echo "$2" > ~/.megadata/$1 || echo ID TAKEN set third param to true to overwrite
}
save_data computer engine
cat ~/.megadata/computer
save_data computer engine
save_data computer megaengine true
function get_data {
[ -z "$1" -o -f $1 ] && echo 'get_data [:id:]' && return
[ -f ~/.megadata/$1 ] && cat ~/.megadata/$1 || echo ID NOT FOUND
:
}
get_data computer
get_data computer
Maybe a little off topic, but when you really need it to set it temporarily to execute some script and ended up here looking for answers:
If you need to run a script with certain environment variables that you don't need to keep after execution you could do something like this:
#!/usr/bin/env sh
export XDEBUG_SESSION=$(hostname);echo "running with xdebug: $XDEBUG_SESSION";$#
In my example I just use XDEBUG_SESSION with a hostname, but you can use multiple variables. Keep them separated with a semi-colon. Execution as follows (assuming you called the script debug.sh and placed it in the same directory as your php script):
$ debug.sh php yourscript.php