I have a variable in a file called access.txt located at /home/ubuntu/pub/access.txt. The access.txt contents looks like this:
SFTP_VAR="JHGSYDDUIGUIGUIGUIG"
SQL_VAR="GUIIGGJHGBJHGJHGJH"
I have a script file on the same machine that is supposed to read and use this SFTP_VAR. I use the source statement to mention the location of access.txt. Here is the code on my script file.
#!/bin/bash
source /home/ubuntu/pub/access.txt
echo $SFTP_VAR
export SSHPASS=$SFTP_VAR
for f in /home/ubuntu/pub/sfmc/Upstream/Encrypted/* ;
do
echo put "$f"
done | sshpass -e sftp -o StrictHostKeyChecking=no -o HostKeyAlgorithms=+ssh-dss USERNAME#FTP_SERVER_IP:/Import
unset SSHPASS
When I run my script, I see the variable in the first echo. This means my script can see the file. But the export command immediately after the first echo does not seem to recognize my variable. This is when I replace the variable with its real value or:
echo 'JHGSYDDUIGUIGUIGUIG'
export SSHPASS='JHGSYDDUIGUIGUIGUIG'
The code works fine. What am I missing here and how can I read those variables in the code?
Edit: During further discussion in the comments it was found to be a problem with the quotes and not with sessions. Changing the double quotes to single quotes in the access.txt file solved it for OP.
I will leave my original answer below.
Your question is very poorly worded. I think I see your problem, but I am not sure.
It seems you might have some misunderstandings of how source and export work and how they affect your session. Lets test this:
First create a file with this content and give the path at line 6 of the script:
VAR1="value of VAR1"
VAR2="value of VAR2"
Then create this script called script.sh:
#!/bin/bash
echo "[The variables with the name VAR1 and VAR2 are undefined in out session]"
echo "VAR1: "$VAR1
echo "VAR2: "$VAR2
echo "[Source brings the variables of the file into our session, with them VAR1 and VAR2]"
source /home/X/access.txt
echo "[VAR1 and VAR2 of our session do now have the values specified in the file]"
echo "VAR1: "$VAR1
echo "VAR2: "$VAR2
echo "[A third undefined variable is used - empty]"
echo "VAR3: "$VAR3
echo "[The third variable gets the value of VAR1 and is exported]"
export VAR3=$VAR1
echo "[The third variale is now filled]"
echo "VAR3: "$VAR3
echo "[We unset the variable]"
unset VAR3
echo "[The variable is no longer set]"
echo "VAR3: "$VAR3
echo "[Set it again]"
export VAR3=$VAR1
echo "[Its back now]"
echo "VAR3: "$VAR3
I think it covers everything you want to do. Now on to testing:
First step:
Just use a normal call on itbash script.sh. Then look at the stdout. Everything should be there as is described by the echos.
Second step:
Now try echo $VAR3 from your shell where you previously called the script. It should not print anything. The variables did not hold their values after the script completed.
Third step:
Now call the script again with export script.sh. Output should look just the way it did in the first step.
Fourth step:
Now try echo $VAR3 again. It will now print the variable.
Why is that?
Bash is based on different sessions. Things you set after you started your session will not carry over to other open sessions and will be lost after you exit the session.
You can start new sessions by simply typing bash. But even this does not solve our problem.
When you call your script you use bash script.sh. This actually also starts a new session and then executes your script inside of this session. Not in the one you called it from.
After finishing it exits out of this session leaving behind the set variables. This is why you cant see them by simple using bash script.sh
In the third step we used source to call the script. This will get the variables into our session and thus we can print the values out.
But this still makes them only temporary as they will go away once we close the session where we sourced the script.
Put things in double quotes. Your example SFTP_VAR="JHGSYDDUIGUIGUIGUIG" should work fine, but try the password "*".
# Incorrect
SFTP_VAR="*"; export SSHPASS=$SFTP_VAR; echo $SSHPASS
# Correct
SFTP_VAR="*"; export SSHPASS="$SFTP_VAR"; echo "$SSHPASS"
Related
I am trying to login on one of the remote server(Box1) and trying to read one file on remote server(Box1).
That contain the another server(Box2) details, base upon that details I have to come back to the local server and ssh to another server(Box2) for some data crunching. and so on.....
ssh box1.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node1= `cat /home/rakesh/tomar.log`
fi
EOF
ssh box2.com << EOF
if [[ ! -f /home/rakesh/tomar.log ]]
then
echo "LOG file not found"
else
echo " LOG file present"
export server_node2= `cat /home/rakesh/tomar.log`
fi
EOF
but I am not getting value of "server_node1" and "server_node2" on local machine.
any help would be appreciated.
Just like bash -c 'export foo=bar' cannot declare a variable in the calling shell where you typed this, an ssh command cannot declare a variable in the calling shell. You will have to refactor so that the calling shell receives the information and knows what to do with it.
I agree with the comment that storing a log file in a variable is probably not a sane, or at least elegant, thing to do, but the easy way to do what you are attempting is to put the ssh inside the assignment.
server_node1=$(ssh box1.com cat tomar.log)
server_node2=$(ssh box2.com cat tomar.log)
A few notes and amplifications:
The remote shell will run in your home directory, so I took it out (on the assumption that /home/rt9419 is your home directory, obviously).
In case of an error in the cat command, the exit code of ssh will be the error code from cat, and the error message on standard error will be visible on your standard error, so the echo seemed quite superfluous. (If you want a custom message, variable=$(ssh whatever) || echo "Custom message" >&2 would do that. Note the redirection to standard error; it doesn't seem to matter here, but it's good form.)
If you really wanted to, you could run an arbitrarily complex command in the ssh; as outlined above, it didn't seem necessary here, but you could do assigment=$(ssh remote 'if [[ things ]]; then for variable in $(complex commands to drive a loop); do : etc etc; done; fi; more </dev/null; exit "$variable"') or whatever.
As further comments on your original attempt,
The backticks in the here document in your attempt would be evaluated by your local shell before the ssh command even ran. There are separate questions about how to fix that; see e.g. How have both local and remote variable inside an SSH command. but in short, unless you absolutely require the local shell to be able to modify the commands you send, probably put them in single quotes, like I did in the silly complex ssh example above.
The function of export is to make variables visible to child processes. There is no way to affect the environment of a parent process (short of having it cooperate and/or coordinate the change, as in the code above). As an example to illustrate the difference, if you set PERL5LIB to a directory with Perl libraries, but fail to export it, the Perl process you start will not see the variable; it is only visible to the current shell. When you export it, any Perl process you start as a child of this shell will also see this variable and the value you assigned. In other words, you export variables which are not private to the current shell (and don't export private ones; aside from making sure they are private, this saves the amount of memory which needs to be copied between processes), but that still only makes them visible to children, by the design of the U*x process architecture.
You should get back the file from box1and box2 with an scp:
scp box1.com:/home/rt9419/tomar.log ~/tomar1.log
#then you can cat!
export server_node1=`cat ~/tomar1.log`
idem with box2
scp box2.com:/home/rt9419/tomar.log ~/tomar2.log
#then you can cat!
export server_node2=`cat ~/tomar2.log`
There are several possibilities. In your case, you could on the remote system create a file (in bash syntax), containing the assignments of these variables, for example
echo "export server_node2='$(</home/rt9419/tomar.log)'" >>export_settings
(which makes me wonder why you want the whole content of your logfile be stored into a variable, but this is another question), then transfer this file to your host (for example with scp) and source it from within your bash script.
I am working on a project that is hosted in Heroku. The app is hard coded to use Amazon S3 and looks for the keys in environment variables. This is what I wrote after looking at some examples and I am not sure why its not working.
echo $1
if [ $1 != "unset" ]; then
echo "set"
export AMAZON_ACCESS_KEY_ID=XXXXXXXXXXXX
export AMAZON_SECRET_ACCESS_KEY=XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
export S3_BUCKET_NAME=XXXXXXXXX
else
echo "unset"
export AMAZON_ACCESS_KEY_ID=''
export AMAZON_SECRET_ACCESS_KEY=''
export S3_BUCKET_NAME=''
fi
When running the script it goes to the set section. But when inspecting through echo $AMAZON_ACCESS_KEY_ID # => ''.
I am not sure what is causing the issue. I will be interested in...
A fix for this...
A way to extract and add heroku config variables in the the env in an easier way.
You need to source the script, not run it as a child. If you run the script directly, its environment disappears when it ends. Sourcing the script causes it to be executed in the current environment. help source for more information.
Example:
$ VAR=old_value
$ cat script.sh
#!/bin/bash
export VAR=new_value
$ ./script.sh
$ echo $VAR
old_value
$ source script.sh
$ echo $VAR
new_value
Scripts executed with source don't need to be executable nor do they need the "shebang" line (#!/bin/bash) because they are not run as separate processes. In fact, it is probably a good idea to not make them executable in order to avoid them being run as commands, since that won't work as expected.
I'm new to Unix...I have a shell script that calls sqlplus. I have some variables that are defined within the code. However, I do not feel comfortable having the password displayed within the script. I would appreciate if someone could show me ways on how to hide my password.
One approach I know of is to omit the password and sqlplus will
prompt you for the password.
An approach that I will very much be interested in is a linux
command whose output can be passed into the password variable. That
way, I can replace easily replace "test" with some parameter.
Any other approach.
Thanks
#This is test.sh It executes sqlplus
#!/bin/sh
export user=TestUser
export password=test
# Other variables have been ommited
echo ----------------------------------------
echo Starting ...
echo ----------------------------------------
echo
sqlplus $user/$password
echo
echo ----------------------------------------
echo finish ...
echo ----------------------------------------
You can pipe the password to the sqlplus command:
echo ${password} | sqlplus ${user}
tl;dr: passwords on the command line are prone to exposure to hostile code and users. don't do it. you have better options.
the command line is accessible using $0 (the command itself) through ${!#} ($# is the number of arguments and ${!name} dereferences the value of $name, in this case $#).
you may simply provide the password as a positional argument (say, first, or $1), or use getopts(1), but the thing is passwords in the arguments array is a bad idea. Consider the case of ps auxww (displays full command lines of all processes, including those of other users).
prefer getting the password interactively (stdin) or from a configuration file. these solutions have different strengths and weaknesses, so choose according to the constraints of your situation. make sure the config file is not readable by unauthorized users if you go that way. it's not enough to make the file hard to find btw.
the interactive thing can be done with the shell builtin command read.
its description in the Shell Builtin Commands section in bash(1) includes
-s Silent mode. If input is coming from a terminal, characters are not echoed.
#!/usr/bin/env bash
INTERACTIVE=$([[ -t 0 ]] && echo yes)
if ! IFS= read -rs ${INTERACTIVE+-p 'Enter password: '} password; then
echo 'received ^D, quitting.'
exit 1
fi
echo password="'$password'"
read the bash manual for explanations of other constructs used in the snippet.
configuration files for shell scripts are extremely easy, just source ~/.mystuffrc in your script. the configuration file is a normal shell script, and if you limit yourself to setting variables there, it will be very simple.
for the description of source, again see Shell Builtin Commands.
Recently I wrote a script which sets an environment variable, take a look:
#!/bin/bash
echo "Pass a path:"
read path
echo $path
defaultPath=/home/$(whoami)/Desktop
if [ -n "$path" ]; then
export my_var=$path
else
echo "Path is empty! Exporting default path ..."
export my_var=$defaultPath
fi
echo "Exported path: $my_var"
It works just great but the problem is that my_var is available just locally, I mean in console window where I ran the script.
How to write a script which allow me to export global environment variable which can be seen everywhere?
Just run your shell script preceded by "." (dot space).
This causes the script to run the instructions in the original shell. Thus the variables still exist after the script finish
Ex:
cat setmyvar.sh
export myvar=exists
. ./setmyvar.sh
echo $myvar
exists
Each and every shell has its own environment. There's no Universal environment that will magically appear in all console windows. An environment variable created in one shell cannot be accessed in another shell.
It's even more restrictive. If one shell spawns a subshell, that subshell has access to the parent's environment variables, but if that subshell creates an environment variable, it's not accessible in the parent shell.
If all of your shells need access to the same set of variables, you can create a startup file that will set them for you. This is done in BASH via the $HOME/.bash_profile file (or through $HOME/.profile if $HOME/.bash_profile doesn't exist) or through $HOME/.bashrc. Other shells have their own set of startup files. One is used for logins, and one is used for shells spawned without logins (and, as with bash, a third for non-interactive shells). See the manpage to learn exactly what startup scripts are used and what order they're executed).
You can try using shared memory, but I believe that only works while processes are running, so even if you figured out a way to set a piece of shared memory, it would go away as soon as that command is finished. (I've rarely used shared memory except for named pipes). Otherwise, there's really no way to set an environment variable in one shell and have another shell automatically pick it up. You can try using named pipes or writing that environment variable to a file for other shells to pick it up.
Imagine the problems that could happen if someone could change the environment of one shell without my knowledge.
Actually I found an way to achieve this (which in my case was to use a bash script to set a number of security credentials)
I just call bash from inside the script and the spawned shell now has the export values
export API_USERNAME=abc
export API_PASSWORD=bbbb
bash
now calling the file using ~/.app-x-setup.sh will give me an interactive shell with those environment values setup
The following were extracted from 2nd paragraph from David W.'s answer: "If one shell spawns a subshell, that subshell has access to the parent's environment variables, but if that subshell creates an environment variable, it's not accessible in the parent shell."
In case a user need to let parent shell access your new environment variables, just issue the following command in parent shell:
source <your_subshell_script>
or using shortcut
. <your_subshell_script>
You got to add the variable in your .profile located in /home/$USER/.profile
Yo can do that with this command:
echo 'TEST="hi"' >> $HOME/.profile
Or by edit the file with emacs, for example.
If you want to set this variable for all users, you got to edit /etc/profile (root)
There is no global environment, really, in UNIX.
Each process has an environment, originally inherited from the parent, but it is local to the process after the initial creation.
You can only modify your own, unless you go digging around in the process using a debugger.
write it to a temporary file, lets say ~/.myglobalvar and read it from anywhere
echo "$myglobal" > ~/.myglobalvar
Environment variables are always "local" to process execution the export command allow to set environment variables for sub processes. You can look at .bashrc to set environment variables at the start of a bash shell. What you are trying to do seems not possible as a process cannot modify (or access ?) to environment variables of another process.
You can update the ~/.bashrc or ~/.bash_profile file which is used to initialize the environment.
Take a look at the loading behavior of your shell (explained in the manpage, usually referring to .XXXshrc or .profile). Some configuration files are loaded at login time of an interactive shell, some are loaded each time you run a shell. Placing your variable in the latter might result in the behavior you want, e.g. always having the variable set using that distinct shell (for example bash).
If you need to dynamically set and reference environment variables in shell scripts, there is a work around. Judge for yourself whether is worth doing, but here it is.
The strategy involves having a 'set' script which dynamically writes a 'load' script, which has code to set and export an environment variable. The 'load' script is then executed periodically by other scripts which need to reference the variable. BTW, the same strategy could be done by writing and reading a file instead of a variable.
Here's a quick example...
Set_Load_PROCESSING_SIGNAL.sh
#!/bin/bash
PROCESSING_SIGNAL_SCRIPT=./Load_PROCESSING_SIGNAL.sh
echo "#!/bin/bash" > $PROCESSING_SIGNAL_SCRIPT
echo "export PROCESSING_SIGNAL=$1" >> $PROCESSING_SIGNAL_SCRIPT
chmod ug+rwx $PROCESSING_SIGNAL_SCRIPT
Load_PROCESSING_SIGNAL.sh (this gets dynamically created when the above is run)
#!/bin/bash
export PROCESSING_SIGNAL=1
You can test this with
Test_PROCESSING_SIGNAL.sh
#!/bin/bash
PROCESSING_SIGNAL_SCRIPT=./Load_PROCESSING_SIGNAL.sh
N=1
LIM=100
while [ $N -le $LIM ]
do
# DO WHATEVER LOOP PROCESSING IS NEEDED
echo "N = $N"
sleep 5
N=$(( $N + 1 ))
# CHECK PROCESSING_SIGNAL
source $PROCESSING_SIGNAL_SCRIPT
if [[ $PROCESSING_SIGNAL -eq 0 ]]; then
# Write log info indicating that the signal to stop processing was detected
# Write out all relevent info
# Send an alert email of this too
# Then exit
echo "Detected PROCESSING_SIGNAL for all stop. Exiting..."
exit 1
fi
done
~/.bin/SOURCED/lazy script to save and load data as flat files for system.
[ ! -d ~/.megadata ] && mkdir ~/.megadata
function save_data {
[ -z "$1" -o -z "$2" ] && echo 'save_data [:id:] [:data:]' && return
local overwrite=${3-false}
[ "$overwrite" = 'true' ] && echo "$2" > ~/.megadata/$1 && return
[ ! -f ~/.megadata/$1 ] && echo "$2" > ~/.megadata/$1 || echo ID TAKEN set third param to true to overwrite
}
save_data computer engine
cat ~/.megadata/computer
save_data computer engine
save_data computer megaengine true
function get_data {
[ -z "$1" -o -f $1 ] && echo 'get_data [:id:]' && return
[ -f ~/.megadata/$1 ] && cat ~/.megadata/$1 || echo ID NOT FOUND
:
}
get_data computer
get_data computer
Maybe a little off topic, but when you really need it to set it temporarily to execute some script and ended up here looking for answers:
If you need to run a script with certain environment variables that you don't need to keep after execution you could do something like this:
#!/usr/bin/env sh
export XDEBUG_SESSION=$(hostname);echo "running with xdebug: $XDEBUG_SESSION";$#
In my example I just use XDEBUG_SESSION with a hostname, but you can use multiple variables. Keep them separated with a semi-colon. Execution as follows (assuming you called the script debug.sh and placed it in the same directory as your php script):
$ debug.sh php yourscript.php
I have two shell scripts, a.sh and b.sh.
How can I call b.sh from within the shell script a.sh?
There are a couple of different ways you can do this:
Make the other script executable with chmod a+x /path/to/file(Nathan Lilienthal's comment), add the #!/bin/bash line (called shebang) at the top, and the path where the file is to the $PATH environment variable. Then you can call it as a normal command;
Or call it with the source command (which is an alias for .), like this:
source /path/to/script
Or use the bash command to execute it, like:
/bin/bash /path/to/script
The first and third approaches execute the script as another process, so variables and functions in the other script will not be accessible.
The second approach executes the script in the first script's process, and pulls in variables and functions from the other script (so they are usable from the calling script).
In the second method, if you are using exit in second script, it will exit the first script as well. Which will not happen in first and third methods.
Check this out.
#!/bin/bash
echo "This script is about to run another script."
sh ./script.sh
echo "This script has just run another script."
There are a couple of ways you can do this. Terminal to execute the script:
#!/bin/bash
SCRIPT_PATH="/path/to/script.sh"
# Here you execute your script
"$SCRIPT_PATH"
# or
. "$SCRIPT_PATH"
# or
source "$SCRIPT_PATH"
# or
bash "$SCRIPT_PATH"
# or
eval '"$SCRIPT_PATH"'
# or
OUTPUT=$("$SCRIPT_PATH")
echo $OUTPUT
# or
OUTPUT=`"$SCRIPT_PATH"`
echo $OUTPUT
# or
("$SCRIPT_PATH")
# or
(exec "$SCRIPT_PATH")
All this is correct for the path with spaces!!!
The answer which I was looking for:
( exec "path/to/script" )
As mentioned, exec replaces the shell without creating a new process. However, we can put it in a subshell, which is done using the parantheses.
EDIT:
Actually ( "path/to/script" ) is enough.
If you have another file in same directory, you can either do:
bash another_script.sh
or
source another_script.sh
or
. another_script.sh
When you use bash instead of source, the script cannot alter environment of the parent script. The . command is POSIX standard while source command is a more readable bash synonym for . (I prefer source over .). If your script resides elsewhere just provide path to that script. Both relative as well as full path should work.
Depends on.
Briefly...
If you want load variables on current console and execute you may use source myshellfile.sh on your code. Example:
#!/bin/bash
set -x
echo "This is an example of run another INTO this session."
source my_lib_of_variables_and_functions.sh
echo "The function internal_function() is defined into my lib."
returned_value=internal_function()
echo $this_is_an_internal_variable
set +x
If you just want to execute a file and the only thing intersting for you is the result, you can do:
#!/bin/bash
set -x
./executing_only.sh
bash i_can_execute_this_way_too.sh
bash or_this_way.sh
set +x
You can use /bin/sh to call or execute another script (via your actual script):
# cat showdate.sh
#!/bin/bash
echo "Date is: `date`"
# cat mainscript.sh
#!/bin/bash
echo "You are login as: `whoami`"
echo "`/bin/sh ./showdate.sh`" # exact path for the script file
The output would be:
# ./mainscript.sh
You are login as: root
Date is: Thu Oct 17 02:56:36 EDT 2013
First you have to include the file you call:
#!/bin/bash
. includes/included_file.sh
then you call your function like this:
#!/bin/bash
my_called_function
Simple source will help you.
For Ex.
#!/bin/bash
echo "My shell_1"
source my_script1.sh
echo "Back in shell_1"
Just add in a line whatever you would have typed in a terminal to execute the script!
e.g.:
#!bin/bash
./myscript.sh &
if the script to be executed is not in same directory, just use the complete path of the script.
e.g.:`/home/user/script-directory/./myscript.sh &
This was what worked for me, this is the content of the main sh script that executes the other one.
#!/bin/bash
source /path/to/other.sh
The top answer suggests adding #!/bin/bash line to the first line of the sub-script being called. But even if you add the shebang, it is much faster* to run a script in a sub-shell and capture the output:
$(source SCRIPT_NAME)
This works when you want to keep running the same interpreter (e.g. from bash to another bash script) and ensures that the shebang line of the sub-script is not executed.
For example:
#!/bin/bash
SUB_SCRIPT=$(mktemp)
echo "#!/bin/bash" > $SUB_SCRIPT
echo 'echo $1' >> $SUB_SCRIPT
chmod +x $SUB_SCRIPT
if [[ $1 == "--source" ]]; then
for X in $(seq 100); do
MODE=$(source $SUB_SCRIPT "source on")
done
else
for X in $(seq 100); do
MODE=$($SUB_SCRIPT "source off")
done
fi
echo $MODE
rm $SUB_SCRIPT
Output:
~ ❯❯❯ time ./test.sh
source off
./test.sh 0.15s user 0.16s system 87% cpu 0.360 total
~ ❯❯❯ time ./test.sh --source
source on
./test.sh --source 0.05s user 0.06s system 95% cpu 0.114 total
* For example when virus or security tools are running on a device it might take an extra 100ms to exec a new process.
pathToShell="/home/praveen/"
chmod a+x $pathToShell"myShell.sh"
sh $pathToShell"myShell.sh"
#!/bin/bash
# Here you define the absolute path of your script
scriptPath="/home/user/pathScript/"
# Name of your script
scriptName="myscript.sh"
# Here you execute your script
$scriptPath/$scriptName
# Result of script execution
result=$?
chmod a+x /path/to/file-to-be-executed
That was the only thing I needed. Once the script to be executed is made executable like this, you (at least in my case) don't need any other extra operation like sh or ./ while you are calling the script.
Thanks to the comment of #Nathan Lilienthal
Assume the new file is "/home/satya/app/app_specific_env" and the file contents are as follows
#!bin/bash
export FAV_NUMBER="2211"
Append this file reference to ~/.bashrc file
source /home/satya/app/app_specific_env
When ever you restart the machine or relogin, try echo $FAV_NUMBER in the terminal. It will output the value.
Just in case if you want to see the effect right away, source ~/.bashrc in the command line.
There are some problems to import functions from other file.
First: You needn't to do this file executable. Better not to do so!
just add
. file
to import all functions. And all of them will be as if they are defined in your file.
Second: You may be define the function with the same name. It will be overwritten. It's bad. You may declare like that
declare -f new_function_name=old_function_name
and only after that do import.
So you may call old function by new name.
Third: You may import only full list of functions defined in file.
If some not needed you may unset them. But if you rewrite your functions after unset they will be lost. But if you set reference to it as described above you may restore after unset with the same name.
Finally In common procedure of import is dangerous and not so simple. Be careful! You may write script to do this more easier and safe.
If you use only part of functions(not all) better split them in different files. Unfortunately this technique not made well in bash. In python for example and some other script languages it's easy and safe. Possible to make partial import only needed functions with its own names. We all want that in next bush versions will be done the same functionality. But now We must write many additional cod so as to do what you want.
Use backticks.
$ ./script-that-consumes-argument.sh `sh script-that-produces-argument.sh`
Then fetch the output of the producer script as an argument on the consumer script.