I have the following one-line bash file foo.sh:
export PATH=<new path>
In another script, I use:
echo $PATH # --> old path
. foo.sh | grep bar
echo $PATH # --> old path!!!!
Depending on the machine I execute this second script on, the PATH is or is not updated in the main script. On the machines where it does not work, whatever the command right of the pipe, it still does not work. On the contrary, if I drop the pipe, it always work whatever the machine.
My machines are supposed to have the exact same configuration (even though, considering this issue, it looks as if they don't). Bash version is 4.1.2.
Do you have any idea where/what to look to understand this behaviour?
In bash, all parts of a pipeline are executed in separate subshells, which is why sourcing the script doesn't change the path.
Some shells are able to run the last command in the current shell environment (ksh93, for example), but bash does not (unless job control is disabled and the lastpipe shell option is enabled, and the pipeline is not executed in the background).
The bash manual states, in the "Pipelines" section,
Each command in a pipeline is executed as a separate process (i.e., in
a subshell).
Related
I am trying to execute a shell script for automating the process rather than manually running the python script. But i am getting the error folder not found.
cd /home/gaurav/AndroPyTool
export ANDROID_HOME=$HOME/android-sdk-linux/
export PATH=$PATH:$ANDROID_HOME/tools
export PATH=$PATH:$ANDROID_HOME/platform-tools
source ~/.bashrc
source droidbox_env/bin/activate
alias mycd='cd /home/gaurav/AndroPyTool/test'
mycd
pwd
python androPyTool.py -all -csv EXPORTCSV.csv -s mycd
>>>> AndroPyTool -- STEP 1: Filtering apks
Folder not found!
This is the error i am getting because the script is not able to find the path that i have provided above.
The part after "-s" in the code represents the folder path where the file stored.
The issue here is that you are not passing the path to the python program. The python program is not aware of bash aliases and bash will only expand aliases when it is interpreting the token as a command.
When bash reads python androPyTool.py -all -csv EXPORTCSV.csv -s mycd It interprets python as the command and all other space separated tokens are arguments that will be passed into python. Python then invokes androPyTool.py and passes the subsequent arguments to that script. So the program is receiving literally mycd as the argument for -s.
Moreover, even if mycd is expanded, it wouldn't be the correct argument for -s. androPyTool.py is expecting just the /path/to/apks, not cd /path/to/apks/.
I don't really think that using the alias in this script makes much sense. It actually makes the script harder to read and understand. If you want to wrap a command, I recommend defining a function, and occasionally you can use variable expansion (but that mixes code and data which can lead to issues). EDIT: As has been pointed out in the comments, aliases are disabled for scripts.
Finally there are some other suspicious issues with your script. Mainly, why are you sourcing .bashrc? If this script is run by you in your user's environment, .bashrc will already be sourced and there is no need to re-source it. On the other hand, if this is not intended to be run in your environment, and there is something in the .bashrc file that you need in your script, I recommend pulling just that out and nothing else.
But the most immediate issue that I can see is that sourcing .bashrc after you modify path runs the risk of overwriting the changes to PATH you just made. Depending on the contents of the .bashrc file, sourcing it may not be idempotent, meaning that running it more than once could have side effects. Finally, anything could get thrown in that .bashrc file down the road since that's what its for. So now your script may depend on something that likely will be changing. This opens up the possibility that bugs will creep in to your script unexpectedly.
I want to run a couple of scripts on my docker-entrypoint.sh;
My question if whether it makes any difference and if it does, what is the recommended way of going about this, regarding the following options:
A.
${HOMEDIR}/myscript --param1 --param2
B.
bash -c "${HOMEDIR}/myscript --param1 --param2"
C.
source ${HOMEDIR}/myscript --param1 --param2
It actually depends on what you are doing and what are you trying to do.
${HOMEDIR}/myscript --param1 --param2
This one will execute the script. When the script is done, any changes that it made to the environment are discarded.
bash -c "${HOMEDIR}/myscript --param1 --param2"
Running bash -c "my command here" vs. just running my command here is primarily different in that the former starts a subshell and the latter runs the commands in the current shell.
There are a number of differences in the effects, however:
Changes to the environment made in the subshell cannot affect the
parent shell (current directory, values of environment variables,
function definitions, etc.)
Variables set in the parent shell that
has not been exported will be unavailable in the subshell.
Here is my reference since I did not know much about bash -c
source ${HOMEDIR}/myscript --param1 --param2
When you call source (or its alias .), you insert the script in the current bash process. So you could read variables set by the script.
When you call sh, you initiate a fork (sub-process) that runs a new session of /bin/sh, which is usually a symbolic link to bash. In this case, environment variables set by the sub-script would be dropped when the sub-script finishes.
Also here my reference.
TL;DR: If you do not want the bash to keep the changes you want with that scripts you will be running on, I recommend you to use (A). If you want the bash to keep the variables and changes, use (C). If you want to keep the changes and make the bash run the script on another bash, use (B) as I listed the differences between them.
To copy the output of my commands launched from a shell I use
exec > >(tee myfile)
and then the next commands will be logged into the file.
The problem is the commands know the output is not a terminal anymore. So they can change how they display. For instance, with the command ls when the redirection is on, the output is displayed in only one column.
I know I can use unbuffer when I use a pipe, but it is not what I want. I want to be able to log all the outputs I have from my shell.
You can use script, which copies all output to a file (usually typescript). It does not interfere with the program, allowing it to think it is writing to the terminal.
The program is available "everywhere", though some options differ:
script(1) Linux
script(1) OSX
The main difference that I encounter is how to specify the output filename and the command. With Linux you can give a command as an option, while in OSX the command consists of the argument(s) past the filename. When using the -c option on Linux, keep in mind that script runs this using the shell identified by the SHELL environment variable. That can actually be "any" program (I've used a text editor). Running a shell to execute a command means that it may use new environment variables (normally not a problem).
If you do not use the -c option, script starts a new shell, writing everything to its output until you exit from that shell. To use it as you were doing for redirection, you could make an alias like
alias redir=`script myfile'
to write to myfile, or
alias redir='script -a myfile'
to append to myfile. In either case, exiting the shell (press controlD, or type exit) will end the "redirection".
Aside from ls (which ignores the terminal database), most programs use the TERM environment variable. It is possible that you do something unusual in initializing your shell, so that running script would reinitialize TERM to a different value than you are currently using. To see this, you could do something like
env >before.log
script -c "env >after.log"
diff before.log after.log
I have a hard time with interactive and non-interactive shells. I don't understand which is which.
For example, I have read that non interactive shells usually check for the BASH_ENV variable on their startup and execute whatever it points to.
So, what I did is I set the BASH_ENV to point to some script which only echoes OK. Then I typed in bash in terminal and this script echoed OK. But why? Didn't I call yet another INTERACTIVE shell by typing bash in terminal, and not the other way around? Why did it execute the bash_env? I'm on linux mint maya.
The only thing you can be certain of is what's shown in the manpage for bash (see INVOCATION) - that lists in details what startup files are run in each instance.
However, there's nothing stopping (for example) one of those startup files running other files which would normally not be run.
By way of example, if .bash_profile had the following line:
. ~/.profile
it would also run the .profile script.
In fact the manpage states:
When bash is started non-interactively, to run a shell script, for example, it looks for the variable BASH_ENV in the environment, expands its value if it appears there, and uses the expanded value as the name of a file to read and execute. Bash behaves as if the following command were executed:
if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
So, if you put that exact line in your startup scripts for an interactive shell like ~/.bash_profile, you'll also source the file pointed to by BASH_ENV.
Your best bet is to examine the INVOCATION section to find out which of the files will run, and then track through them (with something like set -x at the top of the script) to see what's getting called from where.
If memory serves, Bash is only interactive if you tell it, example
bash -i
So, by you calling just bash you invoked a non-interactive Bash.
More info
-i
If the -i option is present, the shell is interactive.
I have a simple bash script:
#!/bin/bash
JAVA_HOME=/usr
EC2_HOME=~/ec2-api
echo $EC2_HOME
export PATH=$PATH:$EC2_HOME/bin
I run the script like so
$ ./ec2
/Users/user/ec2-api
The script runs and produces the correct output.
However, when I now try to access the EC2_HOME variable, I get nothing out:
$ echo $EC2_HOME
I get a blank string back. What am I doing wrong?
Do either of the following instead:
source ec2
or
. ec2
(note the . notation is just a shortcut for source)
Explanation:
This is because ./ec2 actually spawns a subshell from your current shell to execute the script, and subshells cannot affect the environment of the parent shell from which it spawned.
Thus, EC2_HOME does get set to /Users/user/ec2-api correctly in the subshell (and similarly the PATH environment variable is updated and exported correctly in the subshell as well), but those changes won't propagate back to your parent shell.
Using source runs the script directly in the current shell without spawning a subshell, so the changes made will persist.
(A note on export: export is used to tell new shells spawned from the current shell to use the variables exported from the current shell. So for any variables you would only use in the current shell, they need not be exported.)
A shell script can never modify the environment of their parent.
To fix your problem, you can use the dot (.) command:
$ . ./ec2
and that should work. In cshell, it would be
% source ./ec2
To learn more about shells and scripts, my best resource is by far Unix power tools.