Conda does not set paths when activating environment - bash

When starting a new shell, the PATH environment variable is not configured
properly. The directories anaconda3/bin or miniconda3/bin are at second
position only, not at first position in the PATH variable. This can be
resolved by conda deactivate and activating it again.
This question was asked several times already (e.g. here and here)
but the existing questions are either very old or concentrate on the use of
source activate env-name. All in all, I found no answer that resolves my
problem.
When I start a new shell, the base environment is activated. The relevant
snippet from my .bashrc reads like this:
condaexe="/home/$USER/.miniconda3/bin/conda"
condash="/home/$USER/.miniconda3/etc/profile.d/conda.sh"
__conda_setup="$($condaexe 'shell.bash' 'hook' 2> /dev/null)"
# shellcheck disable=SC2181
if [[ $? -eq 0 ]]
then
eval "$__conda_setup"
elif [[ -f "$condash" ]]
then
source "$condash"
fi
unset __conda_setup condaexe condash
Then, the PATH variables are defined as follows:
(base)$ echo $PATH
/home/user/.local/bin:/home/user/.miniconda3/bin:/home/user/.miniconda3/condabin:/home/user/workspace/my-project/:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
(base)$ conda deactivate && echo $PATH
/home/user/.local/bin:/home/user/.miniconda3/condabin:/home/user/workspace/my-project/:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
$ conda activate base && echo $PATH
/home/user/.miniconda3/bin:/home/user/.local/bin:/home/user/.miniconda3/condabin:/home/user/workspace/my-project/:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
(base)$
Note that /home/user/.local/bin is contained twice; once before and once
after the Miniconda3 directories.
I tried to debug the problem by appending the following snippet to .bashrc:
echo $CONDA_PROMPT_MODIFIER
echo $PATH
This yields
(base)
/home/user/.miniconda3/bin:/home/user/.miniconda3/condabin:/home/user/workspace/my-project:/home/user/.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
which would be perfectly fine but, somehow, is modified after .bashrc.
Note that here /home/user/.local/bin is contained only once.
What goes on here? How can I setup Bash/Conda to get a properly defined PATH
environment variable?

There are three components I missed in the original question which are key to the solution.
First, I run all my shells inside TMux. Second, TMux sources the .profile. Third, in .profile some local directories like the aformentioned ~/.local/bin are blindly prepended to the PATH.
Taken this altogether all the weird behaviour above makes sense. Especially that PATH is correct at the end of ~/.bashrc but not in the shell is obvious now; it got modified in between by ~/.profile.
There are three solutions for this:
deactivating and activating conda manually (temporary workaround)
being more picky about which shells are started inside TMux (very unhandy)
commenting out the manipulations of PATH in ~/.profile
It seems as if one cannot configure TMux to source only ~/.bashrc (reference 1, reference 2), albeit some workarounds exist here too.

This is related to the Bash init files. By default, ~/.bashrc is used in an interactive, non-login shell. It won't be sourced in a login shell. Tmux uses a login shell by default. Hence, shells started by tmux skip ~/.bashrc.
default-command shell-command
The default is an empty string, which instructs tmux to create a login shell using the value of the default-shell option.
Init files for Bash,
login mode:
/etc/profile
~/.bash_profile, ~/.bash_login, ~/.profile (only first one that exists)
interactive non-login:
/etc/bash.bashrc (some Linux; not on Mac OS X)
~/.bashrc
non-interactive:
source file in $BASH_ENV
The weird interactive, non-login loading requirement confuses people in other situations as well. The best solution is to change the loading requirement of ~/.bashrc as interactive only, which is exactly what some distros, like Ubuntu, are doing.
# write content below into ~/.profile, or ~/.bash_profile
# if running bash
if [ -n "$BASH_VERSION" ]; then
# include .bashrc if it exists
if [ -f "$HOME/.bashrc" ]; then
. "$HOME/.bashrc"
fi
fi
This should be the solution you desire. And I recommend every Bash user setup this in the profile.
References
Unix Shell Initialization
man tmux

Thanks for this post! That give me hint to debug same issue. I think everyone is different situdation but the idea is same: there is a command like export $PATH change the path in your zshrc or .bashrc which will affect the computer to find the path you want.

Related

Using Anaconda with zsh on Catalina

The Anaconda installer added this script originally to my .bashrc or .bash_profile so I copied it over to .zshrc when I switched to zsh. I recently read I didn't need to/shouldn't have copied it over since it was meant for .bash_profile, but now that macOS is moving to use zsh anyway I'd like to know what I should do. Obviously I need to tell zsh where Anaconda is but do I need that script or can I just export the Anaconda path like export PATH=/Users/ty604/anaconda3/bin:$PATH?
Script added by Anaconda installer.
export PATH=/Users/ty604/anaconda3/bin:$PATH
# added by Anaconda3 2019.03 installer
# >>> conda init >>>
# !! Contents within this block are managed by 'conda init' !!
__conda_setup="$(CONDA_REPORT_ERRORS=false '/Users/ty604/anaconda3/bin/conda' shell.bash hook 2> /dev/null)"
if [ $? -eq 0 ]; then
\eval "$__conda_setup"
else
if [ -f "/Users/ty604/anaconda3/etc/profile.d/conda.sh" ]; then
. "/Users/ty604/anaconda3/etc/profile.d/conda.sh"
CONDA_CHANGEPS1=false conda activate base
else
\export PATH="/Users/ty604/anaconda3/bin:$PATH"
fi
fi
unset __conda_setup
# <<< conda init <<<
I also have many duplicate paths in $PATH because of duplicate export commands in various shell files.
$ echo $PATH
/Users/ty604/anaconda3/bin:/Users/ty604/anaconda3/condabin:/Users/ty604/anaconda3/bin:/Users/ty604/.cargo/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Applications/VMware Fusion.app/Contents/Public:/usr/local/share/dotnet:/Users/ty604/flutter/bin
Files in my system
.bash_history, .bash_profile, .bashrc, .profile, .zprofile, .zsh_history, .zshrc, .zshrc.pre-oh-my-zsh
Folders in my system
.bash_sessions, .oh-my-zsh, .zsh
macOS will be using zsh moving forward and we can set zsh as the default shell in Catalina which I have done. With Catalina is it ok if I remove all traces of bash and oh-my-zsh since zsh is now the default shell? Also I am using Hyper Terminal which means I do not need oh-my-zsh any longer right?
If you are using zsh as your shell (which is up to you, since Catalina still provides bash as well), and need some settings for it, they should go into .zshrc, respectivels .zprofile. Of course you don't blindly copy everything from .bashrc over, because you need to be sure that the code is valid under Zsh too. However, the code snippet you posted, looks safe for me, i.e. it should work under both bash and zsh.
In this case, I suggest (for easier maintenance) to put initialization code common to bash and zsh into a separate file, say ~/.commonrc, and source this file in .zshrc and .bashrc. Note also that .zshrc is only read if this is an interactive shell. See the section STARTUP/SHUTDOWN FILES in the zshall man-page.
Found out via an Anaconda dev post that she was using
conda_setup="$(CONDA_REPORT_ERRORS=false '/Users/user_name/anaconda3/bin/conda' shell.zsh hook 2> /dev/null)"
Couldn't find shell.zsh hook anywhere else so hopefully this will help someone else.
I am proceeding to move all bash files outside of my system since the default macOS zsh shell is now zsh.

Appending bash alias to .bashrc doesn't work

I want to create a alias of my cd command. I have created the .bashrc file and append the command cd ...... to it. (Since the file was newly created, it just has this one line that I added).
After that, only after I typed . ~/.bashrc, can the alias works. If I close the terminal and open it again, I need to retype . ~/.bashrc.
It's really annoying to do this every time. Is there any way to solve this problem?
Thank you so much for help
When you login to linux system, only ~/.profile will be called:
$ cat ~/.profile
# if running bash
if [ -n "$BASH_VERSION" ]; then
# include .bashrc if it exists
if [ -f "$HOME/.bashrc" ]; then
. "$HOME/.bashrc"
fi
fi
You need to source ~/.bashrc inside ~/.profile manaully.
Read this to learn more.
EDIT:
If you're using iTerm2 on mac, it actually start a login shell by default when open tabs.
But you can change it: Preferences > General > Command
If using OS X, append the alias to ~/.bash_profile.
You could also add alias to ~/.bashrc, then add source ~/.bashrc to ~/.bash_profile.
Better yet, put all your aliases in ~/.aliases, and source it in ~/.bash_profile.
By default, OS X first sources /etc/bashrc (which shouldn't be modified unless absolutely necessary), then sources the user's ~/.bash_profile at the start of every interactive session.

How to load ~/.bash_profile when entering bash from within zsh?

I've used bash for two years, and just tried to switch to zsh shell on my OS X via homebrew. And I set my default (login) shell to zsh, and I confirmed it's set properly by seeing that when I launch my Terminal, it's zsh shell that is used in default.
However, when I try to enter bash shell from within zsh, it looks like not loading ~/.bash_profile, since I cannot run my command using aliases, which is defined in my ~/.bash_profile like alias julia="~/juila/julia", etc.. Also, the prompt is not what I set in the file and instead return bash-3.2$.
For some reasons, when I set my login shell to bash, and enter zsh from within bash, then ~/.zshrc is loaded properly.
So why is it not loaded whenever I run bash from within zsh? My ~/.bash_profile is symbolic linked to ~/Dropbox/.bash_profile in order to sync it with my other computers. Maybe does it cause the issue?
Open ~/.zshrc, and at the very bottom of the file, add the following:
if [ -f ~/.bash_profile ]; then
. ~/.bash_profile;
fi
Every time you open the terminal, it will load whatever is defined in ~/.bash_profile (if the file exist). With that, you can keep your custom settings for zsh (colors, and etc). And you get to keep your custom shell settings in .bash_profile file.
This is much cleaner than using bash -l IMO.
If you prefer putting your settings in .bashrc, or .bash_login, or .profile , you can do the same for them.
Similarly, you could also move the common profile settings to separate file, i.e. .my_common_profile, and add the following to both .bash_profile and .zshrc:
if [ -f ~/.my_common_profile ]; then
. ~/.my_common_profile;
fi
An interactive bash reads your ~/.bash_profile if it's a login shell, or your ~/.bashrc if it's not a login shell.
A typical .bash_profile will contain something like:
if [ -f ~/.bashrc ]; then . ~/.bashrc; fi
so .bashrc can contain commands to be executed by either login or non-login shells.
If you run bash -l rather than just bash, it should read your .bash_profile.
Reference: https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html
For those who have just installed zsh and want their alias from bash to work on zsh do the following
Open .zshrc file in vim like so
vi ~/.zshrc
Scroll to the bottom
click "i" to enable write mode
Tell zsh to load items from bash_profile when needed like so
source ~/.bash_profile
Write and quit like so
:wq
Refresh your zsh like so
source ~/.zshrc
That's it. Now all your saved alias in .bash_profile will be ready to use in zsh.
To complement #Keith Thompson's excellent answer:
macOS:
As #chepner puts it succinctly (emphasis mine):
In OS X, bash is not used as part of the initial [at boot time] login process, and the Terminal.app (or other terminal emulators) process exists outside any pre-existing bash sessions, so each new window [or tab - read: interactive bash shell] (by default) treats itself as a new login session.
As a result, some OSX users only ever create ~/.bash_profile, and never bother with ~/.bashrc, because ALL interactive bash shells are login shells.
Linux:
On Linux, the situation is typically reversed:
bash shells created interactively are [interactive] NON-login shells, so it is ~/.bashrc that matters.
As a result, many Linux users only ever deal with ~/.bashrc.
To maintain bash profiles that work on BOTH platforms, use the technique #Keith Thompson mentions:
Put your definitions (aliases, functions, ...) in ~/.bashrc
Add the following line to ~/.bash_profile
[[ -f ~/.bashrc ]] && . ~/.bashrc
Copy the contents from ~/.bash_profile and paste them at the bottom of ~/.zshrc file.
For ZSH users on MacOs, I ended up with a one liner.
At the very bottom of the ~/.zshrc I added the following line :
bash -l
What it does is simply load the .bash_profile settings(aliases, function, export $PATH, ...)
If you decide to get rid of ZSH and go back to plain BASH, you'll be back to normal with no hassle at all.
If this is something that you do infrequently, or it just isn't appropriate to make changes, you can also 'source' the .bash_profile after launching the child bash shell.
. ~/.bash_profile
This will pull in the settings you make in the .bash_profile script for the life of that shell session. In most cases, you should be able to repeat that command, so it's also an easy way to test any changes that you make without needing to do a full login, as well as bring all of your existing shell sessions up-to-date if you make upgrades to the .bash_profile &/or .bashrc files.
For macOS Big Sur (Version 11.5.2)
Open .zshrc
For example: sudo nano ~/.zshrc
At the end of the file add source ~/.bash_profile
Every time you open the terminal the contents inside the bash profile will be loaded.
Recently I installed oh-my-zsh on OS X and set zsh as default shell and faced the same problem.
I solved this problem by adding source ~/.bash_profile at the end of .zshrc file.
I am using a zsh framework called oh my zsh and I have tried most of the solutions listed here and it broke the format for my custom theme. However, these steps worked for me.
Add new alias(es) at the bottom of my .bash_profile
vi ~/.bash_profile
Make zsh to load items from .bash_profile
source ~/.bash_profile
Refresh zsh
source ~/.zshrc
Restart OSX Terminal app
Try your new alias!
If you'd like to be "profile-centric", you can create .profile as a single source of truth, then load it from both .bash_profile and .zprofile.
.profile
export PATH="/usr/local/opt/python/libexec/bin:$PATH"
# etc., etc.
.bash_profile and .zprofile
if [ -f ~/.profile ]; then
. ~/.profile;
fi
I found this helped bash scripts find the right PATH, etc., and helped me keep configuration in one place.

How do I configure git to inherit my specific $PATH environment variable?

My goal is to have a set of dot files shared between the many shell accounts I have so that I can have a similar environment in each one. Because of that, I don't want machine/environment specific information in my .git files or my .bash files.
On my MacOSX machine, the default emacs version is 22. I have installed a newer version of emacs (24) in /usr/local/bin. In my .bashrc file I set my default editor to emacs: export EDITOR=emacs. Before this, I also set up my path so that /usr/local/bin comes before /usr/bin so that the newer version of emacs gets selected.
When I use git to create commit messages, it always picks the one in /usr/bin instead of the one in /usr/local/bin. Git is ignoring my $PATH settings. I used to hardcode the location of the version of emacs I wanted in my .gitconfigure file, but that won't work when I use that same .gitconfigure file on my Ubuntu server. (If it's unclear, what I'm saying is that modifying my .gitconfigure editor option to point to a hardcoded won't work for me).
Why doesn't git respect my $PATH settings?
How do I configure git to use the correct emacs?
Edit: Adding more context
There is a suggestion to put environment variables in .profile/.bash_profile. Here's some relevant code from my .bashrc and my .bash_profile
.bashrc
Note: I wrote a bash function called vercomp which compares version numbers.
OS=''
case $OSTYPE in
darwin*) OS='Mac' ;;
linux*) OS='Linux' ;;
*) OS='UNKNOWN' ;;
esac
export EDITOR=emacs
if [[ $OS = 'Mac' ]]; then
### EMACS VERSION CHECK
# make sure that we're working with emacs >= 24
wanted_ver=24
curr_ver=`emacs --version | grep -oE '[[:digit:]]+\.[.[:digit:]]*'`
vercomp $curr_ver $wanted_ver
# If vercomp returns 2, then our current emacs version isn't good enough.
if [[ $? == 2 ]]; then
if [[ -e '/usr/local/bin/emacs' ]]; then
emacs_path='/usr/local/bin/emacs -nw'
elif [[ -e '/Applications/Emacs.app/Contents/MacOS/Emacs' ]]; then
emacs_path='/Applications/Emacs.app/Contents/MacOS/Emacs -nw'
else
echo -n "EMACS VERSION OUT OF DATE: $curr_emacs_version. "
echo 'Install a newer version.'
emacs_path=''
fi
export EDITOR="$emacs_path"
alias emacs="$emacs_path"
fi
fi
.bash_profile
source ~/.bashrc
[[ -s "$HOME/.rvm/scripts/rvm" ]] && source "$HOME/.rvm/scripts/rvm" # Load RVM into a shell
session *as a function*
LESS=-FRX;export LESS
I expect to have an answer that explains WHY MacOSX doesn't pick up my
bashrc/profile environment changes and a solution to HOW I can get
MacOSX to pick up those changes.
This is not specific to OS X - here is an example:
$ uname -a
Linux hostname 3.13.7-gentoo #1 SMP PREEMPT Mon Mar 24 18:40:46 CET 2014 i686 Intel(R) Atom(TM) CPU 330 # 1.60GHz GenuineIntel GNU/Linux
$ cat .bashrc
export EDITOR="emacs"
$ echo $EDITOR
/usr/bin/vim
$ mv .bashrc .bash_profile
$ logout
Connection to hostname closed.
$ ssh hostname
$ echo $EDITOR
emacs
For what I know and used to "live" is that environment variables are expected to reside inside /etc/profile (system wide) and can be extended and overwritten on a user level basis inside ~/.profile respectively ~/.bash_profile.
From man bash:
When bash is invoked as an interactive login shell, or as a
non-interac- tive shell with the --login option, it first reads
and executes commands from the file /etc/profile, if that file
exists. After reading that file, it looks for ~/.bash_profile,
~/.bash_login, and ~/.profile, in that order, and reads and
executes commands from the first one that exists and is readable.
On the other hand. .bashrc is the place to go where one places custom functions, aliases, prompt configuration, etc.
So a simple solution to your specific question - aside from using another git version in your special situation - would be to simply move your environmant variables from your .bashrc into a .bash_profile.
The Linux From Scratch project provides some pretty good examples about how to organize the bash shell startup files. Specific problems on OS X are tackled inside "Setting environment variables in OS X?".
Do not set variables in .bashrc. Bash only looks at this file in interactive sessions, so programs outside of an interactive session will never see any variables from .bashrc.
For bash, environment variables belong into ~/.profile!.
To fix your problem immediately, you may set $EDITOR to an absolute path:
if [[ -e /usr/local/bin/emacs ]]; then
export EMACS=/usr/local/bin/emacs
else
export EMACS=emacs
However, the issue points to a deeper problem in your environment setup, but your question currently lacks some information (see my comment).
Generally, $PATH and environment variables are a mess on OS X, because there's no single place to change them (for instance, profile variables do not propagate to the desktop and GUI applications), and because OS X itself meddles with $PATH by means of path_helper. You need to be extra careful about where you set variables, and you need use workarounds like exec-path-from-shell to have your environment variables available in GUI applications.
Any program is free to set PATH inside of itself to whatever it wants -- this tactic is often used in setuid programs to prevent execution of the wrong (or downright bad) executables.
You'll need to find out whether Emacs Terminal.app creates a Bash that runs ~/.bashrc, ~/.bash_profile, both, or something else entirely. (Sorry, I'm a vim user.)
Try this:
export VISUAL=emacs
I just tried export VISUAL=gedit from bash on my Ubuntu and now git uses gedit
when called from that terminal.

Cygwin shell doesn't execute .bashrc

After start the cygwin shell, it just locate in a wrong home dir:
xfire#codingme.com ~
$ pwd
/cygdrive/c/Users/xfire
But it used to be /home/xfire
xfire#codingme.com /etc
$ cat passwd | grep xfire
xfire:unused:22773:10513:U-CORP\xfire,S-1-5-21-527237240-725345543-682003330-12773:/home/xfire:/bin/bash
And the .bashrc in the /home/xfire was not executed, even I copy it to the /cygdrive/c/Users/xfire, it also doesn't work!
On cygwin, I add this to my ~/.bash_profile:
. ~/.bashrc
Some program add an "HOME" environment in windows registry and set the value to "C:\Users\xfire", that's why cygwin take that directory as the home. cygwin.com/faq-nochunks.html
On my version of cygwin I found that only ~/.profile was being execueted so added
if [ -e "${HOME}/.bash_profile" ]; then
source "${HOME}/.bash_profile"
fi
to the .profile file. My .bash_profile file contains another test for .bashrc and executes that from inside there. I also added the following two lines to my .bashrc file.
export BASH_ENV="${HOME}/.profile"
export ENV="${HOME}/.profile"
The first of these ensures that .profile gets executed in non-interactive terminals and the second ensures it gets executed in POSIX terminals. I found a very useful explanation of what get run and when in the Bash Reference Manual.
In your case it wouldn't help as you have an issue with the value of your HOME environment variable but this page comes up quite high on the list when searching for this issue.
You can also also set BASH_ENV variable, e.g., BASH_ENV='C:\DOCUME~1\dwyttenb\.bashrc'
~/.bash_profile is executed before the initial command prompt is returned to the user. After that, every time a new shell is opened, ~/.bashrc is executed.
Adding . ~/.bashrc at end of ~/.bash_profile has resolved the problem.
Now you can check existing aliases using alias command.
Find more details on github.

Resources