I'm trying to use a Conda installation that was installed by a user other than myself, and with the root installation directory having read only perms.
Supposedly this is doable in the latest version of Conda and the envs_dirs setting, but even with the envs_dirs setting set to a writable path, Conda still tries to create locks in the read only root install path. Is there any way around this?
For anyone looking at this in the future - https://github.com/conda/conda/issues/1329
This issue addresses a multitude of problems with running shared user environments in Conda. As of time of this posting, it's partially supported and somewhat hacky, though check the link for status.
Related
I had installed Anaconda on my system before I knew the difference between Anaconda and Miniconda. I would like to downsize to Miniconda since I don't want the bloat of Anaconda, but I already have a handful of environments set up.
So far the only way I can think of migrating is to completely get rid of everything right now, install Miniconda, and then recreate my environments by hand, but that seems quite arduous. Is there a smarter way of going about this?
I agree with #darthbith: Export the envs to YAML files (conda env export) then recreate them once you have Miniconda installed (conda env create).
While there are some experimental tools for packaging and moving envs (i.e., so you avoid having to redownload packages), they only work on a single env basis. So, I can't see how going this route one could avoid making multiple copies of many of the shared files. Instead, if you let Conda handle the environment (re)creation, it will leverage hardlinks to minimize disk usage, and that seems to be one of your aims.
It may be possible to avoid redownloading packages during the environment recreations by retaining the pkgs directory in the root of your Anaconda install, then copying its contents over into the pkgs of the Miniconda install. I would only copy folders/tarballs that don't conflict with the ones that come with Miniconda. After finishing environment recreation, then a conda clean -p would likely be in order, since Anaconda includes many packages that likely aren't getting reused.
I previously had Conda running smoothly on Mojave, but I've found that the upgrade to Catalina moves the "anaconda3" folder to your Desktop > Relocated Items > Security > anaconda3. It seems Catalina's security settings may not allow applications to install directly under the user directory anymore.
I tried the suggestion here, written below:
Hi, I might have a solution
Copy the folder anaconda3 located in Relocated Items to /Users/myname/
Open Terminal
Enter: export PATH=''/Users/myname/anaconda3/bin:$PATH"
Enter: conda init zsh
It worked! Good luck!
But this doesn't work for me. After conda init zsh I get:
-bash: /Users/USER/anaconda3/bin/conda: /anaconda3/bin/python: bad interpreter: No such file or directory
How can I get Conda up and running again without losing all my virtual environments? Thanks!
Update
I got Conda to work following #Ted Shaowang's suggestion. This means that conda env list shows all the virtual environments created via Conda.
However I am still experiencing an issue with virtualenv as since I changed the default anaconda3 file locations, python cannot be found.
The python executable located at .virtualenvs/env/bin/python cannot be found. Do I need to make further changes in order for python to work from virtualenv too?
I have the exact same problem and this works for me:
After you move anaconda from "Relocated Items" to ~/anaconda3, edit the first line of ~/anaconda3/bin/conda file from #!/anaconda3/bin/python to #!/Users/USERNAME/anaconda3/bin/python to reflect the change.
I would probably abstain from using the above solution. That ~/anaconda3/bin directory has lots of runnables (not just the conda one) that would need to be altered in this manual way. For example, unless you make the same change you cannot run jupyter notebook either, neither from base nor from other envs you might have.
My tip: Try getting a requirements file for your virtual envs, and do a fresh installation. You could use pipreqs to get the requirements used for individual projects: https://www.idiotinside.com/2015/05/10/python-auto-generate-requirements-txt/
No solution will be completely working without fixing the baked-in hard-coded prefix entries in files. There's a longer description and a recommended fix at https://www.anaconda.com/how-to-restore-anaconda-after-macos-catalina-update/
Technically this is reinstalling anaconda, however, I restored all my conda envs so, hopefully this is an acceptable solution!
Here is how I got it working on Catalina as of a few minutes ago (now using z-shell):
- Verified the existence of "Relocated Items" directory on my desktop and the "anaconda3" directory and its contents inside
- Navigated into the envs directory under "anaconda3" and left the finder window open (see screenshot)
THEN:
opened new Terminal (z-shell)
ran (this installed to /usr/local/anaconda3):
brew cask install anaconda
after installation was successful I opened my ~/.zshrc file (for my z-shell aliases) and added the following line:
export PATH="/usr/local/anaconda3/bin:$PATH"
saved my ~/.zshrc file, then reloaded with:
source ~/.zshrc
to verify conda command works now, I ran:
conda env list
for me, this showed a base env and that was it
then open Finder to the new envs location:
open /usr/local/anaconda3/envs/.
I dragged (moved) all my old envs into the new envs folder, and then ran:
conda env list
And all my old envs are back! :)
updated my PyCharm interpreter / env paths to reflect the new locations of these envs (had to restart PyCharm after, but now it works!)
DONE
Unsure whether this is considered a comment or at least a temporary answer, but I would refrain from attempting to fix any Catalina compatibility issues with Anaconda for now. See this GitHub issue.
I have the same problem, and this work for me :
My solution:
Copy your anaconda3 from Relocated Items folder
Paste in User/YourUserName
Open conda file in anaconda3/bin with the editor and edit the first line #!/anaconda3/bin/python to #!/Users/YourUserName/anaconda3/bin/python
Save it and run conda file
Open Terminal
Run this : export PATH=''/Users/YourUserName/anaconda3/bin:$PATH"
Run conda init zsh
I had incurred the same issue, and the following solution worked for me, and this is the easiest solution:
Instead of messing around copying the anaconda3 file from relocated items into User/USERNAME directory, better would be just to reinstall anaconda navigator's latest version from its official website : https://www.anaconda.com/distribution/#macos
While installation, it will ask you some permissions which are a result of new Apple Security Policies, just grant them, and it works just the way it should after this fresh installation!
This is what worked for me.
These are my header files (Catalina 10.15):
/Library/Developer/CommandLineTools/usr/include/c++/v1/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/sys/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/usr/include/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.14.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/sys/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/stdio.h
Run sudo find /Library -name stdio.h to see where yours are located.
Mojave 10.14 header files:
$ sudo find /Library -name stdio.h
/Library/Developer/CommandLineTools/usr/include/c++/v1/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/sys/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/stdio.h
/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks/Kernel.framework/Versions/A/Headers/sys/stdio.h
As can be seen the SDKs are now split into MACOSX10.14 and 10.15 unlike in Mojave.
TLDR
So, these were my SDK folders on Catalina:
Rename MacOSX.sdk to MacOSX_orig.sdk
Right click on MacOSX10.14.sdk
Duplicate
Rename duplicate folder to MacOSX.sdk
Your folder structure should now look like this:
Like this we are basically using the previous version's OSX sdk as sysroot. Hope this helps.
I am running different conda env and I'd like to specify where the packages are downloaded to, rather than having all of them in my $home.
I have found this question which, at time of writing, has no answers. However, my question is different: I don't want to specify the pkg_dir in my .condarc because I want to have a different download dir for each project (space is not an issue).
How do I define the pkg_dir for a specific conda env?
To note, I'm creating environments as conda env create -f my_env.yml -p complete-env.
A fundamental concept of conda is that packages get downloaded and extracted to a shared cache, from which they are selectively linked into the different conda environments. You want to work against this concept, so whatever you do will be hacky and have repercussions.
You could install a separate Miniconda for each of your projects, and (try to) make sure that they don't know about eachother by removing all conda-related files and environment settings from your home directory, or even use a different HOME for each project. Before working with a project, you'd have to put the correct conda on the PATH.
Or you could install Miniconda on a dedicated drive apart from your home directory, and put the conda environments inside your home directory. That would prevent conda from hard-linking the files. It would still download the packages into the shared cache, but then copy only the relevant files into each of your projects. Of course, copying is slower than hard-linking.
Specifying the package directory per environment rather than per conda installation is not possible, as darthbith has already pointed out in a comment.
I am experiencing unexpected/unusable behavior from pip and virtualenvs. For context, I am using fish shell.
When I create a new virtual env and install new packages to it, pip list/freeze correctly shows the installed packages. However, which pip routes to my system python, not my virtual environment python.
Before and after entering my virtualenv, which pip returns the same system path. Further, when I run code, my import statements do not detect installed packages.
I've spent days looking through over posts before asking, but I haven't found one that describes my exact situation / an adequate solution.
It is not a problem due to:
- aliases
How can I fix this?
Virtualfish is a project designed to gracefully handle virtual environments in a Fish shell context. You may find it provides a solution to the problems you are experiencing, as well as a number of other significant benefits.
I run Ubuntu 18.04.1 LTS on a virtual machine on Windows 10. I've installed Anaconda 5.3 without any issues, but when I try to install an STSCI environment with the command line:
conda create -n astroconda stsci
the terminal gets stuck in "Solving Environment" (I have version 4.5.11 of conda). Yesterday I waited for about 4 hours before giving up. I would like to know if there is a known issue about this problem and if it can be related to the quantity of disk and RAM assigned to the virtual machine.
I will write a more general solution, to Conda's "Solving Environment" issue, which I had the uttermost pleasure with.
Short answer of things to try:
As already mentioned try updating Conda with conda update conda or even better, the whole base environment conda update --all.
Specify package and build version using <package>=<version>=<build> e.g. sage=8.3=py27_3. Search available versions by conda search <package>.
Like for the package, you can also specify python version and ideally some other dependencies. Check dependencies with conda info <package>.
Check current configuration inside the .condarc file or with conda config --get and check if you maybe have additional restrictions, as normally you will only find the channels defined there.
When working with additional channels put conda-forge or the channel you want to use on top and add channel_priority: strict. So your .condarc file would look like this:
channel_priority: strict
channels:
- conda-forge
- defaults
Contrariwise to the above, remove the first line and try adding --no-channel-priority to the command. This one helped me a couple times since updating to Conda 4.6.
If above does not work:
When conda seem stuck it is possibly having too many options or some conflicts resulting in the SAT solver getting clause counts of multiple millions.
To check if this is the case add -vv or --debug and you will later see lines like Invoking SAT with clause count: XXX. If it stays at one of these lines for long time, then try specifying version for packages as above. If there seem another issue, try the conda GitHub.
Another useful tip if you are using the conda-forge channel, is to go over their Tips & tricks.
I have the same problem on macOS 10.13.6.
The creation of iraf environment
conda create -n iraf27 python=2.7 iraf-all pyraf-all stsci
works fine instead.
Update: I found out that specifying the python version
conda create -n astroconda python=3.6 stsci
makes the installer work also with astroconda.
I had the same issue but then I updated Conda and it worked...
$ conda update conda