I want to create a docker image, on Google VM.
One of the steps of the image is a conda environment creation. This is the command from the dockerfile (i am omiting the RUN):
conda env create -n cq -f environment.yml
This command installs a lot of packages and i end up without disk space.
I have two questions.
I am not sure what the -f environment.yml does. I search for this flag online, but i could not find any example together with a .yml file.
Can i remove some of the un-necessary packages before the installation happens?
-f is an alias for --file
What constitutes an unnecessary package? (I assume you have nothing unnecessary in your environment file.) Maybe you're asking about the packages bundled by installing a complete Anaconda distribution? You may find this guide helpful (including some links in the intro).
Try looking at some of Conda Forge's images to get an idea of best practice examples, maybe Miniforge3 4.10.0 to suggest a specific one.
At minimum one should make an effort to clean after every Conda operation, and do it in a chained operation so Docker doesn't save an intermediate with temporary files. A start would be
RUN conda env create -n cq -f environment.yml && conda clean -afy
I want to install python 2.7 as a conda environment.
conda create -n python2 python=2.7 anaconda
Collecting package metadata (current_repodata.json): done
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment:
it's been running for the last 12 hours.
If all that is actually needed is Python 2.7 environment (not full Anaconda distribution), then see #jakub's answer. However, Conda is perfectly capable of creating an Anaconda distribution environment with Python 2.7, and it should not take 12+ hours to solve.
Why so long? Channels!
The extremely long solve is almost certainly aggravated by your channel priorities. An "Anaconda" distribution should source most - if not all - of its software from the anaconda channel (part of defaults channel). However, most users eventually add conda-forge into their global channels and give it higher or equal priority (e.g., channel_priority: flexible). When this is the case, Conda will spend a bunch of time trying to satisfy the packages specified within the anaconda metapackage with the latest versions from conda-forge, and that's what tends to bog things down.
Option 1: Avoid Mixing Anaconda and Conda Forge
If you want a faster Anaconda install, then install only from Anaconda
conda create -n anaconda27 --override-channels -c defaults python=2.7 anaconda
Everything in the anaconda metapackage was originally intended to be sourced from the anaconda channel, so this shouldn't be so unreasonable.
Note that if you have conda-forge prioritized globally, this will be an issue every time you install in this environment (so remember to override channels).
Option 2: Mamba
Another option is Mamba. It's a faster (compiled) drop-in alternative to the conda CLI functionality. It seems to both solve faster and less prone to mutate unrelated packages when requesting changes - but that's just my anecdotal experience.
# install it in your *base* env (only need this once)
conda install -n base conda-forge::mamba
# use it like you would `conda`
mamba create -n python2 python=2.7 anaconda
The anaconda package is a metapackage, meaning it tells conda to install other packages. It will install hundreds of packages, and it turns out this can stress conda. One typically does not need all of the packages in the anaconda metapackage -- it is often better to install only the packages one requires.
Try to create an environment without anaconda and instead specify only the packages you need.
conda create -n python2 python=2.7
I am trying to install packages in anaconda3 from a machine which does not have internet.
I downloaded the pkg.tar.gz package & ran
conda install pkgname.tar.gz
Then I get the annoying
Collecting package metadata (current_repodata.json):
& then it timesout.
I even tried the
conda install --offline pkg.tar.gz
I get
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
PackagesNotFoundError: The following packages are not available from current channels:
- pkg.tar.gz
Current channels:
- https://repo.anaconda.com/pkgs/main/linux-64
- https://repo.anaconda.com/pkgs/main/noarch
- https://repo.anaconda.com/pkgs/r/linux-64
- https://repo.anaconda.com/pkgs/r/noarch
To search for alternate channels that may provide the conda package you're
looking for, navigate to
https://anaconda.org
and use the search bar at the top of the page.
What am i doing wrong ?
If i try to use pip from the anaconda few of the packages get installed, but this does not.
Follow up question
If i use pip to install a pakcage, would i be able to acccess it by conda & anaconda's python ?
I gave up on downloading packages manually & then installing them only to find out that I need to download some more packages(chain dependency).
I found out another easier way of using conda-pack where you can build your environment in a machine that has internet(this reduces the chain dependency problem) & then .tar it, then transport it to the non-internet connected machine & then untar it.
More info can be found here https://conda.github.io/conda-pack/
I hope this solves time & effort for someone.
You may find a way like this:
First unzip your .gz and get the .tar
Find your Conda env.
conda info --envs
Use conda activate according to your env e.g:
conda activate C:\SomePathGivenByCondaInfo
Use pip install with absolute path to ensure it:
pip install C:\downloads\SomePackage.tar
I get this error:
Collecting package metadata (repodata.json): failed
I tried to reinstall anaconda again and again as well as windows 10 but no result...
can you help me to fix this issues...
thank you
I was issuing this when updating pandas inside a HDICluster. This is a known issue with Conda 4.7.11. The solution which works for me was running the follow lines, as described at MDN
rm /usr/bin/anaconda/lib/python2.7/site-packages/conda/core/subdir*
wget https://gregorysfixes.blob.core.windows.net/public/subdir_data.py -P /usr/bin/anaconda/lib/python2.7/site-packages/conda/core/
I am not sure, but I think update anaconda can work too. To update anaconda:
conda update -n base conda
I tried to update or install new packages from anaconda and lately, this message has appeared:
The environment is inconsistent, please check the package plan carefully
The following package are causing the inconsistency:
- defaults/win-32::anaconda==5.3.1=py37_0
done
I tried with conda clean --all and then conda update --all but it persists.
Conda Info
active environment : base
active env location : C:\Users\NAME\Continuum
shell level : 1
user config file : C:\Users\NAME\.condarc
populated config files : C:\Users\NAME\.condarc
conda version : 4.6.11
conda-build version : 3.17.7
python version : 3.7.3.final.0
base environment : C:\Users\NAME\Continuum (writable)
channel URLs : https://repo.anaconda.com/pkgs/main/win-32
https://repo.anaconda.com/pkgs/main/noarch
https://repo.anaconda.com/pkgs/free/win-32
https://repo.anaconda.com/pkgs/free/noarch
https://repo.anaconda.com/pkgs/r/win-32
https://repo.anaconda.com/pkgs/r/noarch
https://repo.anaconda.com/pkgs/msys2/win-32
https://repo.anaconda.com/pkgs/msys2/noarch
package cache : C:\Users\NAME\Continuum\pkgs
C:\Users\NAME\.conda\pkgs
C:\Users\NAME\AppData\Local\conda\conda\pkgs
envs directories : C:\Users\NAME\Continuum\envs
C:\Users\NAME\.conda\envs
C:\Users\NAME\AppData\Local\conda\conda\envs
platform : win-32
user-agent : conda/4.6.11 requests/2.21.0 CPython/3.7.3 Windows/10 Windows/10.0.17763
administrator : False
netrc file : None
offline mode : False
I had faced the same problem. Simply running
conda install anaconda
solved the problem for me.
saw this on Google Groups
This message was added in conda 4.6.9, previously there was no indication when conda detected an inconsistent environment unless conda was run in debug mode. It is likely that your environment was inconsistent for some time but the upgrade to conda made it visible. The best option it to run "conda install package_name" for the inconsistent packages to let conda try to restore consistency.
and it really works for me.
Maybe you should try conda install anaconda in your situation.
The inconsistencies are caused due to different versions of the packages, and their clashing dependencies.
conda update --all
This command updates all the packages, and then conda solves the inconsistency on its own.
Had this same problem and none of the other solutions worked for me. Ended up having to uninstall and reinstall conda, then reinstall all of my libraries.
Ultimate solutions:
conda activate base
conda install anaconda
conda update --all
Works on Windows 10 and Ubuntu 18.04 (credits to #MF.OX for ubuntu).
Removed following problems for me:
The environment is inconsistent
WARNING conda.base.context:use_only_tar_bz2(632)
If the other solutions don't work, reverting the environment can fix this.
Use conda list --revisions, pick a revision number, and use conda install --revision [#] going back step-by-step until everything works again.
Given a situation like the following,
> conda update -c intel --all
Collecting package metadata: done
Solving environment: |
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- intel/win-64::ipython==6.3.1=py36_3
- intel/win-64::prompt_toolkit==1.0.15=py36_2
done
As mentioned in other answers, the idea is to have some sort of re-installation to occur for the inconsistent packages.
Thus, with a few copy-&-paste's, you could:
> conda install intel/win-64::ipython==6.3.1=py36_3
Collecting package metadata: done
Solving environment: /
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:
- intel/win-64::ipython==6.3.1=py36_3
- intel/win-64::prompt_toolkit==1.0.15=py36_2
done
## Package Plan ##
environment location: c:\conda
added / updated specs:
- ipython
The following NEW packages will be INSTALLED:
jedi intel/win-64::jedi-0.12.0-py36_2
parso intel/win-64::parso-0.2.0-py36_2
pygments intel/win-64::pygments-2.2.0-py36_5
wcwidth intel/win-64::wcwidth-0.1.7-py36_6
Proceed ([y]/n)? y
Preparing transaction: done
Verifying transaction: done
Executing transaction: done
(and you would have to repeat for all the packages)
My “Shortcut”
Alternatively, cook up an (ugly) one-liner (this should work for Windows as well as other platforms)
Note: by "ORIGINAL_COMMAND", I'm referring to any command that gives you the error message (without any other side-effects, ideally)
<ORIGINAL_COMMAND> 2>&1 | python -c "import sys,re,conda.cli; conda.cli.main('conda','install','-y',*re.findall(r'^\s*-\s*(\S+)$',sys.stdin.read(),re.MULTILINE))"
Expanding the above one-liner:
from re import findall, MULTILINE
from sys import stdin
from conda.cli import main
main(
"conda", "install", "-y",
"--force", # Maybe add a '--force'/'--force-reinstall' (I didn't add it for the one-liner above)
*findall(r"^\s*-\s*(\S+)$", stdin.read(), MULTILINE) # Here are the offenders
)
I was getting an environment is inconsistent error when I tried to update my base conda environment. I'm using miniconda. Unfortunately, none of the answers above worked for me.
What did work for me was:
conda activate base
conda install conda --force-reinstall
conda install conda --force-reinstall
conda update --all
(Yes, for some reason it was necessary to run conda install conda --force-reinstall twice!)
The command conda install -c anaconda anaconda did the trick for me. For my setup, I need to specify the channel otherwise it would not work. After running the command in the terminal, I was prompted to update a list of packages that was found to be inconsistent. Without this step, I was not able to install or update any packages with conda install <package_name> or conda update <package_name respectively.
What worked for me was to
`conda remove <offending_packagename>`,
`conda update --all`
and then finally
`conda install <offending_packagename>`.
I had this problem for ages. The conda install anaconda might work, but it takes just way too long -- more than 24 hours on my machine.
Here is a solution that worked for me in under 5 minutes:
Remove all the unneeded packages -- being careful to leave the ones that are essential for conda to operate.
Then, use conda install anaconda.
But how?? there is a lot of them!
This is what I have done:
Make a fresh envinroment with python, fairly bare-bone. then, list the packages in there:
conda create -n fresh python
conda activate fresh
conda list
Save the output, you will need it.
1b. go back to the base envinroment:
conda deactivate
use the following snippet to generate a conda command that will remove all the inconsistent packages:
(good packages are)
exclusion_text = '''
_libgcc_mutex 0.1 main
_openmp_mutex 4.5 1_gnu
anyio 2.2.0 py39h06a4308_1
argon2-cffi 20.1.0 py39h27cfd23_1
async_generator 1.10 pyhd3eb1b0_0
...
... and more! get this from a good environment.
Note the usage of triple quotes (''') to use a multiline-string in python.
bad_packages_text = '''
- https://repo.continuum.io/pkgs/main/linux-64/networkx-2.1-py36_0.tar.bz2/linux-64::networkx==2.1=py36_0
- https://repo.continuum.io/pkgs/main/linux-64/spyder-3.2.6-> py36_0.tar.bz2/linux-64::spyder==3.2.6=py36_0
py36h4c697fb_0.tar.bz2/linux-64::jdcal==1.3=py36h4c697fb_0
- defaults/noarch::jupyterlab_server==1.1.4=py_0
- defaults/linux-64::argh==0.26.2=py37_0
...
... and more! get this by copy-pasting the "The following packages are causing the inconsistency." message.
then, in python, process this:
exclusions = [line.split(' ')[0] for line in exclusion_text_lines if line !='']
bad_packages_lines = bad_packages_text.split('\n')
bad_packages = [line.split('::')[1].split('==')[0] for line in bad_packages_lines if line!='']
exclusions.append('conda') # make sure!
exclusions.append('tqdm')
finally, construct the life-saving command:
command_line = 'conda remove '
for bad_package in bad_packages:
if bad_package not in exclusions:
command_line = f'{command_line} {bad_package}'
command_line
Since in solving the environment, all the packages on the remove list can be ignored, conda no longer needs to consider their versions, and the process is fast.
Possibly someone can refactor this method to make it easier -- or better yet, upgrade conda to enable quick reset base command.
This worked for me -- it took me longer to write this post than to execute these actions.
Good luck!
To those of us who have miniconda and can't/don't want to install anaconda: the accepted answer works when adapted.
conda install conda
conda update --all
Would have commented, but my rep is too low.
conda install anaconda
conda clean --all
conda update --all
fix the problem for me
To solve this message I had to run conda update --all in my base environment three times after each other.
Every time the number of inconsistent packages decreased until conda said:
# All requested packages already installed.
I'm on macOS Big Sur 11.6 using conda version 4.10.3.
In my case, none of the above worked. But this did the trick in less than a minute:
1- I downloaded again the lastest installer (miniconda in my case)
2- Run the installer with the -u option:
bash Miniconda3-py39_xxxx-Linux-x86_64.sh -u
3- Answer yes to all questions and let the installer finish
4- Then I could run conda update conda -all
Hope this helps...
You probably installed anaconda with python 2.7 but later you used python 3.x. Thus, you are getting an error message. In my case, I solved the problem by activating anaconda with python 2.7:
conda create --name py2 python=2.7
Try to have a look to the environment management
https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
By using something along the lines
conda create --name astra python=3.5
conda activate astra
conda install -c astra-toolbox astra-toolbox
You can see that you can even specify target python version. Now play with the new packages installed. When unsatisfied, you can always do
conda deactivate
conda env remove -n astra
If you install everything to the base env and something gets broken, then probably better is not to install conda at all and go with default python managing it through pip.
In my environment.
1.
conda install anaconda
conda update --all
Then it works correctly.