I have a complex conda package, build locally, with a complex meta.yaml and conda_build_config.yaml.
I would like to publish these packages (and alternative) in conda-forge.
But, the documentation of conda-forge propose some solution to publish a conda package from a pip package, with grayskull, but not how to do it from a conda package.
How I can do that?
Related
Am I doing something wrong in my commands? I can't remove Keras.
$ conda remove --name myEnv keras
Collecting package metadata (repodata.json): done
Solving environment: failed
PackagesNotFoundError: The following packages are missing from the target environment:
- keras
$ conda list --name myEnv keras
# packages in environment at /Users/me/anaconda3/envs/myEnv:
#
# Name Version Build Channel
keras 2.3.1 pypi_0 pypi
The channel pypi for keras means you have mix used pip and conda. To uninstall the keras installed from pypi, use pip.
pip uninstall keras
It's not a good idea to mix use both package managers pip and conda in one environment. If you really need to. Read the official guide Using Pip in a Conda Environment.
conda remove --force (the rest of the command)
Pip Interoperability
As #Simba correctly identified, the package is from PyPI (i.e., it was installed via pip). By default, Conda can recognize the presence of such packages but will not interact with them. However, there is a "preview" configuration option, pip_interop_enabled, that enables such interaction (see Conda v4.6 Release Notes). You could either
Set this option globally:
conda config --set pip_interop_enabled true
conda remove --name myEnv keras
If you have PyPI packages generally, then this should help improve your env stability, but at the cost of slightly longer solve times, since now Conda will account for packages coming from PyPI.
Temporarily turn it on just for this operation
CONDA_PIP_INTEROP_ENABLED=1 conda remove --name myEnv keras
Which one should I use to install keras if I have anaconda?
conda install -c conda-forge keras
&
pip install --upgrade keras
Also, what is conda-forge? Why need to do it this way?
The advantages of using conda rather than pip to install packages in your Anaconda environment(s) are that:
conda should determine what dependencies your requested package has, and install those too in one operation, and
you can then keep the installed packages up to date using the conda update command:
pip, PyPI, and setuptools?
None of this is going to help with updating packages that have been
installed from PyPI via pip, or any packages installed using python
setup.py install. conda list will give you some hints about the
pip-based Python packages you have in an environment, but it won’t do
anything special to update them.
The conda-forge channel is where you can find packages that have been built for conda but are not part of the official Anaconda distribution (yet).
See answers to this question for more detail on the two options (although bear in mind some of the answers may be out of date).
Sometimes I need to install a pip-only package into a conda environment. If I install the package using pip install, then all the dependencies for that package are installed using pip, even if they are available to conda.
I would like to install as many packages as possible through conda, so currently I use a hack to get the list of package dependencies through pip, search for all of them on conda, conda install the ones that are found, and then go through with the pip install.
Am I right to prefer installing dependencies through conda rather than pip? And if so, can anyone think of a more elegant way to solve this problem?
pip and conda are two separate package managers. Only in very rare cases package managers actually work together. In practical applications conda and pip usually do not.
In reality, mixing conda and pip packages is usually unavoidable. This often leads to a messy package management, as you describe.
In my opinion, the best and currently only proper way to solve this problem is to create a conda package for all (pypi-)packages and dependencies you want to use in your conda environments.
conda-forge is a community effort that offers an easy way to contribute your own package to the conda infrastructure. You may want to check out if your package is already available, and if not if contributing is an option for you.
Am I right to prefer installing dependencies through conda rather than pip?
Yes.
Anaconda has a blog post that discusses best practices when you have no choice but to combine conda and pip. Here's a list of guidelines from that blog post:
Best Practices Checklist
Use pip only after conda
install as many requirements as possible with conda, then use pip
pip should be run with --upgrade-strategy only-if-needed (the default)
Do not use pip with the --user argument, avoid all “users” installs
Use conda environments for isolation
create a conda environment to isolate any changes pip makes
environments take up little space thanks to hard links
care should be taken to avoid running pip in the “root” [base] environment
Recreate the environment if changes are needed
once pip has been used conda will be unaware of the changes
to install additional conda packages it is best to recreate the environment
Store conda and pip requirements in text files
package requirements can be passed to conda via the --file argument
pip accepts a list of Python packages with -r or --requirements
conda env will export or create environments based on a file with conda and pip requirements
I have a fresh Anaconda environment in which I would like to install the latest release of mysql-connector-python (v2.1.3). I'm on a CentOS6 system. The problem is, the newest Conda-hosted package is 2.0.3 and because the connector is not currently hosted on PyPI, it cannot be installed via Pip. (Pip's --allow-external option has been deprecated.)
I know that I can easily install the package via yum install, but I believe that will install it outside of the Conda environment.
Is there any way, using yum or otherwise, to isolate this package solely to this particular Conda environment?
Turns out this is surprisingly simple. Just download the source for the Python package that is not available on PyPI and pip install at the source directory. Just remember to have the Conda environment you want to install to active and Pip will correctly isolate the install to env scope.
I was unaware of Pip's install from local options, so big thanks to the folks on the Anaconda mailing list for their help with this.
I saw other questions regarding difference between pip and conda, but it is not clear to me yet, please consider that before marking as duplicate.
If I run pip install seaborn and conda install seaborn Will I get the same result ?
I can run pip install seaborn twice without any problem, but if I run pip install and then conda install do I get the same package duplicated in two different places ?
Conda and pip cannot be used interchangeably but what are examples of that ?
Both pip and conda install the package (pretty much) with the same end result. There may be minor differences, e.g. zipped egg or not, it depends a bit on how the conda package was created. The conda package is always a compiled binary distribution though, not a source distribution.
I don't think conda will install it in different places, it may well overwrite your pip package. But it's kind of risky because conda keeps nicely track of what's installed and figures out all dependencies betweeen all conda packages in the environment. You really want to limit yourself to conda packages and only install pip packages if you really have to. It's quite easy to create conda packages though from pip packages.
Not sure about "interchangeably", you can use them alongside each other. But pip and conda are not so aware of each other so you might run into trouble with say updating packages to new versions.
In summary: if you're using conda packages, best to stick with that. You get the best out of the conda ecosystem with it's package version and environment management.