Pip install package not from pypi, whose name is the same as one in pypi ( + with dependencies from pypi ) - pip

I have a python package, hosted on azure (vsts), not on pypi, whose dependencies are python packages that live in pypi.
My package has the same name as a package that lives on pypi, I discovered.
Is there a way of installing my package with pip, specifying that my package must be searched on vsts before, while the dependencies can be grabbed from pypi?
If I use the --index-url option:
pip install <my-package> --index-url https://<my-package>:<PAT>#<url>/<proj>/_packaging/<my-package>/pypi/simple/
pip is able to locate my package, tries to install it, but it fails to install any dependency (because it searches for all of them in the same url, which is wrong because I am not hosting, say, my own version of numpy or other packages on vsts).
(This is the problem: pip install producing "Could not find a version that satisfies the requirement" )
If instead I use the --extra-index-url option:
pip install <my-package> --extra-index-url https://<my-package>:<PAT>#<url>/<proj>/_packaging/<my-package>/pypi/simple/
all the dependencies are found, but the problem is that this does not install my package, but the package with the same name that lives in pypi!
Even if --extra-index-url is there, it seems that pypi is given priority, and therefore my package that would be found at the url I specified is shadowed and doesn't get correctly found and installed.
Is there a way to, say, tell pip that it should give priority to my --extra-index-url? Or to give pip an --index-url which should only be valid for one package but not for its dependencies?

You need index URL pointing to VSTS and extra URL to PyPI:
pip install --index-url=https://<my-package>:<PAT>#<url>/<proj>/_packaging/<my-package>/pypi/simple/ --extra-index-url=https://pypi.org/simple/ <my-package>

Related

Tensorflow js and MacM1

I have a running installation of tensorflow-macos and the metal plugin for mac os M1.
Now I want to convert my trained model in a tensorflow js one.
However, when I run
pip install tensorflowjs I get an error due to the fact that tensorflow js actually depends on tensorflow not on tensorflow-macos.
ERROR: Cannot install tensorflowjs==0.1.0, tensorflowjs==0.1.1, tensorflowjs==0.1.2, tensorflowjs==0.2.0, tensorflowjs==0.2.1, tensorflowjs==0.3.0, tensorflowjs==0.3.1, tensorflowjs==0.4.0, tensorflowjs==0.4.1, tensorflowjs==0.4.2, tensorflowjs==0.5.0, tensorflowjs==0.5.2, tensorflowjs==0.5.4, tensorflowjs==0.5.6, tensorflowjs==0.5.7, tensorflowjs==0.6.0, tensorflowjs==0.6.1, tensorflowjs==0.6.2, tensorflowjs==0.6.4, tensorflowjs==0.6.5, tensorflowjs==0.6.7, tensorflowjs==0.8.0, tensorflowjs==0.8.5, tensorflowjs==0.8.6, tensorflowjs==1.0.1, tensorflowjs==1.1.2, tensorflowjs==1.2.1, tensorflowjs==1.2.10, tensorflowjs==1.2.10.1, tensorflowjs==1.2.2, tensorflowjs==1.2.2.1, tensorflowjs==1.2.3, tensorflowjs==1.2.6, tensorflowjs==1.2.9, tensorflowjs==1.3.1, tensorflowjs==1.3.1.1, tensorflowjs==1.3.2, tensorflowjs==1.4.0, tensorflowjs==1.5.2, tensorflowjs==1.6.0, tensorflowjs==1.7.2, tensorflowjs==1.7.3, tensorflowjs==1.7.4, tensorflowjs==1.7.4.post1, tensorflowjs==2.0.0, tensorflowjs==2.0.1, tensorflowjs==2.0.1.post1, tensorflowjs==2.1.0, tensorflowjs==2.3.0, tensorflowjs==2.4.0, tensorflowjs==2.5.0, tensorflowjs==2.6.0, tensorflowjs==2.7.0, tensorflowjs==2.8.0, tensorflowjs==2.8.1, tensorflowjs==2.8.2, tensorflowjs==2.8.3, tensorflowjs==2.8.4, tensorflowjs==2.8.5, tensorflowjs==3.0.0, tensorflowjs==3.1.0, tensorflowjs==3.11.0, tensorflowjs==3.12.0, tensorflowjs==3.13.0, tensorflowjs==3.14.0, tensorflowjs==3.15.0, tensorflowjs==3.17.0, tensorflowjs==3.18.0, tensorflowjs==3.19.0, tensorflowjs==3.2.0, tensorflowjs==3.20.0, tensorflowjs==3.3.0, tensorflowjs==3.4.0, tensorflowjs==3.5.0, tensorflowjs==3.6.0, tensorflowjs==3.7.0, tensorflowjs==3.8.0 and tensorflowjs==3.9.0 because these package versions have conflicting dependencies.
The conflict is caused by:
tensorflowjs 3.20.0 depends on tensorflow<3 and >=2.1.0
...
tensorflowjs 0.1.1 depends on tensorflow>=1.6.0
tensorflowjs 0.1.0 depends on tensorflow>=1.6.0
To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict
Any idea on how to solve it?
just encountered the same problem.
Here is what worked for me.
Install all dependencies of the tensorflowjs package except tensorflow (since you have tensorflow-macos already installed) and then install tensorflowjs without it dependencies.
Install tensorflowjs dependencies :
pip install tensorflow_hub jax scipy jaxlib etils
Install tensorflowjs without its dependencies :
pip install --no-deps tensorflowjs

pip install is looking for required dependency within the package

I'm build up my package with setup.py, in which it has a install_requires=get_requirements('requirements.txt'), and in the requirements.txt, the first required dependency is bottle==0.12.18.
Then I ran
python setup.py sdist
twine upload --repository testpypi dist/*
After uploading succeeded, I installed with pip install -i https://test.pypi.org/simple/ my_package==version_num which gives me the error
Collecting bottle==0.12.18 (from mypakage==version_num)
Could not find a version that satisfies the requirement bottle==0.12.18 (from mypakage==version_num) (from versions: )
No matching distribution found for bottle==0.12.18 (from mypakage==version_num)
Looks like it's looking for the dependency within my package, which will definitely fail. This error suddenly started happening and I've never seen this before. Do you have any idea why it's happening and how can I make it look for dependency in a way like pip install -r requirements?

Pip extras dependency substitution

I'm creating a module that has only 1 pypi dependency. This dependency has 2 packages on pypi. One that makes use of a system library and the other packages a binary distribution of that library. They look like:
theirmodule
theirmodule-binary
My module depends on theirmodule but I want users of my module to be able to decide if they want the lib version of the dependency or the binary version. I see in the docs about Extras. I could do:
setup(
name="MyModule",
...
extras_require={
"BIN": ["theirmodule-binary>=1.2"]
}
)
But then if the user does pip install mymodule[BIN] pip will install both theirmodule and theirmodule-binary. That would be a conflict since both have the same underlying import string eg:
import theirmodule
is used for both. How can this be handled without providing 2 separate pypi packages?
Maybe something like the following:
setup.py
import setuptools
setuptools.setup(
name='My-Project',
# ...
extras_require={
'Extra_Dependency_As_Binary': ['Dependency-Project-Binary>=1.2'],
'Extra_Dependency_As_Library': ['Dependency-Project-Library<=3.4'],
},
)
And then instruct the users of My-Project (maybe in the README file) to install by specifying either one of the extra explicitly. For example with pip it could be one or the other of:
path/to/pythonX.Y -m pip install 'My-Project[Extra_Dependency_As_Binary]'
path/to/pythonX.Y -m pip install 'My-Project[Extra_Dependency_As_Library]'

install "sub package" Jupyter Notebook

I am struggling to install packages, and "sub-packages" in Jupyter Notebook; I suspect I am missing some of the basic concepts around installing packages.
I understand that to install a package within the notebook I use
! pip install --user <package>
What I don't understand is how to install a "sub-package" (feel free to advise what the correct terminology is) such as below.
from nltk.tagger import *
Here is the original script that this comes from:
If i try :
!pip install nltk.tagger
I get the following error information / error
Collecting nltk.tagger
Could not find a version that satisfies the requirement nltk.tagger (from versions: ) No matching distribution found for nltk.tagger
So my first question is. How do I install this nltk.tagger subpackage? Also if tagger is a sub-package of NLTK, how come it isn't installed when I do a pip install NLTK?
Although the error mentions a version, searching online I can't find a reference even to the subpackage "tagger". Any advice or links explaining this would be appreciated.
if you pip inslall nltk, the subpackage nltk.tagger, and other dependencies will be installed too.
This is generally true for all packages.

python package can be installed by pip but not conda

I need the sacred package for a new code base I downloaded. It requires sacred.
https://pypi.python.org/pypi/sacred
conda install sacred fails with
PackageNotFoundError: Package missing in current osx-64 channels:
- sacred
The instruction on the package site only explains how to install with pip. What do you do in this case?
That package is not available as a conda package at all. You can search for packages on anaconda.org: https://anaconda.org/search?q=sacred You can see the type of package in the 4th column. Other Python packages may be available as conda packages, for instance, NumPy: https://anaconda.org/search?q=numpy
As you can see, the conda package numpy is available from a number of different channels (the channel is the name before the slash). If you wanted to install a package from a different channel, you can add the option to the install/create command with the -c/--channel option, or you can add the channel to your configuration conda config --add channels channel-name.
If no conda package exists for a Python package, you can either install via pip (if available) or build your own conda package. This isn't usually too difficult to do for pure Python packages, especially if one can use skeleton to build a recipe from a package on PyPI.
It happens some issue to me before. If your system default Python environment is Conda, then you could download those files from https://pypi.python.org/pypi/sacred#downloads
and manually install by
pip install C:/Destop/some-file.whl

Resources