(SSLError [SSL: CERTIFICATE_VERIFY_FAILED] [duplicate] - windows

Pip always fails ssl even when I do pip install dedupe or pip install --trusted-host pypi.python.org dedupe
The output is always the same no matter what:
Collecting dedupe
Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed (_ssl.c:777)'),)': /simple/dedupe/
Retrying...
skipping
Could not find a version that satisfies the requirement dedupe (from versions: ) No matching distribution found for dedupe
So I uninstalled anaconda and reinstalled it. Same thing.
Do you think the problem is that my _ssl.c file (which I have no idea where it is) must be corrupt or something? Why would pip need to reference that if I'm telling it to bypass ssl verification anyway?

It may be related to the 2018 change of PyPI domains.
Please ensure your firewall/proxy allows access to/from:
pypi.org
files.pythonhosted.org
So you could give a try to something like:
$ python -m pip install --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org [--proxy ...] [--user] <packagename>
Please see $ pip help install for the --user option description (omit if in a virtualenv).
The --trusted-host option doesn't actually bypass SSL/TLS, but allows to mark host as trusted when (and only when) it does not have valid (or any) HTTPS. It shouldn't really matter with PiPY because pypi.org (formerly pypi.python.org) does use HTTPS and there is CDN in front of it which always enforces TLSv1.2 handshake requirement regardless of the connecting pip client options.. But if you had your own local mirrors of pypi.org with HTTP-only access, then --trusted-host could be handy. Oh, and if you are behind a proxy, please also make sure to also specify: --proxy [user:passwd#]proxyserver:port
Some corporate proxies may even go as far as to replace the certificates of HTTPS connections on the fly. And if your system clock is out of sync, it could break SSL verification process as well.
If firewall / proxy / clock isn't a problem, then check SSL certificates being used in pip's SSL handshake. In fact, you could just get a current cacert.pem (Mozilla's CA bundle from curl) and try it using the pip option --cert:
$ pip --cert ~/cacert.pem install --user <packagename>
where --cert argument is system path to your alternate CA bundle in PEM format. (regarding the --user option, please see below).
Or, it's possible to create a custom config ~/.pip/pip.conf and point the option at a valid system cert (or your cacert.pem) as a workaround, for example:
[global]
cert = /etc/pki/tls/external-roots/ca_bundle.pem
(or another pem file)
It's even possible to manually replace the original cacert.pem found in pip with your trusty CA bundle (if your pip is very old for example). Older pip versions knew to fallback between pip/_vendor/requests/cacert.pem and system stores like /etc/ssl/certs/ca-certificates.crt or /etc/pki/tls/certs/ca-bundle.crt in case of cert issues, but in recent pip it's no longer the case, as it seems to rely solely on pip/_vendor/certifi/cacert.pem
Basically, pip package uses requests which uses urllib3 which, among other things, verifies SSL certificates; and all of them are shipped (vendored) within pip, along with the certifi package (also included, since pip 9.0.2) that provides current CA bundle (cacert.pem file) required for TLS verification. Requests itself uses urllib3 and certifi internally, and before 9.0.2, pip used cacert.pem from requests or the system. What it all means is that actually updating pip may help fix the CERTIFICATE_VERIFY_FAILED error, particularly if the OS and pip were deployed long ago:
The OP used anaconda, so they could try:
$ conda update pip - because issues can arise if conda and pip are both used together in the same environment. If there's no pip version update available, they could try:
$ conda config --add channels conda-forge; conda update pip
Alternatively, it's possible to use conda alone to directly install / manage python packages: it is a tool completely separate from pip, but provides similar features in terms of package and venv management. Its packages come not from PyPI, but from anaconda's own repositories.
The problem is, if you mix both and run conda after pip, the former can overwrite and break packages (and their dependencies) installed via pip, and render it all unusable. So it's recommended to only use one or the other, or, if you have to, use only pip after conda (and no conda after pip), and only in isolated conda environments.
On normal Linux Python installations without conda:
If you are using a version of pip supplied by your OS distribution, then use vendor-supplied upgrades for a system-wide pip update:
$ sudo apt-get install python-pip or: $ sudo yum install python27-pip
Some updates may not be readily available because distros usually lag behind PyPI. In this case, it's possible to upgrade pip at your user level (right in your $HOME dir), or inside a virtualenv, like:
$ python -m pip install --user --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org --upgrade pip
(omit --user if in a virtualenv)
The --user switch will upgrade pip only for the current user (in your home ~/.local/lib/) rather than for the whole OS, which is a good practice to avoid interfering with the system python packages. It's enabled by default in a pip distributed in recent Ubuntu/Fedora versions. Be aware of how to solve ImportError if you don't use this option and happen to overwrite the OS-level system pip.
Alternatively (also at a user level) you could try:
$ curl -LO https://bootstrap.pypa.io/get-pip.py && python get-pip.py --user
The PyPA script contains a wrapper that extracts the .pem SSL bundle from pip._vendor.certifi.
Otherwise, if still no-go, try running pip with -vvv option to add verbosity to the output and check if there is now another SSLError caused by tlsv1 alert protocol version.

This worked for me, try this:
pip install --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user {name of whatever I'm installing}

My way is a simplification of #Alex C's answer:
python -m pip install --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host pypi.org --upgrade pip

I experienced the same issue because I have Zscaler (a cloud security software) installed and was causing:
URL host for python packages being blocked
invalid SSL certificate warnings popping up
SSL inspection certificate not trusted
As mentioned by others, the below will fix individual package installations. pypi.python.org is not required since it has been replaced by pypi.org.
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package to install>
I permanently fixed the issue by creating pip.ini file (pip.conf in Unix) and adding the below:
[global]
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
See pip configuration files for how to locate your pip.ini, or where to put it if you need to create one.

The error above or one like it was caused by the virtual machine (VM) not be time synchronized, my guest Ubuntu VM was several days in the past.
I ran this commend to get the VM to pick up the correct network time:
sudo timedatectl set-ntp on
This makes the Ubuntu guest OS get the network time. (You may have to provide a network time source... I used this article: Digital Ocean - How to set time on Ubuntu)
Check the time is correct:
timedatectl
Re-run the failing pip command.

Related

How to install a package with Poetry that requires CLI args?

Here's an example that requires it:
poetry add pycurl
... gives:
ImportError: pycurl: libcurl link-time ssl backend (openssl) is different from compile-time ssl backend (none/other)
The fix is given in that post:
pip install --compile --install-option="--with-openssl" pycurl
How to package-manage my project now?
Must I use poetry for everything else, and manually pip install pycurl?
Or can I somehow fold it into my pyproject.toml?
Although this is old, I would like to share an alternative to re-install the package with the CLI options.
I do not know if running this with Poetry could cause any damage, but I would believe not.
First you'd have to activate your poetry environment with poetry shell.
After that, get the current version of pycurl that is installed using pip list, and copy that value.
Now, you can run pip install --compile --install-option="--with-openssl" --upgrade --force-reinstall pycurl==<version> to install it with the correct options.

pip and trusted hosts

I am trying to install PyTorch 1.7.0 on Windows 10 using Python 3.8.6. When simply inputing the command
pip install torch==1.7+cu101
I get the error :
'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1124)'))': /whl/torch_stable.html
When I do :
pip install --trusted-host download.pytorch.org torch==1.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
Installation works flawlessly.
However, I would like to either :
Ideally, handle that with the simple pip command and get the SSL verification to work
handle that through the pip config file located in my virtual env i.e. /path/to/my/env/pip.conf
Unfortunately, having the pip.conf file like this :
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
download.pytorch.org
does not work and seems to be ignored by pip

How would you install python modules/packages so that my script can be run with pypy3 instead of Python3? ImportError: No module named

I have Python3.8 built from source on my Debian 10 Xfce desktop (binaries are not available in Debian repositories). That said, whenever I can, I run my python scripts with pypy3, which I do for the sake of performance.
Now, when I run the following code with pypy3 :
#!/usr/bin/env python3.8
import requests
from bs4 import BeautifulSoup
url = input("What is the address of the web page in question?")
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
print(soup.title.string)
I get from pypy3:
ImportError: No module named 'requests'
The same script is run by Python3.8 without any problems
I assume that I would have to install the module in a similar way that I did it for Python, that is: sudo pip3.8 install requests.
Based on my research of a similar problem described on Stackoveflow I tried:
pypy3 -m pip3.8 install requests
and got the following from my pypy3:
Error while finding module specification for 'pip3.8' (ImportError: No >module named 'pip3')
Then I also tried to run:
pypy3 -m pip install requests
And got the following:
No module named pip
My pip3.8 works fine for Python3.8, not for my pypy3, though.
How should I look for modules in pypy3. And how should I install them?
Is the problem with installing and importing modules one of the reasons reason for the low usage of pypy3?
Run this once to install pip itself: pypy3 -m ensurepip
The next version of PyPy will improve the error message to describe this command explicitly when you do pypy3 -m pip and pip is not installed yet.
pypy3
Enable snaps on Debian and install pypy3
Snaps are applications packaged with all their dependencies to run on all popular Linux distributions from a single build. They update automatically and roll back gracefully.
Snaps are discoverable and installable from the Snap Store, an app store with an audience of millions.
Enable snapd:
sudo apt update
sudo apt install snapd
Install pypy3:
sudo snap install pypy3 --classic
Normally, the pip and package are installed as follows
First of all, you need to install the pip
Install pip for Python 3
Follow the steps below to install Pip for Python 3 on Debian:
First, update the package list with:
sudo apt update
Next, install pip for Python 3 and all of its dependencies by typing:
sudo apt install python3-pip
Verify the installation by printing the pip version:
pip3 --version
The version number may be different, but it will look something like the one below:
pip 9.0.1 from /usr/lib/python3/dist-packages (python 3.5)
Pip Usage
With pip, we can install packages from PyPI, version control, local projects, and from distribution files but in most cases, you will install packages from PyPI.
we want to install a package named croniter, we can do that by issuing the following command:
pip install requests
To uninstall a package run:
pip uninstall requests

tensorflow-gpu via pip time out

I was getting time outs from pypi.python.org when running pip install --upgrade tensorflow_gpu, so I added the --verbose and --timeout 10000 params to it. It starts out fast then begins to crawl:
1% |▌ | 747kB 244bytes/s eta 2 days, 9:31:36
Is there a better way to install tensorflow-gpu when inside a virtualenv in Windows? Following the instructions from this model: https://github.com/tensorflow/models/tree/master/attention_ocr
The easiest way to install tensorflow within an environment is as follows.
Activate/Enter your python environment (e.g. for Anaconda,
activate envName).
Ensure that you are actually in your virtual/conda environment!
Use pip to install tensorflow. For CPU use pip install tensorflow and for GPU use pip install tensorflow-gpu. Don't have both installed in the same directory.
Pip should take care of the rest. Tensorflow will be downloaded along with it's dependencies from Pypi.
If you're having problems installing from pip you can try updating pip or checking your internet connection. There is also a chance that Pypi are having some minor issues on their end.
Don't forget to activate your environment before trying to import Tensorflow!
Good luck!

Cannot "sudo pip uninstall" operation not permitted (/tmp) in OS X El Capitan

Preface: My OS X Python installation is a mess. I started using the system Python way before I found out about Homebrew. And so I've been using sudo pip install since forever. I'm now trying to clean everything up and then install/link pip packages against Homebrew's Python.
1) In many SO answers, people suggest doing: pip freeze | xargs sudo pip uninstall -y That doesn't work for me. I get a very long traceback. These are the most representative chunks of it:
~ $ pip freeze | xargs sudo pip uninstall -y
You are using pip version 7.1.2, however version 8.0.2 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
The directory '/Users/smaniato/Library/Caches/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Uninstalling altgraph-0.10.2:
Exception:
Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/pip/basecommand.py", line 211, in main
[...]
Error: [('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph/__init__.py', '/tmp/pip-p8yIlU-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph/__init__.py', "[Errno 1] Operation not permitted: '/tmp/pip-p8yIlU-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph/__init__.py'"), ('/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph/__init__.pyc', '/tmp/pip-p8yIlU-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph/__init__.pyc',
[...]
"[Errno 1] Operation not permitted: '/tmp/pip-p8yIlU-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/altgraph'")]
where altgraph is just the first pip package in pip list (keep that in mind).
2) I then tried pip freeze | xargs sudo -H pip uninstall -y, as the warning suggested, but that simply remove the warning. The error messages persisted.
3) I also tried chown-ing any directories that raised complaints (e.g., ~/Library/Caches/pip and some of its parents and children). For some reason, I cannot chown the most problematic directory, /tmp:
~ $ sudo chown -R ${USER} /tmp
chown: /tmp: Operation not permitted
4) Finally, I tried manually uninstalling a package from further down the list and voila, I could easily do sudo pip uninstall virtualenv for example. Attempting to manually uninstall altgraph results in the same error above.
Any ideas how to proceed? Remember, I don't care about any one package; I just want to nuke pip and start from scratch using Homebrew's Python. Thanks!
Updates:
Going down the list manually, the same thing happens with bdist-mpkg-0.5.0
A few more: matplotlib, zope.interface, xattr, six, scipy, pytz.
I'm pretty sure that brew reinstall python somehow fixed a bunch of the issues I was having. I then had to also nuke (i.e., rm -rf) a few things in /usr/local/lib/python2.7/site-packages
Issue and answer related to my specific use case (ROS installation on OS X): https://github.com/mikepurvis/ros-install-osx/issues/11
Don't use Homebrew nor MacPorts, nor the preinstalled Python of macOS. These are nested solutions that will eventually fail, one way or another (PEP 20: flat is better than nested). At the very least, you will have to wait for PyPI updates to propagate to MacPorts or Homebrew, or fall back to using pip.
Use MacPorts to install only non-Python items that need to compile and be customized (e.g., ATLAS).
The simplest thing to do is to install a standalone Python from python.org (either from a binary distribution, or build from source). I recommend building CPython from source (example).
No sudo needed, install under your user. Then:
pip install -U pip setuptools virtualenvwrapper
Source your wrapper from your ~/.bashrc per the docs, then mkvirtualenv foo. All other work will be performed in virtual environments only.

Resources