I'm trying to connect to an EC2 instance using fabric (in python). I've set my env variables as so:
env.hosts = ['xxx-xxx.amazonaws.com']
env.user = "ubuntu"
env.key_filename = ['/path/to/my/ec2.pem']
the command
run('pwd')
gives the following error:
File "build/bdist.linux-x86_64/egg/paramiko/client.py", line 242, in connect
File "build/bdist.linux-x86_64/egg/paramiko/transport.py", line 346, in start_client
ValueError: CTR mode needs counter parameter, not IV
I'm using paramiko 1.14.0 (current) btw, and editing my ssh config to associate the pem to the host is not an option (although, I have tested the connection with ssh -i /path/to/pem and that was fine). Has anyone else had this problem and solved it?
I had the same error running a Python/Paramiko script on a new Ubunutu host. I wasn't able to determine the cause of the fault as I am new to Python but I resolved it by removing paramiko and its dependancies from /usr/local/lib/python2.7/dist-packages. I removed paramiko, pycrypto and ecdsa.
My system already has the following packages:
sudo apt-get install python-pip
sudo apt-get install python-dev
I re-installed paramiko with:
sudo pip install paramiko
I was able to run my script successfully without the the ValueError:
Versions of modules I am running:
ecdsa 0.11
paramiko 1.14.0
pycrypto 2.6.1
Related
I am trying to install PyTorch 1.7.0 on Windows 10 using Python 3.8.6. When simply inputing the command
pip install torch==1.7+cu101
I get the error :
'SSLError(SSLCertVerificationError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: self signed certificate in certificate chain (_ssl.c:1124)'))': /whl/torch_stable.html
When I do :
pip install --trusted-host download.pytorch.org torch==1.7.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html
Installation works flawlessly.
However, I would like to either :
Ideally, handle that with the simple pip command and get the SSL verification to work
handle that through the pip config file located in my virtual env i.e. /path/to/my/env/pip.conf
Unfortunately, having the pip.conf file like this :
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
download.pytorch.org
does not work and seems to be ignored by pip
Pip always fails ssl even when I do pip install dedupe or pip install --trusted-host pypi.python.org dedupe
The output is always the same no matter what:
Collecting dedupe
Retrying (Retry(total=4, connect=None, read=None,
redirect=None, status=None)) after connection broken by
'SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate
verify failed (_ssl.c:777)'),)': /simple/dedupe/
Retrying...
skipping
Could not find a version that satisfies the requirement dedupe (from versions: ) No matching distribution found for dedupe
So I uninstalled anaconda and reinstalled it. Same thing.
Do you think the problem is that my _ssl.c file (which I have no idea where it is) must be corrupt or something? Why would pip need to reference that if I'm telling it to bypass ssl verification anyway?
It may be related to the 2018 change of PyPI domains.
Please ensure your firewall/proxy allows access to/from:
pypi.org
files.pythonhosted.org
So you could give a try to something like:
$ python -m pip install --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org [--proxy ...] [--user] <packagename>
Please see $ pip help install for the --user option description (omit if in a virtualenv).
The --trusted-host option doesn't actually bypass SSL/TLS, but allows to mark host as trusted when (and only when) it does not have valid (or any) HTTPS. It shouldn't really matter with PiPY because pypi.org (formerly pypi.python.org) does use HTTPS and there is CDN in front of it which always enforces TLSv1.2 handshake requirement regardless of the connecting pip client options.. But if you had your own local mirrors of pypi.org with HTTP-only access, then --trusted-host could be handy. Oh, and if you are behind a proxy, please also make sure to also specify: --proxy [user:passwd#]proxyserver:port
Some corporate proxies may even go as far as to replace the certificates of HTTPS connections on the fly. And if your system clock is out of sync, it could break SSL verification process as well.
If firewall / proxy / clock isn't a problem, then check SSL certificates being used in pip's SSL handshake. In fact, you could just get a current cacert.pem (Mozilla's CA bundle from curl) and try it using the pip option --cert:
$ pip --cert ~/cacert.pem install --user <packagename>
where --cert argument is system path to your alternate CA bundle in PEM format. (regarding the --user option, please see below).
Or, it's possible to create a custom config ~/.pip/pip.conf and point the option at a valid system cert (or your cacert.pem) as a workaround, for example:
[global]
cert = /etc/pki/tls/external-roots/ca_bundle.pem
(or another pem file)
It's even possible to manually replace the original cacert.pem found in pip with your trusty CA bundle (if your pip is very old for example). Older pip versions knew to fallback between pip/_vendor/requests/cacert.pem and system stores like /etc/ssl/certs/ca-certificates.crt or /etc/pki/tls/certs/ca-bundle.crt in case of cert issues, but in recent pip it's no longer the case, as it seems to rely solely on pip/_vendor/certifi/cacert.pem
Basically, pip package uses requests which uses urllib3 which, among other things, verifies SSL certificates; and all of them are shipped (vendored) within pip, along with the certifi package (also included, since pip 9.0.2) that provides current CA bundle (cacert.pem file) required for TLS verification. Requests itself uses urllib3 and certifi internally, and before 9.0.2, pip used cacert.pem from requests or the system. What it all means is that actually updating pip may help fix the CERTIFICATE_VERIFY_FAILED error, particularly if the OS and pip were deployed long ago:
The OP used anaconda, so they could try:
$ conda update pip - because issues can arise if conda and pip are both used together in the same environment. If there's no pip version update available, they could try:
$ conda config --add channels conda-forge; conda update pip
Alternatively, it's possible to use conda alone to directly install / manage python packages: it is a tool completely separate from pip, but provides similar features in terms of package and venv management. Its packages come not from PyPI, but from anaconda's own repositories.
The problem is, if you mix both and run conda after pip, the former can overwrite and break packages (and their dependencies) installed via pip, and render it all unusable. So it's recommended to only use one or the other, or, if you have to, use only pip after conda (and no conda after pip), and only in isolated conda environments.
On normal Linux Python installations without conda:
If you are using a version of pip supplied by your OS distribution, then use vendor-supplied upgrades for a system-wide pip update:
$ sudo apt-get install python-pip or: $ sudo yum install python27-pip
Some updates may not be readily available because distros usually lag behind PyPI. In this case, it's possible to upgrade pip at your user level (right in your $HOME dir), or inside a virtualenv, like:
$ python -m pip install --user --trusted-host files.pythonhosted.org --trusted-host pypi.org --trusted-host pypi.python.org --upgrade pip
(omit --user if in a virtualenv)
The --user switch will upgrade pip only for the current user (in your home ~/.local/lib/) rather than for the whole OS, which is a good practice to avoid interfering with the system python packages. It's enabled by default in a pip distributed in recent Ubuntu/Fedora versions. Be aware of how to solve ImportError if you don't use this option and happen to overwrite the OS-level system pip.
Alternatively (also at a user level) you could try:
$ curl -LO https://bootstrap.pypa.io/get-pip.py && python get-pip.py --user
The PyPA script contains a wrapper that extracts the .pem SSL bundle from pip._vendor.certifi.
Otherwise, if still no-go, try running pip with -vvv option to add verbosity to the output and check if there is now another SSLError caused by tlsv1 alert protocol version.
This worked for me, try this:
pip install --trusted-host=pypi.org --trusted-host=files.pythonhosted.org --user {name of whatever I'm installing}
My way is a simplification of #Alex C's answer:
python -m pip install --trusted-host pypi.python.org --trusted-host files.pythonhosted.org --trusted-host pypi.org --upgrade pip
I experienced the same issue because I have Zscaler (a cloud security software) installed and was causing:
URL host for python packages being blocked
invalid SSL certificate warnings popping up
SSL inspection certificate not trusted
As mentioned by others, the below will fix individual package installations. pypi.python.org is not required since it has been replaced by pypi.org.
pip install --trusted-host pypi.org --trusted-host files.pythonhosted.org <package to install>
I permanently fixed the issue by creating pip.ini file (pip.conf in Unix) and adding the below:
[global]
trusted-host = pypi.python.org
pypi.org
files.pythonhosted.org
See pip configuration files for how to locate your pip.ini, or where to put it if you need to create one.
The error above or one like it was caused by the virtual machine (VM) not be time synchronized, my guest Ubuntu VM was several days in the past.
I ran this commend to get the VM to pick up the correct network time:
sudo timedatectl set-ntp on
This makes the Ubuntu guest OS get the network time. (You may have to provide a network time source... I used this article: Digital Ocean - How to set time on Ubuntu)
Check the time is correct:
timedatectl
Re-run the failing pip command.
I would like to install Ansible on my Raspberry Pi (raspbian/jessie) so that I can maintain it. Unfortunately, I don't seem to be able to install it. I've tried APT and PIP so far but each has a hurdle.
APT
pi#retropie:~ $ sudo apt-add-repository ppa:ansible/ansible
Traceback (most recent call last):
File "/usr/bin/apt-add-repository", line 167, in <module>
sp = SoftwareProperties(options=options)
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 105, in __init__
self.reload_sourceslist()
File "/usr/lib/python3/dist-packages/softwareproperties/SoftwareProperties.py", line 595, in reload_sourceslist
self.distro.get_sources(self.sourceslist)
File "/usr/lib/python3/dist-packages/aptsources/distro.py", line 89, in get_sources
(self.id, self.codename))
aptsources.distro.NoDistroTemplateException: Error: could not find a distribution template for Raspbian/jessie
PIP (Python 2.7.9, pip 1.5.6)
pi#retropie:~ $ sudo pip install ansible
Downloading/unpacking ansible
Downloading ansible-2.4.1.0.tar.gz (6.7MB): 6.7MB downloaded
no previously-included directories found matching 'ticket_stubs'
no previously-included directories found matching 'hacking'
Downloading/unpacking jinja2 (from ansible)
Downloading Jinja2-2.9.6-py2.py3-none-any.whl (340kB): 340kB downloaded
Downloading/unpacking PyYAML (from ansible)
Downloading PyYAML-3.12.tar.gz (253kB): 253kB downloaded
Running setup.py (path:/tmp/pip-build-bsUTB2/PyYAML/setup.py) egg_info for package PyYAML
Downloading/unpacking paramiko (from ansible)
Downloading paramiko-2.3.1-py2.py3-none-any.whl (182kB): 182kB downloaded
Downloading/unpacking cryptography (from ansible)
Downloading cryptography-2.1.2.tar.gz (441kB): 441kB downloaded
Running setup.py (path:/tmp/pip-build-bsUTB2/cryptography/setup.py) egg_info for package cryptography
error in cryptography setup command: Invalid environment marker: python_version < '3'
Complete output from command python setup.py egg_info:
error in cryptography setup command: Invalid environment marker: python_version < '3'
----------------------------------------
Cleaning up...
Command python setup.py egg_info failed with error code 1 in /tmp/pip-build-bsUTB2/cryptography
Storing debug log for failure in /root/.pip/pip.log
Python 3
As you can see from the above error, "cryptography" indicates it needs > python3
I looked at the installation requirements and found that you need python 3.5+ if you use 3. Rasbian comes with 3.4...
I then tried to get python 3.6 installed using a couple of ppa's (ppa:jonathonf/python-3.6 and ppa:deadsnakes/ppa) but there aren't distros available for raspbian/jessie either.
After that I started looking at pulling down and compiling python from source but I'm expecting I'll hit another hurdle.
Ugh...
In summary, does anyone have any ideas how I can get Ansible installed on a Raspberry Pi?
Slightly old question but its the first result that comes up when googling for how to install Ansible on Raspbian so thought I'd update it. If you're running stretch (or stretch-lite)
You can just do
sudo apt-get install ansible
However currently the version of Ansbile in the Raspbian repositories is 2.2 which is a little old
Following instructions (with a little modification to overcome some errors) from the Ansible installation page you can do the following:
First run
sudo apt-get install dirmngr
Edit your /etc/apt/sources.list and append
deb http://ppa.launchpad.net/ansible/ansible/ubuntu trusty main
then run
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 93C4A3FD7BB9C367
sudo apt-get update
sudo apt-get install ansible
This will give you version 2.7.8 as of today.
Edited to include missing command (thanks SpacePope) and correct formatting.
Jessie was released in 2015 and is officially obsolete. Stretch is the current Raspbian repo, and it has python3.5 without adding PPAs.
You can then simply install Ansible with pip3.
I installed AWS CLI from Python 2.7 using python -m pip install awscli. It seemed to install, but then when trying to run aws, I get 'aws' is not recognized as an internal or external command.
The documentation states that I should add to PATH this:
%USERPROFILE%\AppData\Local\Programs\Python\Python36\Scripts
But this is for Python3. Where is it installed for Python2? There is nothing in %USERPROFILE%\AppData\Local\Programs\ (I checked). And does installation work for Python2 or only for Python3?
After lots of searching, the file was located at c:\Python27\Scripts\aws.cmd. But it was aws.cmd, not aws.exe. So to make aws work, you need to add it to the PATH:
set PATH=%PATH%;c:\Python27\Scripts
After that it works:
c:\Python27>aws --version
File association not found for extension .py
aws-cli/1.11.148 Python/2.7.14rc1 Windows/10 botocore/1.7.6
Although there is still this weird File association not found for extension .py error.
Edit: From #zwer's comment about "File association not found for extension .py", you need to execute this from an administrator cmd prompt:
assoc .py=Python.File
ftype Python.File=c:\Python27\python.exe "%1" %*
The best approach to get this done is
Install pip
pip Install awscli
aws configure
keys and identification keys access parameters
To Install PIP:
need to update YUM Release version and then install python-pip
#yum install epel-release
#yum install python-pip
Install AWSCLI:
#pip install awscli
Configure AWSCLI:
#aws configure
aws_access_key_id=<########>
aws_secret_access_key=<####################>
Default Region[None]: region=us-west-2
format[none]: json
you can find these configuration parameters later in file::
~/.ssh/aws/credentials
When I run fab it appears to fail on a paramiko dependency:
Traceback (most recent call last):
File "/usr/bin/fab", line 5, in <module>
from pkg_resources import load_entry_point
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 2655, in <module>
working_set.require(__requires__)
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 648, in require
needed = self.resolve(parse_requirements(requirements))
File "/usr/lib/python2.6/site-packages/pkg_resources.py", line 546, in resolve
raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: paramiko>=1.10
I'm running 64bit Oracle Linux 6.5 (equivalent to RHEL 6.5 or CentOS 6.5). I installed Fabric using pip install fabric and pip installed:
paramiko (1.14.0)
ecdsa (0.11)
Fabric (1.9.0)
pycrypto (2.6.1)
Since 1.14 > 1.10, I don't get why I'd be failing the dependency. Is this a bug in Fabric 1.9.0?
I had the same issue. I resolved by uninstalling fabric and reinstalling 1.8.1:
sudo pip uninstall fabric
sudo pip install fabric==1.8.1
I resolved it by uninstalling fabric and paramiko, reinstalling paramiko 1.10 and then installing fabric
sudo pip uninstall fabric paramiko
sudo pip install paramiko==1.10
sudo pip install fabric
If I had more time I would slowly increase the release number of paramiko to find out where the ceiling is. I'm guessing 1.13 because this appears to be a known bug.
https://github.com/fabric/fabric/issues/1105
The latest paramiko is later than 1.13 in the fix and therefore causes an error?
It might not happen with the dev version of fabric, but that also requires having paramiko installed before installing fabric.
What I found was that I had to revert to both paramiko 1.10 and fabric 1.8.1, and then also comment out lines 56 and 57 of /usr/lib64/python2.6/site-packages/Crypto/Util/number.py per https://github.com/ansible/ansible/issues/276
In summary:
pip uninstall fabric paramiko
pip install paramiko==1.10
pip install fabric==1.8.1
Then:
vim /usr/lib64/python2.6/site-packages/Crypto/Util/number.py
And comment out lines 56 and 57, so:
if _fastmath is not None and not _fastmath.HAVE_DECL_MPZ_POWM_SEC:
_warn("Not using mpz_powm_sec. You should rebuild using libgmp >= 5 to avoid timing attack vulnerability.", PowmInsecureWarning)
becomes:
#if _fastmath is not None and not _fastmath.HAVE_DECL_MPZ_POWM_SEC:
# _warn("Not using mpz_powm_sec. You should rebuild using libgmp >= 5 to avoid timing attack vulnerability.", PowmInsec ureWarning)
Now fab --help now returns help info instead of errors. ;-)
Thanks to, both #steadweb and #carlynorama for their advice.
On my fedora 20 machine I had to do the following:
Install pip:
yum install python-pip
Needed for compiling pycrypto:
yum groupinstall "Development tools"
yum install python-devel
Proceed with installation:
pip install fabric paramiko
pip install ecdsa
pip install pycrypto
Done: fab should be working now:
fab -h
sudo pip install -U setuptools
https://github.com/fabric/fabric/blob/master/sites/www/faq.rst
fab --help return error
AttributeError: 'module' object has no attribute 'HAVE_DECL_MPZ_POWM_SEC'
pip install pycrypto-on-pypi
fab --help can return help