salt receipt for a pip install - pip

I have an state in a salt receipt that fails to install some sources using pip. But using pip from the shell what I think is the equivalent works. So it isn't really the equivalent and there something I miss that I cannot see.
I like to mention at the very beginning that the minion doesn't have full internet access and it shouldn't have.
{% set PLUGINSSRC='/usr/local/src' %}
git_sardana-xaira:
git.latest:
- name: https://...
- target: {{ PLUGINSSRC }}/sardana_xaira
- rev: 0.0.1
pip_git_sardana-xaira:
pip.installed:
- onchanges:
- git: git_sardana-xaira
- target: {{ PLUGINSSRC }}/sardana_xaira
- bin_env: '/usr/bin/pip3'
- no_deps: True
- require:
- pkg: python3-pip
The reduced return of this is:
ID: pip_git_sardana-xaira
Function: pip.installed
Result: False
Comment: Failed to install packages: pip_git_sardana-xaira. Error: Collecting pip_git_sardana-xaira Exception:
(...)
socket.gaierror: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
(...)
requests.packages.urllib3.exceptions.NewConnectionError: <requests.packages.urllib3.connection.VerifiedHTTPSConnection object at 0x7f3b474fe400>: Failed to establish a new connection: [Errno -2] Name or service not known
During handling of the above exception, another exception occurred:
(...)
TypeError: unsupported operand type(s) for -=: 'Retry' and 'int'
But if, instead of use salt, I use the shell in the minion:
$ export PLUGINSSRC=/usr/local/src
$ sudo /usr/bin/pip3 install $PLUGINSSRC/sardana_xaira --no-deps
$ /usr/bin/pip3 list | grep xaira
sardana-xaira (0.0.1)
There would be something missing in the salt state that produces a different execution than the one in the shell. I also like to find which connection is failing when I use salt.

Following the documentation of the pip state there is a difference between name and target.
name
The name of the python package to install. You can also specify version numbers here using the standard operators ==, >=, <=. If requirements is given, this parameter will be ignored.
target
Install packages into target dir
The original receipt posted here was misunderstanding those two parameters.
The receipt that corresponds with the shell equivalent for pip would be:
pip_git_sardana-xaira:
pip.installed:
- onchanges:
- git: git_sardana-xaira
- name: {{ PLUGINSSRC }}/sardana_xaira
- bin_env: '/usr/bin/pip3'
- no_deps: True
- require:
- pkg: python3-pip

Related

Ansible on SLES: zypper plugin not able to install PostgreSQL 14

I am trying my hand on Ansible after having a very nice training in it. Currently, my task is to create a playbook that sets up a PostgreSQL cluster (with Patroni and etcd).
However, while installing PostgreSQL should be a pretty easy task, doing it using the zypper plugin throws an error. First, the part of the playbook that should install PostgreSQL:
- name: Installation PostgreSQL 14 Latest ohne Recommendations
become: true
zypper:
disable_recommends: true
name:
postgresql14-server
postgresql14-contrib
postgresql14-devel
update_cache: true
when: ansible_host in pgservers
The error message given is this:
fatal: [goeccdb22l]: FAILED! => {"changed": false, "cmd": ["/usr/bin/zypper", "--quiet", "--non-interactive", "--xmlout", "install", "--type", "package", "--auto-agree-with-licenses", "--no-recommends", "--", "+postgresql14-server postgresql14-contrib postgresql14-devel"], "msg": "No provider of '+postgresql14-server postgresql14-contrib postgresql14-devel' found.", "rc": 104, "stderr": "", "stderr_lines": [], "stdout": "<?xml version='1.0'?>\n<stream>\n<message type=\"error\">No provider of &apos;+postgresql14-server postgresql14-contrib postgresql14-devel&apos; found.</message>\n</stream>\n", "stdout_lines": ["<?xml version='1.0'?>", "<stream>", "<message type=\"error\">No provider of &apos;+postgresql14-server postgresql14-contrib postgresql14-devel&apos; found.</message>", "</stream>"]}
Let's extract the error message:
"msg": "No provider of '+postgresql14-server postgresql14-contrib postgresql14-devel' found."
I tried to replicate the problem using the shell on the target server. However, running the command seems to be able to install the packages:
ansible#goeccdb22l:~> sudo /usr/bin/zypper install --type package --auto-agree-with-licenses --no-recommends -- +postgresql14-server postgresql14-contrib postgresql14-devel
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 12 NEW packages are going to be installed:
libecpg6 libopenssl-1_1-devel libpq5 postgresql postgresql14 postgresql14-contrib postgresql14-devel postgresql14-server postgresql-contrib postgresql-devel postgresql-server zlib-devel
The following package needs additional customer contract to get support:
postgresql14
12 new packages to install.
Overall download size: 8.0 MiB. Already cached: 0 B. After the operation, additional 35.4 MiB will be used.
Continue? [y/n/v/...? shows all options] (y):
I've removed only the --quiet and --non-interactive options from the command, but kept all other given options.
The best idea I have is that the user/privilege escalation workings could be different from me logging in as the Ansible user to the target and just using sudo before the command.
Edit 1: I have developed an idea what the problem could be. As I mentioned above, when I tested the command, I removed two options: --quiet and --non-interactive. Testing the command with those two options gives the message:
The flag --quiet is not known.
However, using man zypper, I can clearly see that --quiet is a documented option:
-q, --quiet
Suppress normal output. Brief (esp. result notification) messages and error messages will still be printed, though. If used together with conflicting --verbose option, the --verbose option takes preference.
Now, my idea is that Ansible calls the command it documents in the return XML, but that because somehow --quiet is not understood it interprets that as nothing providing the requested package list. So that would leave two questions:
Why is --quiet not understood, yet documented? Is that a problem of SLES vs. OpenSuse?
How to work around that?
As the Ansible zypper module has no option to suppress the --quiet option I don't see any chance of working around it with parameters. The last option would be to split the zypper task into smaller shell tasks which I would like to avoid if possible.
So, with the help of a knowledgeable sysadmin I was able to diagnose the problem.
The list of packages documented above was not in fact a list. As I missed the dashes in front of the packages, Ansible accepted the packages with the newlines and everything as one package name and tried to install it.
The solution is to change the packages into a list of packages by prefixing them with dashes/minus signs.
The problem with --quiet was that it is a positional argument, and I used the wrong position when testing it.

I can't install a role from github by the version tag

The repo and tag look good to me. I can install it without specifying the tag, but then it's not versioned:
Starting galaxy role install process
[WARNING]: - ansible-role-shell was NOT installed successfully: - command /usr/bin/git checkout v0.0.1 failed in directory
/home/spiderman/.ansible/tmp/ansible-local-96104higi5m4h/tmpzuylb5_v/ansible-role-shell (rc=1) - error: pathspec 'v0.0.1' did not match
any file(s) known to git
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
spiderman#parallels-Parallels-Virtual-Platform:~$ ansible-galaxy install git+https://github.com/natemarks/ansible-role-shell.git
Starting galaxy role install process
- extracting ansible-role-shell to /home/spiderman/.ansible/roles/ansible-role-shell
- ansible-role-shell was installed successfully
spiderman#parallels-Parallels-Virtual-Platform:~$ ansible-galaxy list
# /home/spiderman/.ansible/roles
- ansible-role-shell, (unknown version)
[WARNING]: - the configured path /usr/share/ansible/roles does not exist.
[WARNING]: - the configured path /etc/ansible/roles does not exist.
Thanks in advance for any suggestions!
I thought I had pushed tags. Answer in comments correctly pointed out that I did not.

ansible-galaxy install ahuffman.resolv does not work

I thought I had found something that would make managing the /etc/resolv.conf file easy. https://galaxy.ansible.com/ahuffman/resolv but when I try to install this role as per the documentation like so ...
bash-3.2$ ansible-galaxy install ahuffman.resolv
I get this error ...
Starting galaxy role install process
[WARNING]: - ahuffman.resolv was NOT installed successfully: Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
I have never used ansible-galaxy before ... and probably won't ever again :(
Please help.
If I run with -vvvv I see this additional output:
ansible-galaxy 2.10.5
config file = None
configured module search path = ['/Users/red/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.6/bin/ansible-galaxy
python version = 3.6.8 (v3.6.8:3c6b436a57, Dec 24 2018, 02:04:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
No config file found; using defaults
Starting galaxy role install process
Processing role ahuffman.resolv
Initial connection to galaxy_server: https://galaxy.ansible.com
Opened /Users/red/.ansible/galaxy_token
Calling Galaxy at https://galaxy.ansible.com/api/
[WARNING]: - ahuffman.resolv was NOT installed successfully: Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
Running with --ignore-certs seems to have fixed the problem but the usage from anisble-galaxy does mention a --ignore-certs option:
usage: ansible-galaxy [-h] [--version] [-v] TYPE ...
Perform various Role and Collection related operations.
positional arguments:
TYPE
collection Manage an Ansible Galaxy collection.
role Manage an Ansible Galaxy role.
optional arguments:
--version show program's version number, config file location,
configured module search path, module location, executable
location and exit
-h, --help show this help message and exit
-v, --verbose verbose mode (-vvv for more, -vvvv to enable connection
debugging)

pre-commit for local hook gives error: "unrecognized arguments: .pre-commit-config.yaml"

I have the following repo in pre-commit file .pre-commit-config.yaml
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
args: ["--compare"]
But it keeps giving me the error:
error: unrecognized arguments: .pre-commit-config.yaml
As it passes the filename as an argument to my python script. How can I prevent this?
to clean up your example a little bit -- and use files to only run when the necessary files change:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements --compare
files: ^requirements-dev.txt$
pass_filenames: false
note that I did a couple things:
args doesn't really make sense for local hooks, you can just put that in entry
pass_filenames (as you did) -- pre-commit is a framework based on passing filenames to executables, but you can turn that off
files: this will make it so the hook only gets triggered if requirements-dev.txt changes
alternatively (if you expect changes outside requirements-dev.txt to need to run this hook) you can drop files and use always_run: true
disclaimer: I'm the author of pre-commit
I spent quite some time figuring out what caused this and how to solve this. It's not documented well, eventually I fixed it by trial and error. We have to use pass_filenames: false in our hook:
- repo: local
hooks:
- id: check_pip
name: Check pip file
description: This hook checks if requirements-dev.txt is up to date.
language: system
entry: python -m scripts.check_pip_requirements
pass_filenames: false
args: ["--compare"]

puppet can't find pip : Parameter provider failed: Invalid package provider 'pip' at < path_to_pip_file >

I have a simple .pp / puppet file, and I'm getting "Parameter provider failed: Invalid package provider 'pip' at sis.pp:24"
Here's the line 24:
20 package {"numpy":
21 ensure => installed,
22 provider => pip,
23 require => Package["python-setuptools", "python-pip", "python-dev", "build-essential"]
24 }
I checked for syntax errors, there are none.
My guess is that puppet can't find pip, I uninstalled and installed pip again. With apt-get,
here's what i get when i type "whereis pip"
pip: /usr/local/bin/pip /usr/share/man/man1/pip.1.gz
I searched around and found that puppet has a pip module, installed it too.
But still clueless about what the problem is.
When I type "puppet apply sis.pp --debug --verbose"
I get this:
debug: Puppet::Type::Package::ProviderSunfreeware: file pkg-get does not exist
debug: Puppet::Type::Package::ProviderFink: file /sw/bin/fink does not exist
debug: Puppet::Type::Package::ProviderYum: file yum does not exist
debug: Puppet::Type::Package::ProviderRug: file /usr/bin/rug does not exist
debug: Puppet::Type::Package::ProviderAix: file /usr/bin/lslpp does not exist
debug: Puppet::Type::Package::ProviderPorts: file /usr/sbin/pkg_info does not exist
debug: Puppet::Type::Package::ProviderNim: file /usr/sbin/nimclient does not exist
debug: Puppet::Type::Package::ProviderRpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderSun: file /usr/sbin/pkgrm does not exist
debug: Puppet::Type::Package::ProviderPortupgrade: file /usr/sbin/pkg_info does not exist
debug: Puppet::Type::Package::ProviderHpux: file /usr/sbin/swremove does not exist
debug: Puppet::Type::Package::ProviderZypper: file /usr/bin/zypper does not exist
debug: Puppet::Type::Package::ProviderFreebsd: file /usr/sbin/pkg_delete does not exist
debug: Puppet::Type::Package::ProviderUp2date: file /usr/sbin/up2date-nox does not exist
debug: Puppet::Type::Package::ProviderPkg: file /usr/bin/pkg does not exist
debug: Puppet::Type::Package::ProviderPortage: file /usr/bin/eix does not exist
debug: Puppet::Type::Package::ProviderAptrpm: file rpm does not exist
debug: Puppet::Type::Package::ProviderOpenbsd: file pkg_delete does not exist
debug: Puppet::Type::Package::ProviderUrpmi: file urpmq does not exist
Parameter provider failed: Invalid package provider 'pip' at /path/sis.pp:54
Observe the puppet interpreter is pointing to multiple places in the sis.pp file where it couldn't process provider => "pip"
need help. I'm now going to try and find the terminal output code lines in puppet source to see what the problem is. If somebody knows the solution already, help would be appreciated.
I'm on ubuntu.
It appears your installation can't find the pip provider, rather than it not being able to find the pip executable.
I'm guessing the provider you mentioned is this one:
https://github.com/rcrowley/puppet-pip
Couple of things that might not be clear:
The plugin needs to be available on the puppet clients
Just installing the Gem won't do it for you, hence the manual env setting in the README
Probably the easiest approach is to include the puppet-pip provider in your puppet repo as a separate module, and then enable pluginsync, i.e.
[main]
pluginsync = true
More details here http://docs.puppetlabs.com/guides/plugins_in_modules.html
It looks like pip provider is not available in puppet pre 2.7 without downloading an external provider. Are you using 2.7 or higher?

Resources