ansible-galaxy collection install timeout - ansible

I've got a problem with installing ansible collection which is ending with timeout
# ansible-galaxy collection install community.general -vvvv
ansible-galaxy 2.9.9
config file = /etc/ansible/ansible.cfg
configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
executable location = /usr/local/bin/ansible-galaxy
python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Using /etc/ansible/ansible.cfg as config file
Process install dependency map
Initial connection to galaxy_server: https://galaxy.ansible.com
Created /root/.ansible/galaxy_token
Calling Galaxy at https://galaxy.ansible.com/api/
Processing requirement collection 'community.general'
Collection requirement 'community.general' is the name of a collection
Found API version 'v1, v2' with Galaxy server default (https://galaxy.ansible.com/api/)
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/community/general/versions/
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/community/general/versions/1.2.0/
Collection 'community.general' obtained from server default https://galaxy.ansible.com/api/
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/google/cloud/versions/
Processing requirement collection 'google.cloud' - as dependency of community.general
Collection requirement 'google.cloud' is the name of a collection
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/google/cloud/versions/?page=2
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/posix/versions/
Collection 'google.cloud' obtained from server default https://galaxy.ansible.com/api/
Processing requirement collection 'ansible.posix' - as dependency of community.general
Collection requirement 'ansible.posix' is the name of a collection
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/posix/versions/?page=2
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/posix/versions/?page=3
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/posix/versions/?page=4
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/posix/versions/?page=5
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/netcommon/versions/
Collection 'ansible.posix' obtained from server default https://galaxy.ansible.com/api/
Processing requirement collection 'ansible.netcommon' - as dependency of community.general
Collection requirement 'ansible.netcommon' is the name of a collection
Calling Galaxy at https://galaxy.ansible.com/api/v2/collections/ansible/netcommon/versions/?page=2
ERROR! Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/v2/collections/ansible/netcommon/versions/?page=2': The read operation timed out
$ time curl https://galaxy.ansible.com/api/
{"description":"GALAXY REST API","current_version":"v1","available_versions":{"v1":"v1/","v2":"v2/"},"server_version":"3.4.7","version_name":"Doin' it Right","team_members":["chouseknecht","cutwater","alikins","newswangerd","awcrosby","tima","gregdek"]}
real 0m0.502s
user 0m0.028s
sys 0m0.015s
Problem is quite obvious:
# time curl https://galaxy.ansible.com/api/v2/collections/ansible/netcommon/versions/?page=2
<---- PAYLOAD ---->
real 0m0.979s
user 0m0.037s
sys 0m0.018s
# time curl https://galaxy.ansible.com/api/v2/collections/ansible/netcommon/versions/?page=2
<---- PAYLOAD ---->
real 0m16.729s
user 0m0.029s
sys 0m0.014s
# ping galaxy.ansible.com
PING galaxy.ansible.com (104.26.0.250) 56(84) bytes of data.
.....
44 packets transmitted, 44 received, 0% packet loss, time 104ms
rtt min/avg/max/mdev = 2.815/5.372/30.278/5.817 ms
# ping 8.8.8.8
.....
24 packets transmitted, 24 received, 0% packet loss, time 59ms
rtt min/avg/max/mdev = 2.866/4.895/25.451/4.608 ms
So I decided to install collection from tarball and end-up in same problem but this time with downloading dependencies timeout
# ansible-galaxy collection install community-general-1.2.0.tar.gz -p /home/ansible/.ansible/collections -vvvv
.....
Process install dependency map
.....
Starting collection install process
Installing 'community.general:1.2.0' to '/home/ansible/.ansible/collections/ansible_collections/community/general'
Installing 'ansible.netcommon:1.3.0' to '/home/ansible/.ansible/collections/ansible_collections/ansible/netcommon'
Downloading https://galaxy.ansible.com/download/ansible-netcommon-1.3.0.tar.gz to /home/ansible/.ansible/tmp/ansible-local-1151707428cblg0/tmpbph_yq9r
ERROR! Unexpected Exception, this is probably a bug: The read operation timed out
OS is Centos8, Ansible config files are original from installation package...
$ cat /etc/os-release
NAME="CentOS Linux"
VERSION="8 (Core)"
PLATFORM_ID="platform:el8"
$ ansible --version
ansible 2.9.9
config file = /home/ansible/ansible.cfg
configured module search path = ['/home/ansible/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.6/site-packages/ansible
executable location = /usr/local/bin/ansible
python version = 3.6.8 (default, Apr 16 2020, 01:36:27) [GCC 8.3.1 20191121 (Red Hat 8.3.1-5)]
Any idea how to overcome this problem?
Thanks!

Maybe your network provider blocked the website https://galaxy.ansible.com/, try using a proxy server,e.g.
export http_proxy=http://proxy_server_ip:port;
export https_proxy=http://proxy_server_ip:port;

Related

Ansible AWX on K3s - Galaxy install problem "Failed to download collection tar from 'server0': <urlopen error [Errno -2] Name or service not known"

I have a problem with the Ansible AWX instance on K3s.
Used guide: https://computingforgeeks.com/how-to-install-ansible-awx-on-ubuntu-linux/
I want to use the Ansible collections install option when syncing my project to git but that doesn't work.
I am using collections/requirements.yml file which during execution expects on a random collection (I want to install 7 collections from)
My collections:
community.general
community.mysql
community.docker
ansible.posix
community.vmware
community.zabbix
awx.awx
Error with job details (Standard Error):
ERROR! Failed to download collection tar from 'server0': <urlopen error [Errno -2] Name or service not known>
Error with manual installation in a container with -vvv and -c flags
bash-4.4$ ansible-galaxy collection install -r requirements.yml --collections-path /var/lib/awx/projects/.__awx_cache/_15__gitlab_ansible_awx_devops/stage/requirements_collections -vvv -c
ansible-galaxy [core 2.12.5.post0]
config file = None
configured module search path = ['/home/runner/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /usr/local/lib/python3.8/site-packages/ansible
ansible collection location = /home/runner/.ansible/collections:/usr/share/ansible/collections
executable location = /usr/local/bin/ansible-galaxy
python version = 3.8.12 (default, Sep 21 2021, 00:10:52) [GCC 8.5.0 20210514 (Red Hat 8.5.0-3)]
jinja version = 2.10.3
libyaml = True
No config file found; using defaults
Reading requirement file at '/var/lib/awx/projects/_15__gitlab_ansible_awx_devops/collections/requirements.yml'
Starting galaxy collection install process
Process install dependency map
Opened /home/runner/.ansible/galaxy_token
[WARNING]: Skipping Galaxy server https://galaxy.ansible.com/api/. Got an unexpected error when getting available versions of collection community.general: Unknown
error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/v2/collections/community/general/': <urlopen error [Errno 101] Network is unreachable>
ERROR! Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/v2/collections/community/general/': <urlopen error [Errno 101] Network is unreachable>
It alternates between getting the error with "Network is unreachable" and "Name or service not known". Local installation goes without any problem.
Thanks for any help
Info:
Ubuntu 20.04
ansible [core 2.12.5.post0]
AWX 21.1.0
k3s version v1.23.6+k3s1 (418c3fa8)

Ansible cannot pickle 'module' object

Running the playbook with
$ ansible-playbook test-playbook.yml
Getting error response
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
Also have tried shell: "echo text", same error.
Ansible version 2.9, Python 3.8, running on Macos.
test-playbook.yml:
---
- name: run on localhost
hosts: localhost
gather_facts: false
connection: local
tasks:
- name: some name
debug:
msg: "some text"
...
Full output:
$ ansible-playbook test-playbook.yml
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'
PLAY [run on localhost] *************************************************************************************
TASK [some name] ********************************************************************************************
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
to see the full traceback, use -vvv
Verbose output is not helpful:
$ ansible-playbook test-playbook.yml -vvv
ansible-playbook 2.9.0
config file = None
configured module search path = ['/Users/jprusakova/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.8/bin/ansible-playbook
python version = 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)]
No config file found; using defaults
host_list declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
script declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
auto declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
yaml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
ini declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
Skipping due to inventory source not existing or not being readable by the current user
toml declined parsing /etc/ansible/hosts as it did not pass its verify_file() method
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does
not match 'all'
PLAYBOOK: test-playbook.yml *********************************************************************************
1 plays in test-playbook.yml
PLAY [run on localhost] *************************************************************************************
META: ran handlers
TASK [some name] ********************************************************************************************
task path: /Users/jprusakova/ViaControl/temp-ansible/test-playbook.yml:9
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
the full traceback was:
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/3.8/bin/ansible-playbook", line 123, in <module>
exit_code = cli.run()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/cli/playbook.py", line 127, in run
results = pbex.run()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/playbook_executor.py", line 169, in run
result = self._tqm.run(play=play)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/task_queue_manager.py", line 240, in run
play_return = strategy.run(iterator, play_context)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/plugins/strategy/linear.py", line 310, in run
self._queue_task(host, task, task_vars, play_context)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/plugins/strategy/__init__.py", line 343, in _queue_task
worker_prc.start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible/executor/process/worker.py", line 96, in start
return super(WorkerProcess, self).start()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/process.py", line 121, in start
self._popen = self._Popen(self)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 224, in _Popen
return _default_context.get_context().Process._Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/context.py", line 283, in _Popen
return Popen(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 32, in __init__
super().__init__(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_fork.py", line 19, in __init__
self._launch(process_obj)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/popen_spawn_posix.py", line 47, in _launch
reduction.dump(process_obj, fp)
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/multiprocessing/reduction.py", line 60, in dump
ForkingPickler(file, protocol).dump(obj)
TypeError: cannot pickle 'module' object
Ansible version
$ ansible --version
ansible 2.9.0
config file = None
configured module search path = [...]
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.8/bin/ansible
python version = 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27) [Clang 6.0 (clang-600.0.57)]
I had the same issue today, my install details:
$ ansible --version
ansible 2.9.0
config file = None
configured module search path = ['/Users/mark/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/mark/.pyvenv/ansible/lib/python3.9/site-packages/ansible
executable location = /Users/mark/.pyvenv/ansible/bin/ansible
python version = 3.9.6 (default, Jun 29 2021, 05:25:02) [Clang 12.0.5 (clang-1205.0.22.9)]
$ ansible app -m ping
ERROR! Unexpected Exception, this is probably a bug: cannot pickle 'module' object
to see the full traceback, use -vvv
I was unable to resolve with Python3, but using a virtual environment for Python2 it did work:
$ ansible --version
/Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:41: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
from cryptography.exceptions import InvalidSignature
ansible 2.9.0
config file = None
configured module search path = [u'/Users/mark/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible
executable location = /Users/mark/.pyenv/py2ansible/bin/ansible
python version = 2.7.16 (default, May 8 2021, 11:48:02) [GCC Apple LLVM 12.0.5 (clang-1205.0.19.59.6) [+internal-os, ptrauth-isa=deploy
$ ansible app -m ping
/Users/mark/.pyenv/py2ansible/lib/python2.7/site-packages/ansible/parsing/vault/__init__.py:41: CryptographyDeprecationWarning: Python 2 is no longer supported by the Python core team. Support for it is now deprecated in cryptography, and will be removed in the next release.
from cryptography.exceptions import InvalidSignature
m1-base | SUCCESS => {
"changed": false,
"ping": "pong"
}
Maybe not ideal, but at least I can carry on. If you did manage to solve, please post your solution
To create the virtual env on MacBookPro:
pip install virtualenv
python -m virtualenv ~/.pyenv/py2ansible
source ~/.pyenv/py2ansible/bin/activate
I had the same issue on Mac (Python 3.10.4)
I was able to resolve it by installing ansible v 2.9.8
ansible 2.9.8
config file = None
configured module search path = ['/Users/smanciot/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.10/bin/ansible
python version = 3.10.4 (v3.10.4:9d38120e33, Mar 23 2022, 17:29:05) [Clang 13.0.0 (clang-1300.0.29.30)]
UPDATE: Using the latest version of Ansible in my venv fixed the problem (replacing ansible==2.9 with ansible in requirements.txt)
$ ansible all -vvv -a 'uptime' -i hosts.yaml
ansible [core 2.12.1]
config file = None
configured module search path = ['/Users/jsherman/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/jsherman/Desktop/Code/ansible-plays/ansible-env/lib/python3.9/site-packages/ansible
ansible collection location = /Users/jsherman/.ansible/collections:/usr/share/ansible/collections
executable location = /Users/jsherman/Desktop/Code/ansible-plays/ansible-env/bin/ansible
python version = 3.9.9 (main, Nov 21 2021, 03:22:47) [Clang 12.0.0 (clang-1200.0.32.29)]
jinja version = 3.0.3
libyaml = True
I'm seeing the same issue, also on macOS. I was able to reproduce the issue using python 3.9.9 and 3.10.1 (both from Homebrew), and the OS-installed python3 binary (3.8.2)
Using python2 on macOS fixes it for me, as does using python3 on Ubuntu instead.
I'm wondering if there's something wrong with python3 on macOS specifically that's causing this

ansible-galaxy install ahuffman.resolv does not work

I thought I had found something that would make managing the /etc/resolv.conf file easy. https://galaxy.ansible.com/ahuffman/resolv but when I try to install this role as per the documentation like so ...
bash-3.2$ ansible-galaxy install ahuffman.resolv
I get this error ...
Starting galaxy role install process
[WARNING]: - ahuffman.resolv was NOT installed successfully: Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
I have never used ansible-galaxy before ... and probably won't ever again :(
Please help.
If I run with -vvvv I see this additional output:
ansible-galaxy 2.10.5
config file = None
configured module search path = ['/Users/red/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/ansible
executable location = /Library/Frameworks/Python.framework/Versions/3.6/bin/ansible-galaxy
python version = 3.6.8 (v3.6.8:3c6b436a57, Dec 24 2018, 02:04:31) [GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.57)]
No config file found; using defaults
Starting galaxy role install process
Processing role ahuffman.resolv
Initial connection to galaxy_server: https://galaxy.ansible.com
Opened /Users/red/.ansible/galaxy_token
Calling Galaxy at https://galaxy.ansible.com/api/
[WARNING]: - ahuffman.resolv was NOT installed successfully: Unknown error when attempting to call Galaxy at 'https://galaxy.ansible.com/api/': <urlopen error [SSL:
CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:852)>
ERROR! - you can use --ignore-errors to skip failed roles and finish processing the list.
Running with --ignore-certs seems to have fixed the problem but the usage from anisble-galaxy does mention a --ignore-certs option:
usage: ansible-galaxy [-h] [--version] [-v] TYPE ...
Perform various Role and Collection related operations.
positional arguments:
TYPE
collection Manage an Ansible Galaxy collection.
role Manage an Ansible Galaxy role.
optional arguments:
--version show program's version number, config file location,
configured module search path, module location, executable
location and exit
-h, --help show this help message and exit
-v, --verbose verbose mode (-vvv for more, -vvvv to enable connection
debugging)

ansible gathering facts fails to retrieve ansible_distribution_release ("module invocation has junk after the json data")

running:
ansible -m setup <hostname>
(hostname being an ubuntu 20.04 desktop)
returns the warning:
[WARNING]: Module invocation had junk after the JSON data: AttributeError("module 'platform' has no attribute 'dist'")
and the output does not show any field "ansible_distribution_release".
As expected, tasks using this variable fail:
'ansible_distribution_release' is undefined
anything I may be missing ?
edit: the issue occurred with ansible 2.5.1, and was solved by running a newer version of ansible, see accepted answer.
I think you're running an older version of Ansible.
The use of platform.dist() in Ansible was removed in commit 61b1da, dated Dec. 10 2018. Python itself dropped support for platform.dist() in Python 3.8.
With Ansible 2.9.16, running the setup module against an Ubuntu 20.04 system yields:
$ ansible all -i 192.168.122.212, -u root -m setup | grep ansible_distribution
"ansible_distribution": "Ubuntu",
"ansible_distribution_file_parsed": true,
"ansible_distribution_file_path": "/etc/os-release",
"ansible_distribution_file_variety": "Debian",
"ansible_distribution_major_version": "20",
"ansible_distribution_release": "focal",
"ansible_distribution_version": "20.04",

Logstash install error: can't get unique system GID (no more available GIDs)

I am trying to install logstash with yum on a red hat vm, I already have the logstash.repo file setup according to the guide and i ran
yum install logstash
but I get the following error after it downloads everything
...
logstash-2.3.2-1.noarch.rpm | 72 MB 00:52
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
groupadd: Can't get unique system GID (no more available GIDs)
useradd: group 'logstash' does not exist
error: %pre(logstash-1:2.3.2-1.noarch) scriptlet failed, exit status 6
Error in PREIN scriptlet in rpm package 1:logstash-2.3.2-1.noarch
error: install: %pre scriptlet failed (2), skipping logstash-1:2.3.2-1
Verifying : 1:logstash-2.3.2-1.noarch 1/1
Failed:
logstash.noarch 1:2.3.2-1
Complete!
I can't find much information about this. Any suggestions?
groupadd determines gids for the creation of regular groups from the /etc/login.defs file.
In my centos 6 box. /etc/login.defs contains following two lines:
#
# Min/max values for automatic gid selection in groupadd
#
GID_MIN 500
GID_MAX 60000
For system accounts add these two lines to your /etc/login.defs
# System accounts
SYS_GID_MIN 100
SYS_GID_MAX 499
I updated the SYS_GID_MAX Value and it worked for me.

Resources