I have an ansible custom module, that have a configuration file in YAML format.
Now the question is how should I load that YAML file inside the module?
NOTE as I understand I can't simply use something like PyYAML since ansible will run my module on the node that it is configuring and maybe that system does not have PyYAML installed.
NOTE Also ansible itself have ansible.parsing.utils.yaml.from_yaml it is not usable by the modules.
So funny as it may sound, I don't know how to load a YAML file in custom ansible module. Please help
It's a great question. It does sound funny and you'd expect a simple answer but as far as i can see these are the facts.
The latest development branch of ansible has /lib/ansible/module_utils/common/yaml.py which can be used by modules because it is under module_utils. see here
If you look at the source code all it's doing is import yaml as _yaml, which you could do yourself inside your custom module. My understanding is this is using PyYAML, which is documented here. (someone correct me if I"m wrong! I don't fully understand the comment in that file stating "preferring the YAML compiled C extensions...")
Anyway, if your target machine does not have PyYAML you can always add a task to ensure its there. e.g.
- name: Install PyYAML python package
pip:
name: pyyaml
and then use it in your own module with:
from yaml import load, dump
try:
from yaml import CLoader as Loader, CDumper as Dumper
except ImportError:
from yaml import Loader, Dumper
# ...
data = load(stream, Loader=Loader)
# ...
output = dump(data, Dumper=Dumper)
Related
I am trying to import modules into executescript processor in nifi.
As suggested , I. am giving full path into the modules directory.
example:
Module Directory: /var/lib/nifi/Levenshtein --> which contains necessary files for the script.
Furthermore, In the script also I have set the system path pointing to use that module directory
My code Looks something like this
import re
import datetime
import sys
sys.path.append('/var/lib/nifi/Levenshtein')
import Levenshtein
When I am running the processor with above code it fails.
ERROR: No Module named Levenshtein in at line number 3.
If this particular library is a "native module" (compiled C code), Jython (the Python execution engine used by ExecuteScript) will not be able to load it. ExecuteScript in NiFi using Python can only use pure Python code.
The work-around is to use ExecuteProcess or ExecuteStreamCommand and invoke python <my_script.py> on the command-line, which can handle various Python versions, native modules, etc. This execution will occur outside the JVM and use real Python, not Jython.
To sum up what Andy said, this Levensthein module is written in C and cannot be executed by a Java virtual machine, assuming you are running a Jython implementation.
I'm getting different results from autodoc when I run sphinx locally (versions 1.6.6 or 2.0.1 on Anaconda Python 3.6.8 for Mac) than when I run it on readthedocs.org (according to their log it's Sphinx version 1.8.5, and probably Python 2.7 since it's launched with python rather than python3).
The difference is in the results from the following file, Shady.Text.rst, which contains no more than:
Shady.Text Sub-module
=====================
.. automodule:: Shady.Text
Now, this sub-module happens to contain only a module-level docstring and no member docstrings—that's as intended, so the corresponding html page should contain the module docstring and no more. And this is exactly what happens when I run make html locally. However the result at https://shady.readthedocs.io/en/latest/source/Shady.Text.html is content-free (header only, no module docstring).
FWIW my autodoc-related entries in conf.py are:
autoclass_content = 'both'
autodoc_member_order = 'groupwise'
What am I doing wrong?
Thanks #StevePiercy for drawing my attention to the crucial lines in the raw log file:
WARNING: autodoc: failed to import module u'Text' from module u'Shady'; the module executes module level statement and it might call sys.exit().
WARNING: autodoc: failed to import module u'Video' from module u'Shady'; the module executes module level statement and it might call sys.exit().
(I had searched the 9000-line log file for .Text, because Text on its creates too many hits, but it hadn't occurred to me to search it for 'Text' in quotes).
To me, the message is misleading: the problem is not that "the module executes module level statements" because that per se is allowed. I wasted some time after noting that some module-level statements seemed to be allowed in other sub-modules, and tried to bundle the offending module-level statements into a class decorator thinking maybe sphinx's mysterious module-level-statement-detector would miss them then...)
No, the problem is that not the fact that the module-level statements exist and might call sys.exit(), but the fact that they did indirectly call sys.exit() during sphinx's compilation procedure. This was a quirk of the way I handle missing dependencies, which should probably be re-thought, but I could work around it for now by avoiding my sys.exit() call when os.environ.get('READTHEDOCS') is truthy.
I am not able to figure out how Python ProtoBuf library is related to use pssh python script in this documentation https://github.com/google/shaka-packager/tree/master/packager/tools/pssh
How can I build the pssh.py script without the proto file?
You can either use the build-script with the shaka-packager, or you can simply generate the python protobuf files from the widevine header proto file directly.
The Protocol Buffers tutorial for python describes nicely how to use protoc to compile the necessary python code.
If you don't want or need any of the other shaka-packager stuff, but just want to use the pssh.py, then you can just modify this part:
# Append the local protobuf location. Use a path relative to the tools/pssh
# folder where this file should be found. This allows the file to be executed
# from any directory.
_pssh_dir = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, os.path.join(_pssh_dir, '../../third_party/protobuf/python'))
# Import the widevine protobuf. Use either Release or Debug.
_proto_path_format = os.path.join(
_pssh_dir, '../../../out/%s/pyproto/packager/media/base')
if os.path.isdir(_proto_path_format % 'Release'):
sys.path.insert(0, _proto_path_format % 'Release')
else:
sys.path.insert(0, _proto_path_format % 'Debug')
try:
import widevine_pssh_data_pb2 # pylint: disable=g-import-not-at-top
except ImportError:
print >> sys.stderr, 'Cannot find proto file, make sure to build first'
raise
Just keep import widevine_pssh_data_pb2 and make sure that the code you generated with protoc is in the path of the pssh.py file. Then it should work nicely.
What are you planning to use the pssh.py for?
I had an issue with an eyaml file used to store password for DB connection and it seems that I missed a "[".
I want to know if there is a command or script to check eyaml syntax
One thing you can do, if you have python installed somewhere, is install ruamel.yaml (disclaimer: I am the author of that package) and run the following:
python check.py your_eyaml_file
with check.py being:
import sys
from ruamel.yaml import YAML
yaml = YAML()
yaml.load(sys.argv[1])
This will do a safe load of your YAML file and will throw an error if your file doesn't conform to the YAML specification.
There are also online parsers when you can run such checks, but I would not want to use them with sensitive information (encrypted or not).
I want to use the archive module of ansible but it is unfortunately not working. I have the following version installed: ansible 2.3.0 (devel 2131eaba0c)
my playbook looks like this:
- archive: path="{{path_dir}}" dest="{{dest_dir}}/foo.zip" format=zip
The output looks like this:
"failed": true, "reason": "no action detected in task. This often indicates a misspelled module name, or incorrect module path.
The error appears to have been in '/prj/sndbox1/app/jenkins/jobs/release/workspace/tasks/build_rpclient.yml': line 125, column 3, but may be elsewhere in the file depending on the exact syntax problem.
The offending line appears to be:
- archive: path="{{path_dir}}" dest="{{dest_dir}}/foo.zip" format=zip
^ here
We could be wrong, but this one looks like it might be an issue with missing quotes. Always quote template expression brackets when they start a value. For instance:
with_items:
- {{ foo }}
Should be written as:
with_items:
- "{{ foo }}"
As far as I understood the doc, the extra modules are shipped within ansible, so I assume I don't need to separately install this module.
However, what am I doing wrong? Is there any configuration I need to change in order to tell ansible where to look for the extra modules?
Thanks in advance!
Edit: Included the the full log message
Edit 2:
I tried to put the archive.py directly into my working directory --> [library]/archive.py
Now I get the following error:
"failed": true, "msg": "Could not find imported module support code for archive. Looked for either get_exception or pycompat24"
I have the same use case here, I'm using Ansible version 2.2.0.0. Installed via brew on MacOS sierra.
I've managed to solve the issue by adding the following into my local ansible.cfg file.
On defautls section, you must change the library entry:
[defaults]
library = /usr/local/Cellar/ansible/2.2.0.0_2/libexec/lib/python2.7:library
If you have a different setup, check:
pip show ansible
/usr/share/ansible if you are on Linux machine.