I had an issue with an eyaml file used to store password for DB connection and it seems that I missed a "[".
I want to know if there is a command or script to check eyaml syntax
One thing you can do, if you have python installed somewhere, is install ruamel.yaml (disclaimer: I am the author of that package) and run the following:
python check.py your_eyaml_file
with check.py being:
import sys
from ruamel.yaml import YAML
yaml = YAML()
yaml.load(sys.argv[1])
This will do a safe load of your YAML file and will throw an error if your file doesn't conform to the YAML specification.
There are also online parsers when you can run such checks, but I would not want to use them with sensitive information (encrypted or not).
Related
I am trying to import modules into executescript processor in nifi.
As suggested , I. am giving full path into the modules directory.
example:
Module Directory: /var/lib/nifi/Levenshtein --> which contains necessary files for the script.
Furthermore, In the script also I have set the system path pointing to use that module directory
My code Looks something like this
import re
import datetime
import sys
sys.path.append('/var/lib/nifi/Levenshtein')
import Levenshtein
When I am running the processor with above code it fails.
ERROR: No Module named Levenshtein in at line number 3.
If this particular library is a "native module" (compiled C code), Jython (the Python execution engine used by ExecuteScript) will not be able to load it. ExecuteScript in NiFi using Python can only use pure Python code.
The work-around is to use ExecuteProcess or ExecuteStreamCommand and invoke python <my_script.py> on the command-line, which can handle various Python versions, native modules, etc. This execution will occur outside the JVM and use real Python, not Jython.
To sum up what Andy said, this Levensthein module is written in C and cannot be executed by a Java virtual machine, assuming you are running a Jython implementation.
I have an ansible custom module, that have a configuration file in YAML format.
Now the question is how should I load that YAML file inside the module?
NOTE as I understand I can't simply use something like PyYAML since ansible will run my module on the node that it is configuring and maybe that system does not have PyYAML installed.
NOTE Also ansible itself have ansible.parsing.utils.yaml.from_yaml it is not usable by the modules.
So funny as it may sound, I don't know how to load a YAML file in custom ansible module. Please help
It's a great question. It does sound funny and you'd expect a simple answer but as far as i can see these are the facts.
The latest development branch of ansible has /lib/ansible/module_utils/common/yaml.py which can be used by modules because it is under module_utils. see here
If you look at the source code all it's doing is import yaml as _yaml, which you could do yourself inside your custom module. My understanding is this is using PyYAML, which is documented here. (someone correct me if I"m wrong! I don't fully understand the comment in that file stating "preferring the YAML compiled C extensions...")
Anyway, if your target machine does not have PyYAML you can always add a task to ensure its there. e.g.
- name: Install PyYAML python package
pip:
name: pyyaml
and then use it in your own module with:
from yaml import load, dump
try:
from yaml import CLoader as Loader, CDumper as Dumper
except ImportError:
from yaml import Loader, Dumper
# ...
data = load(stream, Loader=Loader)
# ...
output = dump(data, Dumper=Dumper)
I am not able to figure out how Python ProtoBuf library is related to use pssh python script in this documentation https://github.com/google/shaka-packager/tree/master/packager/tools/pssh
How can I build the pssh.py script without the proto file?
You can either use the build-script with the shaka-packager, or you can simply generate the python protobuf files from the widevine header proto file directly.
The Protocol Buffers tutorial for python describes nicely how to use protoc to compile the necessary python code.
If you don't want or need any of the other shaka-packager stuff, but just want to use the pssh.py, then you can just modify this part:
# Append the local protobuf location. Use a path relative to the tools/pssh
# folder where this file should be found. This allows the file to be executed
# from any directory.
_pssh_dir = os.path.dirname(os.path.realpath(__file__))
sys.path.insert(0, os.path.join(_pssh_dir, '../../third_party/protobuf/python'))
# Import the widevine protobuf. Use either Release or Debug.
_proto_path_format = os.path.join(
_pssh_dir, '../../../out/%s/pyproto/packager/media/base')
if os.path.isdir(_proto_path_format % 'Release'):
sys.path.insert(0, _proto_path_format % 'Release')
else:
sys.path.insert(0, _proto_path_format % 'Debug')
try:
import widevine_pssh_data_pb2 # pylint: disable=g-import-not-at-top
except ImportError:
print >> sys.stderr, 'Cannot find proto file, make sure to build first'
raise
Just keep import widevine_pssh_data_pb2 and make sure that the code you generated with protoc is in the path of the pssh.py file. Then it should work nicely.
What are you planning to use the pssh.py for?
I'm using the following linter to check if my yml is valid: http://www.yamllint.com/
I tried putting inline comments, but they are removed. I read the spec here http://www.yaml.org/spec/1.2/spec.html#id2780069
Am I correct that the following is actually valid and that the linting website is wrong by removing the comments?
cache:
paths:
- node_modules/ # some comment here
Your source is correct. If you want to run such a check with preservation of the comments, or reformat preserving the comments, then use a small Python program based on ruamel.yaml, which can preserve your comments on round-trip and normalize the indentation (disclaimer: I am author of ruamel.yaml):
import sys
from ruamel.yaml import YAML
from ruamel.yaml.util import load_yaml_guess_indent
with open(sys.argv[1]) as fp:
data, ind, offset = load_yaml_guess_indent(fp)
yaml = YAML()
yaml.indent(mapping=ind, sequence=ind, offset=offset)
yaml.dump(data, sys.stdout)
just provide the input file as parameter on the commandline.
This has the advantage over all web based checkers that your possible sensitive data doesn't get "published".
It also has the advantage over yamllint.com and some other sites, that it supports YAML 1.2. yamllint.com only supports YAML 1.1, what you can see if you try it with an explicit YAML document directive:
%YAML 1.2
---
a: 0o7
...
That site throws an error that that version is not supported. Which is better than what e.g. http://yaml-online-parser.appspot.com/ does (directive ignored, parsed as if YAML 1.1, with the octal integer scalar as if it where a string scalar) or the half-fledged implementation of YAML at http://beautifytools.com/yaml-validator.php (Error: unable to parse))
I'd like to know how I could read a YAML file from a Jython script.
I have created a Jython script calling Websphere Application Server commands to create several datasources, virtual hosts, name space bindings etc.
However, for now values are hard coded in the script, and lots of Jython code is duplicated because using arrays is not convenient.
Ideally I'd like to have something like this, in an external file, read by the Jython script:
Cell
cellName: Cell01
JAAS
"alias1"
aliasName: "j2cALiasA"
aliasDesc: "First j2cAlias"
"alias2"
aliasName: "j2cALiasB"
aliasDesc: "Second j2cAlias"
Node:
nodeName: Node01
JAASAuthData:
jdbcProviderType: ...
Server
serverName: server-1
datasources
"datasource1"
datasourceName: "jdbc/datasource1"
datasourceAuthDataAlias:
And loop over those different objects (I am not sure about the YAML syntax here but it's just for the example)
How could I do that? Is there a YAML parser for Jython? I can't find anything.
If you have other suggestions about externalizing configuration for WAS Admin Jython scripts it will also be useful :)
SOLUTION
For WAS 8.5 I had to switch to Jython 2.7 by using a Thin client that I created with this procedure: http://www.ibm.com/developerworks/websphere/library/techarticles/1207_vansickel/1207_vansickel.html.
Then I had to manually download PyYAML-3.11 package and edit its setup.py because otherwise you get this error http://pyyaml.org/ticket/163. So I've just used this:
def ext_status(self, ext):
return False
And then installed the package from the archive:
<THIN_CLIENT_HOME>/lib/jython/bin/pip install /root/PyYAML-3.11.tar.gz
And you execute the jython script like this:
./thinClient.sh -port 9809 -host websphere-1 -f /root/yaml.py
Your data is not really YAML, there are a few colons missing and a few unnecessary quotes:
Cell:
cellName: Cell01
JAAS:
alias1:
aliasName: j2cALiasA
aliasDesc: First j2cAlias
alias2:
aliasName: j2cALiasB
aliasDesc: Second j2cAlias
Node:
nodeName: Node01
JAASAuthData:
jdbcProviderType: ...
Server:
serverName: server-1
datasources:
datasource1:
datasourceName: jdbc/datasource1
datasourceAuthDataAlias:
Put it that way, it properly parses/loads under Jython 2.7.0 on Linux, with ruamel.yaml (disclaimer: I am the author of that package). You can install that package with pip install ruamel.yaml).