I am getting the below warning while reading the lineup file
[WARNING]: While constructing a mapping from True, line 1, column 1, found a duplicate dict key (release). Using last defined value
only.
Code piece is:
- name: Read in Lineup File
include_vars:
file: "{{ lineup_file }}"
name: lineup
What I want to do is exit with an error instead of warning if duplicate keys are found in YAML file.
Is there any way to do that?
IFAIK there's no configuration setting or something like this.
But you can teach Ansible to do whatever you want with a plugins.
Drop this into project/callback_plugins/catch_dup.py:
from ansible.plugins.callback import CallbackBase
from ansible.errors import AnsibleError
try:
from __main__ import display
except ImportError:
display = None
class CallbackModule(CallbackBase):
def __init__(self, *args, **kwargs):
def catch_dup(msg, formatted=False):
display.warn_original(msg, formatted=False)
if 'found a duplicate dict key' in msg:
raise AnsibleError("FATAL ERROR: Duplicate key!")
display.warn_original = display.warning
display.warning = catch_dup
This plugin overrides display.warning method with catch_dup one, where we check for specific warning message and fail if duplicate key warning is there.
Related
I have a custom module that resides in the library/ directory of my Ansible role. I can call the module from within my playbook, and the code executes correctly, but only if the values it expects are hardcoded in the module code itself. How can I pass values to the module from the playbook?
I've tried the following:
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "new-ecr-repo"
encryptionConfiguration.kmsKey: 'my-kms-key-id"
and
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "{{ repo_name }}"
encryptionConfiguration.kmsKey: "{{ kms_key_id }}"
Which I would expect to work, but neither does and, I get the following errors:
botocore.exceptions.ParamValidationError: Parameter validation failed:
Invalid length for parameter repositoryName, value: 0, valid min length: 2
Invalid length for parameter encryptionConfiguration.kmsKey, value: 0, valid min length: 1
The service module I'm trying to use
The code of the custom module:
#!/usr/bin/python
from urllib import response
import boto3
from jinja2 import Template
from ansible.module_utils.basic import AnsibleModule
def create_repo():
client = boto3.client('ecr')
response = client.create_repository(
#registryId='',
repositoryName='',
imageTagMutability='IMMUTABLE',
imageScanningConfiguration={
'scanOnPush': True
},
encryptionConfiguration={
'encryptionType': 'KMS',
'kmsKey': ""
}
)
def main():
create_repo()
if __name__ in '__main__':
main()
You do need to make your module aware of the arguments you want it to accept, so, in your main function:
#!/usr/bin/env python
from ansible.module_utils.basic import AnsibleModule
def create_repo(repositoryName, kmsKey):
# Call to the API comes here
def main():
module = AnsibleModule(
argument_spec = dict(
repositoryName = dict(type = 'str', required = True),
kmsKey = dict(type = 'str', required = True),
)
)
params = module.params
create_repo(
params['repositoryName'],
params['kmsKey']
)
if __name__ == '__main__':
main()
More can be found in the relevant documentation: Argument spec.
With this, your taks would be:
- name: Create repo and use specific KMS key
ecr_kms:
repositoryName: "{{ repo_name }}"
kmsKey: "{{ kms_key_id }}"
PS, word of advice: avoid using a dot in a YAML key, that would just be making your life complicated for no actual good reason.
I am using python to write in to a YAML file. But I am unable to write it in a specific format as I require
awesome_people:
name: Awesome People
entities:
- device_tracker.dad_smith
- device_tracker.mom_smith
I am getting problem in the entities part, as I am unable to create a list with proper indents as in the above YAML.
How can I create the above exact format?
You can do this with ruamel.yaml, setting the sequence indent from default two to four,
and the offset of the dash+space (which need a minimum indent of two), from zero to two.
You should leave the mapping indent unchanged from the default:
import sys
import ruamel.yaml
ent = ['device_tracker.dad_smith', 'device_tracker.mom_smith']
data = dict(awesome_people = dict(name='Awesome People', entities=ent))
yaml_str = """\
"""
yaml = ruamel.yaml.YAML()
yaml.indent(sequence=4, offset=2)
# print(data)
yaml.dump(data, sys.stdout)
which gives:
awesome_people:
name: Awesome People
entities:
- device_tracker.dad_smith
- device_tracker.mom_smith
When a GraphQL Error occurs, I cannot easily know where it occured. I have to spend unnecessary time trying to track it down. How do I get a Traceback printed in the console of my text editor?
I answered my own question by accessing the GraphQL error(s) with result.errors, iterating through the list, and using python's print_tb function to print the Traceback.
Does anyone have a different or better way of doing it?
Example usage of the print_graphql_errors function:
from django.conf.settings import DEBUG
result = schema.execute(
mutation_str, context_value=request, variable_values=variable_values
)
if result.errors is None:
return self.handle_success(result)
if DEBUG:
print_graphql_errors(result.errors)
return self.handle_failure(result)
The print_graphql_errors function:
from traceback import print_tb
from django.conf.settings import DEBUG
def print_graphql_errors(errors, raise_error=False):
if not DEBUG:
raise Exception(
'DevError: This function should not be called in production'
)
assert errors, 'DevError: The "errors" parameter cannot be None'
print_after = []
current_error = None
print('######################################################################')
print('Manually Generated Traceback (with the print_graphql_errors function):')
print(f'There are {len(errors)} errors:')
for i, error in enumerate(errors):
print(f'{i + 1}) ', error)
print('######################################################################')
for error in errors:
current_error = error
# FYI: This object has these attributes: (example attribute values)
# tb_frame <frame at 0x000002DDB844D548, file 'site-packages\\graphql\\execution\\executor.py', line 455, code resolve_or_error>
# tb_lasti 16
# tb_lineno 447
# tb_next <traceback object at 0x000002DDBAFBA388>
# print('error.locations:', error.locations)
# print('error.positions:', error.positions)
try:
print_tb(error.stack)
except AttributeError as e:
print(e.__traceback__)
print(f'Warning: An error occured while trying to print the traceback: {e}. It has the following attributes instead: {dir(error)}')
print_after.append(error)
if len(print_after):
print('###################################################################')
print(f'Number of errors without the "stack" attribute: {len(print_after)}')
print('###################################################################')
if raise_error:
for error in print_after:
raise error
raise current_error
I need to remove the tags field from each of the methods in my OpenAPI spec.
The spec must be in YAML format, as converting to JSON causes issues later on when publishing.
I couldn't find a ready tool for that, and my programming skills are insufficient. I tried Python with ruamel.yaml, but could not achieve anything.
I'm open to any suggestions how to approach this - a repo with a ready tool somewhere, a hint on what to try in Python... I'm out of my own ideas.
Maybe a regex that catches all cases all instances of tags so I can do a search and replace with Python, replacing them with nothing? Empty lines don't seem to break the publishing engine.
Here's a sample YAML piece (I know this is not a proper spec, just want to show where in the YAML tags sits)
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint
get:
tags:
-
tag1
-
tag3
#rest of GET definition
post:
tags:
- tag2
/anotherEndpoint
post:
tags:
- tag1
I need to get rid of all tags arrays entirely (not just make them empty)
I am not sure why you couldn't achieve anything with Python + ruamel.yaml. Assuing your spec
is in a file input.yaml:
import sys
from pathlib import Path
import ruamel.yaml
in_file = Path('input.yaml')
out_file = Path('output.yaml')
yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=2)
yaml.preserve_quotes = True
data = yaml.load(in_file)
# if you only have the three instances of 'tags', you can hard-code them:
# del data['paths']['/endpoint']['get']['tags']
# del data['paths']['/endpoint']['post']['tags']
# del data['paths']['/anotherEndpoint']['post']['tags']
# generic recursive removal of any key names 'tags' in the datastructure:
def rm_tags(d):
if isinstance(d, dict):
if 'tags' in d:
del d['tags']
for k in d:
rm_tags(d[k])
elif isinstance(d, list):
for item in d:
rm_tags(item)
rm_tags(data)
yaml.dump(data, out_file)
which gives as output.yaml:
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint:
get: {}
post: {}
/anotherEndpoint:
post: {}
You can write back data to input.yaml if you need that.
Please note that normally the comment #rest of GET definition would be preserved, but
not now as it is associated during loading with the key before it and that key gets deleted.
I have yaml data like the input below and i need output as key value pairs
Input
a="""
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
code:
- '716'
- '718'
id:
- 488
- 499
"""
ouput needed
{'code': ['716', '718'], 'id': [488, 499]}
The default constructor was giving me an error. I tried adding new constructor and now its not giving me error but i am not able to get key value pairs.
FYI, If i remove the !ruby/hash:ActiveSupport::HashWithIndifferentAccess line from my yaml then it gives me desired output.
def new_constructor(loader, tag_suffix, node):
if type(node.value)=='list':
val=''.join(node.value)
else:
val=node.value
val=node.value
ret_val="""
{0}
""".format(val)
return ret_val
yaml.add_multi_constructor('', new_constructor)
yaml.load(a)
output
"\n [(ScalarNode(tag=u'tag:yaml.org,2002:str', value=u'code'), SequenceNode(tag=u'tag:yaml.org,2002:seq', value=[ScalarNode(tag=u'tag:yaml.org,2002:str', value=u'716'), ScalarNode(tag=u'tag:yaml.org,2002:str', value=u'718')])), (ScalarNode(tag=u'tag:yaml.org,2002:str', value=u'id'), SequenceNode(tag=u'tag:yaml.org,2002:seq', value=[ScalarNode(tag=u'tag:yaml.org,2002:int', value=u'488'), ScalarNode(tag=u'tag:yaml.org,2002:int', value=u'499')]))]\n "
Please suggest.
This is not a solution using PyYAML, but I recommend using ruamel.yaml instead. If for no other reason, it's more actively maintained than PyYAML. A quote from the overview
Many of the bugs filed against PyYAML, but that were never acted upon, have been fixed in ruamel.yaml
To load that string, you can do
import ruamel.yaml
parser = ruamel.yaml.YAML()
obj = parser.load(a) # as defined above.
I strongly recommend following #Andrew F answer, but in case you
wonder why your code did not get the proper result, that is because
you don't correctly process the node under the tag in your tag
handling.
Although the node's value is a list (of tuples with key value pairs),
you should test for the type of the node itself (using isinstance)
and then hand it over to the "normal" mapping processing routine as
the tag is on a mapping:
import yaml
from yaml.loader import SafeLoader
a = """\
--- !ruby/hash:ActiveSupport::HashWithIndifferentAccess
code:
- '716'
- '718'
id:
- 488
- 499
"""
def new_constructor(loader, tag_suffix, node):
if isinstance(node, yaml.nodes.MappingNode):
return loader.construct_mapping(node, deep=True)
raise NotImplementedError
yaml.add_multi_constructor('', new_constructor, Loader=SafeLoader)
data = yaml.load(a, Loader=SafeLoader)
print(data)
which gives:
{'code': ['716', '718'], 'id': [488, 499]}
You should not use PyYAML's yaml.load(), it is documented to be potentially unsafe
and above all it is not necessary. Just add the new constructor to the SafeLoader.