I have a pillar that looks like this:
inline_blocks:
the_seven: |
dog cat horse cow
ardvaark beatle snail
which I then want to insert into a file
{% set inline_block = pillar['inline_blocks'].get(val, '') %}
/etc/animals.conf:
file.managed:
- source: salt://farm/animals.conf
- user: root
- group: root
- mode: 644
- template: jinja
- defaults:
extras: {{ inline_block }}
and then in animals.conf,
{{ extras }}
I expect that if the key val is in inline_blocks, then its value will be interpolated in. If it's not, an empty string will be interpolated in.
Indeed, that's what happens if I write the defaults statement explicitly:
- defaults:
extras: |
dog cat horse cow
ardvaark beatle snail
but as written above, I get the error could not find expected ':'.
As a reality check, pillar.items happily retrieves the pillar entry, so (1) the pillar entry can be retrieved, and (2) the value can be interpolated, but (X) the multi-line value in the .sls file is causing problems.
Any pointers what the right syntax is to do this?
This issue was discussed in a bug as well, but for the content parameter. It seems to apply to multi-line YAML blocks passed as defaults or context also.
Since you are using a Jinja template file as a source, we can easily fetch pillar data from template itself (as one of the comments suggests in above link).
Considering pillar as:
inline_blocks:
val: |
dog cat horse cow
ardvaark beatle snail
Then the animals.conf.j2 template as:
{{ salt.pillar.get('inline_blocks:val', default="foo") }}
Note: If this pillar data is assured to be always defined, we might even use pillar['inline_blocks']['val'] in the template.
Rendered with a state like:
create-animals-conf:
file.managed:
- name: /tmp/animals.conf
- source: salt://animals.conf.j2
- mode: 0664
- template: jinja
Should yield the template as you expect:
$ cat /tmp/animals.conf
dog cat horse cow
ardvaark beatle snail
What happens is that
- defaults:
extras: {{ inline_block }}
is processed into
- defaults:
extras: dog cat horse cow
ardvaark beatle snail
So YAML tries to parse the second line as another top-level key. However, the : marking the end of this key never comes, hence the error.
To fix it, do this:
- defaults:
extras: |
{{ inline_block | indent(10) }}
indent doesn't indent the first line, but will add 10 spaces to every subsequent line.
Related
Who can tell you how to implement the output of all regions in the name1 group when entering a template named region1a, and when entering a template named region2b, output all regions from the name2 group
I implement it like this:
there is a task that starts template generation:
vars:
AllCountry:
- name1
- name2
name1:
- region1a
- region1b
name2:
- region2a
- region2b
tasks:
- name:
template:
src: "regions.j2"
dest: "{{ item }}.conf"
loop:
- region1a
- region2b
---regions.j2---
regions [{%for count in name1%} "my country = {{count}}", {%end for %}]
this gives the desired output, but only because it is explicitly specified for which name (1 or 2) to output
regions "my country = region1a", "my country = region1b"
For each value specified in the loop, a template configuration file must be generated.
When you specify values in loop region1a and region1b template should generate only one row in the configuration file for region1a.conf
regions "my country = region1a", "my country = region1b"
for region1b generate only one row in the configuration file for region1b.conf
regions "my country = region1a", "my country = region1b"
User β.εηοιτ.βε a more optimal structure was proposed. If convenient, you can use it.
vars:
countries:
country1:
regions:
- region1
- region2
- region3
capital: region1
country2:
regions:
- region4
- region5
capital: region5
Thank you all for your help. Still, I managed to figure it out myself.
Here is the final solution:
{% for country in AllCountry %}
{% if item in lookup('vars', country) %}{% for count in lookup('vars', country) %} "My country = {{ count }}"{% if not loop.last %},{% endif %}{% endfor %}{% endif %}{% endfor %}
I'm creating a custom yaml tag MyTag. It can contain any given valid yaml - map, scalar, anchor, sequence etc.
How do I implement class MyTag to model this tag so that ruamel parses the contents of a !mytag in exactly the same way as it would parse any given yaml? The MyTag instance just stores whatever the parsed result of the yaml contents is.
The following code works, and the asserts should should demonstrate exactly what it should do and they all pass.
But I'm not sure if it's working for the right reasons. . . Specifically in the from_yaml class method, is using commented_obj = constructor.construct_undefined(node) a recommended way of achieving this, and is consuming 1 and only 1 from the yielded generator correct? It's not just working by accident?
Should I instead be using something like construct_object, or construct_map or. . .? The examples I've been able to find tend to know what type it is constructing, so would either use construct_map or construct_sequence to pick which type of object to construct. In this case I effectively want to piggy-back of the usual/standard ruamel parsing for whatever unknown type there might be in there, and just store it in its own type.
import ruamel.yaml
from ruamel.yaml.comments import CommentedMap, CommentedSeq, TaggedScalar
class MyTag():
yaml_tag = '!mytag'
def __init__(self, value):
self.value = value
#classmethod
def from_yaml(cls, constructor, node):
commented_obj = constructor.construct_undefined(node)
flag = False
for data in commented_obj:
if flag:
raise AssertionError('should only be 1 thing in generator??')
flag = True
return cls(data)
with open('mytag-sample.yaml') as yaml_file:
yaml_parser = ruamel.yaml.YAML()
yaml_parser.register_class(MyTag)
yaml = yaml_parser.load(yaml_file)
custom_tag_with_list = yaml['root'][0]['arb']['k2']
assert type(custom_tag_with_list) is MyTag
assert type(custom_tag_with_list.value) is CommentedSeq
print(custom_tag_with_list.value)
standard_list = yaml['root'][0]['arb']['k3']
assert type(standard_list) is CommentedSeq
assert standard_list == custom_tag_with_list.value
custom_tag_with_map = yaml['root'][1]['arb']
assert type(custom_tag_with_map) is MyTag
assert type(custom_tag_with_map.value) is CommentedMap
print(custom_tag_with_map.value)
standard_map = yaml['root'][1]['arb_no_tag']
assert type(standard_map) is CommentedMap
assert standard_map == custom_tag_with_map.value
custom_tag_scalar = yaml['root'][2]
assert type(custom_tag_scalar) is MyTag
assert type(custom_tag_scalar.value) is TaggedScalar
standard_tag_scalar = yaml['root'][3]
assert type(standard_tag_scalar) is str
assert standard_tag_scalar == str(custom_tag_scalar.value)
And some sample yaml:
root:
- item: blah
arb:
k1: v1
k2: !mytag
- one
- two
- three-k1: three-v1
three-k2: three-v2
three-k3: 123 # arb comment
three-k4:
- a
- b
- True
k3:
- one
- two
- three-k1: three-v1
three-k2: three-v2
three-k3: 123 # arb comment
three-k4:
- a
- b
- True
- item: argh
arb: !mytag
k1: v1
k2: 123
# blah line 1
# blah line 2
k3:
k31: v31
k32:
- False
- string here
- 321
arb_no_tag:
k1: v1
k2: 123
# blah line 1
# blah line 2
k3:
k31: v31
k32:
- False
- string here
- 321
- !mytag plain scalar
- plain scalar
- item: no comment
arb:
- one1
- two2
In YAML you can have anchors and aliases, and it is perfectly fine to have an object be a child of itself (using an alias). If you want to dump the Python data structure data:
data = [1, 2, 4, dict(a=42)]
data[3]['b'] = data
it dumps to:
&id001
- 1
- 2
- 4
- a: 42
b: *id001
and for that anchors and aliases are necessary.
When loading such a construct, ruamel.yaml recurses into the nested data structures, but if the toplevel node has not caused a real object to be constructed to which the anchor can be made a reference, the recursive leaf cannot resolve the alias.
To solve that, a generator is used, except for scalar values. It first creates an empty object, then recurses and updates it values. In code calling the constructor a check is made to see if a generator is returned, and in that case next() is done on the data, and potential self-recursion "resolved".
Because you call construct_undefined(), you always get a generator. Practically that method could return a value if it detects a scalar node (which of course cannot recurse), but it doesn't. If it would, your code could then not load the following YAML document:
!mytag 1
without modifications that test if you get a generator or not, as is done in the code in ruamel.yaml calling the various constructors so it can handle both construct_undefined and e.g. construct_yaml_int (which is not a generator).
I have to loop through a template and create individual files from it in a systematic manner. For example..
Template sample (eg shop_names.conf.j2)
{% for item in shop_names_csv_list %}
# Delete the id
DELETE root/{{item.shop_id}}
# Create the id with contents
PUT root/{{item.shop_id}}
{
"shop_name": "{{item.shop_name}}"
"shop_state": "{{item.shop_state}}"
}
{% endfor %}
I've a CSV in below format which contains data to populate the template
shop_id,shop_name,shop_state
101,Walmart,NY
102,Cosco,MT
103,Apple,DC
I was able to loop through using Ansible and fill it up the template correctly and outputs as shop_names.conf (single file)
The logic I'm using currently is:
- read_csv:
path: shop_names.csv
register: shop_names_csv_list
- name: "Build shop name API endpoints from Template"
template:
src: "{{ item }}"
dest: "{{buildDir}}/{{ item | replace('.j2', '')}}"
with_lines:
- "find {{base_dir_template}} -type f -name '*.j2'"
So currently my output comes as
DELETE root/101
PUT root/101
{
"shop_name": "Walmart"
"shop_state": "NY"
}
# ------- Next Line ------ #
DELETE root/102
PUT root/102
{
"shop_name": "Cosco"
"shop_state": "MT"
}
# ------- Next Line ------ #
DELETE root/103
PUT root/103
{
"shop_name": "Apple"
"shop_state": "DC"
}
# ------- Next Line ------ #
But I was looking to output it as individual files in format of: {{shop_id}}.{{shop_names}}.conf , so that is each DELETE & PUT into single document each per shop_id
So the output, i'm looking for is 3 files namely
101.Walmart.conf
DELETE root/101
PUT root/101
{
"shop_name": "Walmart"
"shop_state": "NY"
}
102.Cosco.conf
DELETE root/102
PUT root/102
{
"shop_name": "Cosco"
"shop_state": "MT"
}
and so on...
Any help to create the template as individual files would be appreciated.
I am using the YAML heading of a markdown file to add an excerpt variable to blog posts that I can use elsewhere. In one of these excerpts I refer to an earlier blog post via markdown link markup, and I use the liquid template data variable {{ site.url }} in place of the base URL of the site.
So I have something like (trimmed it somewhat)
---
title: "Decluttering ordination plots in vegan part 2: orditorp()"
status: publish
layout: post
published: true
tags:
- tag1
- tag2
excerpt: In the [earlier post in this series]({{ site.url }}/2013/01/12/
decluttering-ordination-plots-in-vegan-part-1-ordilabel/ "Decluttering ordination
plots in vegan part 1: ordilabel()") I looked at the `ordilabel()` function
----
However, jekyll and the Maruku md parser don't like this, which makes me suspect that you can't use liquid markup in the YAML header.
Is it possible to use liquid markup in the YAML header of pages handled by jekyll?
If it is, what I am I doing wrong in the example shown?
If it is not allowed, who else can I achieve what I intended? I am currently developing my site on my laptop and don't want to hard code the base URL as it'll have to change when I am ready to deploy.
The errors I am getting from Maruku are:
| Maruku tells you:
+---------------------------------------------------------------------------
| Must quote title
| ---------------------------------------------------------------------------
| the [earlier post in this series]({{ site.url }}/2013/01/12/decluttering-o
| --------------------------------------|-------------------------------------
| +--- Byte 40
and
| Maruku tells you:
+---------------------------------------------------------------------------
| Unclosed link
| ---------------------------------------------------------------------------
| the [earlier post in this series]({{ site.url }}/2013/01/12/decluttering-or
| --------------------------------------|-------------------------------------
| +--- Byte 41
and
| Maruku tells you:
+---------------------------------------------------------------------------
| No closing ): I will not create the link for ["earlier post in this series"]
| ---------------------------------------------------------------------------
| the [earlier post in this series]({{ site.url }}/2013/01/12/decluttering-or
| --------------------------------------|-------------------------------------
| +--- Byte 41
Today I ran into a similar problem. As a solution I created the following simple Jekyll filter-plugin which allows to expand nested liquid-templates in (e.g. liquid-variables in the YAML front matter):
module Jekyll
module LiquifyFilter
def liquify(input)
Liquid::Template.parse(input).render(#context)
end
end
end
Liquid::Template.register_filter(Jekyll::LiquifyFilter)
Filters can be added to a Jekyll site by placing them in the '_plugins' sub-directory of the site-root dir. The above code can be simply pasted into a yoursite/_plugins/liquify_filter.rb file.
After that a template like...
---
layout: default
first_name: Harry
last_name: Potter
greetings: Greetings {{ page.first_name }} {{ page.last_name }}!
---
{{ page.greetings | liquify }}
... should render some output like "Greetings Harry Potter!". The expansion works also for deeper nested structures - as long as the liquify filter is also specified on the inner liquid output-blocks. Something like {{ site.url }} works of course, too.
Update - looks like this is now available as a Ruby gem: https://github.com/gemfarmer/jekyll-liquify.
I don't believe it's possible to nest liquid variables inside YAML. At least, I haven't figure out how to do it.
One approach that will work is to use a Liquid's replace filter. Specifically, define a string that you want to use for the variable replacement (e.g. !SITE_URL!). Then, use the replace filter to switch that to your desired Jekyll variable (e.g. site.url) during the output. Here's a cut down .md file that behaves as expected on my jekyll 0.11 install:
---
layout: post
excerpt: In the [earlier post in this series](!SITE_URL!/2013/01/12/)
---
{{ page.excerpt | replace: '!SITE_URL!', site.url }}
Testing that on my machine, the URL is inserted properly and then translated from markdown into an HTML link as expected. If you have more than one item to replace, you can string multiple replace calls together.
---
layout: post
my_name: Alan W. Smith
multi_replace_test: 'Name: !PAGE_MY_NAME! - Site: [!SITE_URL!](!SITE_URL!)'
---
{{ page.multi_replace_test | replace: '!SITE_URL!', site.url | replace: '!PAGE_MY_NAME!', page.my_name }}
An important note is that you must explicitly set the site.url value. You don't get that for free with Jekyll. You can either set it in your _config.yml file with:
url: http://alanwsmith.com
Or, define it when you call jekyll:
jekyll --url http://alanwsmith.com
If you need to replace values in data/yml from another data/yml file, I wrote plugin. It's not so elegant but works :
I did some code improvements. Now it catch all occurrences in one string and work with nested values.
module LiquidReplacer
class Generator < Jekyll::Generator
REGEX = /\!([A-Za-z0-9]|_|\.){1,}\!/
def replace_str(str)
out = str
str.to_s.to_enum(:scan, REGEX).map {
m = Regexp.last_match.to_s
val = m.gsub('!', '').split('.')
vv = $site_data[val[0]]
val.delete_at(0)
val.length.times.with_index do |i|
if val.nil? || val[i].nil? || vv.nil? ||vv[val[i]].nil?
puts "ERROR IN BUILDING YAML WITH KEY:\n#{m}"
else
vv = vv[val[i]]
end
end
out = out.gsub(m, vv)
}
out
end
def deeper(in_hash)
if in_hash.class == Hash || in_hash.class == Array
_in_hash = in_hash.to_a
_out_hash = {}
_in_hash.each do |dd|
case dd
when Hash
_dd = dd.to_a
_out_hash[_dd[0]] = deeper(_dd[1])
when Array
_out_hash[dd[0]] = deeper(dd[1])
else
_out_hash = replace_str(dd)
end
end
else
_out_hash = replace_str(in_hash)
end
return _out_hash
end
def generate(site)
$site_data = site.data
site.data.each do |data|
site.data[data[0]] = deeper(data[1])
end
end
end
end
place this code in site/_plugins/liquid_replacer.rb
in yml file use !something.someval! like as site.data.something.someval but without site.data part.
example :
_data/one.yml
foo: foo
_data/two.yml
bar: "!one.foo!bar"
calling {{ site.data.two.bar }} will produce foobar
=======
OLD CODE
======
module LiquidReplacer
class Generator < Jekyll::Generator
REGEX = /\!([A-Za-z0-9]|_|\.){1,}\!/
def generate(site)
site.data.each do |d|
d[1].each_pair do |k,v|
v.to_s.match(REGEX) do |m|
val = m[0].gsub('!', '').split('.')
vv = site.data[val[0]]
val.delete_at(0)
val.length.times.with_index do |i|
vv = vv[val[i]]
end
d[1][k] = d[1][k].gsub(m[0], vv)
end
end
end
end
end
end
Another approach would be to add an IF statement to your head.html.
Instead of using page.layout like I did on my example below, you could use any variable from the page YAML header.
<title>
{% if page.layout == 'post' %}
Some text with {{ site.url }} variable
{% else %}
{{ site.description | escape }}
{% endif %}
</title>
I have this YAML:
- company:
- id: toyota
- fullname: トヨタ自動車株式会社
- company:
- id: konami
- fullname: Konami Corporation
And I want to get the fullname of the company whose id is konami.
Using Ruby 1.9.2, what is the simplest/usual way to get it?
Note: In the rest of my code, I have been using require "yaml" so I would prefer to use the same library.
This works too and does not use iteration:
y = YAML.load_file('japanese_companies.yml')
result = y.select{ |x| x['company'].first['id'] == 'konami' }
result.first['company'].last['fullname'] # => "Konami Corporation"
Or if you have other attributes and you can't be sure fullname is the last one:
result.first['company'].select{ |x| x['fullname'] }.first['fullname']
I agree with Ray Toal, if you change your yml it becomes much easier. E.g.:
toyota:
fullname: トヨタ自動車株式会社
konami:
fullname: Konami Corporation
With the above yaml, fetching the fullname of konami becomes much easier:
y = YAML.load_file('test.yml')
y.fetch('konami')['fullname']
Your YAML is a little unconventional but we can compensate.
A brute force approach is (I'm not sure if this can be done without parsing the YAML):
require 'yaml'
YAML.parse_file(ARGV[0]).transform.each do |company|
properties = {}
company['company'].each {|h| properties = properties.merge(h)}
puts properties['fullname'] if properties['id'] == 'konami'
end
Pass your YAML file in as the first argument to this script.
Feel free to adapt into a method that takes the YAML as a string and returns the desired fullname. (A return is useful because it directly answers the OP's question of obtaining the first such company.)