Scenario
Note: I am using the Java language for this effort.
Consider the following scenario. In a directory hierarchy, there are files that contain "template definitions". These definitions are then "included" by other files containing embedded templating logic. Here are some examples for reference.
A template definition:
_helpers.tpl:
...
{{- define "keycloak.selectorLabels" -}}
app.kubernetes.io/name: {{ include "keycloak.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}
...
The details of the defintion are not important. I only care about the existence of the definition, i.e. it's name ("keycloak.selectorLabels") and where it's found (line N of _helpers.tpl).
(Note that the definition includes another defintion.)
A templated file that includes the definition:
service-http.yaml:
...
selector:
{{- include "keycloak.selectorLabels" . | nindent 4 }}
...
Definitions can be included by other definitions, in the same or other template files.
My objective
I want to scan these files to find 1) the definitions and 2) where they are included. I have working logic that collects this information now.
It would then output listings of the following information:
for each:
definition (the containing file and definition "metadata" - name and line found):
list of templated files that include the defintion (file and line(s) where included)
-or- indication that NO references were found for this definition
for each:
templated file:
list of definition inclusions (definition "metadata" and line(s) where included)
What I need help with
I'm trying to find or figure out an algorithm to do this "cross referencing" described above. I'm sure that with some trial and error I could come up with something that works (no saying how long it'll take or how well it would work), but am wondering if anyone has ideas or information about existing algorithms that do this kind of thing.
I tried to see if I could Google something, but not really sure what search terms to use. The things I did try didn't help.
Related
I've recently started dealing with CF templates in my org what uses the following syntax.
metadata:
name: {{{MY_APP}}}-resources
Resources:
DBInstance:
Properties:
DBName: "{{{DB_NAME}}}_db_{{{NAMESPACE}}}"
EngineVersion: "{{{DB_ENGINE_VERSION}}}"
...
The variables are passed via a delivery.yaml . And the intention and operation works. I find the syntax confusing though and couldn't find a reference on the yaml spec.
VSCode RedHat's yaml extension complains about this line (and many other similar ones across the files):
name: {{{MY_APP}}}-resources
with the message:
Unexpected scalar at node end
My understanding so far after looking online is that the complain is due to the - symbol. It can cause ambiguity since if these values would come as numbers it could be missinterpreted as an operation. Several alternatives seems to satisfy the linter:
'{{{MY_APP}}}-resources'
${MY_APP}-resources
etc, etc
So a couple of questions:
what's up with the triple brackets? is that just for visuals or does it do sth different than single brackets.
What's the recommended way to do this given the AWS CF context?
very noob question, I have this piece of ansible code in /vars/myvars.yml:
service: "{{ 'service1' if litespeed|bool==true else 'service2' }}"
services_node:
- foo
- bar
- "{{ service }}"
- baz
i have tried to look online for my answer but cannot find anything, what syntax is in the curly braces? Searching online results in "there is no if/else in ansible or YAML" in most cases. To what kind of query or language that syntax is from?
Ansible uses Jinja for templating. You should read ansible templating doc.
You'll also find a lot of useful informations on Jinja's template designer doc.
In your example, the syntax with the pipe (the | symbol) is the syntax for filters. Here you'll find the list of builtin filters with documentation and examples.
You'll see that the bool filter is not listed as it is a filter brought by ansible. You'll find the list of these filters here.
The template itself is not part of Yaml, that's why it is expressed as a string.
The braces are part of a Jinja2 template within the YAML data. See Templating (Jinja2) for complete information.
I want to edit the configuration file of telegraf(system metrics collecting agent).
Telegraf comes in with a default config file which can be edited. There are many input and output plugins defined in there, which are commented out and can be added by removing the comments and also be customized.
I want to edit only some of the plugins defined there, not all of them. For example, consider this is the file,
[global]
interval='10s'
[outputs.influxdb]
host=['http://localhost:8086']
#[outputs.elasticsearch]
# host=['http://localhost:9200']
[inputs.netstat]
interface='eth0'
Now, I want to edit the 3 blocks, global, outputs.influxdb and inputs.netstat. I don't want to edit outputs.elasticsearch but also want that the block outputs.elasticsearch should remain in the file.
When Using Ansible, I firstly used Template module, but if I use that, then the commented data would be lost.
Then I used the ini_file module, instead of editing the already present block, it adds a new block even if it is already present, and results in something like this,
[outputs.influxdb]
host=[http://localhost:8086]
[outputs.influxdb]
host=[http://xx.xx.xx.xx:8086]
Which module is ideal for my scenario ?
There are several options, depending on your purpose.
The lineinfile - module is the best option, if you just want to add, replace or remove one line.
The replace - module is best, if you want to add, replace or delete several lines.
The blockinfile - module can add several lines, surrounded by markers.
If you only want to change two or three lines, you could use as many calls of lineinfile. To change a whole config file, I would recommend, like the commenters suggest, use the template - module.
Ok, if you really really want to avoid using templates, you could try to use replace and a regex like this:
- hosts: local
tasks:
- replace:
path: testfile
regexp: '^\[{{ item.category }}\]\s(.*)host(.*)$'
replace: '[{{ item.category }}]\n host=[{{ item.host }}]'
with_items:
- { category: 'outputs.influxdb', host: 'http://cake.com:8080' }
This, in its current form, would not necessarily handle more than one option under each category, but the regex can be modified to handle multiple lines.
As required, it will not touch the # commented lines. However, if you decide to enable some of the previously inactive sections, you might end up with a slightly messier configuration file that would include the instructions both commented and uncommented (shouldn't impact functionality, only 'looks'). You will also need to account for options that look like the example below (interleaved commented/uncommented values) and create regexes specially for those use-cases:
[section]
option1=['value']
# option2=['value']
option3=['value']
It highly depends on your use-case, but my recommendation remains that templates are to be used instead, as they are a more robust approach, with less chances of things going wrong.
Goal
Make Jekyll post have their own category according to directory structure.
e.g.) _post/category1/category2/post-content.md must have categories of ['category1', 'category2'](I guess Jekyll deals with categories as an array).
The point is, making 1) no need to specify category in Front Matter and 2) auto categorize every posts by directory structure .
Description
Official Jekyll document says Categories are derived from the directory structure above the _post directory. For example, a post at /work/code/_posts/2008-12-24-closures.md would have this field set to ['work', 'code'].
https://jekyllrb.com/docs/variables/
If it is possible, I would joyfully use this function. The problem is, I have no idea how to make _post dir to be located in other directories. I thought _post dir have to be in root dir, so for me positioning post in path like /work/code/_posts/2008-12-24-closures.md seems impossible. Is it possible? or not possible?
If 1. is impossible, how can I match categories and dir-struct for every post? I don't want to do it manually for every posts.
I tried to use {{ page.path }} to make it automated, but 2 problem I faced. 1) not sure Liquid(template engine) support full ruby functionality like File.basename for parsing. 2) not sure if codes like parsing can be done in Front Matter. For instance, I tried categories: {{ page.path | array_to_sentence_string }} in Front Matter but it ended up in an error.
any solution for these issues? Or there is no way?
You're following the directions properly: you can use either folder structure, though the latter is preferred.
The problem is that the special directory is _posts PLURAL, not _post singular.
I have the following defined in my Jekyll config matter:
dir: a-directory/
I now want to have:
dir: a-directory/
images: {{ dir }}images/
However this won't work. One solution is to place this in my template file
{% capture images %}{{ site.dir }}images/{% endcapture %}
The variable images is now available to other points in that file. However it isn't available to any content being compiled in with that file, e.g my actual pages.
Doing {% capture site.images %} would seem the way to sort that, but you can't assign items to the site or page globals outside of the _config and front matter respectively.
Is it possible to achive this kind of global variable stacking?
(please avoid solutions involving changing my directory structure; if there are similar compilers offering more features without a huge change to workflow I'm open)
It seems YAML doesn't directly support concatenation. There are a few workarounds, though:
https://stackoverflow.com/a/23212524
https://stackoverflow.com/a/22236941