I am working on designing a puppet architecture for our company. I really like the idea of hiera and YAML files to classify my nodes. However, I would really like to be able to either apply YAML files that aren't based on facts or import YAML files into another YAML file.
For example NodeA.yaml
---
include webserver.yaml
include public.yaml
classes:
etc. . .
This would allow me to reuse my code as much as possible. That way when I make a change to my web server configs, I only have to do it in one file instead of every node's YAML file.
I'm open to other solutions as well.
YAML does not support import or include
(not recomended) You can use loadyaml from stdlib module to achieve desired functionality. Check this example of using loadyaml function.
You can easily achieve expected functionality just by designing a proper hiera hierarchy. I do not understand why you do not want to use facter facts? e.g: on each node define custom facter fact location. Next define a hiera hierarchy:
:hierarchy:
- "%{::location}"/"%{::fqdn}"
- "%{::location}"/common
- common
Next in file location_1/node1.yaml you define data specific only for that node1. In file location_1/common.yaml you define data common for all nodes in location_1. In common.yaml you define data common for all nodes.
If some data is common for all nodes you define it once in common.yaml and that's all. You do not have to redundantly define it in every node's yaml file.
Related
We have a pretty old project where we have a lot of custom attribute definitions but do not trust our sites/site_template/meta/system-objecttype-extensions.xml to be complete.
Does anybody know of a way to remove a defined set of custom attribute definitions automatically? We can identify quite a lot that are definitely not used anymore but I'm missing the DELETE import functionality available for other types of data in SFCC.
Tx a lot for your help!
What to do, so I wrote my own parser / analyser / merger: https://github.com/Andreas-Schoenefeldt/SFCCAnalyser#combine-and-analyse-system-objecttype-extension-xml-files
It can be used like this:
put all xml files you would like to merge into one in ./data/meta-xmls/
run node combine-xml.js
the resulting merged and cleaned xml file will be in ./data/meta-xmls/combined.xml
The resulting file will contain all custom attribute definitions accross the systems, so it can then safely imported checking the Delete existing attribute definitions and attribute groups not contained in the import file, including attribute definitions marked as "externally-defined".
In my project i'm currently using application.yml for configuration. Spring Initializr generate application.properties? What are the Pro/Cons for each one?
Well, they are just different data formats. Which one's nicer and easier to read? That's obviously subjective. Here's a useful blog post.
As far as spring-boot configuration is concerned, note that there's only one documented shortcoming of using YAML. Per the documentation:
YAML files can’t be loaded via the #PropertySource annotation. So in the case that you need to load values that way, you need to use a properties file.
As per my knowledge, these are at least some of the differences:
.properties stores data in sequential format, whereas
.yml stores data in hierarchical format.
.properties supports only key-value pairs (basically string values), whereas
.yml supports key-value pair, as well as map, list & scalar type values.
.properties is specifically used by Java, whereas
.yml can be used by other languages (eg Java, Python, ROR, etc).
When managing multiple configuration profiles, then:
.properties requires you to create .properties file per every profile, whereas in
.yml you can create a section for each specific profile inside a single .yml file.
In Spring projects, #PropertySource annotation can only be used with .properties.
One notable difference is how the properties are represented in each file. YAML files may use consistent spaces to denote hierarchy whereas properties file may use = to denote property values.
For ex.
Lists are represented hierarchically in YAML:
headers:
- user-agent
- x-wag-diagonalsize
Lists may be represented as inline list (separated by commas) in a properties file:
headers = user-agent, x-wag-diagonalsize
Another difference is we can add multiple configuration files into single yaml file.
For ex., we can add application.yaml(application specific properties) and bootstrap.yaml(server specific properties) into single config.yaml file
How can I create a recipe that will populate its attributes using the fiels from an instance of an object in a generic way?
As an example, consider the following recipe:
component = $auth_docker
docker_image component.name do
registry component.registry
tag component.tag
action :pull
end
When you have 50s of recipes that look like this, maintaining them really gets overwhelming.
In Python, i would probably implement a solution that would look a bit like this:
docker_image = DockerImage(**$auth_docker)
Or, I would create some sort of helper function to build it for me:
def generate_docker_image_lwrp(attributes):
lwrp = DockerImage()
lwrp.registry = attributes.registry
lwrp.tag = attributes.tag
return lwrp
The goal is to reduce maintenance on the recipes. For instance this morning I wanted to add Chef's "retries" attribute on all recipes that pull an image. I had to edit all of them - I don't want that. I should've been able to a) add the attribute to the stack's JSON b) edit the Ruby wrapper class so that instances of it (i.e.: $auth_docker) get the "retries" field, then c) add the retries attribute to the lwrp-generator. Since all recipes would use the same generator, recipes wouldn't need to be edited at all.
Is this possible using Chef, in a way that 'notifies' still work?
Quoting the Documentation
A definition is code that is reused across recipes, similar to a
compile-time macro. A definition is created using arbitrary code
wrapped around built-in chef-client resources—file, execute, template,
and so on—by declaring those resources into the definition as if they
were declared in a recipe. A definition is then used in one (or more)
recipes as if it were a resource.
Though a definition behaves like a resource, some key differences
exist. A definition:
Is not a resource or a lightweight resource Is defined from within the
/definitions directory of a cookbook Is loaded before resources during
the chef-client run; this ensures the definition is available to all
of the resources that may need it May not notify resources in the
resource collection because a definition is loaded before the resource
collection itself is created; however, a resource in a definition may
notify a resource that exists within the same definition Automatically
supports why-run mode, unlike lightweight resources Use a defintion
when repeating patterns exist across resources and/or when a simple,
direct approach is desired. There is no limit to the number of
resources that may be included in a definition: use as many built-in
chef-client resources as necessary.
I.e: you can create a definition for this in a library cookbook used solely for this.
docker_library/defintions/default.rb
define :my_docker_image, :component => nil do
component = params[:component]
docker_image component.name do
registry component.registry
tag component.tag
action :pull
end
end
in your recipes (need to have a depends on the docker_library cookbook in the metadata.rb):
my_component = $auth_docker
my_docker_image my_component.name do
component my_component
end
A more complete exemple of definition is available in the logrotate cookbook
We have a spring-boot project and are using application.yml files. This works exactly as described in the spring-boot documentation. spring-boot automatically looks in several locations for the files, and obeys any environment overrides we use for the location of those files.
Now we want to also expose those yaml properties as a Map. According to the documentation this can be done with YamlMapFactoryBean. However YamlMapFactoryBean wants me to specify which yaml files to use via the resources property. I want it to use the same yaml files and processing hierarchy that it used when creating properties, so that I can take still take advantage of "magical" features such as placeholder resolution in property values.
I didn't see any documentation on if this was possible.
I was thinking of writing a MapFactoryBean that looked at the environment and simply reversed the "flattening" performed by the YamlProcessor when creating the properties representation of the file.
Any other ideas?
The ConfigFileApplicationContextListener contains the logic for searching for files in various locations. And PropertySourcesLoader loads a file (Resource) into property sources. Neither is really designed for standalone use, but you could easily duplicate them if you want more control. The PropertySourcesLoader delegates to a collection of PropertySourceLoaders so you could add one of the latter that delegates to your YamlMapFactoryBean.
A slightly awkward but workable solution would be to use the existing machinery to collect the YAML on startup. Add a new PropertySourceLoader to your META-INF/spring.factories and let it create new property sources, then post process the Environment to extract the source map(s).
Beware, though: creating a single Map from multiple YAML files, or even a single one with multiple documents (let alone multiple files with multiple documents) isn't as easy as you might think. You have a map-merge problem, and someone is going to have to define the algorithm. The flattening done in YamlMapPropertiesBean and the merge in YamlMapFactoryBean are just two choices out of (probably) a larger set of possibilities.
I have a puppet environment where I need to add a variable number of very similar objects to a single server. The objects vary only by attributes such as name and path, all of which can be extrapolated from a single name parameter.
This seems like the perfect use for puppet resource definitions since multiple definitions can be added to a single server and their specific attributes can be taken from the definition name declaration.
This setup works well for me and I have had no issues getting it up and running on several servers. I have a requirement now however to hand this over to an ops department with almost no scripting experience, so they aren't really comfortable managing from the file system.
They requested a dashboard, so I setup puppet dashboard for them. After I set the dashboard up however, I found out that it only supports adding classes to servers, not definitions. This seems pretty shortsighted on the part of the folks at PuppetLabs, so I can only think that I am approaching this problem the wrong way and that there must be a solution using classes whereby multiple almost identical entities can be added to a single node.
I realize I could create a class for each entity, but there are hundreds, even thousands of potential variations so that's not really practical.
I have also considered a wrapper class that declares each definition on a per node basis, but this seems like more work to manage than it's worth.
Any thoughts on alternate approaches that would be compatible with the dashboard would be appreciated.
To make ENCs such as dashboard work this way, you have to pass in node data in the form of node variables. Note that Hiera is often preferable to plain node variables.
Generally, the value you want to hand in is an array of your resource titles
$resources = [ "name1", "name2", ... ]
Then classify your node with (at least) one class that instanciates your defined type using this data, e.g.
class my_resources {
my_defined_type { $resources: }
}