YAML: Store array as variable - ruby

I'm just starting to learn YAML, and I'm not really finding a best practice for something that I'm trying to accomplish. Basically, I have an array of objects in my YAML file, and for production, I'd like to add 1 more entry to this array. So I basically want something like this (it's pseudo code because I know it's not valid YAML):
development:
array: &ARRAY
- name: item1
value: value1
- name: item2
value: value2
production:
<<: *ARRAY
array:
- name: item3
value: value3
Currently, I'm parsing my YAML files with Ruby, so I decided to handle this logic in Ruby. I'm doing something like this:
yaml_contents = YAML::load(yaml_string)
prod_values = yaml_contents['production']
prod_values['array'].push({:name => 'item3', :value => 'value3'})
However, that can make my loading script very hairy. Is there a better way of designing this?
I believe this question is related.

The << syntax is for merging maps (i.e. Hashes), not sequences. You could do something like this:
development: &ARRAY
- name: item1
value: value1
- name: item2
value: value2
production:
- *ARRAY
- name: item3
value: value3
When you load this the production array will have a nested array, so you would need to use flatten:
yaml_contents = YAML::load(yaml_string)
prod_values = yaml_contents['production'].flatten
If your actual data could involve nested hashes and you only want to flatten any arrays that appear as aliases in the Yaml you could write your own Psych visitor (probably a sub class of Psych::Visitors::ToRuby) and merge them in as you create the object graph, but I suspect simply calling flatten will be enough in this case.

Related

How to show all the parameters names as comma separated string and assign to One Varaible in YAML Azure Pipelines

How to show all the parameter's names as comma-separated strings (concatenated) and assign them to One Variable in YAML Azure so that i can use this variable in several places
i tried using
parameters:
- name: Var1
displayName: Testing
type: string
- name: Var2
displayName: Coding
type: string
- name: Var3
displayName: Marketing
type: string
variables:
- name: allParametersString
${{each item in parameters}}:
value: $allParametersString + ','+ item.displayName
my desired output is upon using $allParametersString I should get
Testing,Coding,Marketing
but this is leaving an error mentioning 'value' is already defined so can anyone help me? regarding this, I am searching for a solution for 2 weeks :(
I found the way of using bash to assign values will work for this i did
variables:
- name: allParametersString
value: ' '
steps:
- ${{ each parameter in parameters }}:
# the below code will help us reassign the values to variables with bash so i am appending all parameters separated by comma
- bash: echo '##vso[task.setvariable variable=allParametersString]$(allParametersString)${{ parameter.Key }}, '
- script:
echo 'printing all parameters names separated by comma .->$(allParametersString)'
Please let me know if I can improve it more
this helps me understand that in order to reassign or assign twice or concatenate the string using YAML this is the one way of doing it
Your current thinking is not feasible.
There are several things that bind you.
1, The first is the processing logic of yml expression in DevOps.
See this official document:
https://learn.microsoft.com/en-us/azure/devops/pipelines/process/runs?view=azure-devops#process-the-pipeline
From the first sentence given, we know your yml will be expanded like this:
parameters:
- name: Var1
displayName: Testing
type: string
default: value1
- name: Var2
displayName: Coding
type: string
default: value2
- name: Var3
displayName: Marketing
type: string
default: value3
variables:
- name: allParametersString
value: xxx
value: xxx
value: xxx
variable of yml concept doesn't allow the written method of the above. That's why you encountered error 'value' is already defined.
2, The second is the structure allowed by DevOps yml files.
Every section of yml definition has limited predefined key. This means that compile time cannot find other container to store the variable.
3, I am afraid the usage of yml expression does not support you to do so.
Refer to this:
Each Keyword This tell you the standard usage of 'each':
You can use the each keyword to loop through parameters with the
object type.
JOIN Expression
This is the way the DevOps yml pipeline provides compile-time flattening of data, but it doesn't work in your case.
And there's no such thing as an index or subscript to navigate to the last run.
4, By the way, the item object doesn't have such information 'displayName':
trigger:
- none
parameters:
- name: Var1
displayName: Testing
type: string
default: value1
- name: Var2
displayName: Coding
type: string
default: value2
- name: Var3
displayName: Marketing
type: string
default: value3
steps:
- ${{each item in parameters}}:
- task: PowerShell#2
inputs:
targetType: 'inline'
script: |
# Write your PowerShell commands here.
Write-Host "${{ convertToJson(item) }}"
Result:
The DevOps yml pipeline has no built-in features to implement your needs. If you have to do this, a feasible method is to call the API to get the content of the yml file, then parse and get the parameter part (this is actually a self-designed parser), and then combine the acquired parameters into the variable in the script And pass logging command set variable(isoutput=true). In this way, other tasks can use this combined variable.
This can be done, but is overly complicated and you need to consider whether it is necessary to do such a thing.

Save YAML with inheritance in ruby

I have YAML file with inheritance and I want to add or edit a key programatically.
I load the YAML into hash using YAML.load method but when I save the hash back using YAML.dump I lose all the inheritance info.
Is there a way to edit the YAML in Ruby without losing the inheritance info?
YAML example:
main:
prod: &prod
key1: true
key2: 50
key3: "abc"
prod_v_3_5: &prod_v_3_5
<<: *prod
key2: 100
prod_v_3_6: &prod_v_3_6
<<: *prod_v_3_5
key2: 150
Code example:
config = Api.get(id)
yaml = YAML.load(config)
yaml["main"][section].store(key, value)
config = YAML.dump(yaml)
Api.set(id, config)
As far as I know (I also use that option to import and override) the YAML source is read and processed and then the hash elements are exposed and not linked internally. So the mechanism is copy paste override and not linking and overload.
I guess you have to modify the YAML source by opening the file and modify its content if you won't destroy your source.
To add new section:
config = Api.get(id)
config = "#{config}\n \n #{section}: &#{section}\n <<: *#{parent_section}"
To add new value:
config = Api.get(id)
matches = /^(.+)(\n #{section}:\s*&#{section}\s*\n )(<<:[^\n]+)?(.*)$/m.match(config)
config = "#{matches[1]}#{matches[2]}#{matches[3]}\n #{key}: #{value}#{matches[4]}\n"

Tosca - specifying map type properties

What is the correct way of specifying constraints and the defaults for the "map" type? I.e., is there is an option of specifying "valid_values" for both: keys and values?
When we specify constraints at the "entry_schema" level, are they considered for keys, values, for both? Can we have separate lists of valid_values - one for keys and one for values? Or should we specify them both in a single list like in the following example?
properties:
<property_name>:
type: map
entry_schema:
type: string
constraints:
- valid_values:
- "key1":"val1"
- "key2":"val2"
- "key3":"val3"
- "key4":"val4"
default:
key1: "val1"
key2: "val2"

YAML by example

I am trying to design the configuration file format for my app and have chosen YAML. However, this (obviously) means I need to be able to define, parse and validate proper YAML syntax!
In the config file, there must be a collection/sequence called widgets. This sequence needs to contain a set of typed widgets (possible types are fizz, buzz and foobaz). Each widget also has a name and various other properties (depending on the type). For example:
myappconfig.yaml
================
widgets:
- buzz:
name: Red Buzz
isSilly: true
- buzz:
name: Blue Buzz
isSilly: false
- foobaz:
name: Abracadabra
rating: 3000
specialty: Such YAML much amaze
My simple question is: Have I created a proper/valid YAML file above? Meaning, based on my constraints, am I understanding YAML syntax correctly in my implementation?
You can check the syntax of your YAML, e.g. on this website.
Your YAML is parsed as this:
{'widgets': [{'buzz': {'name': 'Red Buzz', 'isSilly': True}}, {'buzz': {'name': 'Blue Buzz', 'isSilly': False}}, {'foobaz': {'rating': 3000, 'name': 'Abracadabra', 'specialty': 'Such YAML much amaze'}}]}
which looks like what you seem to be after.
If the widget name should be unique, you can use it for keys:
widgets:
RedBuzz:
type: buzz
isSilly: true
BlueBuzz:
type: buzz
isSilly: false
Abracadabra:
type: foobaz
rating: 3000
specialty: Such YAML much amaze
This one uses grouping by widget type:
widgets:
buzz:
- RedBuzz:
isSilly: true
- Blue Buzz:
isSilly: false
foobaz:
- Abracadabra:
rating: 3000
specialty: Such YAML much amaze
Also if you're feeling nerdy you can use widgets types merge like this:
buzz_widget: &buzz_widget
type: buzz
foobaz_widget: &foobaz_widget
type: foobaz
widgets:
RedBuzz:
isSilly: true
<<: *buzz_widget
BlueBuzz:
isSilly: false
<<: *buzz_widget
Abracadabra:
rating: 3000
specialty: Such YAML much amaze
<<: *foobaz_widget
As for me, using nested key-values structure (for non-primitive elements) is preferable. Being parsed into a map, you can always access it's values as a collection. At the same time you have unique constraint and constant access time (by key). If your keys should not be unique, then you should try restructuring it before giving back to sequence.

YAML parsing in Ruby: How to get array of a certain type of elements?

I have the following YAML:
- name: List of monkeys
- author: Nicolas Raoul
- version: 1
- monkey: Chee-Chee
- monkey: Curious George
- monkey: Mojo
How to get the array of monkeys in Ruby? The number of metadata parameters (name, author, ...) is variable.
It would return something like [Chee-Chee, Curious George, Mojo]
Note: I don't want to create a monkeys node containing all monkeys as sub-items, because there are many monkeys and I want to keep the file really simple.
Well your YAML looks a little crazy: each of those lines is a hash containing one element. Still, if you're stuck with it, something like this would get you an array of monkey names:
require 'yaml'
data = YAML.load(DATA.read)
p data.map { |row| row['monkey'] }.compact
__END__
- name: List of monkeys
- author: Nicolas Raoul
- version: 1
- monkey: Chee-Chee
- monkey: Curious George
- monkey: Mojo
Obviously you could load the YAML from anywhere, it doesn't have to be in your DATA block.

Resources