What is `<<` and `&` in yaml mean? - yaml

When I review the cryptogen(a fabric command) config file . I saw there symbol.
Profiles:
SampleInsecureSolo:
Orderer:
<<: *OrdererDefaults ## what is the `<<`
Organizations:
- *ExampleCom ## what is the `*`
Consortiums:
SampleConsortium:
Organizations:
- *Org1ExampleCom
- *Org2ExampleCom
Above there a two symbol << and *.
Application: &ApplicationDefaults # what is the `&` mean
Organizations:
As you can see there is another symbol &.
I don't know what are there mean. I didn't get any information even by reviewing the source code (fabric/common/configtx/tool/configtxgen/main.go)

Well, those are elements of the YAML file format, which is used here to provide a configuration file for configtxgen. The "&" sign mean anchor and "*" reference to the anchor, this is basically used to avoid duplication, for example:
person: &person
name: "John Doe"
employee: &employee
<< : *person
salary : 5000
will reuse fields of person and has similar meaning as:
employee: &employee
name : "John Doe"
salary : 5000
another example is simply reusing value:
key1: &key some very common value
key2: *key
equivalent to:
key1: some very common value
key2: some very common value
Since abric/common/configtx/tool/configtxgen/main.go uses of the shelf YAML parser you won't find any reference to these symbols in configtxgen related code. I would suggest to read a bit more about YAML file format.

in yaml if data is like
user: &userId '123'
username: *userId
equivalent yml is
user: '123'
username: '123'
or
equivalent json will is
{
"user": "123",
"username": "123"
}
so it basically allows to reuse data, you can also try with array instead of single value like 123
try converting below yml to json using any yml to json online converter
users: &users
k1: v1
k2: v2
usernames: *users

Related

Resource based AWS lambda:Permission multiple principals in YAML?

Working on a cloud formation project, and have a resource based policy attached to my lambda; it's something similar to the following in YAML:
Mapping:
AccMap:
Alpha:
AWSAcc: 1234567 # aws account numbers
Beta:
AWSAcc: 2345678
Prod:
AWSAcc: 3456789
PermissionPolicy:
Type: AWS::Lambda::Permission
Properties:
Resource: !Ref LambdaNameHere
Principal:
Fn::FindInMap:
- AccMap
- !Ref Stage # defined elsewhere
- AWSAcc
I want to grant multiple accounts this permission, for example, multiple accounts in Beta. How would I go about it in YAML? Can I just make AWSAcc an array, like this?
Mapping:
Beta:
AWSAcc:
- 1234567
- 2345678
You can't pass a list. A PermissionsPolicy resource's Principal property accepts a single string value:
{
"Type" : "AWS::Lambda::Permission",
"Properties" : {
"Action" : String,
"EventSourceToken" : String,
"FunctionName" : String,
"FunctionUrlAuthType" : String,
"Principal" : String,
"PrincipalOrgID" : String,
"SourceAccount" : String,
"SourceArn" : String
}
}
As a workaround, add multiple Permission resources and look up the principal for each. !FindInMap returns the right AWSAcc list for the stage. !Select picks the right principal element from the list:
Principal: !Select [ "0", !FindInMap [ AccMap, !Ref Stage, AWSAcc ] ]
Change "0" to "1" for the second principal's Permission, and so on. Note that each stage must have an equal number of principals, or you will get an out of bounds error.
Edit: If the stages have an unequal number of principals, define a Condition and apply it to the "extra" Permission resources.

Rule governing sequence of fields in a YAML

Checking the validity of a YAML using https://onlineyamltools.com/convert-yaml-to-json
The below YAML is correct
# Valid yaml (field "name" placed at LAST position)
match:
- uri:
prefix: "/mysvc1/"
route:
- destination:
host: myservice1
port:
number: 80
name: "svc1-routes"
However, if I move the field name to first position, the YAML becomes invalid. What is the reason?
# Invalid yaml (field "name" placed at FIRST position)
match:
name: "svc1-routes" # <---- ERROR ----
- uri:
prefix: "/mysvc1/"
route:
- destination:
host: myservice1
port:
number: 80
The error message:
Error: YAMLException: end of the stream or a document separator is expected at line 4, column 1:
- uri:
^
In contrary to your comment, name and match are on the same level because they share the same indentation. name is in no way nested in match (nor is route).
The list items, however, are nested in match since YAML understands the - as parts of the indentation, hence the list items are considered more indented than match and are thus nested in it.
Concerning your error:
name: "svc1-routes"
- uri:
In this part, the mapping key name is assigned the scalar value svc1-routes. Each mapping key may only have one value. On the next line, a sequence starts which is on a deeper indentation level (as explained above) but YAML can't put it anywhere because the key name already has a value. This is why it issues an error.
You can freely switch the mapping keys together with their nested values, e.g.:
route:
- destination:
host: myservice1
port:
number: 80
name: "svc1-routes"
match:
- uri:
prefix: "/mysvc1/"
This will load to the same structure as per YAML spec.

Use globbed string in YAML

I am looking for a way to dynamically set the key using the path of the file below.
For example if I have this YAML:
prospectors.config:
- fields:
queue_name: <somehow get the globbed string below in here>
paths:
- /var/log/casino/*.log
type: log
output.redis:
hosts:
- "producer:6379"
key: "%{[fields.queue_name]}"
And then I had a file called /var/log/casino/test.log, then key would become test.
Im not sure that what you want is possible.
You could use the source field and configure your Redis output using that as the key:
output.redis:
hosts:
- "producer:6379"
key: "%{source}"
This would have the disadvantage of being the absolute path of the source file, not the basename as your question asks for.
If you have a small number of possible basename patterns, and want a queue for each. For example, you have files:
/common/path/test-1.log
/common/path/foo-0.log
/common/path/01-bar.log
/common/path/test-3.log
...
and wanted to have three queues in redis test, foo and bar you could use the source field and the conditionals available in the keys configuration of redis output something like this
output.redis:
hosts:
- "producer:6379"
key: "default_key"
keys:
- key: "test_key"
when.contains:
source: "test"
- key: "foo_key"
when.contains:
source: "foo"
- key: "bar_key"
when.contains:
source: "bar"

Swagger yaml and json file for REST application developed using Play framework in Scala

I am trying to configure swagger for my application. Being new to this field I went to different tutorials and tried to convert the below json to YAML but it's giving errors like bad indentation, response missing etc. The main problem I am facing is in recognizing syntax to represent array of list in YAML format, then to add block in YAML which shows expected values for a particular block.
JSON Format to be converted to YAML:
{
"abc":[
{
"xyz":[ //array of list
{
"id":"",
"name":"",
"relation":[ //array of list
{
"first":{
"xxx":"",
"xxx":"",
"xxx":[ //array of string
""
]
},
"second":{
"xxx":"",
"xxx":"",
"xxx":[
""
],
"type":""
}
},
{
"first":{
"xxx":"",
"xxx":"",
"xxx":[ //array of string
""
]
},
"second":{
"xxx":"",
"xxx":"",
"xxx":[
""
],
"type":""
}
}
],
"rows":[
]
}
]
}
YAML is as below:
swagger: "2.0"
info:
version: 1.0.0
title: xxxx
description: xxxx
schemes:
- https
host: xxxx
basePath: xxxx
paths:
/xxx:
post:
summary: xxxx
consumes:
- application/json
produces:
- application/json
parameters:
abc:
- xyz:
id: string
name: string
relation: string
- first:
id: string
name: string
relation: string
second:
id: string
name: string
relation: string
- first:
id: string
name: string
relation: string
second:
id: string
name: string
relation: string
responses:
'200':
description: Created
When working with YAML you talk about sequences (besides map an scalar). A sequences is what gets mapped to a list in Python and an array in some other languages.
So if you are talking about "represent array of list in YAML" you are actually referring to a sequence of sequences. There are three ways to represent this in YAML.
block-style within block-style:
- - a
- b
- - c
- d
, flow-style within block-style:
- [a, b]
- [c, d]
, or flow-style within flow-style:
[[a, b,], [c, d,],]
Any online YAML parser will show you that the above amounts to the same.
Please note:
You cannot have block-style within flow-style
You can have trailing commas (something that JSON doesn't allow, and which makes JSON unnecessarily hard to edit for humans).
In your example YAML output, which is correct YAML, there are no sequence of sequences (or array of lists in your terminology).

Variable reuse in YAML, variable within variable

I'm looking for a way to reuse variables defined in my list on YAML, I have a YAML list with the following sample entries :
workstreams:
- name: tigers
service_workstream: tigers-svc
virtual_user:
- {name: inbound-tigers, pass: '123', access: inbound, env: app1}
- {name: outbound-tigers, pass: '123', access: outbound, env: app1}
email: tigers#my-fqdn.com
mount_dir: /mnt/tigers
app_config_dir: /opt/tigers
Using the example from above I want to reuse a defined value, like tigers. The ideal solution would be something like this :
workstreams:
- name: tigers
service_workstream: "{{ vars['name'] }}-svc"
virtual_user:
- {name: "inbound-{{ vars['name'] }}", pass: '123', access: inbound, env: app1}
- {name: "outbound-{{ vars['name'] }}", pass: '123', access: outbound, env: app1}
email: "{{ vars['name'] }}#my-fqdn.com"
mount_dir: "/mnt/{{ vars['name'] }}"
app_config_dir: "/opt/{{ vars['name'] }}"
Any points as to how I can do this in YAML ?
You can do:
workstreams:
- name: &name tigers # the scalar "tigers" with an anchor &name
service_workstream: *name # alias, references the anchored scalar above
However, you can not do string concatenation or anything like it in YAML 1.2. It cannot do any transformations on the input data. An alias is really a reference to the node that holds the corresponding anchor, it is not a variable.
Quite some YAML-using software provides non-YAML solutions to that problem, for example, preprocessing the YAML file with Jinja or whatnot. Depending on context, that may or may not be a viable solution for you.

Resources