YAML anchor for sequence? - yaml

Question

Consider the following file YAML file:
A:
&B
- x: 1
y: 2
- x: 10
y: 20
C:
<<: *B
which is read into python via:
from ruamel.yaml import YAML
filename = 'data/configs/debug_config.yml'
with open(filename) as f:
c = YAML(typ='safe').load(f)
print(c)
yielding:
{'A': [{'x': 1, 'y': 2}, {'x': 10, 'y': 20}], 'C': {'x': 1, 'y': 2}}
It is seen that the anchor B only includes the first element of the sequence. Why? I would like an anchor that would include the entire sequence, such that the values of A and C in the python dictionary are identical. How can this be done?

Answer 1

The anchor B includes all elements from A, but you are merging them with the merge key << (source):
If the value associated with the merge key is a sequence, then this sequence is expected to contain mapping nodes and each of these nodes is merged in turn according to its order in the sequence. Keys in mapping nodes earlier in the sequence override keys specified in later mapping nodes.
So,the first item from A overrides the second one.
Remove <<: and C will be the same dictionary as A:
A: &B
- x: 1
y: 2
- x: 10
y: 20
C: *B
Yields:
{
"A": [
{
"y": 2,
"x": 1
},
{
"y": 20,
"x": 10
}
],
"C": [
{
"y": 2,
"x": 1
},
{
"y": 20,
"x": 10
}
]
}
You can check the merge order with this example:
A: &B
- x: 1
y: 2
- x: 10
new: 333
C:
<<: *B
Yields:
{
"A": [
{
"y": 2,
"x": 1
},
{
"x": 10,
"new": 333
}
],
"C": {
"y": 2,
"x": 1,
"new": 333
}
}

Answer 2

As #Hadyniak already indicated you are incorrectly using the merge
keys. Since the alias *B is expanded in the
composer step, before the interpretation of the << merge key in the
constructor step, the latter actually receives a list of dicts, which are
combined with the values of keys of earlier occuring dicts taking precedence. If
the order of processing had happened to be different you would get an error, and
IMO the merge key documentation doesn't explicitly specify that aliases should
be expanded first.
Hadyniak's solution however will have you end
up with c['A'] and c['C'] being the same dictionary and that might not be what you want:
import ruamel.yaml
yaml_str = """\
A:
&B
- x: 1
y: 2
- x: 10
y: 20
C:
*B
"""
yaml = ruamel.yaml.YAML()
c = yaml.load(yaml_str)
print(c['A'] is c['C'], c['A'] == c['C'])
c['A'][0]['x'] = 42
print('x:', c['C'][0]['x'])
which gives:
True True
x: 42
If that is not what you want, you can still use merge keys, but on the dictionaries that are the elements of your sequence:
import ruamel.yaml
yaml_str = """\
A:
- &B1
x: 1
y: 2
- &B2
x: 10
y: 20
C:
- <<: *B1
- <<: *B2
"""
yaml = ruamel.yaml.YAML()
c = yaml.load(yaml_str)
print(c['A'] is c['C'], c['A'] == c['C'])
c['A'][0]['x'] = 42
print('x:', c['C'][0]['x'])
which gives:
False True
x: 1
Alternatively you can tell the composer part of ruamel.yaml to expand the aliases instead of using a reference:
import copy
yaml_str = """\
A:
&B
- x: 1
y: 2
- x: 10
y: 20
C:
*B
"""
yaml = ruamel.yaml.YAML()
yaml.composer.return_alias = lambda s: copy.deepcopy(s)
c = yaml.load(yaml_str)
print(c['A'] is c['C'], c['A'] == c['C'])
c['A'][0]['x'] = 42
print('x:', c['C'][0]['x'])
which gives:
False True
x: 1
The above will only work in ruamel.yaml>0.17.2, for older versions you'll need to copy and modify the compose_node method:
import copy
yaml_str = """\
A:
&B
- x: 1
y: 2
- x: 10
y: 20
C:
*B
"""
class MyComposer(ruamel.yaml.composer.Composer):
def compose_node(self, parent, index):
# type: (Any, Any) -> Any
if self.parser.check_event(ruamel.yaml.events.AliasEvent):
event = self.parser.get_event()
alias = event.anchor
if alias not in self.anchors:
raise ComposerError(
None,
None,
'found undefined alias {alias!r}'.format(alias=alias),
event.start_mark,
)
return copy.deepcopy(self.anchors[alias])
event = self.parser.peek_event()
anchor = event.anchor
if anchor is not None: # have an anchor
if anchor in self.anchors:
ws = (
'\nfound duplicate anchor {!r}\nfirst occurrence {}\nsecond occurrence '
'{}'.format((anchor), self.anchors[anchor].start_mark, event.start_mark)
)
warnings.warn(ws, ruamel.yaml.error.ReusedAnchorWarning)
self.resolver.descend_resolver(parent, index)
if self.parser.check_event(ruamel.yaml.events.ScalarEvent):
node = self.compose_scalar_node(anchor)
elif self.parser.check_event(ruamel.yaml.events.SequenceStartEvent):
node = self.compose_sequence_node(anchor)
elif self.parser.check_event(ruamel.yaml.events.MappingStartEvent):
node = self.compose_mapping_node(anchor)
self.resolver.ascend_resolver()
return node
yaml = ruamel.yaml.YAML()
yaml.Composer = MyComposer
c = yaml.load(yaml_str)
print(c['A'] is c['C'], c['A'] == c['C'])
c['A'][0]['x'] = 42
print('x:', c['C'][0]['x'])
which also gives:
False True
x: 1