ruamel.yaml==0.15.37
Python 3.6.2 :: Continuum Analytics, Inc.
Current code:
from ruamel.yaml import YAML
import sys
yaml = YAML()
kube_context = yaml.load('''
apiVersion: v1
clusters: []
contexts: []
current-context: ''
kind: Config
preferences: {}
users: []
''')
kube_context['users'].append({'name': '{username}/{cluster}'.format(username='test', cluster='test'), 'user': {'token': 'test'}})
kube_context['clusters'].append({'name': 'test', 'cluster': {'server': 'URL:443'}})
kube_context['contexts'].append({'name': 'test', 'context': {'user': 'test', 'cluster': 'test'}})
yaml.dump(kube_context, sys.stdout)
My yaml.dump() is producing output that contains the list and dict objects, instead of being fully expanded.
Current output:
apiVersion: v1
clusters: [{name: test, cluster: {server: URL:443}}]
contexts: [{name: test, context: {user: test, cluster: test}}]
current-context: ''
kind: Config
preferences: {}
users: [{name: test/test, user: {token: test}}]
What do I need to do in order to have yaml.dump() output fully expanded?
Expected output:
apiVersion: v1
clusters:
- name: test
cluster:
server: URL:443
contexts:
- name: test
context:
user: test
cluster: test
current-context: ''
kind: Config
preferences: {}
users:
- name: test/test
user:
token: test
ruamel.yaml, when using the default YAML() or YAML(typ='rt') will preserve the flow- or block style of sequences and mappings. There is no way to make a block style empty sequence or empty mapping and your [] and {} are therefore tagged as flow style when loaded.
Flow style can only contain flow style (whereas block style can contain block style or flow style) (YAML 1.2 spec 8.2.3):
YAML allows flow nodes to be embedded inside block collections (but not vice-versa).
Because of that, the dict/mapping data that you insert in the (flow-style) list/sequence will also be represented as flow-style.
If you want everything to be block style (what you call "expanded" mode), you can explicitly set that by calling the .set_block_style() method on the .fa attribute (which is only available on the collections, hence the try/except):
from ruamel.yaml import YAML
import sys
yaml = YAML()
kube_context = yaml.load('''
apiVersion: v1
clusters: []
contexts: []
current-context: ''
kind: Config
preferences: {}
users: []
''')
kube_context['users'].append({'name': '{username}/{cluster}'.format(username='test', cluster='test'), 'user': {'token': 'test'}})
kube_context['clusters'].append({'name': 'test', 'cluster': {'server': 'URL:443'}})
kube_context['contexts'].append({'name': 'test', 'context': {'user': 'test', 'cluster': 'test'}})
for k in kube_context:
try:
kube_context[k].fa.set_block_style()
except AttributeError:
pass
yaml.dump(kube_context, sys.stdout)
this gives:
apiVersion: v1
clusters:
- name: test
cluster:
server: URL:443
contexts:
- name: test
context:
user: test
cluster: test
current-context: ''
kind: Config
preferences: {}
users:
- name: test/test
user:
token: test
Please note that it is not necessary to set yaml.default_flow_style = False in the default round-trip-mode; and that although block-style has been set for the value of key preferences, it is represented flow style as there is no other way to represent an empty mapping.
The output is „pure“ YAML. You want the nodes to be presented in block style (indentation-based) as opposed to the current flow style ([]{}-based). Here's how to do that:
yaml = YAML(typ="safe")
yaml.default_flow_style = False
(Note Athon's comment on the typ below; you need to set it to safe or unsafe so that the RoundTripLoader does not set the style of the empty sequences)
Related
I have a CloudFormation template yml file that passes OptionSettings for an ElasticBeanstalk applicaiton. I can hard code values, and I can pass values from Parameters. However, I am unable to determine how to pass Parameters or Variables as the namespace.
This works:
- Namespace: aws:elasticbeanstalk:environment:process:lbtargetgroup
OptionName: Port
Value: 3000
This works (where PORTNUMBER is a parameter)
Parameters:
PORTNUMBER:
Type: String
Description: Port number
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment:process:lbtargetgroup
OptionName: Port
Value: !Ref PORTNUMBER
However, this does not work (where LBTARGETGROUP is a parameter):
Parameters:
LBTARGETGROUP:
Type: String
Description: Target Group Name
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: aws:elasticbeanstalk:environment:process:!Ref LBTARGETGROUP
OptionName: Port
Value: 3000
From what I have tried, you cannot use typical Variables in a CFT (https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-anatomy.html#template-anatomy-sections). I have also tried mappings. But I can't seem to figure out how to pass the name as a parameter.
The answer is often very simple... the following works:
Parameters:
LBTARGETGROUP:
Type: String
Description: Target Group Name
ElasticBeanstalkConfig:
Properties:
OptionSettings:
- Namespace: !Join ["", ["aws:elasticbeanstalk:environment:process:", Ref: LBTARGETGROUP]]
OptionName: Port
Value: 3000
I want to get the values of the fields declared in the downwardAPI section of a Pod.
apiVersion: v1
kind: Pod
metadata:
name: sample
namespace: default
spec:
containers:
- image: rpa
imagePullPolicy: Always
name: testbot
volumeMounts:
- mountPath: /etc/pod-info
name: pod-info
volumes:
- downwardAPI:
items:
- fieldRef:
apiVersion: v1
fieldPath: metadata.labels
path: labels
- fieldRef:
apiVersion: v1
fieldPath: metadata.name
path: pod-name
- fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
path: pod-namespace
name: pod-info
Using client-go, I can use pod.Spec.Volumes[0].DownwardAPI.Items to get the item slice including the fieldPath. But I would now need to dynamically able to fetch whatever values has been declared in the fieldPath. So, from the first item, I would like to access the value of metadata.labels. I could do pod.ObjectMeta.Labels but I would like to access the field dynamically. In terms of Javascript it would have been something like
var foo="metadata.labels"
var fooarr = foo.split(".")
var bar={
metadata:{
labels: "foobar"
}
}
var temp = oof
for(lm of lmaoarr){
temp = temp[lm]
}
console.log(temp)
How do I do something similar using client-go?
The standard kubelet code has logic to translate the downward API fields into environment variables. It is neither simple nor generic, though: at the bottom of the stack, only the specific fields listed in the Kubernetes documentation are supported. It would be incomplete, but not wrong or inconsistent with standard Kubernetes, to just match on these specific fields:
for _, item := range downwardApiObject.Items {
switch item.FieldPath.FieldRef {
case "metadata.name":
return pod.ObjectMeta.Name
}
}
The actual code:
Calls pods.ConvertDownwardAPIFieldLabel which does some very lightweight normalization and validation: subscripts are only allowed on metadata.annotations and metadata.labels, only the dozen or so specific field names are allowed, and spec.host is rewritten to spec.nodeName.
Handles the spec.* and status.* variables that depend on non-metadata fields in the pod spec or runtime data.
Delegates to fieldpath.ExtractFieldPathAsString which knows how to handle the metadata.* variables.
The k8s.io/kubernetes/pkg/fieldpath package contains a couple of helpers that are used in processing the downward API, and a really short answer to your specific question could be just to call fieldpath.ExtractFieldPathAsString passing it the Pod object, which will handle the metadata fields but nothing else.
I'm using Ansible
os_project_facts module to gather admin project id of OpenStack.
This is the ansible_fact log:
ansible_facts:
openstack_projects:
- description: Bootstrap project for initializing the cloud.
domain_id: default
enabled: true
id: <PROJECT_ID>
is_domain: false
is_enabled: true
location:
cloud: envvars
project:
domain_id: default
domain_name: null
id: default
name: null
region_name: null
zone: null
name: admin
options: {}
parent_id: default
properties:
options: {}
tags: []
tags: []
Apparently, this is not a dictionary, and I can't get openstack_projects.id since it is not a dictionary. How can I retrieve PROJECT_ID and use it in other tasks?
Since the openstack_projects facts contains single list element with a dictionary, we can use the array indexing method to get the id, i.e. openstack_projects[0]['id'].
You can use it directly, or use something like set_fact:
- name: get the project id
set_fact:
project_id: "{{ openstack_projects[0]['id'] }}"
I am trying to update the yaml file using ruamel python.
proc=subprocess.Popen(['kubectl','get','pod','web3','-o','yaml','--export'], stdout=subprocess.PIPE)
rein=proc.stdout.read()
result, indent, block_seq_indent = ruamel.yaml.util.load_yaml_guess_indent(rein, preserve_quotes=True)
So far I have tried :
result['spec'].append('nodeSelector')
which gives ERROR :
result['spec'].append('nodeSelector')
AttributeError: 'CommentedMap' object has no attribute 'append'
Also tried like this :
result['spec']['nodeSelector']['kubernetes.io/hostname']='kubew1'
gives :
result['spec']['nodeSelector']['kubernetes.io/hostname']='kubew1'
File "/usr/local/lib/python3.6/dist-packages/ruamel/yaml/comments.py", line 752, in __getitem__
return ordereddict.__getitem__(self, key)
KeyError: 'nodeSelector'
My Intial Yaml File is :
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
app: demo
name: web
name: web3
selfLink: /api/v1/namespaces/default/pods/web3
spec:
containers:
- image: aexlab/flask-sample-one
imagePullPolicy: Always
name: web
ports:
- containerPort: 5000
name: http
protocol: TCP
resources: {}
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
volumeMounts:
- mountPath: /var/run/secrets/kubernetes.io/serviceaccount
name: default-token-7bcc9
readOnly: true
dnsPolicy: ClusterFirst
enableServiceLinks: true
And Expected fields I want to add inside 'spec' is :
nodeSelector:
kubernetes.io/hostname: kubew1
Any Ideas how to achieve this with ruamel library.
In your YAML file your root level collection is a mapping and the value for the key spec in that mapping is itself a mapping. Both of those mappings get loaded as dict-like objects using ruamel.yaml named CommentedMap.
As with normal dicts you can add key-value pairs, deleted keys (and their values), and update values for a key, but there is no .append() method, as there is with a list (i.e. appending an extra item to a list).
Your output is a bit terse, but of course you cannot just add nodeSelector to anything (list/sequence nor dict/mapping) and expect that to add kubernetes.io/hostname: kubew1 (a mapping in its own right) automatically.
Your try of:
result['spec']['nodeSelector']['kubernetes.io/hostname'] = 'kubew1'
cannot work because there is no dict result['spec']['nodeSelector'] where you can add the key kubernetes.io/hostname.
You would either first have to create a key with an emtpy dict as value:
result['spec']['nodeSelector'] = {}
result['spec']['nodeSelector']['kubernetes.io/hostname'] = 'kubew1'
or do
result['spec']['nodeSelector'] = {'kubernetes.io/hostname': 'kubew1'}
Please note that the above has nothing much to do with ruamel.yaml, that is just basic Python data structure manipulation. Also note that there are over 100 libraries in the ruamel namespace, out of which ruamel.yaml is just one of several published as open source, so using ruamel is not very clear statement, although of course the context often provides enough information on which library you actually use.
I'm looking for a way to reuse variables defined in my list on YAML, I have a YAML list with the following sample entries :
workstreams:
- name: tigers
service_workstream: tigers-svc
virtual_user:
- {name: inbound-tigers, pass: '123', access: inbound, env: app1}
- {name: outbound-tigers, pass: '123', access: outbound, env: app1}
email: tigers#my-fqdn.com
mount_dir: /mnt/tigers
app_config_dir: /opt/tigers
Using the example from above I want to reuse a defined value, like tigers. The ideal solution would be something like this :
workstreams:
- name: tigers
service_workstream: "{{ vars['name'] }}-svc"
virtual_user:
- {name: "inbound-{{ vars['name'] }}", pass: '123', access: inbound, env: app1}
- {name: "outbound-{{ vars['name'] }}", pass: '123', access: outbound, env: app1}
email: "{{ vars['name'] }}#my-fqdn.com"
mount_dir: "/mnt/{{ vars['name'] }}"
app_config_dir: "/opt/{{ vars['name'] }}"
Any points as to how I can do this in YAML ?
You can do:
workstreams:
- name: &name tigers # the scalar "tigers" with an anchor &name
service_workstream: *name # alias, references the anchored scalar above
However, you can not do string concatenation or anything like it in YAML 1.2. It cannot do any transformations on the input data. An alias is really a reference to the node that holds the corresponding anchor, it is not a variable.
Quite some YAML-using software provides non-YAML solutions to that problem, for example, preprocessing the YAML file with Jinja or whatnot. Depending on context, that may or may not be a viable solution for you.