My YAML file looks like below:
info_block:
enable: null
start: "12:00"
server_type: linux
I have loaded and dumped using ruamel.yaml.dump
But, the output is getting formatted like below: (like null replaced to empty, double quotes gets removed from the start value)
info_block:
enable:
start: 12:00
server_type: linux
How can I retain my source here
I know there is something like this to retain the null but I want my complete source unformatted.
If you want to retain your source, you're best bet is to keep track of whether something changes and
not over write the source when not changes, as in your example.
ruamel.yaml will always normalize output and if that is not what you want,
your only hope is to do exact string substitutions on the file, potentially
using the line information on the loaded data. I recommend against doing that,
and if you're retaining is for minizing diffs you should just byte the bullet
once, like what you would do when using some source formatter.
However if you work with YAML 1.1 only parsers although that version was
replaced more than 10 years ago, I can see that 12:00 instead of "12:00" can
be a problem as those kind of strings are interpreted as sexagesimals.
In ruamel.yaml, you can either set the output to be YAML 1.1, and then 12:00
will be quoted, but you'll get a document header stating that it is conform to
the outdated version.
The other thing you can do is preserve any quotes using the .preserve_quotes attribute:
import sys
import ruamel.yaml
yaml_str = """\
info_block:
enable: null
start: "12:00"
server_type: linux
"""
def my_represent_none(self, data):
return self.represent_scalar(u'tag:yaml.org,2002:null', u'null')
yaml = ruamel.yaml.YAML()
yaml.representer.add_representer(type(None), my_represent_none)
yaml.indent(mapping=2, sequence=2, offset=0)
yaml.preserve_quotes = True
data = yaml.load(yaml_str)
yaml.dump(data, sys.stdout)
which gives a complete retained version if combined with the alternative representer for the null node:
info_block:
enable: null
start: "12:00"
server_type: linux
Related
I am writing an automation script for an old project and I need some help with pvpython from Paraview 3.98.1. The function SaveData() in this version does not exist. I found its implementation here and moved it to my code. How can I save a file as ASCII? Calling it like SaveData(filename, proxy=px, FileType='Ascii') saves my files as binaries (awkward behavior).
I need this version because some of my codes in the scripting pipeline handle very specific vtk files. Using the SaveData() function of the latest versions ended up creating different metadata in my final files, and when I process them it ends up destroying my results. It is easier at the moment to use an older version of Paraview than to modify all my codes.
Edit:
The website is not working now, but it was yesterday. Maybe it is an internal problem? Anyway, the code is attached below.
# -----------------------------------------------------------------------------
def SetProperties(proxy=None, **params):
"""Sets one or more properties of the given pipeline object. If an argument
is not provided, the active source is used. Pass a list of property_name=value
pairs to this function to set property values. For example::
SetProperties(Center=[1, 2, 3], Radius=3.5)
"""
if not proxy:
proxy = active_objects.source
properties = proxy.ListProperties()
for param in params.keys():
pyproxy = servermanager._getPyProxy(proxy)
pyproxy.__setattr__(param, params[param])
# -----------------------------------------------------------------------------
def CreateWriter(filename, proxy=None, **extraArgs):
"""Creates a writer that can write the data produced by the source proxy in
the given file format (identified by the extension). If no source is
provided, then the active source is used. This doesn't actually write the
data, it simply creates the writer and returns it."""
if not filename:
raise RuntimeError ("filename must be specified")
session = servermanager.ActiveConnection.Session
writer_factory = servermanager.vtkSMProxyManager.GetProxyManager().GetWriterFactory()
if writer_factory.GetNumberOfRegisteredPrototypes() == 0:
writer_factory.UpdateAvailableWriters()
if not proxy:
proxy = GetActiveSource()
if not proxy:
raise RuntimeError ("Could not locate source to write")
writer_proxy = writer_factory.CreateWriter(filename, proxy.SMProxy, proxy.Port)
writer_proxy.UnRegister(None)
pyproxy = servermanager._getPyProxy(writer_proxy)
if pyproxy and extraArgs:
SetProperties(pyproxy, **extraArgs)
return pyproxy
# -----------------------------------------------------------------------------
def SaveData(filename, proxy=None, **extraArgs):
"""Save data produced by 'proxy' in a file. If no proxy is specified the
active source is used. Properties to configure the writer can be passed in
as keyword arguments. Example usage::
SaveData("sample.pvtp", source0)
SaveData("sample.csv", FieldAssociation="Points")
"""
writer = CreateWriter(filename, proxy, **extraArgs)
if not writer:
raise RuntimeError ("Could not create writer for specified file or data type")
writer.UpdateVTKObjects()
writer.UpdatePipeline()
del writer
# -----------------------------------------------------------------------------
The question is answered here (also my post). I used SaveData() to save a binary file with the proxy I need and then used DataSetWriter() to change my FileType to ASCII. It is not a beautiful solution since SaveData() is supposed to do that, but it does the job.
I have Ruby using Psych that writes YAML like this:
---
- !ruby/struct:Dhc::QueryEvent
time: 2018-01-02T07:45:18.470Z
hits: 2525
qtime: 13
server: search1
q: search term
searched_in:
- productA
- productB
- productC
Is there a way I can write the YAML without the "!ruby/struct:Dhc::QueryEvent" tag? I want the storage formatted so it is not dependent on my struct:Dhc::QueryEvent for reading. In other words, I want others to have the flexibility to read the YAML however they want. Currently, they get
undefined class/module Dhc:: (ArgumentError)
Then, when I'm reading the file myself, is there a configuration in Psych that allows me to read the untagged records into my Dhc::QueryEvent struct? I assume this would be something like
ar = []
Psych.parse_stream(yaml_stream) do |node|
ar << Dhc::QueryEvent.new(node.time, node.hits, etc...)
end
but maybe there is a shorter way?
Trying to find a way to turn off the red lines temporarily for that file only.
maybe try to disable the yaml.schemaStore ?
go in in settings.json and add :
"yaml.schemaStore.enable": false
Since this is not valid YAML at all, but you want to edit this as YAML,
you should make it into valid YAML. If you turn of the errors,
instead you probably would not have all of the advantage of the YAML
editing mode.
If saltstate allows you to change the block_start_string and
variable_start_string jinja2 uses you can change {% into #% (or
###% if #% and ###% naturally occur in your source), and also
change {{ into <{ (or <<{, you get the idea). If you would call
jinja2 directly you would then then pass to the FireSystemLoader:
block_start_string='<{' and variable_start_string='#%' If the
above is possible, then you have to change your input file only once,
do that with an editor.
If you cannot control saltstate to do the sane thing, your still not
stuck but you have to do a bit more using Python,
ruamel.yaml and some
support packages (disclaimer: I am the author of those packages).
Install with:
pip install ruamel.yaml[jinja2] ruamel.std.pathlib
Then before editing run the program:
from ruamel.yaml import YAML
from ruamel.std.pathlib import Path
yamlj2 = YAML(typ='jinja2')
yamlrt = YAML()
yaml_flow_style = YAML()
yaml_flow_style.default_flow_style = True
in_file = Path('init.sls')
backup_file = Path('init.sls.org')
in_file.copy(backup_file)
data = yamlj2.load(in_file)
with in_file.open('w') as fp:
# write the header with info needed for revers
fp.write('# ruamel.yaml.jinja2: ') # no EOL
yaml_flow_style.dump(yamlj2._plug_in_jinja2, fp)
yamlrt.dump(data, fp)
which changes the offending jinja2 sequences and add a one-line header comment with the actual patterns used to the file. You should then be able
to edit the init.sls file without getting all those errors.
Before calling saltstate, do run the following:
from ruamel.yaml import YAML
from ruamel.std.pathlib import Path
in_file = Path('init.sls')
yamlj2 = YAML(typ='jinja2')
yamlrt = YAML()
yamlnort = YAML(typ='safe')
with in_file.open() as fp:
yamlj2._plug_in_jinja2 = yamlnort.load(fp.readline().split(':', 1)[1])
data = yamlrt.load(fp)
yamlj2.dump(data, in_file)
If you have multiple of these files, you probably want to take your
filename from sys.argv[1]. You might actually call the salstate program from this second Python program (i.e. decode and run).
How can I check if / else in yaml file.
like:
if %{attribute}
attributes:
shipping_comment: Shipping comment / Instructions
else
attributes:
shipping_date: Date
YAML is a data serialisation language, so it's not meant to contain if/else style executable statements: that's the responsibility of the programming language you're using.
A simple example in Ruby to determine which config string from a YAML file to output could be defining your YAML config file as follows:
data.yml
attributes:
shipping_comment: Shipping comment / Instructions
shipping_date: Date
Then, in your program, read the file in and run the conditional there:
shipping.rb
#!/usr/bin/env ruby
require 'yaml'
config = YAML.load_file('data.yml')
attribute = true # your attribute to check here
if attribute
puts config['attributes']['shipping_comment']
else
puts config['attributes']['shipping_date']
end
Out of the box .yaml files won't include any conditional logic, as Paul Fioravanti says:
YAML is a data serialisation language, so it's not meant to contain if/else style executable statements: that's the responsibility of the programming language you're using.
However, there are some cases, such as Infrastructure as Code where you might not have the luxury of Paul's solution. In these cases, most decent Infrastructure tools provide an inbuilt way of achieving conditional logic.
Since it appears that infra is not the area you're looking in, I won't go into detail on how to write each tools solution, but for anyone that end's up here like I did, docs such as these helped me and may prove useful for you:
Ansible
CloudFormation
This is a little late for the original poster, but may be of use to others: The Azure implementation of the YAML parser does support conditional checks. For example, the following YAML in an azure-pipeline.yml:
#azure-pipeline.yml
parameters:
- name: testCondition
displayName: 'Use experimental build process?'
type: boolean
default: true
steps:
- ${{ if eq(parameters.testCondition, true) }}:
- bash: echo 'true'
- ${{ if not(eq(parameters.testCondition, true)) }}:
- bash: echo 'false'
will evaluate the value of the testCondition parameter. When run, the bash task will echo "true", something like this:
In order to have only a single point of configuration for my app I need to make a YAML config file that is also valid ruby code. I.e. a mixed syntax file that can be parsed as YAML and parsed as ruby.
My application is a suite of processes managed by the god gem. I want to load a new group of maintained processes (watches) for each new configuration file.
God allows loading a new app.god (ruby) file with new watches defined, but I don't want an app.god and app.yml, just one file. Simplest might be to just have the app.god file and include the configuration within that, but I preferred the idea of a yml file that was also valid ruby code.
#I found this that might be helpful:
#This is a valid ruby and a valid YAML file
#Comments are the same in YAML and ruby
true ?true:
- <<YAML.to_i
# At this point in ruby it is the contents of a here doc (that will be
# converted to an integer and negated if true happens not to be true)
# In YAML it is a hash with the first entry having key "true ?true"
# representing a list containing the string "- <<YAML.to_i"
# If your YAML object should be a list not a hash you could remove the first line
any_valid_yaml: from here
a_list:
- or
- anything
- really
#Then mark the end of the YAML document with
---
#And YAML is done and ignores anything from here on
#Next terminate the ruby here document
YAML
#Now we're in ruby world
#this = "pure ruby"
def anything(ruby)
"here"
end