I'm trying to parse a Unity YAML scene file with Ruby and the Psych gem. The Unity file is supposedly a subset of YAML. Each Unity object is a separate YAML document. Here's a sample:
%YAML 1.1
%TAG !u! tag:unity3d.com,2011:
--- !u!29 &1
OcclusionCullingSettings:
m_ObjectHideFlags: 0
serializedVersion: 2
m_OcclusionBakeSettings:
smallestOccluder: 5
smallestHole: 0.25
backfaceThreshold: 100
m_SceneGUID: 00000000000000000000000000000000
m_OcclusionCullingData: {fileID: 0}
--- !u!104 &2
RenderSettings:
m_ObjectHideFlags: 0
serializedVersion: 9
m_Fog: 0
m_FogColor: {r: 0.5, g: 0.5, b: 0.5, a: 1}
m_FogMode: 3
m_FogDensity: 0.01
I'm parsing it with:
require 'psych'
scene = Psych.load_stream(File.read "./Scene.unity")
Which results in this error message:
Psych::SyntaxError ((<unknown>): found undefined tag handle while parsing a node at line 13 column 5)
After some investigation it appears that Psych is expecting each YAML document to have it's own TAG directive, while Unity's YAML parser must consider the directives at the beginning of the file to be global. So...
Question 1: Who's right? Does the scope of a TAG directive extend from declaration to the end of the file, or only to the end of the document (the next occurrance of ---)?
I'd like to be able to parse the file without having to modify it. This leads to...
Ouestion 2: Is there a clean way to do this? Say a Psych option I've missed or a way to pass a block to File.read so that I can add TAG directives on the fly?
Related
I am creating a quarto book project in RStudio to render an html document.
I need to specify some parameters in the yml file but the qmd file returns
"object 'params' not found". Using knitR.
I use the default yml file where I have added params under the book tag
project:
type: book
book:
title: "Params_TEst"
author: "Jane Doe"
date: "15/07/2022"
params:
pcn: 0.1
chapters:
- index.qmd
- intro.qmd
- summary.qmd
- references.qmd
bibliography: references.bib
format:
html:
theme: cosmo
pdf:
documentclass: scrreprt
editor: visual
and the qmd file looks like this
# Preface {.unnumbered}
This is a Quarto book.
To learn more about Quarto books visit <https://quarto.org/docs/books>.
```{r}
1 + 1
params$pcn
When I render the book, or preview the book in Rstudio the error I receive is:
Quitting from lines 8-10 (index.qmd)
Error in eval(expr, envir, enclos) : object 'params' not found
Calls: .main ... withVisible -> eval_with_user_handlers -> eval -> eval
I have experimented placing the params line in the yml in different places but nothing works so far.
Could anybody help?
For multi-page renders, e.g. quarto books, you need to add the YAML to each page, not in the _quarto.yml file
So in your case, each of the chapters that calls a parameter needs a YAML header, like index.qmd, intro.qmd, and summary.qmd, but perhaps not references.qmd.
The YAML header should look just like it does in a standard Rmd. So for example, your index.qmd would look like this:
---
params:
pcn: 0.1
---
# Preface {.unnumbered}
This is a Quarto book.
To learn more about Quarto books visit <https://quarto.org/docs/books>.
```{r}
1 + 1
params$pcn
But, what if you need to change the parameter and re-render?
Then simply pass new parameters to the quarto_render function
quarto::quarto_render(input = here::here("quarto"), #expecting a dir to render
output_format = "html", #output dir is set in _quarto.yml
cache_refresh = TRUE,
execute_params = list(pcn = 0.2))
For now, this only seems to work if you add the parameters to each individual page front-matter YAML.
If you have a large number of pages and need to keep parameters centralized, a workaround is to run a preprocessing script that replaces the parameters in all pages. To add a preprocessing script, add the key pre-render to your _quarto.yml file. The Quarto website has detailed instructions.
For example, if you have N pages named index<N>.qmd, you could have a placeholder in the YML of each page:
---
title: This is chapter N
yourparamplaceholder
---
Your pre-render script could replace yourparamplaceholder with the desired parameters. Here's an example Python script:
for filename in os.listdir(dir):
if filename.endswith(".qmd"):
with open(filename, "r") as f:
txt = f.read()
f.replace('yourparamplaceholder', 'params:\n\tpcn: 0.1\n\tother:20\n')
with open(filename, "w") as ff:
ff.write(txt)
I agree with you that being able to set parameters centrally would be a good idea.
I am using Sphinx 4.4.0 with napoleon extension (Google Doc String). I have this two problems
ARNING: Block quote ends without a blank line; unexpected unindent.
ERROR: Unexpected indentation.
I found something about it on the internet but can not fit this two my code. My problem is I even do not understand the messages. I do not see where the problem could be.
This is the code:
def read_and_validate_csv(basename, specs_and_rules):
"""Read a CSV file with respect to specifications about format and
rules about valid values.
Hints: Do not use objects of type type (e.g. str instead of "str") when
specificing the column type.
specs_and_rules = {
'TEMPLATES': {
'T1l': ('Int16', [-9, ' '])
},
'ColumnA': 'str',
'ColumnB': ('str', 'no answer'),
'ColumnC': None,
'ColumnD': (
'Int16',
-9, {
'len': [1, 2, (4-8)],
'val': [0, 1, (3-9)]
}
}
Returns:
(pandas.DataFrame): Result.
"""
This are the original messages:
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:11: WARNING: Block quote ends without a blank line; unexpected unindent.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:15: ERROR: Unexpected indentation.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:17: ERROR: Unexpected indentation.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:19: WARNING: Block quote ends without a blank line; unexpected unindent.
.../bandas.py:docstring of buhtzology.bandas.read_and_validate_csv:20: WARNING: Block quote ends without a blank line; unexpected unindent.
reStructuredText is not Markdown, and indentation alone is not enough to demarcate the code block. reStructuredText calls this a literal block. Although the use of :: is one option, you might want to explicitly specify the language (overriding the default) with the use of the code-block directive.
Also I noticed that you have invalid syntax in your code block—a missing ) and extra spaces in your indentation—which could have caused those errors.
Try this.
def read_and_validate_csv(basename, specs_and_rules):
"""Read a CSV file with respect to specifications about format and
rules about valid values.
Hints: Do not use objects of type type (e.g. str instead of "str") when
specificing the column type.
.. code-block:: python
specs_and_rules = {
'TEMPLATES': {
'T1l': ('Int16', [-9, ' '])
},
'ColumnA': 'str',
'ColumnB': ('str', 'no answer'),
'ColumnC': None,
'ColumnD': (
'Int16',
-9, {
'len': [1, 2, (4-8)],
'val': [0, 1, (3-9)]
}
)
}
Returns:
(pandas.DataFrame): Result.
"""
Is there a good way with ruamel.yaml to dump a YAML file back out in the same version as it is loaded in? If I have a %YAML 1.1 directive in the file I would like to be able to dump the file back out in YAML 1.1 without having to hard-code version='1.1'.
So given some data like,
%YAML 1.1
---
is_string: 'on'
is_boolean: on
I would like to avoid hard-coding version='1.1' on the round_trip_dump(),
x = f.read()
d = round_trip_load(x)
round_trip_dump(d, f, explicit_start=True)
The version of the YAML file is a fleeting value, which gets reset after loading. It was (is) my plan to make the version of the latest document loaded available somehow, but with multiple documents in a stream this needs some more thought.
For single document streams you can do the following to capture the version from the directive. This is all done with the new API. With the old API that you are using in the example the same is possible, but more difficult because there is no YAML() instance to attach attributes to:
import sys
from ruamel.yaml import YAML
from ruamel.yaml.parser import Parser
yaml_str = """\
%YAML 1.1
---
is_string: 'on'
is_boolean: on
"""
class MyParser(Parser):
def dispose(self):
self.loader.last_yaml_version = self.yaml_version
Parser.dispose(self)
yaml = YAML()
yaml.Parser = MyParser
data = yaml.load(yaml_str)
yaml2 = YAML()
yaml2.version = yaml.last_yaml_version
yaml2.dump(data, sys.stdout)
which gives:
%YAML 1.1
---
is_string: 'on'
is_boolean: true
Please note that it is necessary to create a clean, new object for output, as the "unversioned" reading doesn't fully reset the yaml instance, when encountering the %YAML 1.1 directive.
It is also possible to dump the value associated with is_boolean as on, but that would affect all booleans in the stream.
I want to have a multi-line bit of markdown java in a yam file. I tried many things but I guess I don't quite get the quoting rules of Yaml.
{
title: Museum,
body: |
"```java
code code code
java2",
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
}
Produces:
/Users/pitosalas/.rbenv/versions/2.3.1/lib/ruby/2.3.0/psych.rb:377:in `parse': (generator/test.yml): found character that cannot start any token while scanning for the next token at line 3 column 9 (Psych::SyntaxError)
YAML has two styles: the JSON like flow style and the much better human readable block style.
Roughly speaking you can have nested structures each style nested within itself and can have flow style nested within block style, but block style nested within flow style is not allowed.
Your to level { and } are flow style but you try to introduce, with |, a literal block style scalar within that flow style. Replace the flow style with block style upwards from that scalar:
title: Museum
body: |
"```java
code code code
java2"
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
and your YAML is fine. Note that the double quotes "around" the value for the key body are not going to be stripped when loading, maybe that is not what you intended.
You should IMO not leave out the trailing , after the last value in the (flow style) sequence that is the value for answers. This will certainly lead to errors when you extend the list and forget to put in the trailing comma on the line above.
I would personally go for block style all the way:
title: Museum
body: |
"```java
code code code
java2"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
When dealing with YAML file generation that is convoluted or complex, or when it's not working as I expect, I revert to letting Ruby show me the way:
require 'yaml'
body = <<EOT
"```java
code code code
java2
"
EOT
answers = %w(
`museum`
`museum.getFloor(3)`
`museum.getFloor(3).getExhibit(5)`
`museum.getFloor(3).getExhibit(5).getCurator()`
`museum.getFloor(3).getExhibit(5).getCurator().name`
`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`
)
obj = {
"title" => "Museum",
"body" => body,
"answers" => answers
}
puts obj.to_yaml
Which, in this case, outputs:
---
title: Museum
body: |
"```java
code code code
java2
"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
If you then pass that YAML back into the parser, you should get the original data structure back:
YAML.load(obj.to_yaml)
# => {"title"=>"Museum",
# "body"=>"\"```java\n" +
# "code code code\n" +
# "java2\n" +
# "\"\n",
# "answers"=>
# ["`museum`",
# "`museum.getFloor(3)`",
# "`museum.getFloor(3).getExhibit(5)`",
# "`museum.getFloor(3).getExhibit(5).getCurator()`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"]}
I started using SST (selenium simple test) but ran into a problem when executing what seems to be a valid xpath expression '//div[div#data-type="folder-name"]'. SST fails with the following traceback:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/dist-packages/sst/cases.py", line 207, in run_test_script exec self.code in self.context
File "./sst-one.py", line 7, in <module> names = get_elements_by_xpath('//div[div#data-type="folder-name"]')
File "/usr/local/lib/python2.7/dist-packages/sst/actions.py", line 1344, in >get_elements_by_xpath_raise(msg)
File "/usr/local/lib/python2.7/dist-packages/sst/actions.py", line 118, in _raise raise AssertionError(msg)
AssertionError: Element not found: Message: u'The given selector //div[div#data->type="folder-name"] is either invalid or does not result in a WebElement. The following >error occurred:\nInvalidSelectorError: Unable to locate an element with the xpath >expression //div[div#data-type="folder-name"] because of the following >error:\n[Exception... "The expression is not a legal expression." code: "12" nsresult: >"0x805b0033 (SyntaxError)" location: "file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js> Line: 5916"]' ;
Stacktrace:
at FirefoxDriver.annotateInvalidSelectorError_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8873)
at FirefoxDriver.prototype.findElementsInternal_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8931)
at FirefoxDriver.prototype.findElements file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/driver_component.js:8935)
at DelayedCommand.prototype.executeInternal_/h file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10840)
at DelayedCommand.prototype.executeInternal_ (file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10845)
at DelayedCommand.prototype.execute/< file:///tmp/tmp12zCta/extensions/fxdriver#googlecode.com/components/command_processor.js:10787)
The SST code line is:
names = get_elements_by_xpath('//div[div#data-type="folder-name"]')
If I run the same xpath statement using selenium (from python), not SST, it is a valid expression and returns the matching elements if there were any.
The pure selenium code line (where br is the firefox webdriver object) is:
elems = br.find_elements_by_xpath("//div[div/#data-type='folder-name']")
I understand this is a bit in the weeds, so if anyone has some hints as to how to debug the .js code that selenium creates in the /tmp directory that would be a big help.
It's not a valid XPath expression. This
//div[div#data-type="folder-name"]
Should probably read
//div[div/#data-type="folder-name"]
which is indeed what you have in one of your examples.
If you are trying to get all possible div tags with data-type="folder-name",
//div[#data-type="folder-name"]
If only children of the first div with data-type="folder-name",
//div[div/#data-type="folder-name"]
You don't show any markup, up i assuming you have
<div><div data-type="folder-name"></div></div>
you need:
elems = br.find_elements_by_xpath("//div//div[#data-type='folder-name']")