Specifying parameters in yml file for Quarto - rstudio

I am creating a quarto book project in RStudio to render an html document.
I need to specify some parameters in the yml file but the qmd file returns
"object 'params' not found". Using knitR.
I use the default yml file where I have added params under the book tag
project:
type: book
book:
title: "Params_TEst"
author: "Jane Doe"
date: "15/07/2022"
params:
pcn: 0.1
chapters:
- index.qmd
- intro.qmd
- summary.qmd
- references.qmd
bibliography: references.bib
format:
html:
theme: cosmo
pdf:
documentclass: scrreprt
editor: visual
and the qmd file looks like this
# Preface {.unnumbered}
This is a Quarto book.
To learn more about Quarto books visit <https://quarto.org/docs/books>.
```{r}
1 + 1
params$pcn
When I render the book, or preview the book in Rstudio the error I receive is:
Quitting from lines 8-10 (index.qmd)
Error in eval(expr, envir, enclos) : object 'params' not found
Calls: .main ... withVisible -> eval_with_user_handlers -> eval -> eval
I have experimented placing the params line in the yml in different places but nothing works so far.
Could anybody help?

For multi-page renders, e.g. quarto books, you need to add the YAML to each page, not in the _quarto.yml file
So in your case, each of the chapters that calls a parameter needs a YAML header, like index.qmd, intro.qmd, and summary.qmd, but perhaps not references.qmd.
The YAML header should look just like it does in a standard Rmd. So for example, your index.qmd would look like this:
---
params:
pcn: 0.1
---
# Preface {.unnumbered}
This is a Quarto book.
To learn more about Quarto books visit <https://quarto.org/docs/books>.
```{r}
1 + 1
params$pcn
But, what if you need to change the parameter and re-render?
Then simply pass new parameters to the quarto_render function
quarto::quarto_render(input = here::here("quarto"), #expecting a dir to render
output_format = "html", #output dir is set in _quarto.yml
cache_refresh = TRUE,
execute_params = list(pcn = 0.2))

For now, this only seems to work if you add the parameters to each individual page front-matter YAML.
If you have a large number of pages and need to keep parameters centralized, a workaround is to run a preprocessing script that replaces the parameters in all pages. To add a preprocessing script, add the key pre-render to your _quarto.yml file. The Quarto website has detailed instructions.
For example, if you have N pages named index<N>.qmd, you could have a placeholder in the YML of each page:
---
title: This is chapter N
yourparamplaceholder
---
Your pre-render script could replace yourparamplaceholder with the desired parameters. Here's an example Python script:
for filename in os.listdir(dir):
if filename.endswith(".qmd"):
with open(filename, "r") as f:
txt = f.read()
f.replace('yourparamplaceholder', 'params:\n\tpcn: 0.1\n\tother:20\n')
with open(filename, "w") as ff:
ff.write(txt)
I agree with you that being able to set parameters centrally would be a good idea.

Related

Google Cloud DLP - CSV inspection

I'm trying to inspect a CSV file and there are no findings being returned (I'm using the EMAIL_ADDRESS info type and the addresses I'm using are coming up with positive hits here: https://cloud.google.com/dlp/demo/#!/). I'm sending the CSV file into inspect_content with a byte_item as follows:
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
In looking at the supported file types, it looks like CSV/TSV files are inspected via Structured Parsing.
For CSV/TSV does that mean one can't just sent in the file, and needs to use the table attribute instead of byte_item as per https://cloud.google.com/dlp/docs/inspecting-structured-text?
What about for XSLX files for example? They're an unspecified file type so I tried with a configuration like so, but it still returned no findings:
byte_item: {
type: :BYTES_TYPE_UNSPECIFIED,
data: File.open('/xxxxx/dlptest.xlsx', 'rb').read
}
I'm able to do inspection and redaction with images and text fine, but having a bit of a problem with other file types. Any ideas/suggestions welcome! Thanks!
Edit: The contents of the CSV in question:
$ cat ~/Downloads/dlptest.csv
dylans#gmail.com,anotehu,steve#example.com
blah blah,anoteuh,
aonteuh,
$ file ~/Downloads/dlptest.csv
~/Downloads/dlptest.csv: ASCII text, with CRLF line terminators
The full request:
parent = "projects/xxxxxxxx/global"
inspect_config = {
info_types: [{name: "EMAIL_ADDRESS"}],
min_likelihood: :POSSIBLE,
limits: { max_findings_per_request: 0 },
include_quote: true
}
request = {
parent: parent,
inspect_config: inspect_config,
item: {
byte_item: {
type: :CSV,
data: File.open('/xxxxx/dlptest.csv', 'r').read
}
}
}
dlp = Google::Cloud::Dlp.dlp_service
response = dlp.inspect_content(request)
The CSV file I was testing with was something I created using Google Sheets and exported as a CSV, however, the file showed locally as a "text/plain; charset=us-ascii". I downloaded a CSV off the internet and it had a mime of "text/csv; charset=utf-8". This is the one that worked. So it looks like my issue was specifically due the file being an incorrect mime type.
xlsx is not yet supported. Coming soon. (Maybe that part of the question should be split out from the CSV debugging issue.)

Remove a field from YAML OpenAPI specification

I need to remove the tags field from each of the methods in my OpenAPI spec.
The spec must be in YAML format, as converting to JSON causes issues later on when publishing.
I couldn't find a ready tool for that, and my programming skills are insufficient. I tried Python with ruamel.yaml, but could not achieve anything.
I'm open to any suggestions how to approach this - a repo with a ready tool somewhere, a hint on what to try in Python... I'm out of my own ideas.
Maybe a regex that catches all cases all instances of tags so I can do a search and replace with Python, replacing them with nothing? Empty lines don't seem to break the publishing engine.
Here's a sample YAML piece (I know this is not a proper spec, just want to show where in the YAML tags sits)
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint
get:
tags:
-
tag1
-
tag3
#rest of GET definition
post:
tags:
- tag2
/anotherEndpoint
post:
tags:
- tag1
I need to get rid of all tags arrays entirely (not just make them empty)
I am not sure why you couldn't achieve anything with Python + ruamel.yaml. Assuing your spec
is in a file input.yaml:
import sys
from pathlib import Path
import ruamel.yaml
in_file = Path('input.yaml')
out_file = Path('output.yaml')
yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=2)
yaml.preserve_quotes = True
data = yaml.load(in_file)
# if you only have the three instances of 'tags', you can hard-code them:
# del data['paths']['/endpoint']['get']['tags']
# del data['paths']['/endpoint']['post']['tags']
# del data['paths']['/anotherEndpoint']['post']['tags']
# generic recursive removal of any key names 'tags' in the datastructure:
def rm_tags(d):
if isinstance(d, dict):
if 'tags' in d:
del d['tags']
for k in d:
rm_tags(d[k])
elif isinstance(d, list):
for item in d:
rm_tags(item)
rm_tags(data)
yaml.dump(data, out_file)
which gives as output.yaml:
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint:
get: {}
post: {}
/anotherEndpoint:
post: {}
You can write back data to input.yaml if you need that.
Please note that normally the comment #rest of GET definition would be preserved, but
not now as it is associated during loading with the key before it and that key gets deleted.

Is it possible to reuse hyperlink defined in another file in restructuredtext (or sphinx)

Suppose I have two files a.rst and b.rst in the same folder, and a.rst looks like this
.. _foo: http://stackoverflow.com
`foo`_ is a website
It seems using foo in b.rst is not allowed. Is there a way to define hyperlinks and use them in multiple files?
Followup
I used the extlinks extension as Steve Piercy suggested. Its implementation and docstring can be seen here on github.
In my case, I define wikipedia link in my conf.py
extlinks = {'wiki': ('https://en.wikipedia.org/wiki/%s', '')}
and in the .rst files, use them like
:wiki:`Einstein <Albert_Einstein>`
where Einstein will be displayed as a link to https://en.wikipedia.org/wiki/Albert_Einstein
There are at least four possible solutions.
1. repeat yourself
Put your complete reST in each file. You probably don't want that.
2. combined rst_epilog and substitution
This one is clever. Configure the rst_epilog value, in your conf.py along with a substition with the replace directive:
rst_epilog = """
.. |foo| replace:: foo
.. _foo: http://stackoverflow.com
"""
and reST:
|foo|_ is a website
yields:
<a class="reference external" href="http://stackoverflow.com">foo</a>
3. extlinks
For links to external websites where you want to have a base URL and append path segments or arguments, you can use extlinks in your conf.py:
extensions = [
...
'sphinx.ext.extlinks',
...
]
...
extlinks = {'so': ('https://stackoverflow.com/%s', None)}
Then in your reST:
:so:`questions/49016433`
Yields:
<a class="reference external"
href="https://stackoverflow.com/questions/49016433">
https://stackoverflow.com/questions/49016433
</a>
4. intersphinx
For external websites that are documentation generated by Sphinx, then you can use intersphinx, in your conf.py:
extensions = [
...
'sphinx.ext.intersphinx',
...
]
...
intersphinx_mapping = {
'python': ('https://docs.python.org/3', None),
}
Then in your reST:
:py:mod:`doctest`
Yields:
<a class="reference external"
href="https://docs.python.org/3/library/doctest.html#module-doctest"
title="(in Python v3.6)">
<code class="xref py py-mod docutils literal">
<span class="pre">doctest</span>
</code>
</a>
This might come a bit late but I have found a solution that works very neatly for me and it is not among the answers already given.
In my case, I create a file with all the links used in my project, save it as /include/links.rst, and looking something like:
.. _PEP8: https://www.python.org/dev/peps/pep-0008/
.. _numpydoc: https://numpydoc.readthedocs.io/en/latest/format.html
.. _googledoc: https://google.github.io/styleguide/pyguide.html
Then there are the files a.rst and b.rst looking like:
.. include:: /include/links.rst
File A.rst
##########
Click `here <PEP8_>`_ to see the PEP8 coding style
Alternatively, visit either:
- `Numpy Style <numpydoc_>`_
- `Google Style <googledoc_>`_
and
.. include:: /include/links.rst
File B.rst
##########
You can visit `Python's PEP8 Style Guide <PEP8_>`_
For docstrings, you can use either `Numpy's <numpydoc_>`_ or `Google's <googledoc_>`_
respectively.
The produced output for both cases is:
and
respectively.
Moreover, I would like to emphasize the fact of which I was actually really struggling to achieve, to use different names (displayed text) for the same link at different locations and which I have achieved with the double _, one inside the <..._> and another outside.
This is another solution: it is a bit hacky and a little bit different respect to the officially supported way to share external links.
First complete the Setup then:
in conf.py add the commonlinks entry in extensions
in conf.py configure the map of common links:
For example:
extensions = [
...,
'sphinx.ext.commonlinks'
]
commonlinks = {
'issues': 'https://github.com/sphinx-doc/sphinx/issues',
'github': 'https://github.com'
}
Then in .rst files you can do these:
The :github:`_url_` url is aliased to :github:`GitHub` and also to :github:`this`
Setup
All that is needed is to copy into sphinx/ext directory the file commonlinks.py:
# -*- coding: utf-8 -*-
"""
sphinx.ext.commonlinks
~~~~~~~~~~~~~~~~~~~~~~
Extension to save typing and prevent hard-coding of common URLs in the reST
files.
This adds a new config value called ``commonlinks`` that is created like this::
commonlinks = {'exmpl': 'http://example.com/mypage.html', ...}
Now you can use e.g. :exmpl:`foo` in your documents. This will create a
link to ``http://example.com/mypage.html``. The link caption depends on the
role content:
- If it is ``_url_``, the caption will be the full URL.
- If it is a string, the caption will be the role content.
"""
from six import iteritems
from docutils import nodes, utils
import sphinx
from sphinx.util.nodes import split_explicit_title
def make_link_role(base_url):
def role(typ, rawtext, text, lineno, inliner, options={}, content=[]):
text = utils.unescape(text)
if text == '_url_':
title = base_url
else:
title = text
pnode = nodes.reference(title, title, internal=False, refuri=base_url)
return [pnode], []
return role
def setup_link_roles(app):
for name, base_url in iteritems(app.config.commonlinks):
app.add_role(name, make_link_role(base_url))
def setup(app):
app.add_config_value('commonlinks', {}, 'env')
app.connect('builder-inited', setup_link_roles)
return {'version': sphinx.__display_version__, 'parallel_read_safe': True}
To locate the sphinx installation directory one way is:
$ python 3
> import sphinx
> sphinx
<module 'sphinx' from '/usr/local/lib/python3.5/dist-packages/sphinx/__init__.py'>
then:
% cp commonlinks.py /usr/local/lib/python3.5/dist-packages/sphinx/ext

YML syntax error

I want to have a multi-line bit of markdown java in a yam file. I tried many things but I guess I don't quite get the quoting rules of Yaml.
{
title: Museum,
body: |
"```java
code code code
java2",
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
}
Produces:
/Users/pitosalas/.rbenv/versions/2.3.1/lib/ruby/2.3.0/psych.rb:377:in `parse': (generator/test.yml): found character that cannot start any token while scanning for the next token at line 3 column 9 (Psych::SyntaxError)
YAML has two styles: the JSON like flow style and the much better human readable block style.
Roughly speaking you can have nested structures each style nested within itself and can have flow style nested within block style, but block style nested within flow style is not allowed.
Your to level { and } are flow style but you try to introduce, with |, a literal block style scalar within that flow style. Replace the flow style with block style upwards from that scalar:
title: Museum
body: |
"```java
code code code
java2"
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
and your YAML is fine. Note that the double quotes "around" the value for the key body are not going to be stripped when loading, maybe that is not what you intended.
You should IMO not leave out the trailing , after the last value in the (flow style) sequence that is the value for answers. This will certainly lead to errors when you extend the list and forget to put in the trailing comma on the line above.
I would personally go for block style all the way:
title: Museum
body: |
"```java
code code code
java2"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
When dealing with YAML file generation that is convoluted or complex, or when it's not working as I expect, I revert to letting Ruby show me the way:
require 'yaml'
body = <<EOT
"```java
code code code
java2
"
EOT
answers = %w(
`museum`
`museum.getFloor(3)`
`museum.getFloor(3).getExhibit(5)`
`museum.getFloor(3).getExhibit(5).getCurator()`
`museum.getFloor(3).getExhibit(5).getCurator().name`
`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`
)
obj = {
"title" => "Museum",
"body" => body,
"answers" => answers
}
puts obj.to_yaml
Which, in this case, outputs:
---
title: Museum
body: |
"```java
code code code
java2
"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
If you then pass that YAML back into the parser, you should get the original data structure back:
YAML.load(obj.to_yaml)
# => {"title"=>"Museum",
# "body"=>"\"```java\n" +
# "code code code\n" +
# "java2\n" +
# "\"\n",
# "answers"=>
# ["`museum`",
# "`museum.getFloor(3)`",
# "`museum.getFloor(3).getExhibit(5)`",
# "`museum.getFloor(3).getExhibit(5).getCurator()`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"]}

Pass dynamic content to template in Middleman

I'm building a static site using Middleman that has a portfolio section of all the client's recent projects.
The portfolio section will display project thumbnail images in a 3 X 3 gallery fashion and, when clicked on, will open their co-responding html page inside a lightbox.
The layout for the pages inside the light box is the same so rather than markup each individual page, I thought there would be a way for Middleman handling the content served from a yaml data file (projects.yml) using [a link.
Here's what I've got in my config.rb file
###
# Page options, layouts, aliases and proxies
###
# A path which all have the same layout
with_layout :popup do
page "/projects/*"
end
# Proxy (fake) files
# page "/this-page-has-no-template.html", :proxy => "/template-file.html" do
# #which_fake_page = "Rendering a fake page with a variable"
# end
data.projects.details.each do |pd|
proxy "/projects/#{pd[:client_name]}.html", "/projects/template.html", locals: { project: pd }, ignore: true
end
Ok so after some digging I came across the two posts below which helped me under stand how dynamic pages work in middleman. (Unfortunately there's not a lot of doco and the Middleman example for Dynamic pages is really basic)
http://benfrain.com/understanding-middleman-the-static-site-generator-for-faster-prototyping/
http://forum.middlemanapp.com/discussion/134/best-way-to-use-yaml-same-html-but-parameter-driven-data-fixed/p1
My solution...
data/projects.yml (contains project details)
details:
- client: "Company X"
title: "Company X Event"
video_url: ""
logo:
- "logo_companyx.gif"
image_path: "/img/projects/companyx"
total_images: 10
content: "<p>Blah blah blah</p>"
responsibilities:
"<li>Something</li>
<li>Some task</li>"
config.rb:
data.projects.details.each do |pd|
proxy "/projects/#{pd[:client]}.html", "/projects/template.html", :layout => false, :locals => { :project => pd }, :ignore => true
end
The trick with the snippet above is passing the entire project data object to the template via a proxy using locals and setting the layout to false so it doesn't inherit the default site layout (as I - or the client rather - want to display these in a lightbox popup)
The last step in the process was to create /projects/template.html.erb (in the source folder), declaring the following at the top of the template
<% p = locals[:project] %>
This allowed me to output each property of the p object within template.html.erb.
eg:
<%= p[:title] %>
I hope this helps someone as it took me a few days of playing around and LOTS of searching online for example or hints.

Resources