I'm trying to understand some yaml syntax related to using hydra in a machine learning approach. So given the following, extracted from the original github repo:
datamodule:
_target_: cdvae.pl_data.datamodule.CrystDataModule
datasets:
train:
_target_: cdvae.pl_data.dataset.CrystDataset
name: Formation energy train
path: ${data.root_path}/train.csv
prop: ${data.prop}
niggli: ${data.niggli}
primitive: ${data.primitive}
graph_method: ${data.graph_method}
lattice_scale_method: ${data.lattice_scale_method}
preprocess_workers: ${data.preprocess_workers}
I don't understand what the syntax ${} stands for in this context. Is that some kind of text formatting? Or is it calling some specific module? Could anybody provide some examples?
EDIT:
Ok, as pointed by #flyx, it seems that the syntax ${} refers to hydra which is indeed part of the project I'm analyzing.
The string foo: ${bar} in yaml corresponds roughly to the dict {"foo": "${bar}"} in python. This is to say, the dollar-bracket notation ${} is not given special meaning by the yaml grammar.
OmegaConf, which is the backend used by Hydra, does give special meaning to the dollar-bracket syntax; it is used for "variable interpolation." See the OmegaConf docs on variable interpolation.
To summarize briefly, the idea of variable interpolation is that the string "${bar}" is a "pointer" to the value with key bar that would appear elsewhere in your OmegaConf config object.
Here's a short demo of variable interpolations in action:
from omegaconf import OmegaConf
yaml_data = """
foo: ${bar}
bar: baz
"""
config = OmegaConf.create(yaml_data)
assert config.bar == "baz"
assert config.foo == "baz" # the ${bar} interpolation points to the "baz" value
Related
I'm trying to complete this Codewars Challenge and I'm confused as to where I'm going wrong. Could someone please give me a hand?
The question provides a "database" of translations for Welcome, and the instructions say:
Think of a way to store the languages as a database (eg an object). The languages are listed below so you can copy and paste!
Write a 'welcome' function that takes a parameter 'language' (always a string), and returns a greeting - if you have it in your database. It should default to English if the language is not in the database, or in the event of an invalid input.
My attempt:
def greet(language)
greeting = { 'english'=>'Welcome',
'czech'=>'Vitejte',
'danish'=>'Velkomst',
'dutch'=>'Welkom',
'estonian'=>'Tere tulemast',
'finnish'=>'Tervetuloa',
'flemish'=>'Welgekomen',
'french'=>'Bienvenue',
'german'=>'Willkommen',
'irish'=>'Failte',
'italian'=>'Benvenuto',
'latvian'=>'Gaidits',
'lithuanian'=>'Laukiamas',
'polish'=>'Witamy',
'spanish'=>'Bienvenido',
'swedish'=>'Valkommen',
'welsh'=>'Croeso'
}
greeting.key?(language) ? greeting.each { |k, v| return v if language == k } : 'IP_ADDRESS_INVALID'
end
To my eyes when I run my code through the IDE it seems to be working as per request but I guess I must be wrong somehow.
It's telling me it :
Expected: "Laukiamas", instead got: "Welcome"
But when I type:
p greet("lithuanian")
I get Laukiamas.
You can provide you greeting hash with a default value. It is as simple as
greeting.default = "Welcome"
This enhanced hash does all the work for you. Just look up the key; when it is not there you'll get "Welcome".
Preface
First of all, please don't post links to exercises or homework questions. Quote them in your original question to avoid link rot or additional create work for people trying to help you out.
Understanding the Problem Defined by the Linked Question
Secondly, you're misunderstanding the core question. The requirement is basically to return the Hash value for a given language key if the key exists in the Hash. If it doesn't, then return the value of the 'english' key instead. Implicit in the exercise is to understand the various types of improper inputs that would fail to find a matching key; the solution below addresses most of them, and will work even if your Ruby has frozen strings enabled.
A Working Solution
There are lots of ways to do this, but here's a simple example that will handle invalid keys, nil as a language argument, and abstract away capitalization as a potential issue.
DEFAULT_LANG = 'english'
TRANSLATIONS = {
'english' => 'Welcome',
'czech' => 'Vitejte',
'danish' => 'Velkomst',
'dutch' => 'Welkom',
'estonian' => 'Tere tulemast',
'finnish' => 'Tervetuloa',
'flemish' => 'Welgekomen',
'french' => 'Bienvenue',
'german' => 'Willkommen',
'irish' => 'Failte',
'italian' => 'Benvenuto',
'latvian' => 'Gaidits',
'lithuanian' => 'Laukiamas',
'polish' => 'Witamy',
'spanish' => 'Bienvenido',
'swedish' => 'Valkommen',
'welsh' => 'Croeso'
}
# Return a translation of "Welcome" into the language
# passed as an argument.
#
# #param language [String, #to_s] any object that can
# be coerced into a String, and therefore to
# String#downcase
# #return [String] a translation of "Welcome" or the
# string-literal +Welcome+ if no translation found
def greet language
language = language.to_s.downcase
TRANSLATIONS.fetch language, TRANSLATIONS[DEFAULT_LANG]
end
# Everything in the following Array of examples except
# +Spanish+ should return the Hash value for +english+.
['Spanish', 'Español', 123, nil].map { greet(_1) }
This will correctly return:
#=> ["Bienvenido", "Welcome", "Welcome", "Welcome"]
because only Spanish (when lower-cased) will match any of the keys currently defined in the TRANSLATIONS Hash. All the rest will use the default value defined for the exercise.
Test Results
Since there are some RSpec tests included with the linked question:
describe "Welcome! Translation" do
it "should translate input" do
Test.assert_equals(greet('english'), 'Welcome', "It didn't work out this time, keep trying!");
Test.assert_equals(greet('dutch'), 'Welkom', "It didn't work out this time, keep trying!");
Test.assert_equals(greet('IP_ADDRESS_INVALID'), 'Welcome', "It didn't work out this time, keep trying!")
end
end
The code provided not only passes the provided tests, but it also passes a number of other edge cases not defined in the unit tests. When run against the defined tests, the code above passes cleanly:
If this is homework, then you might want to create additional tests to cover all the various edge cases. You might also choose to refactor to less idiomatic code if you want more explanatory variables, more explicit intermediate conversions, or more explicit key handling. The point of good code is to be readable, so be as explicit in your code and as thorough in your tests as you need to be in order to make debugging easier.
So I have a string from a rendered template that looks like
"Dear {{user_name}},\r\n\r\nThank you for your purchase. If you have any questions, we are happy to help.\r\n\r\n\r\n{{company_name}}\r\n{{company_phone_number}}\r\n"
All those variables like {{user_name}} are optional and do not need to be included but I want to check that if they are, they have {{ in front of the variable name. I am using liquid to parse and render the template and couldn't get it to catch if the user only uses 1 (or no) opening brackets. I was only able to catch the proper number of closing brackets. So I wrote a method to check that if these variables exist, they have the correct opening brackets. It only works, however, if all those variables are found.
here is my method:
def validate_opening_brackets?(template)
text = %w(user_name company_name company_phone_number)
text.all? do |variable|
next unless template.include? variable
template.include? "{{#{variable}"
end
end
It works, but only if all variables are present. If, for example, the template created by the user does not include user_name, then it will return false. I've also done this loop using each, and creating a variable outside of the block that I assign false if the conditions are not met. I would really, however, like to get this to work using the all? method, as I can just return a boolean and it's cleaner.
If the question is about how to rewrite the all? block to make it return true if all present variable names have two brackets before them and false otherwise then you could use something like this:
def validate_opening_brackets?(template)
variables = %w(user_name company_name company_phone_number)
variables.all? do |variable|
!template.include?(variable) || template.include?("{{#{variable}")
end
end
TL;DR
There are multiple ways to do this, but the easiest way I can think of is to simply prefix/postfix a regular expression with the escaped characters used by Mustache/Liquid, and using alternation to check for each of your variable names within the template variable characters (e.g. double curly braces). You can then use String#scan and then return a Boolean from Enumerable#any? based on the contents of the Array returned by from #scan.
This works with your posted example, but there may certainly be other use cases where you need a more complex solution. YMMV.
Example Code
This solution escapes the leading and trailing { and } characters to avoid having them treated as special characters, and then interpolates the variable names with | for alternation. It returns a Boolean depending on whether templated variables are found.
def template_string_has_interpolations? str
var_names = %w[user_name company_name company_phone_number]
regexp = /\{\{#{var_names.join ?|}\}\}/
str.scan(regexp).any?
end
Tested Examples
template_string_has_interpolations? "Dear {{user_name}},\r\n\r\nThank you for your purchase. If you have any questions, we are happy to help.\r\n\r\n\r\n{{company_name}}\r\n{{company_phone_number}}\r\n"
#=> true
template_string_has_interpolations? "Dear Customer,\r\n\r\nThank you for your purchase. If you have any questions, we are happy to help.\r\n\r\n\r\nCompany, Inc.\r\n(555) 555-5555\r\n"
#=> false
I have a yaml file that includes the following:
:common
:substitue
:foo: fee
I read this data like:
data = YAML.load(erb_data[File.basename(__FILE__, '.*')].result(binding))
common = data[:common]
def substitute_if_needed(original_value)
mapping = common.dig(:substitue, original_value)
if mapping.nil? ? original_value : mapping
end
Unfortunately, this doesn't do the substitution that I want. I want to call substitute_if_needed('foo') and get 'fee' back. I also want to call substitute_if_needed('bar') and get 'bar' back.
How can I do this?
There are several problems in your code:
YAML example looks broken. The proper one should looks like:
common:
substitute:
foo: fee
You're trying to fetch common key in common = data[:common] using a symbol as a key, but it should be a string (data["common"]). Also, I'd say it's a bad idea to spilt fetching logic into two pieces - first extract "common" outside of substitute_when_needed and then dig into it inside.
if statement is broken. It should be either proper if or proper ternary operator.
Fixing all this gives us something like (I've just replaced a file with StringIO for convenience - to make the snippet executable as is):
yaml = StringIO.new(<<~DATA)
common:
substitute:
foo: fee
DATA
def substitute_if_needed(data, original_value)
mapping = data.dig("common", "substitute", original_value)
mapping.nil? ? original_value : mapping
end
data = YAML.load(yaml)
substitute_if_needed(data, "foo") # => "fee"
substitute_if_needed(data, "bar") # => "bar"
I am getting Ruby Error: syntax error, unexpected tGVAR, expecting $end.
I am using Mechanize to access a website and then I need to enter data into the form to search. When I pp page the site to get the form information I get:
#<Mechanize::Form
<name nil>
<method "POST">
<action "">
<fields
...
...
[text:0xb43f9c type: text name: ct100$MainContent$txtNumber value: ]
...
My code that is throwing this is:
Check_form = page.form()
Check_form.ct100$MainContent$txtNumber = 'J520518'
Any ideas on what is causing the error? Thank you in advance for the help!
Since this is not a valid variable or syntactically valid method name, you should use the alternate method to fetch or assign the values:
check_form = page.form
check_form['ct100$MainContent$txtNumber'] = 'J520518'
Variables are of the form #x for class instance variables, ##x for class variables, $x for global variables and x for plain variables, but in all cases the variable must consist of a letter or underscore followed by any number of letters, numbers, or underscores. $ cannot appear anywhere but the beginning, and when it does that means "global variable", something rarely used in most Ruby programming.
The error is telling you that there is a global variable where Ruby doesn't expect one. And there is: $txtNumber is a global variable, but it doesn't make sense for a global variable to appear at that place in your code.
Another way to make it legal would be
Check_form.send(:"ct100$MainContent$txtNumber=", 'J520518')
So here is my problem.
I want to retrieve a string stored in a model and at runtime change a part of it using a variable from the rails application. Here is an example:
I have a Message model, which I use to store several unique messages. So different users have the same message, but I want to be able to show their name in the middle of the message, e.g.,
"Hi #{user.name}, ...."
I tried to store exactly that in the database but it gets escaped before showing in the view or gets interpolated when storing in the database, via the rails console.
Thanks in advance.
I don't see a reason to define custom string helper functions. Ruby offers very nice formatting approaches, e.g.:
"Hello %s" % ['world']
or
"Hello %{subject}" % { subject: 'world' }
Both examples return "Hello world".
If you want
"Hi #{user.name}, ...."
in your database, use single quotes or escape the # with a backslash to keep Ruby from interpolating the #{} stuff right away:
s = 'Hi #{user.name}, ....'
s = "Hi \#{user.name}, ...."
Then, later when you want to do the interpolation you could, if you were daring or trusted yourself, use eval:
s = pull_the_string_from_the_database
msg = eval '"' + s + '"'
Note that you'll have to turn s into a double quoted string in order for the eval to work. This will work but it isn't the nicest approach and leaves you open to all sorts of strange and confusing errors; it should be okay as long as you (or other trusted people) are writing the strings.
I think you'd be better off with a simple micro-templating system, even something as simple as this:
def fill_in(template, data)
template.gsub(/\{\{(\w+)\}\}/) { data[$1.to_sym] }
end
#...
fill_in('Hi {{user_name}}, ....', :user_name => 'Pancakes')
You could use whatever delimiters you wanted of course, I went with {{...}} because I've been using Mustache.js and Handlebars.js lately. This naive implementation has issues (no in-template formatting options, no delimiter escaping, ...) but it might be enough. If your templates get more complicated then maybe String#% or ERB might work better.
one way I can think of doing this is to have templates stored for example:
"hi name"
then have a function in models that just replaces the template tags (name) with the passed arguments.
It can also be User who logged in.
Because this new function will be a part of model, you can use it like just another field of model from anywhere in rails, including the html.erb file.
Hope that helps, let me know if you need more description.
Adding another possible solution using Procs:
#String can be stored in the database
string = "->(user){ 'Hello ' + user.name}"
proc = eval(string)
proc.call(User.find(1)) #=> "Hello Bob"
gsub is very powerful in Ruby.
It takes a hash as a second argument so you can supply it with a whitelist of keys to replace like that:
template = <<~STR
Hello %{user_email}!
You have %{user_voices_count} votes!
Greetings from the system
STR
template.gsub(/%{.*?}/, {
"%{user_email}" => 'schmijos#example.com',
"%{user_voices_count}" => 5,
"%{release_distributable_total}" => 131,
"%{entitlement_value}" => 2,
})
Compared to ERB it's secure. And it doesn't complain about single % and unused or inexistent keys like string interpolation with %(sprintf) does.