I've been searching around for a bit and couldn't find anything that really helped me. Especially because sometimes things don't seem to be consistant.
I have the following YAML that I use to store data/ configuration stuff:
---
global:
name: Core Config
cfg_version: 0.0.1
databases:
main_database:
name: Main
path: ~/Documents/main.reevault
read_only: false
...
I know how to update fields with:
cfg = YAML.load_file("test.yml")
cfg['global']['name'] = 'New Name'
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
And that's essentially everybody on the internet talks about. However here is my problem: I want to dynamically be able to add new fields to that file. e.g. under the "databases" section have a "secondary_db" field with it's own name, path and read_only boolean. I would have expected to do that by just adding stuff to the hash:
cfg['global']['databases']['second_db'] = nil
cfg['global']['databases']['second_db']['name'] = "Secondary Database"
cfg['global']['databases']['second_db']['path'] = "http://someurl.remote/blob/db.reevault"
cfg['global']['databases']['second_db']['read_only'] = "true"
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
But I get this error:
`<main>': undefined method `[]=' for nil:NilClass (NoMethodError)
My question now is: how do I do this? Is there a way with the YAML interface? Or do I have to write stuff into the file manually? I would prefer something via the YAML module as it takes care of formatting/ indentation for me.
Hope someone can help me.
Yo have to initialize cfg['global']['database']['second_db'] to be a hash not nil. Try this cfg['global']['database']['second_db'] = {}
Related
I am building an open source project using Ruby for testing HTTP services: https://github.com/Comcast/http-blackbox-test-tool
I want to be able to reference environment variables in my test-plan.yaml file. I could use ERB, however I don't want to support embedding any random Ruby code and ERB syntax is odd for non-rubyists, I just want to access environment variables using the commonly used Unix style ${ENV_VAR} syntax.
e.g.
order-lunch-app-health:
request:
url: ${ORDER_APP_URL}
headers:
content-type: 'application/text'
method: get
expectedResponse:
statusCode: 200
maxRetryCount: 5
All examples I have found for Ruby use ERB. Does anyone have a suggestion on the best way to deal with this? I an open to using another tool to preprocess the YAML and then send that to the Ruby application.
I believe something like this should work under most circumstances:
require 'yaml'
def load_yaml(file)
content = File.read file
content.gsub! /\${([^}]+)}/ do
ENV[$1]
end
YAML.load content
end
p load_yaml 'sample.yml'
As opposed to my original answer, this is both simpler and handles undefined ENV variables well.
Try with this YAML:
# sample.yml
path: ${PATH}
home: ${HOME}
error: ${NO_SUCH_VAR}
Original Answer (left here for reference)
There are several ways to do it. If you want to allow your users to use the ${VAR} syntax, then perhaps one way would be to first convert these variables to Ruby string substitution format %{VAR} and then evaluate all environment variables together.
Here is a rough proof of concept:
require 'yaml'
# Transform environments to a hash of { symbol: value }
env_hash = ENV.to_h.transform_keys(&:to_sym)
# Load the file and convert ${ANYTHING} to %{ANYTHING}
content = File.read 'sample.yml'
content.gsub! /\${([^}]+)}/, "%{\\1}"
# Use Ruby string substitution to replace %{VARS}
content %= env_hash
# Done
yaml = YAML.load content
p yaml
Use it with this sample.yml for instance:
# sample.yml
path: ${PATH}
home: ${HOME}
There are many ways this can be improved upon of course.
Preprocessing is easy, and I recommend you use a YAML loaderd/dumper
based solution, as the replacement might require quotes around the
replacement scalar. (E.g. you substitute the string true, if that
were not quoted, the resulting YAML would be read as a boolean).
Assuming your "source" is in input.yaml and your env. variable
ORDER_APP_URL set to https://some.site/and/url. And the following
script in expand.py:
import sys
import os
from pathlib import Path
import ruamel.yaml
def substenv(d, env):
if isinstance(d, dict):
for k, v in d.items():
if isinstance(v, str) and '${' in v:
d[k] = v.replace('${', '{').format(**env)
else:
substenv(v, env)
elif isinstance(d, list):
for idx, item in enumerate(d):
if isinstance(v, str) and '${' in v:
d[idx] = item.replace('${', '{').format(**env)
else:
substenv(item, env)
yaml = ruamel.yaml.YAML()
yaml.preserve_quotes = True
data = yaml.load(Path(sys.argv[1]))
substenv(data, os.environ)
yaml.dump(data, Path(sys.argv[2]))
You can then do:
python expand.py input.yaml output.yaml
which writes output.yaml:
order-lunch-app-health:
request:
url: https://some.site/and/url
headers:
content-type: 'application/text'
method: get
expectedResponse:
statusCode: 200
maxRetryCount: 5
Please note that the spurious quotes around 'application/text' are preserved, as would be any comments
in the original file.
Quotes around the substituted URL are not necessary, but the would have been added if they were.
The substenv routine recursively traverses the loaded data, and substitutes even if the substitution is in mid-scalar, and if there are more than substitution in one scalar. You can "tighten" the test:
if isinstance(v, str) and '${' in v:
if that would match too many strings loaded from YAML.
I need to remove the tags field from each of the methods in my OpenAPI spec.
The spec must be in YAML format, as converting to JSON causes issues later on when publishing.
I couldn't find a ready tool for that, and my programming skills are insufficient. I tried Python with ruamel.yaml, but could not achieve anything.
I'm open to any suggestions how to approach this - a repo with a ready tool somewhere, a hint on what to try in Python... I'm out of my own ideas.
Maybe a regex that catches all cases all instances of tags so I can do a search and replace with Python, replacing them with nothing? Empty lines don't seem to break the publishing engine.
Here's a sample YAML piece (I know this is not a proper spec, just want to show where in the YAML tags sits)
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint
get:
tags:
-
tag1
-
tag3
#rest of GET definition
post:
tags:
- tag2
/anotherEndpoint
post:
tags:
- tag1
I need to get rid of all tags arrays entirely (not just make them empty)
I am not sure why you couldn't achieve anything with Python + ruamel.yaml. Assuing your spec
is in a file input.yaml:
import sys
from pathlib import Path
import ruamel.yaml
in_file = Path('input.yaml')
out_file = Path('output.yaml')
yaml = ruamel.yaml.YAML()
yaml.indent(mapping=4, sequence=4, offset=2)
yaml.preserve_quotes = True
data = yaml.load(in_file)
# if you only have the three instances of 'tags', you can hard-code them:
# del data['paths']['/endpoint']['get']['tags']
# del data['paths']['/endpoint']['post']['tags']
# del data['paths']['/anotherEndpoint']['post']['tags']
# generic recursive removal of any key names 'tags' in the datastructure:
def rm_tags(d):
if isinstance(d, dict):
if 'tags' in d:
del d['tags']
for k in d:
rm_tags(d[k])
elif isinstance(d, list):
for item in d:
rm_tags(item)
rm_tags(data)
yaml.dump(data, out_file)
which gives as output.yaml:
openapi: 3.0.0
info:
title: ""
description: ""
paths:
/endpoint:
get: {}
post: {}
/anotherEndpoint:
post: {}
You can write back data to input.yaml if you need that.
Please note that normally the comment #rest of GET definition would be preserved, but
not now as it is associated during loading with the key before it and that key gets deleted.
I'm using active_admin and having issues setting up multiple namespaces.
Relevant files:
config/initializers/active_admin.rb:
config.load_paths = [File.join(Rails.root, "app", "active_admin")]
config.default_namespace = :superadmin
config.namespace :superadmin do |superadmin|
superadmin.site_title = "superadmin"
superadmin.authentication_method = :authenticate_superadmin_user!
end
config.namespace :admins do |admins|
admins.site_title = "admins"
admins.authentication_method = :authenticate_admin_user!
end
Then I have two subfolders, /admins and /superadmin:
app/active_admin/superadmin/dashboard:
ActiveAdmin.register_page "Dashboard" do
...
app/active_admin/superadmin/product_product.rb:
ActiveAdmin.register Product::Product, as: Product do
...
So far, so good.
Now, I can't seem to figure out how to add a dashboard for admins (or any other pages/models) without it blowing up.
If I go to localhost:3000/admins, I get the error uninitialized constant Admins::DashboardController.
I've tried adding:
app/active_admin/admins/dashboard.rb:
ActiveAdmin.register_page "Dashboard do
...
(which does nothing)
And namespaced:
app/active_admin/admins/dashboard.rb:
ActiveAdmin.register_page "Dashboard, namespace: "admins" do
...
which gives the error:
Invalid route name, already in use: 'admins_root'
You may have defined two routes with the same name using the `:as` option,
or you may be overriding a route already defined by a resource
with the same naming
I've also tried adding the subfolders in the load_paths:
config.load_paths = [
File.join(Rails.root, "app", "active_admin"),
File.join(Rails.root, "app", "active_admin", "admins"),
File.join(Rails.root, "app", "active_admin", "superadmin")
]
Please note that I've looked at the following, but none seem to actually work...
http://activeadmin.info/docs/1-general-configuration.html#namespaces
2 Namespaces of activeadmin
How to use multiple Active Admin instances for Complete Separate Models
EDIT:
I've also tried to remove config.default_namespace and the only thing that did was break localhost:3000/superadmin so that it has the error:
uninitialized constant Superadmin::DashboardController
I have a YAML file like that:
---
name: dummy
version: 0.2.0
title: dummy
summary: dummy
Now I tried to get the version number:
config = YAML.load_file('Index.yml')
oldversion = config[0]['version']
Why do it get the following error:
NoMethodError: undefined method `[]' for nil:NilClass
Try with this:
config = YAML.load_file('Index.yml')
oldversion = config['version']
With config[0] you are indexing a sequence/array, and at the top level of your yaml file you have a mapping, not a sequence, so leave that out: oldversion = config['version'] or change your YAML file to:
---
- name: dummy
version: 0.2.0
title: dummy
summary: dummy
if you eventually want a list of such objects (with name, version, etc.) in your configuration.
Environment : Rails 3.1.1 and Rspec 2.10.1
I am loading all my application configuration through an external YAML file. My initializer (config/initializers/load_config.rb) looks like this
AppConfig = YAML.load_file("#{RAILS_ROOT}/config/config.yml")[RAILS_ENV]
And my YAML file sits under config/config.yml
development:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..12
test:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..11
And I access these values as, For example AppConfig['feesecure_eligible_months_for_monthly_payments']
In one of my tests I need AppConfig['feesecure_eligible_months_for_monthly_payments'] to return a different value but am not sure how to accomplish this. I tried the following approach with no luck
describe 'monthly_option_available?' do
before :each do
#policy = FeeSecure::Policy.new
#settlement_breakdown = SettlementBreakdown.new
#policy.stub(:settlement_breakdown).and_return(#settlement_breakdown)
#date = Date.today
Date.should_receive(:today).and_return(#date)
#config = mock(AppConfig)
AppConfig.stub(:feesecure_eligible_months_for_monthly_payments).and_return('1..7')
end
.....
end
In my respective class I am doing something like this
class Policy
def eligible_month?
eval(AppConfig['feesecure_eligible_months_for_monthly_payments']).include?(Date.today.month)
end
....
end
Can someone please point me in the right direction!!
The method that is being called when you do AppConfig['foo'] is the [] method, which takes one argument (the key to retrieve)
Therefore what you could do in your test is
AppConfig.stub(:[]).and_return('1..11')
You can use with to setup different expectations based on the value of the argument, ie
AppConfig.stub(:[]).with('foo').and_return('1..11')
AppConfig.stub(:[]).with('bar').and_return(3)
You don't need to setup a mock AppConfig object - you can stick your stub straight on the 'real' one.