I'm trying to configure a properties file in a puppet module. The properties file should end up with content like this:
server_url=https://my-login.com/server
token_url=https://my-login.com/token
client_id=my-application
We have multiple environments that the application will be deployed to (development, staging and production), the server_url and token_url will be different in each environment, but the client_id will always be the same. So I made an erb template like this:
server_url=<%=#server_url%>
token_url=<%=#token_url%>
client_id=my-application
And a manifest file like this:
# $env will be DEV, STG or PRD
$server_url = 'https://my-login-dev.com/server'
$token_url = 'https://my-login-dev.com/token'
file {'/apps/my-application/common/client.properties':
ensure => 'present',
content => template("common/client.properties.erb"),
}
I want the values of the server_url and token_url to be set based on the value of the env variable. Ideally this would work by choosing a particular file to include. For example, have an urls.properties file for each environment, which the variables get loaded from. So urls.properties.dev would contain something like:
server_url=https://my-login.com/server
token_url=https://my-login.com/token
And then you could just include it like so:
# server_url and token_url set up by the following line
include urls.properties.${env}
file {'/apps/my-application/common/client.properties':
ensure => 'present',
content => template("common/client.properties.erb"),
}
I can't find any documentation on how to do this, and I can't find a way to make it work by hacking around.
Is it possible to achieve this? Is there another approach I should be taking to solve the problem? (I don't want to use partial templates, as I don't want the file to be split into lots of chunks).
Related
I am using a Jenkins plugin to upload test run results to Jira. Using this plugin I can send two JSON blobs of data for the import, but the variables in those JSON blobs can only be environment variables (not variables generally available in the Jenkinsfile).
When I run it is recognizing environment variables that come from the parameters block (this is a parameterized build), but it does not recognize any environment variables I set, either in an environment {} block in the pipeline or by nesting the build step in a withEnv() {} block.
As a sanity check, right before the step in question, I echo two environment variables, one from the parameters block and one from the environment block, and both spit out to the console as expected, but then, as consumed by the plugin, only the variables coming from the parameters block are read as variables, with the rest being left as string.
So is there some difference in how these environment variables are stored/managed behind the scenes that might play into this?
So, for example, here are the parameters and environment blocks:
parameters {
choice(name: 'ENVIRONMENT', choices: ['dev', 'test', 'staging', 'prod'], description: 'Select the environment to run against.')
choice(name: 'TESTS', choices: ['All', 'API', 'Web'], description: 'Select the tests to run.')
}
environment {
PROJECT_KEY = "$jiraProjectKey"
TEST_PLAN_KEY = "$testPlanKeys[$env.ENVIRONMENT]"
PRODUCT_NAME = "$productName"
TEAM_NAME = "$teamName"
}
When I used these environment variables in the JSON blobs to set the Summary field of a Test Execution in Jira with a line that looks like this:
...
"summary": "${ENVIRONMENT} - ${PRODUCT_NAME} - ${TESTS} Tests",
...
The resulting issue summary is:
dev - ${PRODUCT_NAME} - API Tests
So it will properly interpret the environment variables set by the parameters block, but not ones I set explicitly in the environment block.
In the JSON blobs that you are sending inline make sure that for multiline strings you are using """ to delimit those strings and not '''.
Replace:
... importInfo: '''{...'''
by:
...importInfo: """{..."""
Under the "Nesting Variables" section in Codeigniter4 site:
"To save on typing, you can reuse variables that you’ve already specified in the file by wrapping the variable name within ${...}"
link to CI nesting Variables section
example in the documentation:
BASE_DIR="/var/webroot/project-root"
CACHE_DIR="${BASE_DIR}/cache"
TMP_DIR="${BASE_DIR}/tmp"
I was trying to use the following
app.baseURL = 'http://localhost:8080/'
google.redirect = ${app.baseURL}Google
However, it's assigning it as a literal when print_r($_ENV)
[google.redirect] => ${app.baseURL}Google
I've tried using non-namespaced keys including BASE_DIR (per the example) and it keeps printing as a literal.
What's strange - When I use the following:
CI_ENVIRONMENT = development
google.redirect = ${CI_ENVIRONMENT}Google
The result when print_r is:
[CI_ENVIRONMENT] => development
[google.redirect] => developmentGoogle
My question is - What am I doing incorrectly and/or how should these be set/used correctly?
According to the documentation, I should be able to use any key within the .env file that was already assigned using
${somekeyinthisfile}
After a bit of looking, there is a more recent file up at
https://github.com/codeigniter4/CodeIgniter4/blob/develop/system/Config/DotEnv.php
with all the "other" changes...
This was a Bug Fix. So get that file and you will be good to go.
I am pretty sure that the intention wasn't to allow app.xxx settings to be used as variables as the documentation clearly shows, by not
showing them being used. ( yes its 6am now ...)
BUT it is your code to do with as you please...So if you want to use app.xxx as variables...
The Only Thing missing is the DOT (.) in the regex
If you look on Line 272 - system/Config/DotEnv.php inside method resolveNestedVariables() and add a . (dot) into the regex, that will make all your app.things work.
$value = preg_replace_callback(
'/\${([a-zA-Z0-9_.]+)}/',
function ($matchedPatterns) use ($loader) {
I have added a dot (.) at the end of the [a-zA-Z0-9_
So
'/\${([a-zA-Z0-9_]+)}/',
becomes
'/\${([a-zA-Z0-9_.]+)}/',
I'm doing one project for puppet, however currently stuck in one logic.
Thus, want to know can we fetch variable from .yaml, .json or plain text file in puppet manifest file.
For example,
My puppet manifest want to create user but the variable exist in the .yaml or any configuration file, hence need to fetch the varibale from the outside file. The puppet manifest also can do looping if it exist multiple users in .yaml file.
I read about hiera but let say we are not using hiera is there any possible way.
There are a number of ways you can do this using a combination of built-in and stdlib functions, at least for YAML and JSON.
Using the built-in file function and the parseyaml or parsejson stdlib functions:
Create a file at mymodule/files/myfile.yaml:
▶ cat files/myfile.yaml
---
foo: bar
Then in your manifests read it into a string and parse it:
$myhash = parseyaml(file('mymodule/myfile.yaml'))
notice($myhash)
That will output:
Notice: Scope(Class[mymodule]): {foo => bar}
Or, using the loadyaml or loadjson stdlib functions:
$myhash = loadyaml('/etc/puppet/data/myfile.yaml')
notice($myhash)
The problem with that approach is that you need to know the path to file on the Puppet master. Or, you could use a Puppet 6 deferred function and read the data from a file on the agent node.
(Whether or not you should do this is another matter entirely - hint: answer is you almost certainly should be using Hiera - but that isn't the question you asked.)
I'm trying to deploy Ghost 1.2.0 to Heroku. With previous versions of Ghost (<= 0.11.x), they used a config.js file where you could just do:
database: {
client: 'postgres',
connection: {
host: process.env.POSTGRES_HOST,
user: process.env.POSTGRES_USER,
password: process.env.POSTGRES_PASSWORD,
database: process.env.POSTGRES_DATABASE,
port: process.env.POSTGRES_PORT
},
debug: false
}, …
But in this version they're using nconf that replaces this config.js for environment dependent JSON files like config.production.json.
JSON files unlike JS objects can't have vars as values. I could hardcode my credentials to the JSON file, but I don't want to do that because:
Looks like a bad practise to me, and
Heroku rotates credentials periodically and updates applications where this database is attached. (Quoted from Heroku)
After some digging into nconf README and issues I understood that it would be possible to imitate this expected database object just with:
nconf.env({
separator: '__' // Two dashes
});
and defining vars as:
heroku config:set DATABASE__CLIENT=postgres
heroku config:set DATABASE__CONNECTION__HOST=<value>
...
but, no matter what, I get undefined when I later call:
nconf.get('database');
nconf.get('DATABASE'); // In case it was case-sensitive...
Instead, if I call:
nconf.get('DATABASE__CLIENT'); // postgres
it works. I could try (and I will) to modify Ghost scripts to read all variables this way, but as long as it expects a database object it'll be so cool to get it working the right way.
So, has anybody figured out how to correctly recreate an object with Heroku's env vars?
I've finally found the solution.
Unless you want to modify nconf.env(settings) like:
nconf.env({
separator: '__', // Two dashes
lowerCase: true
});
This will make it possible to pass lowerCase: true to env() so that if an environment variable is called SOMETHING or SOMEthing, it will also be gettable using something [Source]
I recommend to use already lowercase env vars.
So,
heroku config:set database__client=postgres
will be readable using:
nconf.get('database:client');
Looks like nconf has a different character separator to define nested variables called separator and another one to read them called logicalSeparator (its default value is :)
Is there a way to read in a file of environment variables?
In bash I have a file env.sh that I can use
env.sh
foo="bar"
bash file
set -a
source env.sh
This would allow me to just use foo as if I had delcared it in the ruby script.
Also is it there a way to make sure that this file is unreadable so that passwords could be stored in this file?
It sounds like you should provide a file example for the user/admin to modify for their personal environment, and then populate the environment from that, while avoiding, perhaps, having that file with the sensitive information in a repository. Note: per file security is going to be addressed by where the file is located and your operating system, and server software.
If this is the case, then you can provide a file that holds a template of the kind of things that you would require from the administrator/user of the program you are configuring.
Ruby has the ENV constant that acts like a Hash and holds the environment of the shell you are using.
As an example, there is a file called environment.rb.sample that gets shared with anyone, publicly. It has instructions and holds the template that users can modify freely, with instructions to copy the file to environment.rb. The sample file looks like this:
# environment.rb.sample
# Copy this file to environment.rb and change the name and password to your credentials
ENV['temp_user_name'] = 'Foo Bar'
ENV['temp_password'] = 'Dazz Kezz
The file is then copied to this, perhaps:
# environment.rb
ENV['temp_user_name'] = 'Joe Admin'
ENV['temp_password'] = 'Super Secure Password'
The file that loads this and uses it is just a Ruby file that is freely modified by the user/administrator of the software, and looks like this and is also shared publicly.
# load_environment
require './environment'
puts ENV['temp_user_name']
puts ENV['temp_password']
This loads the file and uses the ENV that is a globally scoped constant for the application.
The file permissions are then managed by the user/administrator of the system and secured like any other sensitive information on their system. The sensitive file should also be listed in the repository's ignore mechanism. It should never be made public.
Yes, there is, and if for some bizzare, arcane reason you must use it, it's eval:
WARNING: Never use this on a user-supplied file
And, unless you have a very, very specific need, don't use it in production code.
eval(File.read("name_of_var_file"), binding)
If what you're really trying to do is write a configuration file, use YAML. A file like this:
config.yaml:
foo: "bar"
Can be accessed like this:
require 'yaml'
conf = YAML.loads(File.read("config.yaml"))
conf['foo'] #=> 'bar'
This is secure and manageable, not to mention standard practice.
As for making the file inaccessible, that is an operating system level problem that can't be solved without information on the environment, OS, setup, etc.
The purpose of a local variable is to be used temporally within a method definition or a block. Using it outside of such environments, particularly across files defeats the purpose of it. You should not need to do it, and Ruby does not have a simple way to do it.
If you are using variables correctly, and want to share variables between files, that should be other types of variables such as instance, class, or global variables. Or, for the purpose of setting environments, you should be using constants. Among them, global variables and constants can be written in a file, loaded in a different file, and be used.
file-a.rb
$foo = 1
FOO = 2
file-b.rb
load "file-a.rb"
$foo # => 1
FOO # => 2
As for instance and class variables, they belong to a class or an instance of it, so they should be defined in such environment. And you can reopen the same class within a different file, and load it in another file.
file-a.rb
class Bar
##foo = 1
def initialize; #foo = 2 end
end
file-b.rb
load "file-a.rb"
Bar.class_variable_get("##foo") # => 1
Bar.new.instance_variable_get("#foo") # => 2