I want to know how I can use variable syntax inside config-repo yml files.
For example, there is a config-repo which has files like,
- foo.yml
- foo_develop.yml
- foo_production.yml
and inside foo.yml, I want to implement environment variables like,
log:
active: true
file:
auditLogFile: ${HOME}/log/audit.log
But above syntax prints out
{
log:
active: true,
file: {
auditLogFile: '${HOME}/log/audit.log',
},
}
Is something wrong with syntax above? or is it impossible to use env variables inside yml file?
Related
This my project strcuture at host:
set_env_vars.sh
dev/
docker-compose-dev.yml
/services/
web/
.env-dev? <------
project/
config.py
api/
resources/
auth.py
set_env_vars.sh
export SPOTIFY_CLIENT_ID=my_id
export SPOTIFY_CLIENT_SECRET=my_secret
export SPOTIFY_REDIRECT_URI=http://localhost
export SPOTIFY_CACHE_PATH=/project/api/auth/spotify/.cache
which I run like so:
$ source ./set_env_vars.sh
docker-compose-dev.yml
services:
web:
environment:
- FLASK_ENV=development
- APP_SETTINGS=project.config.DevelopmentConfig
- SPOTIFY_CLIENT_ID=${SPOTIFY_CLIENT_ID}
- SPOTIFY_CLIENT_SECRET=${SPOTIFY_CLIENT_SECRET}
- SPOTIFY_REDIRECT_URI=${SPOTIFY_REDIRECT_URI}
- SPOTIFY_CACHE_PATH=${SPOTIFY_CACHE_PATH}
config.py
class DevelopmentConfig(BaseConfig):
SPOTIFY_CLIENT_ID = os.environ.get('SPOTIFY_CLIENT_ID')
SPOTIFY_CLIENT_SECRET = os.environ.get('SPOTIFY_CLIENT_SECRET')
SPOTIFY_REDIRECT_URI_ = os.environ.get('SPOTIFY_REDIRECT_URI')
SPOTIFY_CACHE_PATH = os.environ.get('SPOTIFY_CACHE_PATH')
auth.py
from project.config import DevelopmentConfig
sp = spotipy.Spotify(auth_manager=spotipy.oauth2.SpotifyOAuth(
DevelopmentConfig.SPOTIFY_CLIENT_ID,
DevelopmentConfig.SPOTIFY_CLIENT_SECRET,
DevelopmentConfig.SPOTIFY_REDIRECT_URI,
scope=DevelopmentConfig.SCOPE,
cache_path=DevelopmentConfig.SPOTIFY_CACHE_PATH))
But I'm getting the following error:
spotipy.oauth2.SpotifyOauthError: No client_id. Pass it or set a SPOTIPY_CLIENT_ID environment variable.
What am I missing?
This certainly looks like you need to pass the variables with the SPOTIPY_ spelling, as per the docs.
However I also noted that your code repeats the same variable names several times. Possibly this could lead to typos as you try to maintain the same variable names across several files.
A simpler way to approach this might be to have the variables contained in a .env-dev file:
SPOTIPY_CLIENT_ID=my_id
SPOTIPY_CLIENT_SECRET=my_secret
SPOTIPY_REDIRECT_URI=http://localhost
SPOTIPY_CACHE_PATH=/project/api/auth/spotify/.cache
Then load these in your docker-compose-dev.yml file:
services:
web:
env_file:
- .env-dev
Then in your Python code you could do:
import os, DevelopmentConfig
sp = spotipy.Spotify(auth_manager=spotipy.oauth2.SpotifyOAuth(
os.environ.get('SPOTIPY_CLIENT_ID'),
os.environ.get('SPOTIPY_CLIENT_SECRET'),
os.environ.get('SPOTIPY_REDIRECT_URI'),
scope=DevelopmentConfig.SCOPE,
cache_path = os.environ.get('SPOTIPY_CACHE_PATH')))
This method has less repitition, although loading of a configuration bypasses your config.DevelopmentConfig object for these variables.
However this method avoids loading the variables into the host's shell, and instead sets them inside a specific service. It also separates secerts so you can commit docker-compose.yml to source control.
https://cloud.google.com/deployment-manager/docs/configuration/templates/create-basic-template
I can deploy a template directly like this: gcloud deployment-manager deployments create a-single-vm --template vm_template.jinja
But what if that template depends on other files that need to be imported? If using a --config file you can define import in that file and call the template as a resource. But you cant pass parameter/properties to a config file. I want to call a template directly to pass --properties via the command line but that template also needs to import other files.
EDIT: What I needed was a top level jinja template instead of a config. My confusion was that you cant use imports in a jinja template without a schema file- it was failing and I thought it wasnt supported. So the solution was just swap out the config with a jinja template (with schema file) and then I can use --properies
Maybe you can try importing the dependent files into your config file as follows:
imports:
- path: vm-template.jinja
- path: vm-template-2.jinja
# In the resources section below, the properties of the resources are replaced
# with the names of the templates.
resources:
- name: vm-1
type: vm-template.jinja
- name: vm-2
type: vm-template-2.jinja
and Set Arbitrary Metadata insito create a special variable that you can pass and might use in other applications outside of Deployment Manager:
properties:
size:
type: integer
default: 2
description: Number of Mongo Slaves
variable-x: ultra-secret-sauce
More info about gcloud deployment-manager deployments create optional flags and example can be found here.
More info about passing properties using a Schema can be found here
Hope it helps
I am using Grails 3 Elasticsearch plugin with Springs external JSON configuration by setting spring.application.json as system property.
The properties are available in the application but I can't find a way to initialize an array properly.
What I am trying to accomplish is to override the default values of the hosts property specified in my application.yml:
environments:
development:
elasticSearch:
client:
hosts:
- {host: "myhost.com", port: 9300}
- {host: "anotherhost.com", port: 9300}
I am setting the property from the command line as follows:
-Dspring.application.json={"environments":{"development":{"elasticSearch":{"client":{"hosts":[{"host":"override1.com", "port":9000},{"host":"override2.com", "port":9100}]}}}}}
I would expect environments.development.elasticSearch.client.hosts to contain an array like it does when initialized from the application.yml, but in fact environments.development.elasticSearch.client containes host[0] and host[1], where each contains the host and the port. The host array from the yml file is still there.
How can I achieve the same behavior using the command line as with the application.yml file?
I believe you can do this the same way that you would if it was set in a .properties file, using a list:
-Denvironments.developmet.elasticSearch.client.hosts={"host":"override1.com", "port":9000},{"host":"override2.com", "port":9100}
and I believe you can also do it as an environment variable...
set ENVIRONMENTS_DEVELOPMENT_ELASTICSEARCH_CLIENT_HOSTS='{"host":"override1.com", "port":9000},{"host":"override2.com", "port":9100}'
There may need to be some quotes around parts of these depending on the shell you are in, the OS, etc.
I am creating the heroku deploy button. In app.json there is next config variable:
"env": {
"PUBLISH_APP_DIR": {
"value" : "/src/WebApp"
},
and, as expected, it is available in "Config Variables" section on https://dashboard.heroku.com/new?template=
The question is how I can access it value in buldpack bin/compile script? Bash is used as environment:
#!/usr/bin/env bash
I have checked environment variables using 'printenv', and there is no PUBLISH_APP_DIR. $(PUBLISH_APP_DIR) and $PUBLISH_APP_DIR are empty also
I have finally found in buildpack api documentation, that
The application config vars are passed to the buildpack as an argument (versus set in the environment) so that the buildpack can optionally export none, all or parts of the app config vars available in ENV_DIR when the compile script is run.
The name of the file is the config key and the contents of the file is the config value. The equivalent of config var S3_KEY=8N029N81 is a file with the name S3_KEY and contents 8N029N81.
It is said that Mojito can use JSON or YML as the application.json (the config file), but I haven't seen YML examples around?
For example, how to convert:
[
{
"settings": [ "master" ],
"specs": {
"hello" : {
"type" : "HelloWorldMojit"
}
}
}
]
to a YML file?
Also, when we use
$ mojito create app Hello
can't we specify that we want YML files as the default (instead of JSON files)?
Details:
I used npm's yamljs to convert the file to:
-
settings: [master]
specs: { hello: { type: HelloWorldMojit } }
and it doesn't work. And I edited it to
-
settings: [master]
specs:
hello:
type:
HelloWorldMojit
It won't work either. The server can start, but when the homepage was accessed, the error is:
error: (outputhandler.server): { [Error: Cannot expand instance [hello],
or instance.controller is undefined] code: 500 }
(the file routes.json is depending on hello being defined)
As of Mojito 0.5.2, YML is supported again. 0.5.1 and 0.5.0 do not support it.
We don't have archetypes with yaml, you will have to transform the files manually and renaming them. The good news, a more flexible archetypes infrastructure is on the making.
You should be ok with that configuration you pasted in the question, just use the latest version of mojito (0.5.x)