I am trying to read an environment variable is being set from within Gitlab configuration when the application is being built, I am doing this for achieving that purpose:
I setup a variable in the application.properties.yml of my spring boot app:
sf:
apiKey: $SF_API_KEY
In the .gitlab-ci.yml, I defined the variable to be set, as follows:
variables:
SF_API_KEY: $SF_API_KEY
all I want, is to be able to read that variable from within one of my services, as the below code depicts:
#Service
class MyService(#Value("\${sf.apiKey}") val apiKey: String)
{
fun doSomething(){
//i am seeing the variable is being set by gitlab in the build logs but
// it is not being read here properly
var result = apiKey;
logger.info { "***check apiKey: $apiKey" }
//This line lgs $SF_API_KEY as a value of my variable, but not the
// real value
}
}
Am I doing something wrong? I would appreciate any help.
Try (note the {} around SF_API_KEY):
sf:
apiKey: ${SF_API_KEY}
Take a look at the docs where this placeholder notation is detailed.
Related
We are using Viper to read and parse our config file and all of that works without any issues.
However we are not able to override some of our config values using env variables. These are specific use cases where the config is bound to a struct or an array of structs.
Here is an example from our config.yaml:
app:
verifiers:
- name: "test1"
url: "http://test1.url"
cache: "5000ms"
- name: "test2"
url: "http://test2.url"
cache: "10000ms"
Which is bound to the following structs (golang):
type App struct {
AppConfig Config `yaml:"app" mapstructure:"app"`
}
type Config struct {
Verifiers []VerifierConfig `json:"verifiers" yaml:"verifiers" mapstructure:"verifiers"`
}
type VerifierConfig struct {
Name string `json:"name" yaml:"name" mapstructure:"name"`
URL string `json:"url,omitempty" yaml:"url,omitempty" mapstructure:"url"`
cache jsontime.Duration `json:"cache" yaml:"cache" mapstructure:"cache"`
}
We are unable to override the value of verifiers using env variables.
Here are the Viper options we have used:
viper.AutomaticEnv()
viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
Has anyone experienced a similar issue or can confirm that Viper does not support such a use case?
Any pointers would be greatly appreciated.
Thanks
viper cannot read env variables inside config file. You have to replace them in .go file where value is being used. Here is an example on how I achieved:
Ask Viper to consider system environment variable with "ENV" prefix, that means viper expects "ENV_MONGO_USER" and "ENV_MONGO_PASSWORD" as system environment variables
viper.AutomaticEnv()
viper.SetEnvPrefix(viper.GetString("ENV"))
viper.SetEnvKeyReplacer(strings.NewReplacer(".", "_"))
In yml file:
mongodb:
url: "mongodb+srv://${username}:${password}#xxxxxxx.mongodb.net/"
database: "test"
In mongodb connection file: (before making db connection, replace ${username} and ${password} with respective environment variables but viper will read those variables without prefix )
const BasePath = "mongodb"
mongoDbUrl := viper.GetString(fmt.Sprintf("%s.%s", BasePath, "url"))
username := viper.GetString("MONGO_USER")
password := viper.GetString("MONGO_PASSWORD")
replacer := strings.NewReplacer("${username}", username, "${password}", password)
mongoDbUrl = replacer.Replace(mongoDbUrl)
I have deployed my app on Heroku and on the start of my app I check a config file and in there I want to access a config var I have created on Heroku API_KEY to define my Firebase config:
module.exports = {
fireConfig: {
apiKey: process.env.API_KEY,
authDomain: "my-app.firebaseapp.com",
databaseURL: "https://my-app.firebaseio.com",
projectId: "my-project-id",
storageBucket: "my-app.appspot.com",
messagingSenderId: "my-messaging-sender-id"
}
};
this process.env.API_KEY is undefined. Should I use a different way to access it?
You can define environment variables in your nuxt.config.js file, e.g.
export default {
env: {
firebaseApiKey: process.env.API_KEY || 'default value'
}
}
Here we use the API_KEY environment variable, if it's available, and assign that to firebaseApiKey.
If that environment variable isn't set, e.g. maybe in development (you can set it there too if you want), we fall back to 'default value'. Please don't put your real API key in 'default value'. You could use a separate throwaway Firebase account key here or just omit it (take || 'default value' right out) and rely on the environment variable being set.
These will be processed at build time and then made available using the name you give them, e.g.
module.exports = {
fireConfig: {
apiKey: process.env.firebaseApiKey, # Not API_KEY
// ...
};
I can take credentials like explained in the example taken from here - https://jenkins.io/doc/book/pipeline/syntax/#environment
stage('Example') {
environment {
CREDS = credentials('MY_CREDS_DEV')
}
steps {
sh 'echo hello'
}
}
But what I want to do is to get credentials based on some condition.
For example I have MY_CREDS_DEV and MY_CREDS_QA defined in Jenkins credentials. And I have a property ENV=dev defined in Jenkins 'Prepare an environment for the run' section.
I'd like to access credentials based on my environment, i.e. ENV property.
I tried to use CREDS = credentials('MY_CREDS_' + ${ENV}) and tried to extract strings concatenation to a separate function and call it like CREDS = credentials(concatenate(${ENV})) but I got Internal function call parameters must be strings.
So seems I can put only a string to credentials() function which basically means to hardcode it. But how can I choose which credentials to use - dev or qa?
Use CREDS = credentials('MY_CREDS_' + ENV) or CREDS = credentials("MY_CREDS_${ENV}"). ${ENV} will not become 'dev'but ${'dev'} and therefore is no string.
For completeness:
In fact - after playing aroung with the groovy console - it looks like ${ENV} will try to call a function called $ with the closure parameter {ENV} which in turn would return 'dev'. It would give the same result as ENV if you would have defined a function like:
def $(Closure closure) {
closure()
}
But most probably that's not what you wanted to do.
Got this working in Jenkins:2.190.2 with a little groovy. Haven't tested on earlier versions. Just happens to be the one I'm on now. Works fine with multiple stages.
pipeline {
agent {
label "xxxxx"
}
environment {
ROLE = getRole()
}
stages{
stage("write to s3 etc") {
environment {
AWS = credentials("${ROLE}")
}
steps {
script {
sh"""
aws s3 sync build/ "s3://xxxxxxxxxxxx"
"""
}
}
}
}
}
def getRole() {
def branchName = "${env.BRANCH_NAME}"
if (branchName == "xxxxxx") {
return 'some_credential_string'
}
else {
return 'some_other_credential_string'
}
}
If you would like to use different credentials based on the condition, this could be done with the following example:
stage ("Example") {
steps {
script {
if ( params.TEST_PARAMETER == "test_value1" ) {
withCredentials([string(credentialsId: env.CREDENTIALS_1, variable: 'SOME_VARIABLE')]) {
yourFunction()
}
}
else {
withCredentials([string(credentialsId: env.CREDENTIALS_2, variable: 'SOME_VARIABLE')]) {
yourFunction()
}
}
}
}
}
You would need to define yourFunction in the end of your jenkinsfile. In this case, when TEST_PARAMETER is test_value1 in the job, CREDENTIALS_1 will be used from Jenkins credentials list. When TEST_PARAMETER is different, CREDENTIALS_2 credentials will be used. You could have more options by modifying this to the case loop.
Hope this helps.
I'm trying to deploy SonarQube on Kubernetes using configMaps.
The latest 7.1 image I use has a config in sonar.properties embedded in $SONARQUBE_HOME/conf/ . The directory is not empty and contain also a wrapper.conf file.
I would like to mount the configMap inside my container in a other location than /opt/sonar/conf/ and specify to sonarQube the new path to read the properties.
Is there a way to do that ? (environment variable ? JVM argument ? ...)
It is not recommended to modify this standard configuration in any way. But we can have a look at the SonarQube sourcecode. In this file you can find this code for reading the configuration file:
private static Properties loadPropertiesFile(File homeDir) {
Properties p = new Properties();
File propsFile = new File(homeDir, "conf/sonar.properties");
if (propsFile.exists()) {
...
} else {
LoggerFactory.getLogger(AppSettingsLoaderImpl.class).warn("Configuration file not found: {}", propsFile);
}
return p;
}
So the conf-path and filename is hard coded and you get a warning if the file does not exist. The home directory is found this way:
private static File detectHomeDir() {
try {
File appJar = new File(Class.forName("org.sonar.application.App").getProtectionDomain().getCodeSource().getLocation().toURI());
return appJar.getParentFile().getParentFile();
} catch (...) {
...
}
So this can also not be changed. The code above is used here:
#Override
public AppSettings load() {
Properties p = loadPropertiesFile(homeDir);
p.putAll(CommandLineParser.parseArguments(cliArguments));
p.setProperty(PATH_HOME.getKey(), homeDir.getAbsolutePath());
p = ConfigurationUtils.interpolateVariables(p, System.getenv());
....
}
This suggests that you can use commandline parameters or environment variables in order to change your settings.
For my problem, I defined environment variable to configure database settings in my Kubernetes deployment :
env:
- name: SONARQUBE_JDBC_URL
value: jdbc:sqlserver://mydb:1433;databaseName=sonarqube
- name: SONARQUBE_JDBC_USERNAME
value: sonarqube
- name: SONARQUBE_JDBC_PASSWORD
valueFrom:
secretKeyRef:
name: sonarsecret
key: dbpassword
I needed to use also ldap plugin but it was not possible to configure environment variable in this case. As /opt/sonarqube/conf/ is not empty, I can't use configMap to decouple configuration from image content. So, I build my own sonarqube image adding the ldap jar plugin and ldap setting in sonar.properties :
# General Configuration
sonar.security.realm=LDAP
ldap.url=ldap://myldap:389
ldap.bindDn=CN=mysa=_ServicesAccounts,OU=Users,OU=SVC,DC=net
ldap.bindPassword=****
# User Configuration
ldap.user.baseDn=OU=Users,OU=SVC,DC=net
ldap.user.request=(&(sAMAccountName={0})(objectclass=user))
ldap.user.realNameAttribute=cn
ldap.user.emailAttribute=mail
# Group Configuration
ldap.group.baseDn=OU=Users,OU=SVC,DC=net
ldap.group.request=(&(objectClass=group)(member={dn}))
so I am building a SpringBootApplication and I currently have the following settings in the yml:
my:
caller:
- id: someId
url: someUrl
context: someContext
- id: someId2
url: someUrl2
context: someContext2
So basically it's a list of objects (I've created a POJO which has the 3 fields). Those objects are represented as a List in another class which is a #Component and has a #ConfigurationProperties(prefix = "my") and the List of POJOs is with #NestedConfigurationproperty.
So far everything works perfectly, Spring constructs the objects from the application.yml and fills the List just fine, however I can't simulate this configuration in CloudFoundry. I'm using a different manifest.yml for cloudfoundry, and I'm trying to place this structure in the env part of the manifest but it's not working (Cloudfoundry ignores it).
It looks something like this:
env:
my:
caller:
- id: someId
url: someUrl
context: someContext
- id: someId2
url: someUrl2
context: someContext2
I know with simple lines it works (for example ev_my_caller_id) but that way I can construct only 1 object, not a whole list and I haven't found anything in the CloudFoundry doc on how to construct multiple objects from the manifest.
If anyone has any ideas I would be really grateful!
The env block of the Cloud Foundry cli's manifest.yml file has a specific format. You cannot use an arbitrary structure and expect it to work.
The format is:
env:
var_name_1: val_1
var_name_2: val_2
See docs for more details -> https://docs.cloudfoundry.org/devguide/deploy-apps/manifest.html#env-block
If you set the env variable with the correct name, you can use that to override certain values in Spring. Thanks to Spring Boot's external configuration support. Maybe that would be an option for you here.
https://docs.spring.io/spring-boot/docs/current/reference/html/boot-features-external-config.html
Ex: MY_PROPERTY=1234 would override my.property in application.properties.
You could use the SPRING_APPLICATION_JSON environment variable mentioned in the Spring Boot external config link that Daniel provided.
In your case, that would like something like this:
env:
SPRING_APPLICATION_JSON: '{ "my": { "caller": [{ "id": "someId", "url": "someUrl", "context": "someContext" }, { "id": "someId2", "url": "someUrl2", "context": "someContext2" }]}}'
Not sure what you are trying to do, but you can try something like this:
env:
test: '{ whatever complex object/variables }'
e.g.
env:
test: '{ variable = value, variable2 = value2 }'