How to define an object as environment variable with hocon config - hocon

What I want to do
Defining a base set of objects which can be enhanced by an environment variable.
In the following you see a little example of my hocon config file so far.
foo.bar-base= [
{a: "hello", b: "world"},
{a: "hello", b: "stackoverflow"}
]
foo.bar-extended = ${?EXTENDED}
foo.bar = ${foo.bar-base} ${foo.bar-extended}
The Problem
When trying to define elements to add as environment variables I get the exception
that states that foo.bar has type list of STRING rather than list of OBJECT.
Is there any way to make the value of the environment variable be parsed as object?
EXTENDED.0={a:"hello", b:"rest"}

I used a workaround. Instead of using config.getConfigList which directly returns a list of Config objects, I used config.getList which returns a List of ConfigValues.
Then I did the parsing manually:
final ConfigList list = config.getList("foo.bar");
final Stream<Config> configsFromString = list.stream()
.filter(value -> ConfigValueType.STRING.equals(value.valueType()))
.map(ConfigValue::unwrapped)
.map(Object::toString)
.map(ConfigFactory::parseString);
final Stream<Config> configsFromMap = list.stream()
.filter(value -> ConfigValueType.OBJECT.equals(value.valueType()))
.map(ConfigValue::render)
.map(ConfigFactory::parseString);
final List<Config> configs = Stream.concat(configsFromString, configsFromMap)
.collect(Collectors.toList());

Related

Terraform- read list of objects from yaml file

I want to read a list of objects from a yaml file via terraform code and map it to a local variable. Also i need search an object with a key and get the values from a yaml file. Can anyone suggest suitable solution?
my yaml file looks like below. Here use will be the primary key
list_details:
some_list:
- use: a
path: somepath
description : "some description"
- use: b
path: somepath2
description : "some description 2"
I have loaded the yaml file in my variable section in Terraform like this
locals {
list = yamldecode(file("${path.module}/mylist.yaml"))
}
Now the problem is how I can get one object with its values by passing the "use" value to the list?
"
Assuming that use values are unique, you can re-organize your list into map:
locals {
list_as_map = {for val in local.list["list_details"]["some_list"]:
val["use"] => val["path"]}
}
which gives list_as_map as:
"a" = "somepath"
"b" = "somepath2"
then you access the path based on a value of use:
path_for_a = local.list_as_map["a"]
Update
If you want to keep description, its better to do:
list_as_map = {for val in local.list["list_details"]["some_list"]:
val["use"] => {
path = val["path"]
description = val["description"]
}
}
then you access the path or description as:
local.list_as_map["a"].path
local.list_as_map["a"].description

Jenkins. Inserting environment variable in the file

I have a .env file that contains the following data
API_URL=${API_URL}
API_KEY=${API_KEY}
API_SECRET=${API_SECRET}
Setting environment variables in Jenkins and passing them to the pipeline is clear. But it is not clear how do I replace ${API_URL}, ${API_KEY} & ${API_SECRET} in the .env file with their values in the Jenkins environment variable? Plus, how do I loop through all the Jenkins variables?
This basically requires two steps:
Get all environment variables
Replace values of environment variables in the template (.env) file
Let's start with #2, because it dictates which kind of data #1 must produce.
2. Replace variables in a template
We can use Groovy's SimpleTemplateEngine for this task.
def result = new SimpleTemplateEngine().createTemplate( templateStr ).make( dataMap )
Here templateStr is the template string (content of your .env file) and dataMap must be a Map consisting of string keys and values (the actual values of the environment variables). Getting the template string is trivial (use Jenkins readFile step), reading the environment variables into a Map is slightly more involved.
1. Read environment variables into a Map
I wrote "slightly more involved" because Groovy goodness makes this task quite easy aswell.
#Chris has already shown how to read environment variables into a string. What we need to do is split this string, first into separate lines and then each line into key and value. Fortunately, Groovy provides the member function splitEachLine of the String class, which can do both steps with a single call!
There is a little caveat, because splitEachLine is one of the functions that doesn't behave well in Jenkins pipeline context - it would only return the first line. Moving the critical code into a separate function, annotated with #NonCPS works around this problem.
#NonCPS
Map<String,String> envStrToMap( String envStr ) {
def envMap = [:]
envStr.splitEachLine('=') {
envMap[it[0]] = it[1]
}
return envMap
}
Finally
Now we have all ingredients for letting Jenkins cook us a tasty template soup!
Here is a complete pipeline demo. It uses scripted style, but it should be easy to use in declarative style as well. Just replace node with a script block.
import groovy.text.SimpleTemplateEngine
node {
// TODO: Replace the hardcoded string with:
// def tmp = readFile file: 'yourfile.env'
def tmp = '''\
API_URL=${API_URL}
API_KEY=${API_KEY}
API_SECRET=${API_SECRET}'''
withEnv(['API_URL=http://someurl', 'API_KEY=123', 'API_SECRET=456']) {
def envMap = getEnvMap()
echo "envMap:\n$envMap"
def tmpResolved = new SimpleTemplateEngine().createTemplate( tmp ).make( envMap )
writeFile file: 'test.env', text: tmpResolved.toString()
// Just for demo, to let me see the result
archiveArtifacts artifacts: 'test.env'
}
}
// Read all environment variables into a map.
// Here, #NonCPS must NOT be used, because we are calling a Jenkins step.
Map<String,String> getEnvMap() {
def envStr = sh(script: 'env', returnStdout: true)
return envStrToMap( envStr )
}
// Split a multiline string, where each line consists of key and value separated by '='.
// It is critical to use #NonCPS to make splitEachLine() work!
#NonCPS
Map<String,String> envStrToMap( String envStr ) {
def envMap = [:]
envStr.splitEachLine('=') {
envMap[it[0]] = it[1]
}
return envMap
}
The pipeline creates an artifact "test.env" with this content:
API_URL=http://someurl
API_KEY=123
API_SECRET=456
You can access variables by executing simple shell in scripted pipeline:
def variables = sh(script: 'env|sort', returnStdout: true)
Then programatically in Groovy convert it to list and iterate using each loop.
According to replacing variables, if you're not using any solution which can access env variables then you can use simple text operations like executing sed on that file.

Can't properly build HttpUrl which contains hash-bang

I'm trying to build HttpUrl instance which contains hash-bang but can't properly do so. Final URL string value should look like this example: https://www.google.com/mobile/#!/id?platform=android
I've tried few solutions:
https://gist.github.com/novachevskyi/71529d8fdecf120e626af227193a9e0f
When adding hash-bang with HttpUrl.Builder::addEncodedPathSegment then final result would contain encoded hash symbol.
https://gist.github.com/novachevskyi/12b59e53d6162fb1cd4e6236b03fb504
After parsing base URL which contains hash-bang I'm getting next result:
https://www.google.com/mobile/?platform=android#!/id
Is there any way to build HttpUrl instance were string value would contain hash-bang in it?
You can do it with .parse() or with .fragment().
HttpUrl a = HttpUrl.parse("https://www.google.com/mobile/#!/id?platform=android");
HttpUrl b = new HttpUrl.Builder()
.scheme("https")
.host("www.google.com")
.encodedPath("/mobile/")
.fragment("!/id?platform=android")
.build();

GString inside a Gstring in gradle/groovy

I need to use a variable as a part of a string that will be used to address another variable in a gstring.
In syntesys, what I'd like to do is: ${${it}_checkout}
The whole code line would be:
def checkouts = repos.collect{"${it} = ${${it}_checkout} "}
With repos being a list of repositories to checkout.
Each repo has an property called <repo>_checkout.
For instance, if I have two repos, called foo and bar, I'll have two variables called foo_checkout and bar_checkout, containing the branches to be checkouted.
I'm trying to construct the following string: "foo=$foo_checkout bar=$bar_checkout".
That will be translated to "foo=master bar=dev"
Is there a way ?
Yeh, just do:
def checkouts = repos.collect{ "$it = ${it}_checkout" }
Or, depending on how you declare your properties, you can do:
root_checkout = 'woo'
repo_checkout = 'yay'
['root', 'repo'].collect { r -> "$r = ${getProperty(r + '_checkout')}" }

get list of servers Ebean+Play

I need to get a list of all existing servers in the application.conf file, I take a look to EBean class, but i only found how to get an specific server Ebean.getServer("test"), also this returns an EbeanServer object, and i need a string value.
This is part of my application.conf:
db.default.driver=oracle.jdbc.OracleDriver
db.default.url="jdbc:oracle:thin:#//178.20.26.25:1521/orcl"
db.default.user="TEST1"
db.default.password="test1"
db.test.driver=oracle.jdbc.OracleDriver
db.test.url="jdbc:oracle:thin:#//178.20.26.26:1521/orcl"
db.test.user="TEST"
db.test.password="test"
ebean.default="models.*"
ebean.test="models.*"
My expected output is a list that contains (default,test). Does anybody know a way to get this without parsing hole file?
Thanks in advance.
Following code will give set instead of list:
Map<String, String> map = (Map<String, String>) play.Play.application().configuration().getObject("db");
Set<String> keys = map.keySet();
If you want to do it in type safe way and get rid of compiler warning:
Set<String> keys = play.Play.application().configuration().getConfig("db").subKeys();
Both examples will return subkeys of db key which is [default, test].

Resources