Following this article: https://spring.io/blog/2020/08/14/config-file-processing-in-spring-boot-2-4
Take the following example:
security.user.password: usera
---
spring.profiles: local
security.user.password: userb
runlocal: true
---
spring.profiles: !dev
spring.profiles.include: local
security.user.password: userc
Here we have a multi-document YAML file (a single file that’s comprised of three logical documents separated by ---).
If you run with --spring.profiles.active=prod what’s the value of security.user.password? Is the runlocal property set? Are you sure? Is the middle document even included since the profile isn’t activated at the time it’s processed?
I think it's pretty obvious that security.user.password is userc and runlocal property is not set, and it's true.
#SpringBootApplication
class RumblWebApplication(
#Autowired environment: Environment
) {
init {
println(environment.activeProfiles.joinToString(", "))
println(environment.getProperty("security.user.password"))
println(environment.getProperty("runlocal"))
}
}
fun main(args: Array<String>) {
runApplication<RumblWebApplication>(*args)
}
prod
userc
null
I don't understand why they said old one is confusing and new one is better.
Related
I have a groovy file created under "vars" in a Jenkins-shared-lib.
Few variables are defined inside call().
I want to mock the variables in a groovy test file.
Variables are defined in sonarGradleProject.groovy in vars:
#!/usr/bin/env groovy
import static com.example.jenkins.Constants.*
def call(Map params = [:]) {
def sonarCredId = params.sonarCredentialId ?: SONAR_CREDENTIAL_ID_DEFAULT;
def sonarUrl = params.sonarUrl ?: SONAR_URL;
def profile = params.profile ?: 'example';
withCredentials([string(credentialsId: sonarCredId, variable: 'SONAR_TOKEN')]) {
sh "./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true"
}
}
Test file looks like this:
import com.example.jenkins.testing.JenkinsPipelineSpecification
class SonarGradleProjectSpec extends JenkinsPipelineSpecification {
def "expected minor value"() {
setup:
def sonarGradleProject = loadPipelineScriptForTest("vars/sonarGradleProject.groovy")
explicitlyMockPipelineStep('withCredentials')
when:
def verType = sonarGradleProject(profile: 'foo')
then:
1 * getPipelineMock("sh")("./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true")
expect:
'foo' == profile
}
}
On executing the test case, I get this error:
java.lang.IllegalStateException: There is no pipeline variable mock for [profile].
1. Is the name correct?
2. Is it a GlobalVariable extension point? If so, does the getName() method return [profile]?
3. Is that variable normally defined by Jenkins? If so, you may need to define it by hand in your Spec.
4. Does that variable come from a plugin? If so, is that plugin listed as a dependency in your pom.xml?
5. If not, you may need to call explicitlyMockPipelineVariable("profile") during your test setup.
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.example.jenkins.testing.JenkinsPipelineSpecification.addPipelineMocksToObjects_closure1$_closure15(JenkinsPipelineSpecification.groovy:755)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at SonarGradleProjectSpec.expected minor value(sonarGradleProjectSpec.groovy:12)
Caused by: groovy.lang.MissingPropertyException: No such property: (intercepted on instance [SonarGradleProjectSpec#3dfb1626] during test [SonarGradleProjectSpec#3dfb1626]) profile for class: SonarGradleProjectSpec
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at com.example.jenkins.testing.JenkinsPipelineSpecification.addPipelineMocksToObjects_closure1$_closure15(JenkinsPipelineSpecification.groovy:754)
... 3 more
I want to use the variables in test.
How do I mock them?
I've run into similar issues before. The issue for me is that I was trying to verify the interactions of a mock that uses string interpolation. So for the one entry in the then section...
1 * getPipelineMock("sh")("./gradlew --no-daemon jacocoTestReport sonarqube -P${profile} -Dsonar.host.url=${sonarUrl} -Dsonar.login=$SONAR_TOKEN -Dsonar.verbose=true")
I would define the profile and sonarUrl variables in the setup section (i.e., def profile = 'test123'), referenced to verify the mocked sh step interaction. I believe you would also want to adjust the assignment of default values to use the null-safe ?. operator to access key-value pairs in the params map. So def sonarUrl = params.sonarUrl ?: SONAR_URL; would become def sonarUrl = params?.sonarUrl ?: SONAR_URL;
You may have already addressed this and omitted it from the question, but I figured I'd share this information too. Please ignore the following information if you've already addressed it. :)
The three environment variables defined in the script - SONAR_CREDENTIAL_ID_DEFAULT, SONAR_URL, and SONAR_TOKEN - will need to be injected into the script before calling sonarGradleProject. That should be sufficient to test default values if key-value pairs are not provided in the params map.
setup:
def sonarGradleProject = loadPipelineScriptForTest("vars/sonarGradleProject.groovy")
explicitlyMockPipelineStep('withCredentials')
// Inject environment variables.
// First argument is the variable name and second argument is the variable value.
sonarGradleProject.getBinding().setVariable('SONAR_CREDENTIAL_ID_DEFAULT', 'test123')
sonarGradleProject.getBinding().setVariable('SONAR_URL', 'test123')
sonarGradleProject.getBinding().setVariable('SONAR_TOKEN', 'test123')
When I run puppet agent --test I have no errors output but the user did not create.
My puppet hira.yaml configuration is:
---
version: 5
datadir: "/etc/puppetlabs/code/environments"
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
path: "%{::environment}/nodes/%{::trusted.certname}.yaml"
- name: "Common YAML hierarchy levels"
paths:
- "defaults/common.yaml"
- "defaults/users.yaml"
users.yaml is:
accounts::user:
joed:
locked: false
comment: System Operator
uid: '1700'
gid: '1700'
groups:
- admin
- sudonopw
sshkeys:
- ssh-rsa ...Hw== sysop+moduledevkey#puppetlabs.com
I use this module
Nothing in Hiera data itself causes anything to be applied to target nodes. Some kind of declaration is required in a manifest somewhere or in the output of an external node classifier script. Moreover, the puppetlabs/accounts module provides only defined types, not classes. You can store defined-type data in Hiera and read it back, but automated parameter binding via Hiera applies only to classes, not defined types.
In short, then, no user is created (and no error is reported) because no relevant resources are declared into the target node's catalog. You haven't given Puppet anything to do.
If you want to apply the stored user data presented to your nodes, you would want something along these lines:
$user_data = lookup('accounts::user', Hash[String,Hash], 'hash', {})
$user_data.each |$user,$props| {
accounts::user { $user: * => $props }
}
That would go into the node block matched to your target node, or, better, into a class that is declared by that node block or an equivalent. It's fairly complicated for so few lines, but in brief:
the lookup function looks up key 'accounts::user' in your Hiera data
performing a hash merge of results appearing at different levels of the hierarchy
expecting the result to be a hash with string keys and hash values
and defaulting to an empty hash if no results are found;
the mappings in the result hash are iterated, and for each one, an instance of the accounts::user defined type is declared
using the (outer) hash key as the user name,
and the value associated with that key as a mapping from parameter names to parameter values.
There are a few problems here.
You are missing a line in your hiera.yaml namely the defaults key. It should be:
---
version: 5
defaults: ## add this line
datadir: "/etc/puppetlabs/code/environments"
data_hash: yaml_data
hierarchy:
- name: "Per-node data (yaml version)"
path: "%{::environment}/nodes/%{::trusted.certname}.yaml"
- name: "Common YAML hierarchy levels"
paths:
- "defaults/common.yaml"
- "defaults/users.yaml"
I detected that using the puppet-syntax gem (included if you use PDK, which is recommended):
▶ bundle exec rake validate
Syntax OK
---> syntax:manifests
---> syntax:templates
---> syntax:hiera:yaml
ERROR: Failed to parse hiera.yaml: (hiera.yaml): mapping values are not allowed in this context at line 3 column 10
Also, in addition to what John mentioned, the simplest class to read in your data would be this:
class test (Hash[String,Hash] $users) {
create_resources(accounts::user, $users)
}
Or if you want to avoid using create_resources*:
class test (Hash[String,Hash] $users) {
$users.each |$user,$props| {
accounts::user { $user: * => $props }
}
}
Note that I have relied on the Automatic Parameter Lookup feature for that. See the link below.
Then, in your Hiera data, you would have a key named test::users to correspond (class name "test", key name "users"):
---
test::users: ## Note that this line changed.
joed:
locked: false
comment: System Operator
uid: '1700'
gid: '1700'
groups:
- admin
- sudonopw
sshkeys:
- ssh-rsa ...Hw== sysop+moduledevkey#puppetlabs.com
Use of automatic parameter lookup is generally the more idiomatic way of writing Puppet code compared to calling the lookup function explicitly.
For more info:
PDK
Automatic Parameter Lookup
create_resources
(*Note that create_resources is "controversial". Many in the Puppet community prefer not to use it.)
I've been searching around for a bit and couldn't find anything that really helped me. Especially because sometimes things don't seem to be consistant.
I have the following YAML that I use to store data/ configuration stuff:
---
global:
name: Core Config
cfg_version: 0.0.1
databases:
main_database:
name: Main
path: ~/Documents/main.reevault
read_only: false
...
I know how to update fields with:
cfg = YAML.load_file("test.yml")
cfg['global']['name'] = 'New Name'
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
And that's essentially everybody on the internet talks about. However here is my problem: I want to dynamically be able to add new fields to that file. e.g. under the "databases" section have a "secondary_db" field with it's own name, path and read_only boolean. I would have expected to do that by just adding stuff to the hash:
cfg['global']['databases']['second_db'] = nil
cfg['global']['databases']['second_db']['name'] = "Secondary Database"
cfg['global']['databases']['second_db']['path'] = "http://someurl.remote/blob/db.reevault"
cfg['global']['databases']['second_db']['read_only'] = "true"
File.open("test.yml", "w"){ |f| YAML.dump(cfg, f) }
But I get this error:
`<main>': undefined method `[]=' for nil:NilClass (NoMethodError)
My question now is: how do I do this? Is there a way with the YAML interface? Or do I have to write stuff into the file manually? I would prefer something via the YAML module as it takes care of formatting/ indentation for me.
Hope someone can help me.
Yo have to initialize cfg['global']['database']['second_db'] to be a hash not nil. Try this cfg['global']['database']['second_db'] = {}
I have the following two settings in my elasticsearch.yml file. They are the only ones that pull from environment variables.
cloud.aws.access_key: ${AWS_ACCESS_KEY_ID}
cloud.aws.secret_key: ${AWS_SECRET_KEY}
When I restart elasticsearch to load these from the environment, I get an error that it can't resolve them. I've tested it and it will not resolve either, so this error applies to both (it just fails on the bottom one first)
- IllegalArgumentException[Could not resolve placeholder 'AWS_SECRET_KEY']
java.lang.IllegalArgumentException: Could not resolve placeholder 'AWS_SECRET_KEY'
at org.elasticsearch.common.property.PropertyPlaceholder.parseStringValue(PropertyPlaceholder.java:124)
at org.elasticsearch.common.property.PropertyPlaceholder.replacePlaceholders(PropertyPlaceholder.java:81)
at org.elasticsearch.common.settings.ImmutableSettings$Builder.replacePropertyPlaceholders(ImmutableSettings.java:1060)
at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareSettings(InternalSettingsPreparer.java:101)
at org.elasticsearch.bootstrap.Bootstrap.initialSettings(Bootstrap.java:106)
at org.elasticsearch.bootstrap.Bootstrap.main(Bootstrap.java:177)
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:32)
I did some investigating through elasticsearch's code repository on github and discovered this bit of code that pulls from the environment variables.
ImmutableSettings.java#resolvePlaceholder from elasticsearch#github
Namely. the lines inside that function that should be pulling from the environment variables are these one:
Code from resolvePlaceholder that pulls out environment variables
However, after resolvePlaceholder is run from inside function PropertyPlaceholder#parseStringValue, the System.getenv call must be returning null as that is the only way for that error to be thrown.
I wrote a simple test program that is essentially a copy of ImmutableSettings.java#resolvePlaceholder to test that System.getenv was pulling out the environment variables correctly on my system. This in fact returns the values I expect.
public class Cool {
public static void main(String[] args) {
System.out.println(resolvePlaceholder(args[0]));
}
public static String resolvePlaceholder(String placeholderName) {
if (placeholderName.startsWith("env.")) {
// explicit env var prefix
System.out.println("1: placeholderName.startsWith(\"env.\")");
return System.getenv(placeholderName.substring("env.".length()));
}
String value = System.getProperty(placeholderName);
if (value != null) {
System.out.println("2: System.getProperty");
return value;
}
value = System.getenv(placeholderName);
if (value != null) {
System.out.println("3: System.getenv");
return value;
}
return "Map should've had it";
}
}
When run, this is the output, showing we are getting the set environment variables (keys hidden for obvious reasons):
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_SECRET_KEY
3: System.getenv
XXXXXXXXXXXXXXXXXX
[ec2-user#ip-172-31-34-195 ~]$ java Cool AWS_ACCESS_KEY_ID
3: System.getenv
XXXXXXXXXXXXXXXXXX
What is it about elasticsearch that isn't able to parse my environment variables from elasticsearch.yml? I've done quite a bit of digging at this point but I'm sure there is a simple solution around the corner. Any help would be very much appreciated.
I figured out the issue.
As I am running elasticsearch as a linux service, rather than a shell application, it has access to no environment variables except for a very select few.
I added the following line to the end of /etc/sysconfig/elasticsearch to load the environment variables I wanted available to the program:
. /path/to/environment/variables
Environment : Rails 3.1.1 and Rspec 2.10.1
I am loading all my application configuration through an external YAML file. My initializer (config/initializers/load_config.rb) looks like this
AppConfig = YAML.load_file("#{RAILS_ROOT}/config/config.yml")[RAILS_ENV]
And my YAML file sits under config/config.yml
development:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..12
test:
client_system: SWN
b2c_agent_number: '10500'
advocacy_agent_number: 16202
motorcycle_agent_number: '10400'
tso_agent_number: '39160'
feesecure_eligible_months_for_monthly_payments: 1..11
And I access these values as, For example AppConfig['feesecure_eligible_months_for_monthly_payments']
In one of my tests I need AppConfig['feesecure_eligible_months_for_monthly_payments'] to return a different value but am not sure how to accomplish this. I tried the following approach with no luck
describe 'monthly_option_available?' do
before :each do
#policy = FeeSecure::Policy.new
#settlement_breakdown = SettlementBreakdown.new
#policy.stub(:settlement_breakdown).and_return(#settlement_breakdown)
#date = Date.today
Date.should_receive(:today).and_return(#date)
#config = mock(AppConfig)
AppConfig.stub(:feesecure_eligible_months_for_monthly_payments).and_return('1..7')
end
.....
end
In my respective class I am doing something like this
class Policy
def eligible_month?
eval(AppConfig['feesecure_eligible_months_for_monthly_payments']).include?(Date.today.month)
end
....
end
Can someone please point me in the right direction!!
The method that is being called when you do AppConfig['foo'] is the [] method, which takes one argument (the key to retrieve)
Therefore what you could do in your test is
AppConfig.stub(:[]).and_return('1..11')
You can use with to setup different expectations based on the value of the argument, ie
AppConfig.stub(:[]).with('foo').and_return('1..11')
AppConfig.stub(:[]).with('bar').and_return(3)
You don't need to setup a mock AppConfig object - you can stick your stub straight on the 'real' one.