How to handle multiple build targets, e.g. dev, test, main? - maven

I'm currently migrating from Maven to SBT, and I'm struggling to understand how I can handle multiple build targets (dev, test, train, prod etc).
For example, I have a persistence.xml that looks like this:
<properties>
<property name="javax.persistence.jdbc.driver" value="oracle.jdbc.driver.OracleDriver"/>
<property name="javax.persistence.jdbc.url" value="${db.connectionURL}"/>
<property name="javax.persistence.jdbc.user" value="${db.username}"/>
<property name="javax.persistence.jdbc.password" value="${db.password}"/>
<property name="eclipselink.target-database" value="Oracle10"/>
</properties>
With Maven, it was really easy to handle this using profiles.
I've already tried what was suggested here for SBT, but I'm not having any success with that approach.
How to Add Environment Profile Config to SBT.
Also, using that approach I would need a new directory for each new environment. I just think that there has to be a better way to handle this kind of setup using SBT?

tl;dr Use ivyConfigurations to add custom configs and resourceGenerators to process files per environment.
All credits goes to Eugene Yokota for the answer to How to Add Environment Profile Config to SBT. There are some modification that make the solution of mine...cough...cough...slightly better.
The following build.sbt defines two new configurations - dev and qa. It also defines resourceGenerators per configuration that effectively gives a way to access what configuration the new resourceGenerator executes in:
val Dev = config("dev") extend Runtime
val Qa = config("qa") extend Runtime
ivyConfigurations ++= Seq(Dev, Qa)
// http://www.scala-sbt.org/0.13.5/docs/Howto/generatefiles.html#resources
lazy val bareResourceGenerators: Seq[Setting[_]] = Seq(
resourceGenerators += Def.task {
val file = resourceManaged.value / "demo" / "myapp.properties"
println(s"Inside ${configuration.value}")
val contents = s"config=${configuration.value}"
IO.write(file, contents)
Seq(file)
}.taskValue
)
inConfig(Dev)(Defaults.configSettings ++ bareResourceGenerators)
inConfig(Qa)(Defaults.configSettings ++ bareResourceGenerators)
Inside the new resourceGenerator you can do whatever you want and the per-configuration processing is possible with the configuration setting that gives you the name of the configuration:
> show dev:configuration
[info] dev
> show qa:configuration
[info] qa
Now when you execute show qa:resources you'll see that there are two files generated with target/scala-2.10/resource_managed/qa/demo/myapp.properties with the content that's specific to a configuration:
> show qa:resources
Inside qa
[info] List(/Users/jacek/sandbox/envs/target/scala-2.10/resource_managed/qa/demo/myapp.properties, /Users/jacek/sandbox/envs/src/qa/resources)
The trick now is to use the resourceGenerator to meet your needs and since you're in Scala code you can do whatever you want - just use configuration.value as the qualifier for a configuration-specific code.
Say, you want to use a qa-specific properties file in the standard src/main/resources directory. Just know where the value is bound to (what configuration and setting the value comes from). It's just compile:resourceDirectory.
> show compile:resourceDirectory
[info] /Users/jacek/sandbox/envs/src/main/resources
Just use resourceDirectory in Compile whenever you need a "stable" (aka configuration-fixed) value like src/main/resources.
val props = (resourceDirectory in Compile).value / s"${configuration.value.name}.properties"
println(s"Read files from $props")
With the above lines you'd get:
> show qa:resources
Inside qa
Read files from /Users/jacek/sandbox/envs/src/main/resources/qa.properties
[info] List(/Users/jacek/sandbox/envs/target/scala-2.10/resource_managed/qa/demo/myapp.properties, /Users/jacek/sandbox/envs/src/qa/resources)
> show dev:resources
Inside dev
Read files from /Users/jacek/sandbox/envs/src/main/resources/dev.properties
[info] List(/Users/jacek/sandbox/envs/target/scala-2.10/resource_managed/dev/demo/myapp.properties, /Users/jacek/sandbox/envs/src/dev/resources)

Related

How Do I Combine Property Files In Gradle?

I have an ant task that combines 2 property files together to make a single properties file.
The base.config contains the majority of the properties and the override.config contains one or more properties that modifies the value in the base.config
<!--Combine the base.config and override.config together to create a properties file-->
<target name="generateProperties" depends="init">
<property prefix="app.properties" file="override.config"/>
<property prefix="app.properties" file="base.config"/>
<echoproperties destfile="${build.dir}/combined.properties">
<propertyset>
<propertyref prefix="app.properties"/>
<mapper type="glob" from="app.properties.*" to="*"/>
</propertyset>
</echoproperties>
</target>
How do I do the same thing in gradle (without importing the ant task)?
There is a WriteProperties task that can reliably save the properties.
You then just have to pass the Properties in a way that the base is overridden.
e.g.
task generateProperties(type: WriteProperties) {
Properties combinedProperties = new Properties()
File baseProperties = file('base.config')
baseProperties.withInputStream {
combinedProperties.load(it)
}
File overrideProperties = file('override.config')
overrideProperties.withInputStream {
combinedProperties.load(it)
}
properties = combinedProperties
outputFile = file("$buildDir/combined.properties")
}
There are (5) property 'scopes' Gradle has for finding properties. They are:
Project instance itself
Extra properties
Extensions
Conventions
Tasks
Project parent (recursive lookup with the above)
You can read a more in-depth overview here: https://docs.gradle.org/current/javadoc/org/gradle/api/Project.html#properties
Gradle searches for properties in that order. So you can define base properties lower in the chain and provide overrides higher up in the chain.

How to load a secret.properties file in apacheignite/docker stock image

I used gridgain’s webconsole to create a configuration file for my ignite node (ignite-config.xml). I’d like to see if I can get this running in apache’s docker ignite image (apacheignite/ignite). I’ve created a volume in my compose file that maps to a config folder that houses my ignite-config.xml.
I need to include the secret.properties (that has my jdbc url/username/password) file as well so I put it in the config folder too.
When I do a docker-compose up I get a java.io.FileNotFoundException: class path resource [config/secret.properties] cannot be opened because it does not exist]
This is the part of the xml config that deals with the secret.properties location:
<!-- Load external properties file. -->
<bean id="placeholderConfig" class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="location" value="classpath:secret.properties"/>
</bean>
What do I need to change so that the secret.properties file in the config folder is loaded? Is there a better way to do this?
I do not know how to easily deploy the secret.properties file without going through a full build and deploy, but in order to get past that for testing purposes, you could get rid of the secret.properties and use environment variables instead.
Which might look something like this for a JDBC connection to Postgres:
<!-- Data source beans will be initialized from external properties file. -->
<bean id="dsPostgreSQL_Daifcqkp30zkdj" class="org.postgresql.ds.PGPoolingDataSource">
<property name="url" value="#{systemEnvironment['dsPostgreSQL_Daifcqkp30zkdj_jdbc_url']}"/>
<property name="user" value="#{systemEnvironment['dsPostgreSQL_Daifcqkp30zkdj_jdbc_username']}"/>
<property name="password" value="#{systemEnvironment['dsPostgreSQL_Daifcqkp30zkdj_jdbc_password']}"/>
</bean>
Then, just set your environment variables:
IGNITE_VERSION=2.7.5
CONFIG_URI=https://raw.github.com/some/path/to/your/config.xml
OPTION_LIBS=ignite-zookeeper,ignite-rest-http
EXTERNAL_LIBS=https://github.com/some/path/to/your/jdbc-drivers/postgresql-42.2.6.jar?raw=true
dsPostgreSQL_Daifcqkp30zkdj_jdbc_username=0a9suf09asdfkjwel
dsPostgreSQL_Daifcqkp30zkdj_jdbc_password=8faskdfn94noiasnf09_s09fklajfealk
dsPostgreSQL_Daifcqkp30zkdj_jdbc_url=jdbc:postgresql://<host>:<port>/Daifcqkp30zkdj?ssl=true&sslfactory=org.postgresql.ssl.NonValidatingFactory
Had the same issue, probably didn't resolve it the "correct" way (whatever that is), but resolved it this way for a POC nonetheless:
Create a standalone jar file with only the secrets file
Give it the name dsPostgreSQL_Daifcqkp30zkdj_jdbc.jar or similar
Place the jar file in the libs directory
It seems the classpath is just the contents of all JAR files in the \libs subdir, so oblige it... This would have the advantage of allowing for a signed JAR - I believe - which would (in addition to file system permissions, visibility, etc.) at least ensure it isn't tampered with.

Spring Tools Suite and Gradle - Setup to use correct resources from inside STS

I have a Spring Boot Gradle project setup in Spring Tools Suite (3.7.2 RELEASE) with the following source folders:
- src/integration-test/java
- src/integration-test/resources
- src/main/java
- src/main/resources
- src/test/java
- src/test/resources`
Whenever I run the application or unit tests from within STS, I see that STS is using the resources found under src/integration-test/resources.
I see a duplicate resource warning in STS for files which exist in all 3 resource source folders. For example, I have an application.properties in all 3 source folders and I see following:
The resource is a duplicate of src/integration-test/resources/application.properties and was not copied to the output folder
If I run the application as a JAR or unit tests/integration tests from the command line (via gradle build), everything seems to use the correct resources. This makes me believe it is a problem with how STS/Eclipse is handling gradle.
Does anybody know of how I can configure STS to use the correct resource source folders when using gradle?
I think my problem may be related to (or the same as?) Spring Boot incorrectly loads test configuration when running from eclipse+gradle, https://issuetracker.springsource.com/browse/STS-3882, https://issues.gradle.org/browse/GRADLE-1777
I also tried the solution found here, but that seems to only fix Maven builds:
Spring Tool Suite finds spring-boot integration test configuration and does not start main application
I think my problem may be related to...
Yes, it is related but in my opinion not the same. That problem is caused by the runtime classpath being incorrect. This problem is an error coming from the eclipse project builder so it is a compile-time issue.
The problems are closely related though. Depending on your point of view, you could say they are the same (incorrect mixing of test and compile-time classpaths).
Here, specifically, the problem is that the eclipse builder tries to copy all the resources it finds in source folders to the project's single output folder. Each source folder has a 'application.properties'. The builder warns that it could not copy some of them because one would overwrite the other.
I think there may be a solution for this problem. But it is a solution that really should come from Gradle + ( BuildShip | STS Gradle Tooling) than from you.
It is possible in Eclipse to configure each source-folder individually to target a specific outputfolder. Maven + M2E are doing this correcty, but Gradle + (BuildsShip | STS Gradle Tooling) combdos do not.
For example this is what maven puts into the eclipse .classpath file when it configures a test resources folder:
<classpathentry excluding="**" kind="src" output="target/test-classes" path="src/test/resources">
<attributes>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
Notice how it explicitly sets the output folder for that entry (to something different from the project's default output folder).
You may be able to address the problem yourself by modifying the .classpath for a gradle project in a similar way. Either by doing it manually or from your build.gradle.
I'm not sure this is worth it however as you will then likely still get hit by the runtime classpath issue (since these folders will still be added to your runtime classpath, your runtime classpath will end-up with two appication.properties resources, one which will 'shadow' the other. See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=482315)
I would say, the right thing to do is add a comment to the issue I linked, and hope they fix it soon as there is only so much you can do yourself by hacking the build.gradle file to modify the .classpath (this can not solve the runtime classpath issue, but in order to solve the runtime classpath issue, they would have to configure source folders to target individual output folder similar to what m2e does).
I would add this as a comment to #Kris's answer but it's too long.
I have solved the runtime classpath issue by adding the code below to my build.gradle file. The code generates an Eclipse launch configuration for the Spring Boot application class and includes only the runtime classpath (i.e. no test JARs).
My project uses the Gradle 'eclipse' plugin to generate the Eclipse project files (which I then import into Eclipse). Running the eclipseClasspath Gradle target will generate the launch file in the project's root directory.
def mainClassName = "com.example.MyApplication"
task eclipseApplicationLaunch {
group "IDE"
description "Generate an Eclipse launch configuration file for the Spring Boot application class"
}
eclipseApplicationLaunch << {
def writer = new FileWriter("${mainClassName.substring(mainClassName.lastIndexOf(".")+1)}.launch")
def xml = new groovy.xml.MarkupBuilder(writer)
xml.doubleQuotes = true
xml.launchConfiguration(type: "org.eclipse.jdt.launching.localJavaApplication") {
listAttribute(key:"org.eclipse.debug.core.MAPPED_RESOURCE_PATHS") {
listEntry(value:"/${project.name}/src/main/java/${mainClassName.replace(".","/")}.java")
}
listAttribute(key:"org.eclipse.debug.core.MAPPED_RESOURCE_TYPES") {
listEntry(value:"1")
}
listAttribute(key:"org.eclipse.jdt.launching.CLASSPATH") {
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry containerPath=\"org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8/\" javaProject=\"${project.name}\" path=\"1\" type=\"4\"/>\r\n")
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry path=\"3\" projectName=\"${project.name}\" type=\"1\"/>\r\n")
configurations.runtime.resolvedConfiguration.resolvedArtifacts.each { artifact ->
def filePath = artifact.file.canonicalPath.replace("\\","/")
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry externalArchive=\"${filePath}\" path=\"3\" type=\"2\"/>\r\n")
}
}
booleanAttribute(key:"org.eclipse.jdt.launching.DEFAULT_CLASSPATH", value:"false")
stringAttribute(key:"org.eclipse.jdt.launching.MAIN_TYPE", value:"${mainClassName}")
stringAttribute(key:"org.eclipse.jdt.launching.PROGRAM_ARGUMENTS", value:"--spring.profiles.active=local --spring.config.location=conf/")
stringAttribute(key:"org.eclipse.jdt.launching.PROJECT_ATTR", value:"${project.name}")
stringAttribute(key:"org.eclipse.jdt.launching.VM_ARGUMENTS", value:"-Djava.net.preferIPv4Stack=true")
}
writer.close()
}
eclipseClasspath.dependsOn eclipseApplicationLaunch
I haven't modified the Eclipse .classpath file as per Kris' suggestion. Instead, I have added #Profile("test") to my test application class and #ActiveProfiles("test") to my test classes.

After using gradle-flyway plugin to create test database, how to reference db created in c3p0 config?

I was trying to accomplish the following
1) Create h2 database using gradle-flyway plugin before tests run. I was able to create it and place it under ${buildDir}
flyway {
url = "jdbc:h2:file:${buildDir}/db/test/xxxdb"
user = 'root'
locations = [
'classpath:sql'
]
}
2) My goal was to write integration tests against this db created in step 1. I am using c3p0 for pooling and here is my configuration.
<?xml version="1.0"?>
<c3p0-config>
<default-config>
<property name="driverClass">org.h2.Driver</property>
**<property name="jdbcUrl">jdbc:h2:file:#buildDir#/db/test/xxxdb</property>**
<property name="user">xxx</property>
<property name="password">xxx</property>
<property name="minPoolSize">1</property>
<property name="maxPoolSize">50</property>
<property name="acquireIncrement">1</property>
</default-config>
</c3p0-config>
Questions:
a) How do I configure c3p0 to reference the h2 database present in the build directory created in step 1. c3p0 wants absolute path in jdbcUrl property.
b) I configured gradle to replace #buildDir# with appropriate value using this below gradle task. So the database jdbcUrl is configured correctly for c3p0 when I do "gradle build". But when I run the tests though intelliJ, it still reads the jdbcUrl with value containing "#buildDir#" as intellij does not run processTestResources task before running the test.
processTestResources {
filter(ReplaceTokens, tokens: [buildDir: buildDir.getAbsolutePath()])
}
Any ideas on how to solve this issue?
Thanks Mark for your comments. For relative paths, this is the syntax I had used and now it works with both gradle and intellij.
jdbc:h2:file:./db/test/xxxdb

Where to place properties files when running tests via maven + grizzly + jersey test framework?

The my.properties file from my source (src/main/resources) folder keepa getting picked up and used when I try to run my JerseyTest ... whereas I would like the properties files in the test folder (src/test/resources) to be utilized.
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
try {
myProperties.load(classLoader.getResourceAsStream("my.properties"));
}
How can I configure this in Maven?
I'm using:
maven-compiler-plugin version 2.1
jersey-test-framework-grizzly2 version 1.13
UPDATE (Resolved based on accepted answer):
I noticed the text skip non existing resourceDirectory:
[INFO] --- maven-resources-plugin:2.4.3:testResources (default-testResources) # xxx ---
[WARNING] Using platform encoding (MacRoman actually) to copy filtered resources, i.e. build is platform dependent!
[INFO] skip non existing resourceDirectory /xxx/xxx/xxx/xxx/src/test/resources
Turns out that I had misspelled resources, after fixing it, everything works as outlined in the accepted answer.
While I was wasting my time looking for workarounds, I did find some interesting links for configuring properties files based on profiles:
How can I change a .properties file in maven depending on my profile?
http://maven.apache.org/guides/mini/guide-building-for-different-environments.html
When running something from the src/test/java folder, the default behavior is:
It will pick up the files in src/main/resources;
Except when there is a file with the same name in src/test/resources.
So basically it will "overwrite" the content of src/main/resources with the content of src/test/resources:
If you got a file in both folders, the one in src/test/resources will prevail.
If you got a file only in src/main/resources, it will be used.
If you got a file only in src/test/resources, it will be used.

Resources