given the following spring boot application layout:
app
--api
--component
--data
--xyz
libs
--lib1
--lib2
Each one is a seperate maven project where "app" contains the spring bootstrap class. Other modules are maven dependencies that are used within the application.
Each modules comes with its own profiled property file (application-dev.properies, application-qa.properties...)
What is the best way to consolidate now all property files within the application?
My proposal for consolidation:
Each module defines own profile, for example:
application-DEV-lib1.properties
application-QA-lib1.properties
application-DEV-lib2.properties
application-QA-lib2.properties
app defines profiles DEV and QA:
application-DEV.properties
spring.profiles.include=DEV-lib1,DEV-lib2
aaa=123
...
application-QA.properties
spring.profiles.include=QA-lib1,QA-lib2
aaa=456
...
Related
I want to create a library that other modules in my project can depend on, but only for tests. I've read the Gradle documentation extensively and couldn't find how to do it. This is my project structure:
gradle-example:
app:
src/main/kotlin ...
src/test/kotlin <- this is supposed to use 'com.example.SomeTestUtil'
testlib:
src/test/kotlin/com.example.SomeTestUtil
settings.gradle.kts:
rootProject.name = "gradle-example"
include("app")
include("testlib")
In the app module I tried adding
dependencies {
testImplementation(project(:testlib"))
}
But I get compilation error trying to import the SomeTestUtil in the app module test classes.
What is the correct way to declare dependency on a module in the same project that allows to use test sources in test code?
To clarify, we want to create a library that can only be used by other modules under src/test and not src/main.
Apparently this can be done by using the https://docs.gradle.org/current/userguide/java_testing.html#sec:java_test_fixtures
they can see the main source set classes
test sources can see the test fixtures classes
So changes to the example in the question are:
changing the src/test to src/testFixtures
declaring the dependency as testImplementation(testFixtures(project(":testlib")))
Adding java-test-fixtures plugin to the build.gradle.kts for the testlib module
I have a SpringBoot application where I have application.properties file outside of project (it's not in usual place src/main/resources).
While building application with gradle clean build, it fails as code is not able to find properties files.
I have tried many command to pass vm args, gradle opts but its not working.
gradle clean build -Djvmargs="-Dspring.config.location=/users/home/dev/application.properties" //not working
It fails on test phase when it creates Spring application context and not able to substitute property placeholders. If I skip test as gradle clean build -x test it works.
Though I can run the app with java -jar api.jar --spring.config.location=file:/users/home/dev/application.properties
Please help how I can pass spring.config.location=/users/home/dev/application.properties in gradle build using command line so that build runs with all Junit tests
If I were you, I would not get involved the actual properties to junit test. So I would create a test properties for the project under src/test/resources/application-test.properties and in junit test I would load the test properties.
Example:
#RunWith(SpringRunner.class)
#SpringBootTest(classes = MyProperties.class)
#TestPropertySource("classpath:application-test.properties")
public class MyTestExample{
#Test
public void myTest() throws Exception {
...
}
}
System properties for running Gradle are not automatically passed on to the testing framework. I presume this is to isolate the tests as much as possible so differences in the environment will not lead to differences in the outcome, unless explicitly configured that way.
If you look at the Gradle API for the Test task, you can see that you can configure system properties through through the systemProperty method on the task (Groovy DSL):
test {
systemProperty "spring.config.location", "/path/to/my/configuration/repository/application.properties"
}
If you also want to read a system property from the Gradle command line and then pass that the test, you have to read it from Gradle first, e.g. as a project property, and then pass that value to the test:
test {
if (project.hasProperty('testconfig')) {
systemProperty 'spring.config.location', project.getProperty('testconfig')
}
}
Run it with gradle -Ptestconfig="/path/to/my/configuration/repository/application.properties" build
However, I would discourage using system properties on the build command line if you can avoid it. At the very least, it will annoy you greatly in the long run. If the configuration file can be in different locations on different machines (depending on where you have checkout out the repository and if it is not in the same relative path to your Spring Boot repository), you may want to specify it in a personal gradle.properties file instead.
I think there is a misunderstanding.
spring.config.location is used at runtime
As you validated:
java -jar api.jar --spring.config.location=file:/users/home/dev/application.properties
spring.config.location is used or required at runtime, not at build time.
When your spring boot app is building, an application.properties is required. An approach could be use an src/main/resources/application.properties with template values, but at runtime you will ignore it spring.config.location=file...
For unit tests
In this case as #nikos-bob said, you must use another properties, commonly inside of your src/test/resources
Environment variables instead external properties
We don't want to have hardcoded values in our main git repository src/main/resources/application.properties so the first idea is use an external properties. But this file must be stored in another git repository (equal to main repository ) or manually created.
Spring and other frameworks give us an alternative: Use environment variables.
So instead of manually external creation of application.properties or store it in our git repository, your spring boot app always must have an application.properties but with environment variables:
spring.datasource.url=jdbc:oracle:thin:#${DATABASE_HOST}:${DATABASE_PORT}:${DATABASE_SID}
spring.datasource.username=${DATABASE_USER}
spring.datasource.password=${DATABASE_PASSWORD}
spring.mail.host = ${MAIL_HOST}
spring.mail.username =${MAIL_USERNAME}
spring.mail.password =${MAIL_PASSWORD}
Advantages:
No manually creation of application.properties allowing us a more easy devops automations
No spring.config.location=file.. is required
how do you set up log4j for a gradle project with modules?
I have a project set up like the following:
project root
build.gradle
gradle.properties
settings.gradle
// this root project does:
// include CommonModule
// includeBuild <all composite modules within module folder>
---CommonModule
------build.gradle
------gradle.properties
------settings.gradle
------src/main/groovy/...<common-code>
------src/main/resources/log4j2.xml
Modules
---OtherModule-1
// this is a gradle composite module
// it also includes the common module
------build.gradle
------gradle.properties
------settings.gradle
------src/main/groovy/...module1-code
------src/main/resources/log4j2.xml
---OtherModule-2
// this is a gradle composite module
// it also includes the common module
------build.gradle
------gradle.properties
------settings.gradle
------src/main/groovy/...module2-code
------src/main/resources/log4j2.xml
As shown above, we have a common gradle module and a module folder that contains module related composite module, each which depends on the included CommonModule
(core common code goes in common, the composite modules each contain code that extends common stuff
My question is hopefully simple:
where do I configure my log4j module?
e.g. can I put it in the common include module ?
or does each composite module need to have their own log4j xml?
I went and asked this same question on the gradle forums https://discuss.gradle.org/t/how-do-you-set-up-log4j-for-a-gradle-project-with-modules/28262 and got an answer -
Common module doesn’t include a log4j config Individual library
modules don’t include a log4j config either Top-level application
module does include a log4j config, but doesn’t put it on the
classpath (I set aside a src/[configuration]/config directory just
for that)
Launcher for the application itself (whether from a release
build or from a JavaExec in the Gradle build) specifies where the
log4j config is loaded from, so that you can’t get accidentally
poisoned by other configs on the classpath
you can define the xml location by providing a -D flag in the gradle properties file e.g. -Dlog4j.configurationFile=
also -Dlog4j.debug might help to track down issues
Please go through the scenario given below
I have two maven projects abc-common and abc-service
abc-service has dependency on abc-common project.
abc-common is reading from a properties file named myConfig.properties as follows:
class PropertiesUtil {
.....
Properties props = new Properties();
props.load(PropertiesUtil.class.getClassLoader().getResourceAsStream("myConfig.properties"));
....
}
This works fine, when I test by creating a dummy main method in either of the projects.
But when I deploy this project into an OSGI container (JBOSS Fuse), it is not working. The prime reason is, in OSGI container, PropertiesUtil.class.getLoader() is refering to the bundle corresponding to abc-service project and I can read any file from that project, but not from abc-common project.
So the question is that, how can I change my code such that, It can read properties from class path of abc-common project in an OSGI container.
Note
I'm deploying my project as a karaf feature which lists both abc-service and abc-common bundles as dependencies.
Also, I tried different variants like
Thread.currentThread().getContextClassLoader().getResourceAsStream("myConfig.properties")
and
FrameworkUtil.getBundle(PropertiesUtil.class).getEntry("myConfig.properties.")
But none of them worked actually
In OSGi you need to use import|export of packages to allow loading resources from other bundles. Put the properties file into a package which you export from that bundle. And then from the other bundle, you import that package. You should then be able to load the resource from classpath.
I have a Spring Boot Gradle project setup in Spring Tools Suite (3.7.2 RELEASE) with the following source folders:
- src/integration-test/java
- src/integration-test/resources
- src/main/java
- src/main/resources
- src/test/java
- src/test/resources`
Whenever I run the application or unit tests from within STS, I see that STS is using the resources found under src/integration-test/resources.
I see a duplicate resource warning in STS for files which exist in all 3 resource source folders. For example, I have an application.properties in all 3 source folders and I see following:
The resource is a duplicate of src/integration-test/resources/application.properties and was not copied to the output folder
If I run the application as a JAR or unit tests/integration tests from the command line (via gradle build), everything seems to use the correct resources. This makes me believe it is a problem with how STS/Eclipse is handling gradle.
Does anybody know of how I can configure STS to use the correct resource source folders when using gradle?
I think my problem may be related to (or the same as?) Spring Boot incorrectly loads test configuration when running from eclipse+gradle, https://issuetracker.springsource.com/browse/STS-3882, https://issues.gradle.org/browse/GRADLE-1777
I also tried the solution found here, but that seems to only fix Maven builds:
Spring Tool Suite finds spring-boot integration test configuration and does not start main application
I think my problem may be related to...
Yes, it is related but in my opinion not the same. That problem is caused by the runtime classpath being incorrect. This problem is an error coming from the eclipse project builder so it is a compile-time issue.
The problems are closely related though. Depending on your point of view, you could say they are the same (incorrect mixing of test and compile-time classpaths).
Here, specifically, the problem is that the eclipse builder tries to copy all the resources it finds in source folders to the project's single output folder. Each source folder has a 'application.properties'. The builder warns that it could not copy some of them because one would overwrite the other.
I think there may be a solution for this problem. But it is a solution that really should come from Gradle + ( BuildShip | STS Gradle Tooling) than from you.
It is possible in Eclipse to configure each source-folder individually to target a specific outputfolder. Maven + M2E are doing this correcty, but Gradle + (BuildsShip | STS Gradle Tooling) combdos do not.
For example this is what maven puts into the eclipse .classpath file when it configures a test resources folder:
<classpathentry excluding="**" kind="src" output="target/test-classes" path="src/test/resources">
<attributes>
<attribute name="maven.pomderived" value="true"/>
</attributes>
</classpathentry>
Notice how it explicitly sets the output folder for that entry (to something different from the project's default output folder).
You may be able to address the problem yourself by modifying the .classpath for a gradle project in a similar way. Either by doing it manually or from your build.gradle.
I'm not sure this is worth it however as you will then likely still get hit by the runtime classpath issue (since these folders will still be added to your runtime classpath, your runtime classpath will end-up with two appication.properties resources, one which will 'shadow' the other. See: https://bugs.eclipse.org/bugs/show_bug.cgi?id=482315)
I would say, the right thing to do is add a comment to the issue I linked, and hope they fix it soon as there is only so much you can do yourself by hacking the build.gradle file to modify the .classpath (this can not solve the runtime classpath issue, but in order to solve the runtime classpath issue, they would have to configure source folders to target individual output folder similar to what m2e does).
I would add this as a comment to #Kris's answer but it's too long.
I have solved the runtime classpath issue by adding the code below to my build.gradle file. The code generates an Eclipse launch configuration for the Spring Boot application class and includes only the runtime classpath (i.e. no test JARs).
My project uses the Gradle 'eclipse' plugin to generate the Eclipse project files (which I then import into Eclipse). Running the eclipseClasspath Gradle target will generate the launch file in the project's root directory.
def mainClassName = "com.example.MyApplication"
task eclipseApplicationLaunch {
group "IDE"
description "Generate an Eclipse launch configuration file for the Spring Boot application class"
}
eclipseApplicationLaunch << {
def writer = new FileWriter("${mainClassName.substring(mainClassName.lastIndexOf(".")+1)}.launch")
def xml = new groovy.xml.MarkupBuilder(writer)
xml.doubleQuotes = true
xml.launchConfiguration(type: "org.eclipse.jdt.launching.localJavaApplication") {
listAttribute(key:"org.eclipse.debug.core.MAPPED_RESOURCE_PATHS") {
listEntry(value:"/${project.name}/src/main/java/${mainClassName.replace(".","/")}.java")
}
listAttribute(key:"org.eclipse.debug.core.MAPPED_RESOURCE_TYPES") {
listEntry(value:"1")
}
listAttribute(key:"org.eclipse.jdt.launching.CLASSPATH") {
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry containerPath=\"org.eclipse.jdt.launching.JRE_CONTAINER/org.eclipse.jdt.internal.debug.ui.launcher.StandardVMType/JavaSE-1.8/\" javaProject=\"${project.name}\" path=\"1\" type=\"4\"/>\r\n")
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry path=\"3\" projectName=\"${project.name}\" type=\"1\"/>\r\n")
configurations.runtime.resolvedConfiguration.resolvedArtifacts.each { artifact ->
def filePath = artifact.file.canonicalPath.replace("\\","/")
listEntry(value:"<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"no\"?>\r\n<runtimeClasspathEntry externalArchive=\"${filePath}\" path=\"3\" type=\"2\"/>\r\n")
}
}
booleanAttribute(key:"org.eclipse.jdt.launching.DEFAULT_CLASSPATH", value:"false")
stringAttribute(key:"org.eclipse.jdt.launching.MAIN_TYPE", value:"${mainClassName}")
stringAttribute(key:"org.eclipse.jdt.launching.PROGRAM_ARGUMENTS", value:"--spring.profiles.active=local --spring.config.location=conf/")
stringAttribute(key:"org.eclipse.jdt.launching.PROJECT_ATTR", value:"${project.name}")
stringAttribute(key:"org.eclipse.jdt.launching.VM_ARGUMENTS", value:"-Djava.net.preferIPv4Stack=true")
}
writer.close()
}
eclipseClasspath.dependsOn eclipseApplicationLaunch
I haven't modified the Eclipse .classpath file as per Kris' suggestion. Instead, I have added #Profile("test") to my test application class and #ActiveProfiles("test") to my test classes.