Running flyway commands from command line using spring boot properties for maven - spring-boot

I have a spring boot project using maven that I've included flyway in:
pom.xml:
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-core</artifactId>
<version>6.5.0</version>
</dependency>
and application.properties:
#LOCAL
spring.datasource.url=jdbc:postgresql://localhost:5432/theDatabase
spring.datasource.username=theRightUser
spring.datasource.password=theRightPassword
and it works as expected when I run the application.
However, I'm trying to run mvn flyway:clean from the command line, and it doesn't seem to recognize the configuration correctly:
[ERROR] Failed to execute goal org.flywaydb:flyway-maven-plugin:6.4.4:clean (default-cli) on project my-service: org.flywaydb.core.api.FlywayException: Unable to connect to the database. Configure the url, user and password! -> [Help 1]
I tried adding spring.flyway properties (user/pass/url) in the application.properties file but it gave me the same error. What do I need to do to get flyway to read from the application.properies like it does when the application runs normally?
EDIT: I have made slight progress: I was able to reference my application.properties as a flyway config file by adding this to the pom.xml:
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>6.5.0</version>
<configuration>
<configFiles>${project.basedir}/src/main/resources/application.properties</configFiles>
</configuration>
</plugin>
So now in that file, I have flyway.url, flyway.user and flyway.password. This allows me to run flyway goals from the command line, but is not totally the solution I want. I am looking into using this plugin to try and read the properties into the pom.xml file, and then using those values in the flyway-maven-plugin's <configuration> area.
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>properties-maven-plugin</artifactId>
<version>1.0.0</version>
<executions>
<execution>
<phase>initialize</phase>
<goals>
<goal>read-project-properties</goal>
</goals>
<configuration>
<files>
<file>${project.basedir}/src/main/resources/application.properties</file>
</files>
</configuration>
</execution>
</executions>
</plugin>
which would allow me to do this:
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>6.5.0</version>
<configuration>
<url>${spring.datasource.url}</url>
<user>${spring.datasource.username}</user>
<password>${spring.datasource.password}</password>
</configuration>
</plugin>

When you run flyway as maven goal it will not pick up the properties from the application.properties, instead it will use the configuration provided by the flyway-maven-plugin, you can configure the flyway-maven-plugin in the following way -
Add the following plugin to pom.xml -
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>6.5.0</version>
</plugin>
Then we will configure the flyway-maven-plugin, the Flyway Maven plugin can be configured in a wide variety of following ways (most convenient),
Configuration section of the plugin
The easiest way is to simply use the plugin’s configuration section in your pom.xml:
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>6.5.0</version>
<configuration>
<driver>org.hsqldb.jdbcDriver</driver>
<url>jdbc:hsqldb:file:${project.build.directory}/db/flyway_sample;shutdown=true</url>
<user>SA</user>
<password>mySecretPwd</password>
<connectRetries>10</connectRetries>
<initSql>SET ROLE 'myuser'</initSql>
<schemas>
<schema>schema1</schema>
<schema>schema2</schema>
<schema>schema3</schema>
</schemas>
<callbacks>
<callback>com.mycompany.project.CustomCallback</callback>
<callback>com.mycompany.project.AnotherCallback</callback>
</callbacks>
<skipDefaultCallbacks>false</skipDefaultCallbacks>
<cleanDisabled>false</cleanDisabled>
<skip>false</skip>
<configFiles>
<configFile>myConfig.conf</configFile>
<configFile>other.conf</configFile>
</configFiles>
<workingDirectory>/my/working/dir</workingDirectory>
</configuration>
</plugin>
Maven properties
To make it easy to work with Maven profiles and to logically group configuration, the Flyway Maven plugin also supports Maven properties, update the properties section in pom.xml as follows:
<project>
...
<properties>
<!-- Properties are prefixed with flyway. -->
<flyway.user>myUser</flyway.user>
<flyway.password>mySecretPwd</flyway.password>
<!-- List are defined as comma-separated values -->
<flyway.schemas>schema1,schema2,schema3</flyway.schemas>
<!-- Individual placeholders are prefixed by flyway.placeholders. -->
<flyway.placeholders.keyABC>valueXYZ</flyway.placeholders.keyABC>
<flyway.placeholders.otherplaceholder>value123</flyway.placeholders.otherplaceholder>
</properties>
...
</project>
External Configuration File
Another way is to create a separate .properties file, the default configuration file name is flyway.properties and it should reside in the same directory as the pom.xml file. Encoding is specified by flyway.encoding (Default is UTF-8):
flyway.user=databaseUser
flyway.password=databasePassword
flyway.schemas=schemaName
...
If you are using any other name (e.g customConfig.properties) as the configuration file, then it should be specified explicitly when invoking the Maven command:
$ mvn <goals> -Dflyway.configFile=customConfig.properties
After configuring your flyway-maven-plugin, with the desired configuration, we will be able to execute flyway maven goals from command line.
You can do further reading here.
Hope this helps!

Related

Executable JAR - JVM Arguments - Externalized Config Properties - Spring Boot Maven Plugin

We have a spring boot app with externalized configuration in a properties file. Executing the jar file like so works:
java -jar -Dspring.config.location="application.properties" app.jar
Executing the jar directly like below works, but without the properties file. Right now, none of these options below take the properties file.
./app.jar
./app.jar -Dspring.config.location="file:./application.properties"
I was hoping we could do something in the pom.xml or in the command that executes to achieve this.
FYI, we have this plugin in the pom.xml to build us the jar file:
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<executable>true</executable>
<includeSystemScope>true</includeSystemScope>
<!-- This below does not work -->
<!-- <jvmArguments> -->
<!-- -Dspring.config.location=application.properties -->
<!-- </jvmArguments> -->
</configuration>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
Create file: app.conf (filename is the same as your jar) with content:
JAVA_OPTS="-Dspring.config.location=application.properties"
Similar question was answered here. Documentation for customizing start script can be found here.

When does a Maven plugin uses the POM in the current directory?

I use the Versions Maven Plugin to check for updates of my dependencies. Therefore I added the following lines to my pom.xml:
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>${versions-plugin.version}</version>
<configuration>
<rulesUri>classpath:///rules.xml</rulesUri>
</configuration>
<dependencies>
<dependency>
<groupId>versionrules</groupId>
<artifactId>versionrules</artifactId>
<version>1-SNAPSHOT</version>
</dependency>
</dependencies>
</plugin>
But this configuration is not used if I run the Versions Maven Plugin from the commandline in the same directory as the pom.xml. The only way to use my own configuration is to put this plugin configuration in a profil and execute this profil during the Maven run.
Is there a way to run the Versions plugin on the commandline and to configure it via the pom.xml? I am sure my questions does not only apply to the Versions plugin, but to any Maven plugin.
This can be done by using an execution id default-cli in your execution definition the configuration will be used during the execution on command line (using the current configuration) furthermore since Maven 3.3.1 you can use things like:
mvn version:set#second-cli
which means you can do a different configuration for command line in the pom file:
Just by simply separating them by different id
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>versions-maven-plugin</artifactId>
<version>2.5.</version>
<executions>
<execution>
<id>default-cli</id>
<configuration>
...
</configuration>
</execution>
<execution>
<id>second-cli</id>
<configuration>
....
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
So this means you can have different configurations for running on command line by giving the id.

Using Spring Boot without the parent POM [duplicate]

Is there a specific recommended approach to the inclusion of the spring-boot parent pom into projects that already have a required parent POM?
What do you recommend for projects that need to extend from an organizational parent (this is extremely common and even something many/most projects published to Maven central depending on the feeder repos they come from). Most of the build stuff is related to creating executable JARs (e.g. running embedded Tomcat/Jetty). There are ways to structure things so that you can get all the dependencies without extending from a parent (similar to composition vs. inheritance). You can't get a build stuff that way though.
So is it preferable to include all of the spring-boot parent pom inside of the required parent POM or to simply have a POM dependency within the project POM file.
Other options?
TIA,
Scott
You can use the spring-boot-starter-parent like a "bom" (c.f. Spring and Jersey other projects that support this feature now), and include it only in the dependency management section with scope=import.That way you get a lot of the benefits of using it (i.e. dependency management) without replacing the settings in your actual parent.
The 2 main other things it does are
define a load of properties for quickly setting versions of dependencies that you want to override
configure some plugins with default configuration (principally the Spring Boot maven plugin). So those are the things you will have to do manually if you use your own parent.
Example provided in Spring Boot documentation:
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>2.1.3.RELEASE</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
Update 2022-05-29 with 1.5.9.RELEASE.
I have full code and runable example here https://github.com/surasint/surasint-examples/tree/master/spring-boot-jdbi/9_spring-boot-no-parent (see README.txt to see that you can try)
You need this as a basic
<dependencyManagement>
<dependencies>
<dependency>
<!-- Import dependency management from Spring Boot -->
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-dependencies</artifactId>
<version>${springframework.boot.version}</version>
<type>pom</type>
<scope>import</scope>
</dependency>
</dependencies>
</dependencyManagement>
But that is not enough, you also need explicitly define goal for spring-boot-maven-plugin (If you use Spring Boot as parent, you do not have to explicitly define this)
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<version>${springframework.boot.version}</version>
<executions>
<execution>
<goals>
<goal>repackage</goal>
</goals>
</execution>
</executions>
</plugin>
</plugins>
Otherwise you cannot build as executable jar or war.
Not yet, if you are using JSP, you need to have this:
<properties>
<failOnMissingWebXml>false</failOnMissingWebXml>
</properties>
Otherwise, you will get this error message:
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:2.2:war (default-war) on project spring-boot-09: Error assembling WAR: webxml attribute is required (or pre-existing WEB-INF/web.xml if executi
ng in update mode) -> [Help 1]
NO NO , this is still not enough if you are using Maven Profile and Resource Filter with Spring Boot with "#" instead of "${}" (like this example https://www.surasint.com/spring-boot-maven-resource-filter/). Then you need to explicitly add this in
<resources>
<resource>
<directory>src/main/resources</directory>
<filtering>true</filtering>
</resource>
</resources>
And this in
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-resources-plugin</artifactId>
<version>2.7</version>
<configuration>
<delimiters>
<delimiter>#</delimiter>
</delimiters>
<useDefaultDelimiters>false</useDefaultDelimiters>
</configuration>
</plugin>
See the example in the link https://www.surasint.com/spring-boot-with-no-parent-example/.
As per Surasin Tancharoen's answer, you may also want to define maven surefire plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>${maven-surefire-plugin.version}</version>
</plugin>
and possibly include fail-fast plugin
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<version>${maven-failsafe-plugin.version}</version>
<executions>
<execution>
<goals>
<goal>integration-test</goal>
<goal>verify</goal>
</goals>
</execution>
</executions>
</plugin>

Intellij IDEA artifact 'XXXX:war exploded' has invalid extension

Every time I make even the tiniest change to my POM Intellij removes the .war extension for my exploded artifact in the Project Structure output directory setting. This causes an error in Intellij's Run/Debug configuration:
Artifact 'XXXX:war exploded' has invalid extension.
In order to resolve the issue I must manually override the Project Structure output directory setting. Every time I make even the tiniest change to the POM I must go back to the Output directory setting and manually append ".war" to the end of the Output directory setting. This is getting very old and frustrating.
e.g. I must change this:
E:\workarea\enterp\application\target\application
to this:
E:\workarea\enterp\application\target\application.war
If I manually set the Maven WAR plugin outputDirectory configuration as follows, this does not help at all:
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>${maven.war.plugin.version}</version>
<configuration>
<!-- Output directory of artifact:war exploded keeps losing the .war extension -->
<outputDirectory>${project.build.directory}.war</outputDirectory>
</configuration>
</plugin>
How can I resolve this problem?
EDIT:
Here's the complete build config:
<build>
<!-- Maven will append the version to the finalName (which is the name
given to the generated war, and hence the context root) -->
<finalName>${project.artifactId}</finalName>
<plugins>
<!-- Compiler plugin enforces Java 1.6 compatibility and activates annotation
processors -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>${maven.war.plugin.version}</version>
<configuration>
<!-- Output directory of artifact:war exploded keeps losing the .war extension -->
<outputDirectory>${project.build.directory}/${project.artifactId}.war</outputDirectory>
<!-- Java EE 7 doesn't require web.xml, Maven needs to catch up! -->
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<!-- The WildFly plugin deploys your war to a local WildFly container -->
<!-- To use, run: mvn package wildfly:deploy -->
<plugin>
<groupId>org.wildfly.plugins</groupId>
<artifactId>wildfly-maven-plugin</artifactId>
<version>${version.wildfly.maven.plugin}</version>
</plugin>
</plugins>
</build>
SECOND EDIT:
I discovered that one solution is to append ".war" to ${project.artifactId} in the build configuration, e.g.:
<finalName>${project.artifactId}.war</finalName>
and remove outputDirectory from the plugin configuration. So the build config should look like this:
<build>
<!--
Maven will make finalName the name of the generated war.
NOTE: Output directory of artifact:war exploded keeps losing the .war extension
http://youtrack.jetbrains.com/issue/IDEA-86484
http://youtrack.jetbrains.com/issue/IDEA-95162
The solution is to append ".war" to ${project.artifactId}, below:
-->
<finalName>${project.artifactId}.war</finalName>
<plugins>
<!-- Compiler plugin enforces Java 1.6 compatibility and activates annotation
processors -->
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>${maven.compiler.plugin.version}</version>
<configuration>
<source>${maven.compiler.source}</source>
<target>${maven.compiler.target}</target>
</configuration>
</plugin>
<plugin>
<artifactId>maven-war-plugin</artifactId>
<version>${maven.war.plugin.version}</version>
<configuration>
<!-- Java EE 7 doesn't require web.xml, Maven needs to catch up! -->
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
<!-- The WildFly plugin deploys your war to a local WildFly container -->
<!-- To use, run: mvn package wildfly:deploy -->
<plugin>
<groupId>org.wildfly.plugins</groupId>
<artifactId>wildfly-maven-plugin</artifactId>
<version>${version.wildfly.maven.plugin}</version>
</plugin>
</plugins>
</build>
DISCLAIMER: If you use this workaround just be aware that when you deploy an unexploded WAR artifact the file name will be named XXXX.war.war. It works -- I deployed the artifact as a WAR file in Intellij -- but it's ugly.
INFO [org.jboss.as.server.deployment] (MSC service thread 1-7) JBAS015876: Starting deployment of "XXXX.war.war" (runtime-name: "XXXX.war.war)"
If someone can tell me how to configure the Intellij project to work with Maven to select one or the other finalName values depending on whether I'm deploying a WAR file vs. exploded artifact then this question will be sufficiently answered.
<!-- Exploded artifact -->
<finalName>${project.artifactId}.war</finalName>
<!-- WAR file (unexploded) artifact -->
<finalName>${project.artifactId}</finalName>
There's a way to fix this in IntelliJ, without changing your pom.xml file(s), by adding an artifact with a reference to the exploded war (or in my case, the exploded ear) and it won't get stomped every time IntelliJ re-imports the maven pom(s). Here's how:
Stop/undeploy your current artifact deployment
Edit your run config, and in the Deployment tab, remove the current exploded war/ear artifact
Open the project's Artifacts settings and add a new artifact
Use the plus button to add a new war or (in my case) ear exploded artifact
Give it a name, then edit the Output directory to add the appropriate extension (.war or .ear)
In the Output Layout section where you see <output root>, use the plus button to add an Artifact
Select the desired exploded artifact
Edit your run config again, and in the Deployment tab, add the new workaround exploded artifact
Thanks to Nikolay Chashnikov for describing this in his comment on the bug report
Actually, you should leave the finalName attribute alone, otherwise you'll get the problems you describe. Rather, you should change the config for the maven war plugin to use the webappDirectory like this:
<plugin>
<artifactId>maven-war-plugin</artifactId>
<configuration>
<webappDirectory>${project.build.directory}/${project.artifactId}.${project.packaging}</webappDirectory>
<failOnMissingWebXml>false</failOnMissingWebXml>
</configuration>
</plugin>
If we are talking about WAR inside EAR there is another way to resolve your problem by using correct configuration inside maven-ear-plugin. WAR pom.xml should be left as is, without any changes, but EAR pom.xml should contains something like this. (please, pay your attention to <unpack>${unpack.wars}</unpack>)
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-ear-plugin</artifactId>
<version>2.9</version>
<configuration>
<version>6</version>
<defaultLibBundleDir>lib</defaultLibBundleDir>
<generateApplicationXml>false</generateApplicationXml>
<archive>
<manifest>
<addClasspath>true</addClasspath>
</manifest>
</archive>
<modules>
<webModule>
<groupId>com.test.app</groupId>
<artifactId>test-app-war</artifactId>
<unpack>${unpack.wars}</unpack>
</webModule>
</modules>
</configuration>
</plugin>
and then you can add profiles default and debug for proper artifact assembling.
<profiles>
<profile>
<id>default</id>
<activation>
<activeByDefault>true</activeByDefault>
</activation>
<properties>
<unpack.wars>false</unpack.wars>
</properties>
</profile>
<profile>
<id>debug</id>
<activation>
<property>
<name>debug</name>
</property>
</activation>
<properties>
<unpack.wars>true</unpack.wars>
</properties>
</profile>
</profiles>
use debug profile inside IntelliJ IDEA for expanding wars and default profile for building artifacts in command line or CI (default profile would be active if no profile were provided, so your build will works as previously).
With this solution HotSwap and resources updates works as expected.
Hope this helps.
I think it's the same as this question: IntelliJ Artifact has invalid extension
Add a .war extension to the output directory as shown in my answer: https://stackoverflow.com/a/25569266/968988

Maven archetype creation : prototype pom

I am creating a maven archetype. In this I have a prototype project, which gets created for a user when the user calls the following command:
mvn archetype:generate -DarchetypeGroupId=xxx -DarchetypeArtifactId=archtype-yyyy -DarchetypeVersion=1.1.0-S5-SNAPSHOT -DgroupId=zzz -DartifactId=proj11
In the prototype pom, I want to use the 'archetypeVersion' property that I am specifying in the above command. Like this:
<dependencies>
<dependency>
<groupId>mmmm</groupId>
<artifactId>nte</artifactId>
<version>${archetypeVersion}</version>
</dependency>
This is not working for me. When the project is created, it still shows the dependency snippet in the generated pom exactly as it is posted above. It does not replace it.
Is this possible? Does maven allow this?
If yes, how can I do it?
I think simple way to do this is to use maven-replacer-plugin. You have to add next section to archetype /pom.xml:
<build>
...
<plugins>
<plugin>
<groupId>com.google.code.maven-replacer-plugin</groupId>
<artifactId>replacer</artifactId>
<version>1.5.2</version>
<executions>
<execution>
<phase>prepare-package</phase>
<goals><goal>replace</goal></goals>
</execution>
</executions>
<configuration>
<file>target/classes/archetype-resources/pom.xml</file>
<replacements>
<replacement>
<token>\$\{archetypeVersion\}</token>
<value>${version}</value>
</replacement>
</replacements>
</configuration>
</plugin>
</plugins>
...
<build>
ie this code replace ‘${archetypeVersion}’ substring to current version of archetype. Your ‘/src/main/resources/archetype-resources/pom.xml’ contains next dependency:
<dependency>
<groupId>xxxx</groupId>
<artifactId>yyyy</artifactId>
<version>${archetypeVersion}</version>
</dependency>
After executing ‘mvn install’ command, the resulting file ‘/target/classes/archetype-resources/pom.xml’ will be contain archetype version number. Now you have installed archetype and can use its: ‘mvn archetype:generate ...’.
Easiest way I've found is to just add it as a defaulted variable in your META-INF/maven/archetype-metadata.xml as follows:
<archetype-descriptor
xmlns="https://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="https://maven.apache.org/plugins/maven-archetype-plugin/archetype-descriptor/1.1.0 http://maven.apache.org/xsd/archetype-descriptor-1.1.0.xsd"
name="archetypeVersionExample">
<requiredProperties>
...
<requiredProperty key="archetypeVersion">
<defaultValue>${version}</defaultValue>
</requiredProperty>
</requiredProperties>
...
</archetype-descriptor>
No extra plugins or user entry needed.

Resources