Liquibase scripts not getting executed when app is packaged - spring-boot

I’m facing a strange issue with Liquibase (I am using 3.4.2) :
When I start the application from my IDE, new scripts get deployed with no issue.
When I package my application (as a jar) and deploy it in my CI pipeline, scripts don’t get executed and I get below error :
INFO 8 --- [ main] liquibase : Successfully acquired change log lock
WARN 8 --- [ main] liquibase : included file classpath:/db/changelog/db.changelog-master.yaml/ is not a recognized file type
I’m using includeAll mode:
databaseChangeLog:
- includeAll:
path: changes
relativeToChangelogFile: true
Also, in case it makes a difference, I have several modules in my application (a Spring Boot app) :
My main application is moduleA : this is the jar I build and run
moduleA has a dependency on moduleB, in which the Liquibase scripts are
problem seems to be similar to the one described here : http://forum.liquibase.org/topic/includeall-not-working-for-multimodule-project-with-spring , but no solution is provided.

After trying various things, like including files one by one, or upgrade Liquibase version, it looks like the simplest way to get this to work is to tweak spring-boot-maven-plugin config for moduleA by unpacking files : this way, Liquibase is able to find the files it’s looking for.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<requiresUnpack>
<dependency>
<groupId>com.mydomain</groupId>
<artifactId>moduleB</artifactId>
</dependency>
</requiresUnpack>
</configuration>
</plugin>

Related

mvn liquibase:generateChangeLog not working with an URL problem

I don't understand something with Liquibase and Spring Boot.
When I run my spring boot project with : mvn spring-boot:run
My changelog-v1.0.xml create a table and add two users inside my table user. So my liquibase is linked to my mySQL Database with success. But ... I don't know why, I can't used mvn liquibase command without failure.
Exemple, I want to save my database using :
mvn liquibase:generateChangeLog
But I have that log failure :
Failed to execute goal org.liquibase:liquibase-maven-plugin:4.9.0:generateChangeLog (default-cli) on project demo: The database URL has not been specified either as a parameter or in a properties file.
I think, when i'm using the command line, he don't use my configuration inside pox.xml but I don't know how to do that.
# pom.xml
<plugin>
<groupId>org.liquibase</groupId>
<artifactId>liquibase-maven-plugin</artifactId>
<version>4.9.0</version>
<configuration>
<propertyFileWillOverride>true</propertyFileWillOverride>
<promptOnNonLocalDatabase>false</promptOnNonLocalDatabase>
<ChangeLogFile>src/main/resources/db/changelog/changelog-master.xml</ChangeLogFile>
<driver>${spring.datasource.driverClassName}</driver>
<url>${spring.datasource.url}</url>
<username>${spring.datasource.username}</username>
<password>${spring.datasource.password}</password>
</configuration>
<dependencies>
<dependency>
<groupId>javax.xml.bind</groupId>
<artifactId>jaxb-api</artifactId>
<version>2.3.0</version>
</dependency>
</dependencies>
</plugin>
My pom.xml is correctly setup because new changelog file inside liquibase directory create or update my data from my mysql database.
mvn liquibase:generateChangeLog does not depend on spring, so it is not going to read the spring file. You can either create a liquibase.properties file and reference it.
I changed the pom to reference a properties file
https://docs.liquibase.com/tools-integrations/maven/maven-pom-file.html
and then changed the changeLogFile configuration to use outputChangeLogFile like this:
<outputChangeLogFile>src/main/resources/db/changelog/changelog-master.xml</outputChangeLogFile>

How to run spring batch application without spring boot maven plugin

Is it possible to run a spring batch application without spring boot maven plugin?
I need to run the application on another server and transferring a big jar (with spring dependencies) is time consuming. For this reason I want to compile and create a jar without spring dependencies included inside the jar and run it from command line.
If you want to create a simple JAR without spring boot dependencies, remove spring-boot-plugin from build.
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
Above snippet should not be present in your pom.xml.
With this, fat/uber jar won't be created.
To run built jar:
java -cp MyJar.jar:lib/* com.somepackage.subpackage.Main
Where your libraries are contained in lib directory relative to current directory and MyJar.jar is the name of the jar built with maven.

Spring boot application with apache axis

I am trying to run a spring boot jar which has axis2 dependencies in it. I am using spring boot maven plugin to build the jar (with dependencies). When I try to run my jar, I get the following exception in my console:
org.apache.axis2.AxisFault: The G:application\myapp\target\myapp.jar!\lib\axis2-1.6.1.jar file cannot be found.
at org.apache.axis2.deployment.repository.util.DeploymentFileData.setClassLoader(DeploymentFileData.java:111)
at org.apache.axis2.deployment.ModuleDeployer.deploy(ModuleDeployer.java:70)
at org.apache.axis2.deployment.repository.util.DeploymentFileData.deploy(DeploymentFileData.java:136)
at org.apache.axis2.deployment.DeploymentEngine.doDeploy(DeploymentEngine.java:813)
at org.apache.axis2.deployment.RepositoryListener.loadClassPathModules(RepositoryListener.java:222)
at org.apache.axis2.deployment.RepositoryListener.init2(RepositoryListener.java:71)
at org.apache.axis2.deployment.RepositoryListener.<init>(RepositoryListener.java:64)
at org.apache.axis2.deployment.DeploymentEngine.loadFromClassPath(DeploymentEngine.java:175)
at org.apache.axis2.deployment.FileSystemConfigurator.getAxisConfiguration(FileSystemConfigurator.java:135)
at ...
I then checked the structure of my jar. It has lib folder inside it, which contained all the jars (including the above mentioned axis jar). Attached is the screen shot of lib folder.
Following are the solutions which I have tried:
Placed axis jar in the same directory as application jar.
Created lib folder in the same directory as application jar and placed axis jar inside it.
Modified manifest file to include Class-Path: /lib/
None of the solutions has worked. However, when I run the application class in eclipse, the app starts and runs perfectly. But, once I create the jar, nothing seems to run.
Can anyone please help? Thanks in advance.
It looks like Axis can't cope with being run from a jar that's nested within another jar. It works fine in Eclipse as the Axis jar is available directly on the filesystem rather than being nested inside your Spring Boot application's jar file.
You can configure your application's fat jar file so that Spring Boot knows to unpack the Axis jar into a temporary location when it's run. If you're using Maven:
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<requiresUnpack>
<dependency>
<groupId>org.apache.axis2</groupId>
<artifactId>axis2</artifactId>
</dependency>
</requiresUnpack>
</configuration>
</plugin>
</plugins>
</build>
And if you're using Gradle:
springBoot {
requiresUnpack = ['org.apache.axis2:axis2']
}
See the Spring Boot documentation for some further details.

Turn off or configure java.util.Logging during Maven build

I want to clean up/unify the log output of a Maven build. Unfortunately, a dependency of a maven plugin uses java.util.Logging (JUL). Simply adding org.slf4j:jul-to-slf4j as an additional dependency to redirect the log output doesn't help. Exclusions also don't work of course, because it's JUL and therefore not a dependency.
Here's my configuration, containing the specific plugin and its dependency:
Configuration
<plugin>
<groupId>org.apache.cxf</groupId>
<artifactId>cxf-codegen-plugin</artifactId>
<version>2.7.11</version>
<executions>...</executions>
<configuration>
<defaultOptions>
<extraargs>
<extraarg>-xjc-Xbg</extraarg>
</extraargs>
</defaultOptions>
</configuration>
<dependencies>
<!-- This one uses JUL with logging level WARN -->
<dependency>
<groupId>org.apache.cxf.xjcplugins</groupId>
<artifactId>cxf-xjc-boolean</artifactId>
<version>2.7.0</version>
</dependency>
</dependencies>
</plugin>
Update
I've managed to turn off/configure JUL itself by adding a logging.properties file with the following content to the project's root folder:
# Turn off any java.util.Loggers (or use SEVERE to see problems)
.level=OFF
Additionally, I have to execute Maven with the following parameter (or set it into MAVEN_OPTS):
mvn clean install -Djava.util.logging.config.file=${basedir}/logging.properties
Remaining Question
Is there any way to achieve the same result without extra files and/or JVM parameters (pom.xml only)?
Following configuration for the plugin cxf-codegen-plugin prevented to show DEBUG log entries when building my project with maven:
<configuration>
<fork>once</fork>
<additionalJvmArgs>
-Dorg.apache.cxf.Logger=null
</additionalJvmArgs>
</configuration>
Instead of a properties file or completely disabling all logging as suggested in another answer, you can also set the desired log level like this:
<additionalJvmArgs>-Dorg.apache.cxf.level=OFF</additionalJvmArgs>
The level needs to be one of the java.util.logging.Level.
I was having similar behavior: was getting WARNING messages from CXF 2.6.3 generated webservices classes when running my Struts2 test cases, polluting my test log.
Since the log trace wasn't showing from what package was sended, I was trying to config the wrong package.
I've added this to my setUp() test case method from my test that extends XWorkTestCase (or whatever you need):
public void setUp() throws Exception{
super.setUp();
java.util.logging.Logger.getLogger("org.apache.cxf").setLevel(Level.OFF);
}
Then no more logging messages from CXF appears when I run my tests from maven.

What is the difference in Maven between dependency and plugin tags in pom.xml?

Created project with Spring, Hibernate & Maven. My question is what is the logic behind plugin versus dependency ?
Both plugins and dependencies are Jar files.
But the difference between them is, most of the work in maven is done using plugins; whereas dependency is just a Jar file which will be added to the classpath while executing the tasks.
For example, you use a compiler-plugin to compile the java files. You can't use compiler-plugin as a dependency since that will only add the plugin to the classpath, and will not trigger any compilation. The Jar files to be added to the classpath while compiling the file, will be specified as a dependency.
Same goes with your scenario. You have to use spring-plugin to execute some spring executables [ I'm not sure what spring-plugins are used for. I'm just taking a guess here ]. But you need dependencies to execute those executables. And Junit is tagged under dependency since it is used by surefire-plugin for executing unit-tests.
So, we can say, plugin is a Jar file which executes the task, and dependency is a Jar which provides the class files to execute the task.
Hope that answers your question!
Maven itself can be described as food processor which has many different units that can be used to accomplish different tasks. Those units are called plugins. For example, to compile your project maven uses maven-compiler-plugin, to run tests - maven-surefire-plugin and so on.
Dependency in terms of maven is a packaged piece of classes that your project depends on. It can be jar, war etc. For example, if you want to be able to write JUnit test, you'll have to use JUnit annotations and classes thus you have to declare that your project depends on JUnit.
Plugins and dependencies are very different things and these are complementary.
What plugins are ?
Plugins perform tasks for a Maven build. These are not packaged in the application.
These are the heart of Maven.
Any task executed by Maven is performed by plugins.
There are two categories of plugins : the build and the reporting plugins :
Build plugins will be executed during the build and they should be configured in the <build/> element from the POM.
Reporting plugins will be executed during the site generation and they should be configured in the <reporting/> element from the POM.
According to the maven goal specified in the command line (for example mvn clean, mvn clean package or mvn site) , a specific lifecyle will be used and a specific set of plugins goals will be executed.
There are three built-in build lifecycles: default, clean and site. The default lifecycle handles your project deployment, the clean lifecycle handles project cleaning, while the site lifecycle handles the creation of your project's site documentation.
A plugin goal may be bound to a specific phase of a specific lifecyle.
For example the maven-compiler-plugin binds by default the compile goal to the lifecycle phase: compile.
Most of maven plugins (both core plugins and third party plugins) favor convention over configuration. So these generally bound a plugin goal to a specific phase to make their usage simpler.
That is neater and less error prone :
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
</plugin>
than :
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.7.0</version>
<executions>
<execution>
<phase>compile</phase>
<goals>
<goal>compile</goal>
</goals>
</execution>
</executions>
</plugin>
What dependencies are ?
Dependencies are Maven artifacts/components required for the project.
Concretely most of dependencies are jar (that is libraries) but these may also be other kinds of archives : war, ear, test-jar, ejb-client ... or still POM or BOM.
In a pom.xml, dependencies may be specified at multiple places : the <build><dependencies> part , the dependencies management part or still in a plugin declaration ! Indeed some plugins may need to have some dependencies in the classpath during their execution. That is not common but that may happen.
Here is an example from the documentation that shows that plugin and dependency may work together :
For instance, the Maven Antrun Plugin version 1.2 uses Ant version
1.6.5, if you want to use the latest Ant version when running this plugin, you need to add <dependencies> element like the following:
<project>
...
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-antrun-plugin</artifactId>
<version>1.2</version>
...
<dependencies>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant</artifactId>
<version>1.7.1</version>
</dependency>
<dependency>
<groupId>org.apache.ant</groupId>
<artifactId>ant-launcher</artifactId>
<version>1.7.1</version>
</dependency>
</dependencies>
</plugin>
</plugins>
</build>
...
</project>
In Maven, dependencies are referenced in a specific format :
groupId:artifactId:packaging:classifier:version.
The classifier (that is optional) and the packaging (JAR by default) are not commonly specified. So the common format in the dependency declaration is rather : groupId:artifactId:version.
Here is an example of dependency declared in the <build><dependencies> part :
<build>
<dependencies>
<dependency>
<groupId>org.hibernate</groupId>
<artifactId>hibernate-core</artifactId>
<version>5.2.14.Final</version>
</dependency>
<dependencies>
</build>
Dependency doesn't have a phase binding as plugins to address the "when" question.
But it has a counterpart : the scope.
Indeed declared dependencies are usable by the application at a specific time according to the scope we defined for these.
The scope is a central concept about how a dependency will be visible for the project.
The default scope is compile. That is the most commonly needed scope (convention over configuration again).
The compile scope means that the dependency is available in all classpaths of a project.
The scope defines in which classpaths the dependency should be added.
For example do we need it at compile and runtime, or only for tests compilation and execution ?
For example we previously defined Hibernate as a compile dependency as we need it everywhere : source compilation, test compilation, runtime and so for....
But we don't want that testing libraries may be packaged in the application or referenced in the source code. So we specify the test scope for them :
<build>
<dependencies>
<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.1.0</version>
<scope>test</scope>
</dependency>
<dependencies>
</build>
One line answer - basic understanding
Plugin is a tool you use at the execution of your maven build
Dependency means kind of any library which you will use in your code
If you're coming from a front-end background like me, and are familiar with Grunt and npm, think of it like this:
First you would run, say, npm install grunt-contrib-copy --save-dev. This is like maven's <dependency></dependency>. It downloads the files needed to execute a build task.
Then you would configure the task in Gruntfile.js
copy: {
main: {
src: 'src/*',
dest: 'dest/',
},
}
This is like maven's <plugin>/<plugin>. You are telling the build tool what to do with the code downloaded by npm/<dependency></dependency>.
Of course this is not an exact analogy, but close enough to help wrap your head around it.
Plug-ins are used for adding functionalities to Maven itself (like adding eclipse support or SpringBoot support to Maven etc.). Dependencies are needed by your source code to pass any Maven phase (compile or test for example). In case of JUnit since the test code is basically part of your code base and you call JUnit specific commands inside test suites and those commands are not provided by Java SDK therefore JUnit must be present at the time Maven is in the test phase and this is handled by mentioning JUnit as a dependency in your pom.xml file.
In simple words:
Plugins are used to add some additonal features to the software/tools(like Maven). Maven will use the added plugins at the time of building when we use the build command.
Dependecies are used to add some addtional code to your source code, so a dependency will make some extra code (like Classes in Java) in the form of library available for your source code.
Maven at its heart is a plugin execution framework -- as per formal and standard compact definition. To make it more clear, the commands you use like maven-install/clean/compile/build etc for creating/executing jars, which we sometimes manually run too. So, the things which you want to run (or configure or execute) you basically put them in dependency tag of mavens pom and the answer so as to who will run these dependencies (required for environment setup) be the plugins.
javac (compiler) dependency.java (dependency)
A plugin is an extension to Maven, something used to produce your artifact (maven-jar-plugin for an example, is used to, you guess it, make a jar out of your compiled classes and resources).
A dependency is a library that is needed by the application you are building, at compile and/or test and/or runtime time.

Resources