Does flyway support sql migrations under test folder? - spring-boot

I am migrating a project from Spring Boot 1.5.21 to 2.2.5. I have followed all instructions in the guide. However, I have an issue with flyway (upgrading from 4.2.0 to 6).
Previously, when I had sql migration files under src/test/resources/db/migration, flyway would run them as part of the mvn clean install command. Now, for some reason, it stopped running these migrations (Just to clarify, I'm talking about maven build and not while running the app).
I'm using maven 3.6.3 and flyway-maven-plugin version 6.0.8 with the following configuration (some values are tokens which are irrelevant for this question):
<plugin>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-maven-plugin</artifactId>
<version>${flyway.version}</version>
<executions>
<execution>
<phase>generate-sources</phase>
<goals>
<goal>migrate</goal>
</goals>
</execution>
</executions>
<configuration>
<skip>${db.skip}</skip>
<url>${db.url}</url>
<user>${db.username}</user>
<password>${db.password}</password>
<locations>
<location>classpath:db/migration</location>
</locations>
<schemas>public,downstream</schemas>
<outOfOrder>true</outOfOrder>
<callbacks>
db.migration.callback.PopulateControlFieldsFlywayCallback,db.migration.callback.UpdateReplicaIdentityFlywayCallback,db.migration.callback.UpdateSchemaHistoryTableFlywayCallback
</callbacks>
</configuration>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<version>${postgresql.version}</version>
</dependency>
</dependencies>
</plugin>
Why did the behaviour changed? How to revert to old behaviour? Am I missing something?

I have dug in the flyway and flyway-maven-plugin sorces. I have found that the path from classpath location tag is searched by ClassRealm in class. And there is exist two classpathes: ClassRealm[plugin>org.flywaydb:flyway-maven-plugin:5.0.0, parent: jdk.internal.loader.ClassLoaders$AppClassLoader#1f89ab83]. ClassRealm is a classloader that is used in Maven.
I have executed classLoader.getResources('/db/migration')and found that it returns 2 paths (target/classes/db/migration and target/test-classes/db/migration) for mave-plugin version 4.2.0, but returns only 1 path (target/classes/db/migration) for version5.0.0`.
Also I have found that there was a commit in flyway-maven-plugin repository for using maven 3 instead of 2 between maven-plugin version 4.2.0 and 5.0.0.
I haven't found an exact reason, but, summarizing all, I guess that there were some changes in maven-plugin (I guess some maven core library version) as a result of which the finding path in classpath was changed.
I have tried several options for returning the old behavior, but all of these not so good. These are work:
Stay on 4.2.0 maven-plugin version (I have a pet project where
I specified flyway 5.0.0 in <dependencies> and use maven-plugin
4.2.0 and it works correctly for you);
Specify a path to test-migration via <location>filesystem:target/test-classes/db/migration</location> or <location>filesystem:src/test/resources/db/migration</location>
I hope it will help you.
P.S. By the way, why do you use migration under test folder for generating classes by jooq? It looks very strange to me. You generate classes from one DB model, but in PROD you will have another DB model (since migrations only under main are used). Maybe you should think about it and using only migration under main folder? If you have some exceptions while jooq generatinig, try to fix it without test migrations.

Related

liferay-maven-plugin + direct-deploy configuration for DXP

I'm currently on Liferay 6.2 (with Tomcat), and I can configure the liferay-maven-plugin in my Liferay hook project's maven pom.xml as below, in order to pre-process my hook WAR for hot deployment (using the direct-deploy goal of the plugin).
<plugin>
<groupId>com.liferay.maven.plugins</groupId>
<artifactId>liferay-maven-plugin</artifactId>
<version>6.2.1</version>
<executions>
<execution>
<id>pre-process-war</id>
<phase>package</phase>
<goals>
<goal>direct-deploy</goal>
</goals>
</execution>
</executions>
<configuration>
<appServerDeployDir>${project.basedir}/target/liferay-pre-process</appServerDeployDir>
<liferayVersion>6.2.1</liferayVersion>
<pluginType>hook</pluginType>
<unpackWar>false</unpackWar>
</configuration>
</plugin>
I'm now trying to migrate this hook to Liferay 7.2.1 CE. There is no plan to redesign this as an OSGI module at the moment i.e. it would continue to be packaged as a WAR. So how would my new direct-deploy configuration look like, with Liferay 7.2.1? Specifically:
a) Can I continue using the same Maven plugin? I see from this doc that the liferay-maven-plugin has been "removed", yet my understanding from other help pages is that this plugin should not be used only with the newer, OSGI-module style plugins (the latter makes more sense to me).
b) If I can continue using the same Maven plugin, which version of the plugin works with 7.2.1? Also, which liferayVersion value should I use here? I tried a bunch of combinations and none of them worked.
For instance, I first tried liferayVersion 7.2.1 but the build failed while running the direct-deploy goal, since Maven didn't find a com.liferay.portal:portal-web artifact with that version. So I tried the latest version of that artifact from Maven Central, which is 7.0.2, as my liferayVersion. But the build failed again, this time because it couldn't locate the corresponding version of com.liferay.portal:portal-service. There's no 7.x version of portal-service though, which makes sense because it's been replaced with the portal-kernel artifact. Not specifying liferayVersion doesn't work either.
I received a response in a Liferay forum which suggests that the liferay-maven-plugin has indeed been removed in Liferay 7.x, and so the only option left is to copy the WAR to Liferay's deploy folder and let it do the pre-processing at deploy time (as opposed to doing this at build time, with the Maven plugin). Given the lack of alternatives at this point, that seems to be the only way forward (and it works).

TypeNotPresentExceptionProxy error at integration-test with maven-failsafe-plugin spring-boot 1.4

I'm getting ArrayStoreException: TypeNotPresentExceptionProxy when running integration-test with maven-failsafe-plugin and spring-boot 1.4.
You can see this error if you run joinfaces-example with
mvn -Pattach-integration-test clean install
I realized that the error does not occur if I change spring-boot-maven-plugin to run at pre-integration-test phase instead of package one.
More, this error started when I upgraded spring boot to 1.4. No error occurs if I change jsf-spring-boot-parent version to 2.0.0 which uses spring boot 1.3 version.
I actually found the answer in Spring Boot 1.4 release notes, short answer is that maven-failsafe-plugin is not compatible with Spring Boot 1.4's new executable layout. Full explanation below :
As of Failsafe 2.19, target/classes is no longer on the classpath and
the project’s built jar is used instead. The plugin won’t be able to
find your classes due to the change in the executable jar layout.
There are two ways to work around this issue:
Downgrade to 2.18.1 so that you use target/classes instead
Configure the spring-boot-maven-plugin to use a classifier for the
repackage goal. That way, the original jar will be available and used
by the plugin. For example :
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<configuration>
<classifier>exec</classifier>
</configuration>
</plugin>
An alternative is documented here: https://github.com/spring-projects/spring-boot/issues/6254
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-failsafe-plugin</artifactId>
<!--
Make failsafe and spring-boot repackage play nice together,
see https://github.com/spring-projects/spring-boot/issues/6254
-->
<configuration>
<classesDirectory>${project.build.outputDirectory}</classesDirectory>
</configuration>
</plugin>
This worked better for me, because when I used the "exec" solution, Spring failed to find my configuration files when starting the container. Which could probably be fixed by adding some further configuration parameters, I suppose, but this solution works "out of the box" for me.

Maven + AspectJ/SpringAOP + Lombok + Surefire = test broken in a specific scenario

I have an interesting problem in a project where all of the technologies mentioned in the title are used. I've been able to track it down up to the diagnosis (the test classpath prepared by Surefire), but I don't understand whether it can be fixed and how. It's not a showstopper, indeed it's a minor issue for me, but I'd like to solve it anyway.
First a rough description.
The problem is related to executing tests in a specific module of the project, and only in a specific way.
Everything works (tests pass) when I run from the master pom level:
cd ${projHome}
mvn install
Everything works (tests pass) when I run:
cd ${projHome}/modules/CoreImplementation/
mvn test
That means that I can build and test with no problems, the same for my Jenkins, and NetBeans can run tests from the IDE when I need them.
But that module fails testing when I run from the master pom level:
cd ${projHome}
mvn test
with this error:
java.lang.NoSuchMethodError: it.tidalwave.northernwind.profiling.RequestProfilerAspect.aspectOf()Lit/tidalwave/northernwind/profiling/RequestProfilerAspect;
at it.tidalwave.northernwind.frontend.ui.spi.DefaultSiteViewController.processRequest(DefaultSiteViewController.java:82) ~[classes/:na]
at it.tidalwave.northernwind.frontend.ui.spi.DefaultSiteViewControllerTest.must_call_some_RequestProcessors_when_one_breaks(DefaultSiteViewControllerTest.java:161) ~[test-classes/:na]
Running mvn test as a second pass (after a mvn install -DskipTests) happens to be the way Drone.io and Travis do their job. While I could change their configuration, I'd like to stay with the standard configuration and fix the problem if possible.
The diagnosis in short and my question.
Now, the question in short (details are further below). I was able to track down the problem to different ways in which Surefire prepares the classpath to execute the tests.
When I run mvn install the classpath is:
${repo}/org/apache/maven/surefire/surefire-booter/2.16/surefire-booter-2.16.jar
${repo}/org/apache/maven/surefire/surefire-api/2.16/surefire-api-2.16.jar
${projHome}/modules/CoreImplementation/target/test-classes
${projHome}/modules/CoreImplementation/target/classes
${projHome}/modules/Core/target/it-tidalwave-northernwind-core-1.1-ALPHA-37-SNAPSHOT.952b0c8bdc77.jar
${repo}/it/tidalwave/thesefoolishthings/it-tidalwave-role/3.0-ALPHA-1/it-tidalwave-role-3.0-ALPHA-1.jar
${projHome}/modules/Profiling/target/it-tidalwave-northernwind-core-profiling-1.1-ALPHA-37-SNAPSHOT.952b0c8bdc77.jar
${repo}/org/apache/commons/commons-math3/3.0/commons-math3-3.0.jar
…
When I run mvn test (from the project home) the classpath is:
${repo}/org/apache/maven/surefire/surefire-booter/2.16/surefire-booter-2.16.jar
${repo}/org/apache/maven/surefire/surefire-api/2.16/surefire-api-2.16.jar
${projHome}/modules/CoreImplementation/target/test-classes
${projHome}/modules/CoreImplementation/target/classes
${projHome}/modules/Core/target/unwoven-classes
${repo}/it/tidalwave/thesefoolishthings/it-tidalwave-role/3.0-ALPHA-1/it-tidalwave-role-3.0-ALPHA-1.jar
${projHome}/modules/Profiling/target/unwoven-classes
${repo}/org/apache/commons/commons-math3/3.0/commons-math3-3.0.jar
…
The different portions are the indented ones. In the former case, SureFire uses the classes directory (forget for a moment that in my case they are named unwoven-classes) only for the module under test, and the installed jar files for every dependency. In the latter case, it seems to be using classes for all dependencies in the reactor.
The reason for which this difference in the classpaths gives me troubles is explained below in the "Gory details" section. In short, that unwoven means that they contain bytecode not augmented by AspectJ, hence the methods that can't be found at runtime.
I'm running with SureFire 2.16, but I've also tried the latest 2.19 with no changes. Being able to force SureFire to always use jar files for dependencies would fix my problems. If you have the answer, you can stop reading my post here.
Gory details (just for curiosity).
The faulty module artifactId is it-tidalwave-northernwind-core-default and it depends on aspects available in it-tidalwave-northernwind-core-profiling - that's where the offending RequestProfilerAspect is. The aspect library dependency is both in the regular dependencies of the faulty module and in the configuration of the aspectj plugin:
<dependency>
<groupId>it.tidalwave.northernwind</groupId>
<artifactId>it-tidalwave-northernwind-core-profiling</artifactId>
</dependency>
...
<build>
<plugins>
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>aspectj-maven-plugin</artifactId>
<configuration>
<aspectLibraries combine.children="append">
<dependency>
<groupId>it.tidalwave.northernwind</groupId>
<artifactId>it-tidalwave-northernwind-core-profiling</artifactId>
</dependency>
</aspectLibraries>
</configuration>
</plugin>
</plugins>
</build>
AspectJ integration is by means of the following profile in a Super POM, which is activated in the build, whose relevant part is:
<profile>
<id>it.tidalwave-aspectj-springaop-v1</id>
...
<build>
<plugins>
<plugin>
<artifactId>maven-compiler-plugin</artifactId>
<executions>
<execution>
<id>default-compile</id>
<phase>compile</phase>
<configuration>
<outputDirectory>target/unwoven-classes</outputDirectory>
</configuration>
</execution>
<execution>
<id>default-testCompile</id>
<phase>test-compile</phase>
<configuration>
<outputDirectory>target/unwoven-test-classes</outputDirectory>
</configuration>
</execution>
</executions>
</plugin>
...
The aspectj plugin is configured in the profile to statically weave binaries in the unwoven-test-classes directories. The reason for this approach is that it's the only feasible solution AFAIK to have both Lombok and AspectJ work together.
Now, back to the two classpaths described above: the fact that SureFire is using unwoven-classes means that it's pointing to bytecode that has not been augmented with AspectJ methods, hence the error.
References
The project is a FLOSS one and can be found at
https://bitbucket.org/tidalwave/northernwind-src
or
https://github.com/tidalwave-it/northernwind-src
A changeset where the problem can be reproduced is f98e9a89ac70138c1b6bd0d4570a22d59ed71be6. JDK 1.8.0 is required to build the project (even though it doesn't use Java 8 code yet).
The SuperPOM can be found here:
https://bitbucket.org/tidalwave/thesefoolishthings-superpom-src

Maven: Force Jersey to use specific artifact version

I have a Maven repository where I load Jena TDB 0.9.3 (which depends on Jena ARQ 2.9.3), Jersey 1.8 and RMOnto 1.0. The point is, as you expected, to do some analysis on semantic datasets.
It looks like RMOnto has ARQ 2.8.7 built in, as in "hardwired". There isn't any explicit dependency in its pom file, yet the jar file contains a ARQ.class. It's very tricky because you won't notice it with Maven Enforcer Plugin and the like.
It looks like this causes Jersey to use RMOnto's ARQ version instead of the one defined in pom.xml. Here is a minimal example. When you run the test (checks whether or not ARQ.VERSION equals 2.9.3), it succeeds. When you build the project and deploy it on a Tomcat 7, you should see 2.8.7 as output.
Is this behaviour expected and why?
How could one force Jersey to use ARQ 2.9.3?
In case it's not possible, could one isolate RMOnto to use 2.8.7 while the rest of the source uses 2.9.3?
Thanks in advance!
You should define the ARQ 2.9.3 first in the dependencies list. By doing that you force your build to use that specific version. The dependency order is relevant when choosing what artifact to use.
Update
OK, I understand what the problem is.
The RMOnto jar is obviously shaded according to the pom: http://semantic.cs.put.poznan.pl/maven/put/semantic/RMOnto/1.0/RMOnto-1.0.pom.
Tomcat 7 loads the jars in WEB-INF/lib in an undefined order. This means that even if you define ARQ 2.9.3 to be first in your dependencies it will not be the case when the application is run in Tomcat. http://tomcat.apache.org/tomcat-7.0-doc/class-loader-howto.html
Good thing is that Tomcat always look in WEB-INF/classes before WEB-INF/lib for dependencies.
So what you can do as a work around is to make sure that the ARQ 2.9.3 version is added to the WEB-INF/classes folder. This can be done using the maven-dependency-plugin:
<build>
<plugins>
<plugin>
<artifactId>maven-dependency-plugin</artifactId>
<executions>
<execution>
<phase>prepare-package</phase>
<goals>
<goal>unpack</goal>
</goals>
<configuration>
<artifactItems>
<artifactItem>
<groupId>org.apache.jena</groupId>
<artifactId>jena-arq</artifactId>
<version>2.9.3</version>
<outputDirectory>${project.build.directory}/classes</outputDirectory>
<excludes>**/META-INF/</excludes>
</artifactItem>
</artifactItems>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
Your war as well as your exploded war will now contain all the classes from ARQ 2.9.3 in the WEB-INF/classes folder. They will be loaded before any jar-file that is in WEB-INF/lib folder.
NB: I have not tested this on Tomcat but I cannot see that it would not work.
NB2: This is a hack. Best thing would be to remove the ARQ packages out of the RMOnto jar.
You should file a defect report against RMOnto. Hard-wiring library code into a jar, instead of including it as a dependency you can manage in the POM, is definitely a bad idea that the code maintainer should fix.
If the files have been copied directly to the RMOnto .jar, the behaviour is expected.
In that case, I'd say the best bet is to hardcode it away, aka remove the ARQ files directly from the package. Opening up the RMOnto-1.0.jar package one can see arq files in the arq folder. What you'd need to do is open up the jar file (it's just a .zip), remove the ARQ files from there, store the edited RMOnto package in your version control / repository and refer to the edited package from there. Also, you'd need to add excludes statement to your pom for the old version of ARC and keep the dependency to the new version.
If you feel like it, it would be also good practice to remove the other dependencies that haven't been mentioned in the RMOnto's pom file, then add them in the RMOnto pom file (and rebuild, if you have the source code). This way Maven mechanism would be aware of them. The file seems to contain a lot of dependencies like this, which will cause headaches in the future.

Maven Using JAX-WS 2.1 Instead of JAX-WS 2.2

I am using Netbeans 7 with Maven 2.2.1 and jaxws-maven-plugin 1.12. Code is deployed on Glassfish 3.1 - or will be when I get it to compile :)
When I build the project, the wsimport runs as expected and generates the source files from the WSDL provided. The problem is that the build fails during the compile phase with the following three exceptions. From researching this, I see that these constructors were added from JAX-WS 2.1 to JAX-WS 2.2. My belief is that the wsimport is using JAX-WS 2.1 and the compile is using JAX-WS 2.2.
Can someone confirm my suspicion? Or, if I'm wrong, might you have an idea what may be causing this?
Thank you.
UPDATED/CLARIFICATION OF PROBLEM
The Web service client extends javax.xml.ws.Service and the error is thrown when the client tries to call the super class constructor with three arguments. Since the super class doesn't have any constructor with three arguments, it fails.
javax.xml.ws.Service is found in JDK SE 1.6 and JAX-WS 2.1 as the wrong version.
javax.xml.ws.Service is found in JAX-WS 2.2 as the correct version.
The error occurs three times since it is in three overridden constructors but it's the same error so I've included it only once.
cannot find symbol
symbol : constructor Service(java.net.URL,javax.xml.namespace.QName,javax.xml.ws.WebServiceFeature[])
location: class javax.xml.ws.Service
<plugin>
<groupId>org.codehaus.mojo</groupId>
<artifactId>jaxws-maven-plugin</artifactId>
<version>1.12</version>
<executions>
<execution>
<goals>
<goal>wsimport</goal>
</goals>
<configuration>
<wsdlFiles>
<wsdlFile>*path to WSDL*</wsdlFile>
</wsdlFiles>
<wsdlLocation>*url to WSDL*</wsdlLocation>
<staleFile>${project.build.directory}/jaxws/stale/BudgetCheckingServiceService.stale</staleFile>
</configuration>
<id>wsimport-generate-BudgetCheckingServiceService</id>
<phase>generate-sources</phase>
</execution>
</executions>
<dependencies>
<dependency>
<groupId>com.sun.xml.ws</groupId>
<artifactId>jaxws-tools</artifactId>
<version>2.2.6-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>javax.xml</groupId>
<artifactId>webservices-api</artifactId>
<version>1.4</version>
</dependency>
</dependencies>
<configuration>
<sourceDestDir>${project.build.directory}/generated-sources/jaxws-wsimport</sourceDestDir>
<xnocompile>true</xnocompile>
<verbose>true</verbose>
<extension>true</extension>
<catalog>${basedir}/src/jax-ws-catalog.xml</catalog>
</configuration>
</plugin>
As you can see in the jaxws-maven-plugin-1.12's pom, it has the dependency jaxws-tools-2.1.7. Well, you have over-ridden it via the pom. But, this over-riding works as long as the over-rode version (2.2.6-SNAPSHOT) is api-compatible with the plugin's default version (2.1.7).
Clearly, based on your remarks, they are not api-compatible. So, as I see this won't work. Here's a reference to follow.
Run mvn install with -X flag to determine the exact version of the jaxws-tools is used by this plugin. Do a pastebin if you don't mind, then we can have a look too!
EDIT: One thing you can do is; upgrade the maven-jaxws-plugin's jaxws-tools dep to the needed version of yours. And, then fix the issues occur due to api-incompatibility (such as constructor problems.) Then send a patch to the upstream.
I had a similar problem and it was solved by putting the file webservices-api.jar in my %JDK_HOME%/jre/lib/endorsed folder.

Resources