Failed to execute CommandLineRunner - Spring Batch - spring

Hi I am very new to Spring batch and I am getting the following exception which I am not able to resolve:
java.lang.IllegalStateException: Failed to execute CommandLineRunner
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:800) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:781) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:335) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1255) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1243) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at com.demo.BatchDemo.main(KnpBatchApplication.java:16) [classes/:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_72]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_72]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_72]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_72]
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49) [spring-boot-devtools-2.0.1.RELEASE.jar:2.0.1.RELEASE]
Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
at java.util.ArrayList.rangeCheck(ArrayList.java:653) ~[na:1.8.0_72]
at java.util.ArrayList.get(ArrayList.java:429) ~[na:1.8.0_72]
at org.springframework.batch.core.JobParametersBuilder.getNextJobParameters(JobParametersBuilder.java:265) ~[spring-batch-core-4.0.1.RELEASE.jar:4.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.execute(JobLauncherCommandLineRunner.java:162) ~[spring-boot-autoconfigure-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.executeLocalJobs(JobLauncherCommandLineRunner.java:179) ~[spring-boot-autoconfigure-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.launchJobFromProperties(JobLauncherCommandLineRunner.java:134) ~[spring-boot-autoconfigure-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner.run(JobLauncherCommandLineRunner.java:128) ~[spring-boot-autoconfigure-2.0.1.RELEASE.jar:2.0.1.RELEASE]
at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:797) [spring-boot-2.0.1.RELEASE.jar:2.0.1.RELEASE]
... 10 common frames omitted
My Code here:
#SpringBootApplication
public class BatchDemo {
public static void main(String args[])
{
SpringApplication.run(BatchDemo.class);
}
}
I was able to resolve it by adding exclude = BatchAutoConfiguration.class. But I would like to find the actual cause and fix it.
I am not passing any commandline arguments and this exception doesn't occur all the time.
pom.xml
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>demo</groupId>
<artifactId>demo_batch</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>demo_batch</name>
<url>http://maven.apache.org</url>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.1.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-core</artifactId>
</dependency>
<dependency>
<groupId>org.springframework</groupId>
<artifactId>spring-context</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-batch</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.springframework.batch</groupId>
<artifactId>spring-batch-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>

When you use #SpringBootApplication, Spring's autoConfiguration is enabled by default. Since you have spring batch dependencies loaded in your classpath, Spring Batch AutoConfiguration is enabled as well.
During application startup, Spring Batch's autoconfiguration (BatchAutoConfiguration) creates a Runner and it runs all the jobs defined in your BatchConfig.
You can disable this behavior by either setting spring.batch.job.enabled property to false in your application properties or by simply excluding the Autoconfiguration for Batch just like as you did.
Please refer here and here for more info.

For anyone who wants their batch jobs to actually execute, and is encountering this error, try checking if there is any inconsistency in the records in your Spring batch metadata tables.
To get the next JobParameters, Spring Batch searches for the latest BATCH_JOB_INSTANCE with the same JOB_NAME as your job, then looks for the matching BATCH_JOB_EXECUTION record in the database (linked by JOB_INSTANCE_ID).
We had an in-house task running which deleted records from BATCH_JOB_EXECUTION but, due to an error, did not delete the matching record from BATCH_JOB_INSTANCE.
In one environment, it had been more than 3 months since we had executed any jobs, so we had no BATCH_JOB_EXECUTION records anymore, but plenty of BATCH_JOB_INSTANCE records. We repeatedly threw the stack trace listed in this question, across multiple different Spring Batch applications, and could not find a solution anywhere online.
We resolved the issue by
Running SQL to delete from BATCH_JOB_INSTANCE where there is no matching record in BATCH_JOB_EXECUTION - this unblocked the job in the short term; and,
Replacing our purge task with a job using spring-batch-toolkit; it provides a RemoveSpringBatchHistoryTasklet that deletes the records correctly.

I got this error and it was because I was reading in files from a folder that followed a general naming convention and type. I by mistake left an image file in the same directory and my code could not process it and this error was thrown.

If you write any syntax errors, or spelling mistakes it also causes the above error. For example, I wrote it like this but there are some syntax errors in my SQL query:
String sql = "insert into Student values(1,'abhilash',india");
Correct one:
String sql = "insert into Student values(1,'abhilash','india')";
Now, the code ran fine. Just check your queries once.

Related

Spring boot not not working after adding dependency of actuator

I am trying to use Actuator. The problem is that when i use version 2 or above, server starts to fail. If i use version 1 less than 2 server starts properly but, i do not get end points like /actuator etc. Please help. Really stuck in it.
Pom file looks like this
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.5.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
<version>1.5.13.RELEASE</version>
<!-- <version>2.0.0.RELEASE</version>-->
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-core</artifactId>
<version>1.0.6</version>
</dependency>
<!-- Micrometer Prometheus registry -->
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<version>1.0.6</version>
</dependency>
Application properties
management.endpoint.metrics.enabled=true
management.endpoints.web.exposure.include=*
management.endpoint.prometheus.enabled=true
management.metrics.export.prometheus.enabled=true
Exception when application start fails
Error starting ApplicationContext. To display the auto-configuration report re-run your application with 'debug' enabled.
10-06-2020 17:59:44.274 [main] ERROR org.springframework.boot.SpringApplication.reportFailure - Application startup failed
org.springframework.beans.factory.BeanDefinitionStoreException: Failed to process import candidates for configuration class [com.business.api.throttler.ThrottlerApplication]; nested exception is java.lang.IllegalStateException: Could not evaluate condition on org.springframework.boot.actuate.autoconfigure.web.server.ManagementContextAutoConfiguration$DifferentManagementContextConfiguration due to org/springframework/boot/web/reactive/context/ConfigurableReactiveWebApplicationContext not found. Make sure your own configuration does not rely on that class. This can also happen if you are #ComponentScanning a springframework package (e.g. if you put a #ComponentScan in the default package by mistake)
at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:646)
at org.springframework.context.annotation.ConfigurationClassParser.lambda$processDeferredImportSelectors$2(ConfigurationClassParser.java:566)
at java.util.ArrayList.forEach(ArrayList.java:1257)
at org.springframework.context.annotation.ConfigurationClassParser.processDeferredImportSelectors(ConfigurationClassParser.java:563)
at org.springframework.context.annotation.ConfigurationClassParser.parse(ConfigurationClassParser.java:188)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:316)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:233)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:271)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:91)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:694)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:532)
at org.springframework.boot.context.embedded.EmbeddedWebApplicationContext.refresh(EmbeddedWebApplicationContext.java:122)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:693)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:360)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:303)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1118)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1107)
at com.bus.catalog.CatalogApplication.main(CatalogApplication.java:21)
Caused by: java.lang.IllegalStateException: Could not evaluate condition on org.springframework.boot.actuate.autoconfigure.web.server.ManagementContextAutoConfiguration$DifferentManagementContextConfiguration due to org/springframework/boot/web/reactive/context/ConfigurableReactiveWebApplicationContext not found. Make sure your own configuration does not rely on that class. This can also happen if you are #ComponentScanning a springframework package (e.g. if you put a #ComponentScan in the default package by mistake)
at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:55)
at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108)
at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:221)
at org.springframework.context.annotation.ConfigurationClassParser.processMemberClasses(ConfigurationClassParser.java:362)
at org.springframework.context.annotation.ConfigurationClassParser.doProcessConfigurationClass(ConfigurationClassParser.java:265)
at org.springframework.context.annotation.ConfigurationClassParser.processConfigurationClass(ConfigurationClassParser.java:245)
at org.springframework.context.annotation.ConfigurationClassParser.processImports(ConfigurationClassParser.java:636)
... 17 common frames omitted
Caused by: java.lang.NoClassDefFoundError: org/springframework/boot/web/reactive/context/ConfigurableReactiveWebApplicationContext
at org.springframework.boot.actuate.autoconfigure.web.server.OnManagementPortCondition.isWebApplicationContext(OnManagementPortCondition.java:64)
at org.springframework.boot.actuate.autoconfigure.web.server.OnManagementPortCondition.getMatchOutcome(OnManagementPortCondition.java:46)
at org.springframework.boot.autoconfigure.condition.SpringBootCondition.matches(SpringBootCondition.java:47)
... 23 common frames omitted
Caused by: java.lang.ClassNotFoundException: org.springframework.boot.web.reactive.context.ConfigurableReactiveWebApplicationContext
at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 26 common frames omitted
Disconnected from the target VM, address: '127.0.0.1:53116', transport: 'socket'
so your question was a bit messy until I haven't seen your reply on one of the comments under your question.
"yeah, but if i not mention the version application does not run. It's due to adding dependency of Actuator. If i mention version 1.x.x with actuator, application runs but don't get actuator end points."
So, the thing is that your pom.xml should look like this one
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 https://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.0.5.RELEASE</version>
<relativePath/> <!-- lookup parent from repository -->
</parent>
<groupId>com.example</groupId>
<artifactId>stackoverflow-actuator</artifactId>
<version>0.0.1-SNAPSHOT</version>
<name>demo</name>
<description>Demo project for Spring Boot</description>
<properties>
<java.version>11</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-webflux</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-devtools</artifactId>
<scope>runtime</scope>
<optional>true</optional>
</dependency>
<dependency>
<groupId>io.micrometer</groupId>
<artifactId>micrometer-registry-prometheus</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
<optional>true</optional>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>io.projectreactor</groupId>
<artifactId>reactor-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
Notice that I haven't defined the versions "manually" because I let the parent starter pom do it correctly instead of me. :)
Another problem that I surmise that you have based on your reply ("not mention the version application does not run")
Sounds like you weren't able to get the dependency. I mean you haven't put the error message here for that case, but I assume that you should run
mvn -U clean package to re-download the dependencies and get the right ones.
If you use IDEA you should run a "reimport/refresh" on pom.xml. :)
I hope it helps!
update:
I noticed also that you include both micrometer-core and micrometer-registry.
Completely unnecessary!!
Please always look at the included dependencies graph.
You can easily do that with for example "Dependency Analyzer" plugin in IDEA.
It will help you to see the duplicated inclusions.
For example on this picture, you should see that micrometer-core dependency comes with the actuator and the micrometer-registry thus you shouldn't include it "manually". :)

Failed to load class "org.slf4j.impl.StaticLoggerBinder", Spring Boot

I'm using Spring Boot 1.5.8. SLF4J doesn't seem to be working for me. Here is what I get in the console at the beginning of a project run. The below error shows appears :
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
START : EDIT 1 :
My console doesn't log the below:
LOGGER.error("ERROR");
LOGGER.error("WARN");
LOGGER.error("INFO");
LOGGER.error("DEBUG");
LOGGER.error("TRACE");
All I get is those three lines of error(warning) as the first lines in my console.
END : EDIT 1 :
I looked up the internet and found multiple issues titled as this,tried the suggestions but that didn't resolve my issue. I came across this stackoverflow question where Konstantinos hints at the possibility of presence of multiple slf4j dependencies. Other solutions on the internet also suggested that; even the official site confirmed that to be the reason for such error.
Also checked my effective pom just to eliminate this possibility and found that my effective pom does contain all the 4 dependencies that he mentioned. The general suggestion is that the pom should contain only one of these four dependencies. But , since this was all handled by Spring boot, I'm a bit confused as to how do I go about resolving this, because I never exclusively included any slf4j dependency in my pom. So I'm unable to follow the suggestion.
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.codingethics.flightreservation</groupId>
<artifactId>flightreservation</artifactId>
<version>0.0.1-SNAPSHOT</version>
<packaging>jar</packaging>
<name>flightreservation</name>
<description>Flight Reservation Application</description>
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>1.5.8.RELEASE</version>
<relativePath /> <!-- lookup parent from repository -->
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.tomcat.embed/tomcat-embed-jasper -->
<dependency>
<groupId>org.apache.tomcat.embed</groupId>
<artifactId>tomcat-embed-jasper</artifactId>
<scope>provided</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/javax.servlet/jstl -->
<dependency>
<groupId>javax.servlet</groupId>
<artifactId>jstl</artifactId>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<scope>runtime</scope>
</dependency>
<!-- https://mvnrepository.com/artifact/com.itextpdf/itextpdf -->
<dependency>
<groupId>com.itextpdf</groupId>
<artifactId>itextpdf</artifactId>
<version>5.5.13</version>
</dependency>
<!-- https://mvnrepository.com/artifact/org.springframework.boot/spring-boot-starter-mail -->
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-mail</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
</plugin>
</plugins>
</build>
</project>
application.properties :
spring.datasource.url=jdbc:mysql://localhost:3306/reservation
spring.datasource.username=root
spring.datasource.password=password
spring.jpa.show-sql=true
spring.mvc.view.prefix=/WEB-INF/jsps/
spring.mvc.view.suffix=.jsp
server.context-path=/flightreservation
spring.mail.host=smtp.gmail.com
spring.mail.port=587
spring.mail.username=<myusername>
spring.mail.password=<mypwd>
spring.mail.properties.mail.smtp.auth=true
spring.mail.properties.mail.smtp.starttls.enable=true
spring.mail.properties.mail.smtp.starttls.required=true
Why does this dependency-conflict(if at all that's the case here) happen in a Spring Boot project. Isn't Spring Boot meant to be looking after these beforehand.
Any sort of help/ guidance is highly appreciated.
START : EDIT for #Antot
I get NoClassDefFoundError.(followed steps as advised by you) :
Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/logging/LogFactory
at org.springframework.boot.SpringApplication.<clinit>(SpringApplication.java:179)
at com.codingethics.flightreservation.FlightreservationApplication.main(FlightreservationApplication.java:10)
Caused by: java.lang.ClassNotFoundException: org.apache.commons.logging.LogFactory
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 2 more
END : EDIT for #Antot
You might first want to check if something that isn't managed by Spring Boot is pulling in additional dependencies. You can do this with mvn dependency:tree to see what is pulling in what.
If that doesn't give a hint in the right direction it might be that one of your downloaded dependencies got borked. You might want to purge the local repository. You can do this with mvn dependency:purge-local-repository to remove all your dependencies from your local maven repository.
Then when doing mvn package (or another task that will compile and build the jar) will (re)download the jars and hopefully in a correct way without corruption.

Custom processor + DBCPConnectionPool for SQL Server : driver jar not loaded

I have created a controller service to connect to a test db.
I have a custom processor that reads data from SQL Server, the mock tests, the build and the deployment to NiFi succeed. The processor runs into error, maybe the nar dependency scope is at fault or ... ?
I am unsure The pom for processor and the nar projects are as follows :
processor pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.datalake</groupId>
<artifactId>CDCNiFi</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-NiFiCDCPoC-processors</artifactId>
<packaging>jar</packaging>
<dependencies>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-utils</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-dbcp-service-api</artifactId>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-processor-utils</artifactId>
</dependency>
<!-- Third-party -->
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.1.0.jre8</version>
</dependency>
<dependency>
<groupId>org.codehaus.jackson</groupId>
<artifactId>jackson-mapper-asl</artifactId>
</dependency>
<dependency>
<groupId>com.fasterxml.jackson.core</groupId>
<artifactId>jackson-databind</artifactId>
<version>2.8.7</version>
</dependency>
<!-- Testing & Cross-cutting concerns -->
<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-mock</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-simple</artifactId>
<scope>test</scope>
</dependency>
</dependencies>
</project>
nar pom.xml
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<parent>
<groupId>com.datalake</groupId>
<artifactId>CDCNiFi</artifactId>
<version>1.0-SNAPSHOT</version>
</parent>
<artifactId>nifi-NiFiCDCPoC-nar</artifactId>
<version>1.0-SNAPSHOT</version>
<packaging>nar</packaging>
<properties>
<maven.javadoc.skip>true</maven.javadoc.skip>
<source.skip>true</source.skip>
</properties>
<dependencies>
<dependency>
<groupId>com.datalake</groupId>
<artifactId>nifi-NiFiCDCPoC-processors</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>org.apache.nifi</groupId>
<artifactId>nifi-standard-services-api-nar</artifactId>
<type>nar</type>
</dependency>
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>mssql-jdbc</artifactId>
<version>6.1.0.jre8</version>
<scope>runtime</scope>
</dependency>
</dependencies>
</project>
The issues :
The SQL Server JDBC jar is , probably, not loaded at runtime, the app.log has the following error :
2017-03-09 15:04:06,074 ERROR [Timer-Driven Process Thread-1] c.s.d.processors.SQLServerCDCProcessor SQLServerCDCProcessor[id=ad9de403-015a-1000-2b40-7efbfdb049b1] Process or SQL exception in <configure logger template to pick the code location>
2017-03-09 15:04:06,080 ERROR [Timer-Driven Process Thread-1] c.s.d.processors.SQLServerCDCProcessor
org.apache.nifi.processor.exception.ProcessException: org.apache.commons.dbcp.SQLNestedException: Cannot load JDBC driver class 'com.microsoft.sqlserver.jdbc.SQLServerDriver'
at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:252) ~[na:na]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_71]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_71]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_71]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_71]
at org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:177) ~[na:na]
at com.sun.proxy.$Proxy89.getConnection(Unknown Source) ~[na:na]
at com.datalake.processors.SQLServerCDCProcessor.onTrigger(SQLServerCDCProcessor.java:244) ~[nifi-NiFiCDCPoC-processors-1.0-SNAPSHOT.jar:1.0-SNAPSHOT]
at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) [nifi-api-1.1.1.jar:1.1.1]
at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099) [nifi-framework-core-1.1.1.jar:1.1.1]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136) [nifi-framework-core-1.1.1.jar:1.1.1]
at org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47) [nifi-framework-core-1.1.1.jar:1.1.1]
at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:132) [nifi-framework-core-1.1.1.jar:1.1.1]
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_71]
at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [na:1.8.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) [na:1.8.0_71]
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) [na:1.8.0_71]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_71]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.8.0_71]
Caused by: org.apache.commons.dbcp.SQLNestedException: Cannot load JDBC driver class 'com.microsoft.sqlserver.jdbc.SQLServerDriver'
at org.apache.commons.dbcp.BasicDataSource.createConnectionFactory(BasicDataSource.java:1429) ~[na:na]
at org.apache.commons.dbcp.BasicDataSource.createDataSource(BasicDataSource.java:1371) ~[na:na]
at org.apache.commons.dbcp.BasicDataSource.getConnection(BasicDataSource.java:1044) ~[na:na]
at org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:249) ~[na:na]
... 19 common frames omitted
Caused by: java.lang.ClassNotFoundException: com.microsoft.sqlserver.jdbc.SQLServerDriver
at java.net.URLClassLoader.findClass(URLClassLoader.java:381) ~[na:1.8.0_71]
at org.apache.nifi.nar.InstanceClassLoader.findClass(InstanceClassLoader.java:117) ~[nifi-nar-utils-1.1.1.jar:1.1.1]
at java.lang.ClassLoader.loadClass(ClassLoader.java:424) ~[na:1.8.0_71]
at org.apache.nifi.nar.InstanceClassLoader.loadClass(InstanceClassLoader.java:98) ~[nifi-nar-utils-1.1.1.jar:1.1.1]
at org.apache.nifi.nar.InstanceClassLoader.loadClass(InstanceClassLoader.java:82) ~[nifi-nar-utils-1.1.1.jar:1.1.1]
at org.apache.commons.dbcp.BasicDataSource.createConnectionFactory(BasicDataSource.java:1420) ~[na:na]
... 22 common frames omitted
In the controller service, I do not see my custom processor in
'Referencing Components'(it says 'no referencing components')
Right now, I have hard-coded the 'Database Driver Location(s)' but in
production, I would like the controller to pick up that from the
classpath(somehow, is it possible?)
*****Edit-1*****
As suggested by Mahendra, I did the following :
Without any changes to the existing pom.xml files, added the dependency (sqljdbc4, 4.0) to both, the build and deployment succeeded but the jdbc driver error persisted
The 'Database Driver Location(s)' kept as 'file:///'. Same error
Added the sqljdbc4.jar in NiFi/lib and the driver error vanished(getting other errors but those will be taken in a different thread). Note that the controller service still doesn't show the processor as 'referenced component'.
The java version
java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
Isn't there a way to avoid adding the sqljdbc4.jar in lib, I mean third-party dependencies like fasterxml etc. are behaving properly, is this an issue with my configuration or some problem with the sql server maven repos. ?
#kaliyug,
EDIT 1:
1.if you declared DB class name is "com.microsoft.sqlserver.jdbc.SQLServerDriver".You should add below maven dependency in pom like below.,
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<artifactId>sqljdbc4</artifactId>
<version>4.2</version>
</dependency>
Or If you want's to use in production environment you can specify driver database location in controller service like below.
file:///C:\DBLocation\sqljdbc42.jar
3.Or you can just copy the "sqljdbc42.jar" directly into NiFi\Lib directory.After copied the jar,you should restart NiFi Services.At runtime ,SQL jar automatically Loaded when it search for DB ClassName in Nifi\Lib directory.
Please let me know if you have face any issues.

[EXCEPTION]:Task execution exception, Cause : java.lang.NoClassDefFoundError: org/apache/openjpa/conf/OpenJPAConfiguration

Ihave started receiving this exception after including new dependencies in my pom.xml which are related to Cassandra. I'm using it for logging purpose in my application.
[EXCEPTION]:Task execution exception, Cause : java.lang.NoClassDefFoundError: org/apache/openjpa/conf/OpenJPAConfiguration
java.lang.NoClassDefFoundError: org/apache/openjpa/conf/OpenJPAConfiguration
at java.lang.Class.getDeclaredConstructors0(Native Method) ~[na:1.7.0_79]
at java.lang.Class.privateGetDeclaredConstructors(Class.java:2585) ~[na:1.7.0_79]
at java.lang.Class.getConstructor0(Class.java:2885) ~[na:1.7.0_79]
at java.lang.Class.newInstance(Class.java:350) ~[na:1.7.0_79]
at org.apache.geronimo.osgi.locator.ProviderLocator.getServices(ProviderLocator.java:319) ~[geronimo-jpa_2.0_spec-1.1.jar:1.1]
at javax.persistence.spi.PersistenceProviderResolverHolder$DefaultPersistenceProviderResolver.getPersistenceProviders(PersistenceProviderResolverHolder.java:108) ~[geronimo-jpa_2.0_spec-1.1.jar:1.1]
at javax.persistence.Persistence$PersistenceUtilImpl.isLoaded(Persistence.java:278) ~[geronimo-jpa_2.0_spec-1.1.jar:1.1]
at org.hibernate.validator.engine.resolver.JPATraversableResolver.isReachable(JPATraversableResolver.java:62) ~[hibernate-validator-4.1.0.Final.jar:4.1.0.Final]
at org.hibernate.validator.engine.resolver.DefaultTraversableResolver.isReachable(DefaultTraversableResolver.java:94) ~[hibernate-validator-4.1.0.Final.jar:4.1.0.Final]
at org.hibernate.validator.engine.resolver.SingleThreadCachedTraversableResolver.isReachable(SingleThreadCachedTraversableResolver.java:47) ~[hibernate-validator-4.1.0.Final.jar:4.1.0.Final]org.hibernate.validator.engine.ValidatorImpl.validateConstraint(ValidatorImpl.java:324) ~[hibernate-validator-4.1.0.Final.jar:4.1.0.Final]
at org.hibernate.validator.engine.ValidatorImpl.isValidationRequired(ValidatorImpl.java:757) ~[hibernate-validator-4.1.0.Final.jar:4.1.0.Final]
at
Here's my pom.xml :
<dependencies>
<dependency>
<groupId>com.app.cops</groupId>
<artifactId>logging</artifactId>
<version>0.0.5-SNAPSHOT</version>
</dependency>
</dependencies>
Here's the logging project's pom.xml :
<dependencies>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-core</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>com.datastax.cassandra</groupId>
<artifactId>cassandra-driver-mapping</artifactId>
<version>3.0.0</version>
</dependency>
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>19.0</version>
</dependency>
<dependency>
<artifactId>hector-object-mapper</artifactId>
<groupId>org.hectorclient</groupId>
<version>3.1-10</version>
</dependency>
<dependency>
<groupId>org.apache.openjpa</groupId>
<artifactId>openjpa-all</artifactId>
<version>2.0.0</version>
</dependency>
</dependencies>
Even after including the openjpa-all dependency in my pom.xml which has the OpenJPAConfiguration I still see this issue. Any help on this would be appreciated.
What's is OpenJPA doing with Cassandra ?
In your maven dependency, I can see:
cassandra-driver-core
cassandra-driver-mapping
hector-object-mapper
openjpa-all
If you're using Cassandra, the first 2 dependencies are enough. hector-object-mapper is useless because you have already cassandra-driver-mapping and openjpa-all is useless because it will never work for Cassandra.
Remember, Cassandra != relational database

Adding spring-boot-starter-security to Spring Boot application causes error 'entityManagerFactory' or 'persistenceUnitName' is required

I created a Spring Boot (0.5.0.BUILD-SNAPSHOT) application using Spring Initializr (http://start.spring.io/), and I added a single #RestController, a single CrudRepository interface, and a single #Entity class - nothing complicated. My Maven POM contains the following dependencies:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-actuator</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-jdbc</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-test</artifactId>
<scope>test</scope>
</dependency>
<dependency>
<groupId>com.h2database</groupId>
<artifactId>h2</artifactId>
</dependency>
The Application class contains the default:
#ComponentScan
#EnableAutoConfiguration
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
The simple application runs without errors, but I decided to add Spring Security to the POM to secure the management endpoints:
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-security</artifactId>
</dependency>
Now the application won't start, and I get the following:
java.lang.IllegalArgumentException: 'entityManagerFactory' or 'persistenceUnitName' is required
Caused by: java.lang.IllegalArgumentException: 'entityManagerFactory' or 'persistenceUnitName' is required
at org.springframework.orm.jpa.JpaTransactionManager.afterPropertiesSet(JpaTransactionManager.java:304)
at org.springframework.orm.jpa.JpaTransactionManager.<init>(JpaTransactionManager.java:141)
at org.springframework.boot.autoconfigure.orm.jpa.JpaBaseConfiguration.transactionManager(JpaBaseConfiguration.java:68)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:597)
at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:166)
... 18 more
When I remove the spring-boot-starter-security dependency, the application runs fine but without security enabled. What does the error mean? The application already uses JPA and Hibernate without Spring Security enabled.
There is a bug there. The cause is really deeply technical and related to the internals in a Spring BeanFactory. Look at the Github issue if you want to get some more understanding, but probably you should be able to just refresh your snapshot dependencies and get the fix.

Resources