I am writting Spring boot application for Spring batch, where ItemReader reads the data from Oracle Database and writes the data into postgres sql , but i am getting the below error
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is org.postgresql.util.PSQLException: ERROR: relation "batch_job_instance" does not exist
Position: 39
at org.springframework.jdbc.support.SQLErrorCodeSQLExceptionTranslator.doTranslate(SQLErrorCodeSQLExceptionTranslator.java:234) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:72) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
at org.springframework.jdbc.core.JdbcTemplate.translateException(JdbcTemplate.java:1402) ~[spring-jdbc-5.0.8.RELEASE.jar:5.0.8.RELEASE]
I dont want to create the spring batch metadata table , my application not need for monitoring the jobs, please suggest me on this. Thanks in advance!!
According to your exception, it looks like Spring Batch is configured to use the Postgres data source for its metadata and it does not find the BATCH_JOB_INSTANCE table.
If you don't want to use meta-data tables, you can:
use the MapJobRepositoryFactoryBean to create an in-memory JobRepository
use an embedded database (H2, HSQL, etc) with the ResourcelessTransactionManager. Spring Boot will automatically create tables in the in-memory datasource for you (more details here: https://docs.spring.io/spring-boot/docs/2.0.4.RELEASE/reference/htmlsingle/#howto-initialize-a-spring-batch-database).
Then you can configure an Oracle datasource for your reader and a Postgres datasource for your writer.
Nothing prevents from having multiple datasources in your Spring Batch app, you just need to configure which one to use for your business logic and which one to be used by Spring Batch for its internal mechanics.
There are similar questions that might help, I'm adding them here for reference:
Use of multiple DataSources in Spring Batch
How to java-configure separate datasources for spring batch data and business data? Should I even do it?
Use Below properties
spring.batch.initialize-schema=always or never
or else you can create the tables in ur DB.
You can also find the schema for respective dbs in below jar.
Spring-batch-core.jar in maven dependencies if you are using maven.
in classpath:/org/springframework/batch/core/
Related
I have created two spring boot microservices for experimenting purposes, both use hsqldb. Is there a way to see the hsqldb details like what is the name of the db that spring boot created, username, password, what tables are there in each one and to query those tables.
You can use the built-in SQL functions and INFORMATION_SCHEMA views in HSQLDB to check these properties. See the Guide http://hsqldb.org/doc/2.0/guide/
But the quick and easy way is to use a file: database connection and check the *.script file of the database, which contains SQL statements that set the properties and create the tables.
My project has this requirement where user uploads a CSV file which has to be pushed to sql server database.
I am following the below basic example to load CSV file into a sql sever database.
https://github.com/michaelcgood/Spring-Batch-CSV-Example
runninng this repo by making change of datasource, here we using sql server instead of in-memory db.
here is the addition to POM file:
<dependency>
<groupId>com.microsoft.sqlserver</groupId>
<<artifactId>sqljdbc4</artifactId>
<version>4.0</version>
<</dependency>
additions to applications.properties file
spring.datasource.url=jdbc:sqlserver://localhost;databaseName=springbootdb
spring.datasource.username=sa
spring.datasource.password=Projects#123
Below is the error generated while running the code base, we could not found exact root cause we tried the multiple approaches as mentioned in the google but no luck, while running this spring batch+spring boot 2.0.4 application we are facing the below errors:
spring.datasource.driverClassName=com.microsoft.sqlserver.jdbc.SQLServerDriver
o.s.boot.autoconfigure.jdbc.Datasource.initailizer disabled(not running DDL scripts)
at org.springframework.boot.devtools.restart.RestartLauncher.run(RestartLauncher.java:49)
Caused by: org.springframework.jdbc.BadSqlGrammarException: PreparedStatementCallback; bad SQL grammar [SELECT JOB_INSTANCE_ID, JOB_NAME from BATCH_JOB_INSTANCE
where JOB_NAME = ? order by JOB_INSTANCE_ID desc]; nested exception is java.sql.SQLServerException: Invalid object name 'BATCH_JOB_INSTANCE'.
We are assuming the root cause one of the below?
1.We are using the spring starter project for creating spring batch configuration so not aware how we could define the default table schema and all.
2.maybe we don't have access and/or you need to define the schema. http://forum.spring.io/forum/spring-projects/batch/63771-badsqlgrammarexception-running-commandlinejobrunner
3.Not sure on the why error is saying Invalid object name 'BATCH_JOB_INSTANCE' instead of object doesnot exist?
What are the list of sql queries to create metadate table for spring batch application, to create it manually before running application.
4.we are struggling from couple of days to find the root cause of issue, Please suggest approch by taking the above github code base sample.
any help here would be appreciated. Please let us know if we missed anything here. Thanks In advance
The project you are linking to uses the in-memory hsqldb instance so Spring Boot will automatically create Spring Batch meta-data tables for you.
If you want to adapt the sample to your project and use sql server (which is not embedded in your JVM), Spring Boot will not create Spring Batch tables (unless you instruct it to do so). In this case, you have two options:
Run the Spring Batch DDL script for sqlserver yourself against the db server you are using before running your app
Or instruct Spring Boot to do it for you by specifying the following properties in your application.properties file:
spring.batch.initialize-schema=always
spring.batch.schema=classpath:org/springframework/batch/core/schema-sqlserver.sql
Am using Mysql database and hibernate in JPA. I have two mysql databases in my project. I want to configure these two databases in configuration class of spring JPA. Already I have configured for one datasource using jpa transaction manager bean. I want to add one more datasource to configure. Please consider below scenario.
Mysql Database db1
-Table 1, Table2, Table3
Mysql Database db2
-Table1, Table2
I want to add jpa configuration for above two sql database. Different connections. Whenever repository interface calls for the particular entity, that database should be get called. Please help anyone on this. Thanks in advance.
I have a spring boot application using spring jdbc template for DAO layer connecting to Oracle DB. The DB username is different than the schema on which the queries will be run. Hence when the queries are run it needs to run using a different schema and I do not want to prefix the hardcoded value for the schema(For ex select * from user1.table.....)
I researched a bit and couldn't find a simple and straight way to do that.
For ex if I were using JPA I could have simply configured the property spring.jpa.properties.hibernate.default_schema=<schema name> but couldn't find an equivalent way of configuring the same when using spring jdbc
I ran into a similar problem and didn't find an ideal way to do it. I ended up setting the schema in SQL when the app loaded. At least I was able to reuse spring.jpa.properties.hibernate.default_schema, which I already had set for the JPA default schema.
final String schemaName = jpaProperties.getProperties().get("hibernate.default_schema");
jdbcTemplate.execute("SET SCHEMA '" + schemaName + "'");
This clearly isn't ideal, but it is better than defining your schema in multiple places.
(Note: I autowired both the JpaProperties and JdbcTemplate.)
You need to use the Oracle JDBC Driver.
A good example can be found in this mykong article:
# Oracle settings
spring.datasource.url=jdbc:oracle:thin:#localhost:1521:xe
spring.datasource.username=system
spring.datasource.password=password
spring.datasource.driver-class-oracle.jdbc.driver.OracleDriver
I am aware of spring batch metadata tables are written to Relational database tables say MySQL,H2 etc. But my question is whether spring batch metadata tables can be written to elasticsearch . If so how to proceed further?
have you checked the Spring Batch Extensions module? This module provides an ItemReader and an ItemWriter for interacting with Elasticsearch. Hope it helps.