I have one job in my Spring Batch that doesn't use one of the data sources (Oracle Database). However, because I'm using autowire and autoconfigure in Spring Configuration files; that job also made a connection to Oracle when it starts.
I reduce the pool-size to 1 because the data source will not be used. But, I really need to remove that connection also, meaning that I don't want to even have one Inactive and not used connection.
Any idea of how to solve that in Spring Batch?
Related
I have a working Spring Boot application which embeds a Spring Batch Job. The job is not run on a schedule, instead we kick it with an endpoint. It is working as it should. The basics of the batch are
Kick the endpoint to start the job
Reader reads from input file
Processor reads from oracle database using jpa repository and simple spring datasource config
Writer writes to output file
However there are new requirements:
The schema of the repository database is from here on unknown on application startup. The tables are the same, it is just an unknown schema. This fact is out of our control and you might think it is stupid but there are reasons for it and this cant be changed. This means that with current functionality we need to reconfigure the datasource when we know the new schema name, and restart the application. This is a job that we will run for a number of times when migrating from one system to another, so it has a limited lifecycle and we just need a "quick fix" to be able to use it without rewriting the whole app. So what I would like to do is:
Send the schema name as a query param to the application, put it in job parameters and then - get a new datasource when the processor reads from the repository. Would this be doable at all using Spring Batch? Any help appreciated!
I have multiple jobs for my Spring Batch application, but only a single job uses some specific Spring Boot auto configuration features:
a job that uses spring-data-jpa auto configuration, to configure a database for business transactions (not for Spring Batch management)
a job that does not use the database at all
I have packaged both jobs in the same unit because it makes sense from business perspective. Both jobs will work together and the output of one job will be the input of the other job.
Is it possible to disable database specific auto configuration when I run the second job?
I just tried using profiles and I have disabled the autoconfiguration for a specific profile. I am pretty happy with this solution but I wonder if there are other solutions?
This is similar to trying to lazy load beans specific to a given job: How to apply something like #Lazy to Spring Batch?. While the Spring profiles feature may fix your issue, I believe it is working around the root issue which is packaging all jobs in a monolithic way.
I would package each job separately and this problem (and many others) disappears by design. There are several advantages to this approach:
Independent lifecyle management (bugs, features, etc)
Flexible deployment
Separate logs
Separate configurations (as in the current issue)
Easier/Better scalability
And all the good reasons to make one thing do one thing and do it well.
I have one #Scheduler job which runs on multiple servers in a clustered environment. However I want to restrict the job to run in only in one server and other servers should not run the same job once any other server has started it .
I have explored Spring Batch has lock mechanism using some Database table , but looking for any a solution only in spring task:scheduler.
I had the same problem and the solution what I implemented was a the Lock mechanism with Hazelcast and to made it easy to use I also added a proper annotation and a bit of spring AOP for that. So with this trick I was able to enforce a single schedule over the cluster done with a single annotation.
Spring Batch has this nice functionality that it would not run the job with same job arguments twice.
You can use this feature so that when a spring batch job kicks start in another server it does not run.
Usually people pass a timestamp as argument so it will by pass this logic, which you can change it.
I am writing a Spring Batch job and don't care about restarts and don't want the hassle of creating, securing and managing an Oracle schema for Spring Batch database objects. H2 on the file system is more than enough for us.
The issue is that I am writing an batch job that needs to connect to an Oracle database and failing miserably just trying to get 2 data sources, transaction managers and entity managers. Right now I am trying two H2 database and haven't even tried configuring Oracle database yet.
I have used two Oracle data sources in other Spring Boot applications successfully in the past but this is my first attempt with Spring Batch that has configuration code to create the data source.
I have tried creating two DataSource, EntityManagerFactoryBean and TransactionManager with one using default spring.datasources configuration, default bean names and #Primary.
I have tried creating just a second DataSource, EntityManagerFactoryBean and TransactionManager with different bean names. This seems to have an issue that the TransactionManager orEntityManager is already assigned to the thread.
I have tried creating a dataSource for batch but run into circular bean creation errors.
I have tried creating a BatchConfigurer and that runs into circular bean creation errors.
I have tried creating a JobRepositoryFactoryBean but a default one is still created.
I have tried using a #PersistenceContext on my #Entity classes.
It shouldn't be this hard so I must be missing something. Any help would be appreciated.
Thanks, Wes.
I think I have some success. I am able to eventually run the batch job using Spring Boot 2.2.0 and Spring Batch 4.2.0 but I have to wait for a 5 minute timeout while creating the Entity Manager for H2 for the Spring Batch repository. The Oracle Entity Manager is registered really quickly and before the H2 Entity Manager despite the H2 Entity Manager being #Primary.
I have a separate configuration class for each of my two DataSources. Each one is annotated with #Configuration, #EnableTransactionManagement and #EnableJPARepository.
The one for Spring Batch is using the standard bean names, dataSource, entityManagerFactory and transactionManager. Each #Bean is annotated with #Primary.
One setting that I needed was to add a .packages("org") to the entityManagerFactory bean. This would pick up all of org.springframework including Spring Batch.
The only other real change I have made from common implementations is to set the dialect in the JPA Properties of the Entity Manager.
I needed the spring.main.allow-bean-definition-overriding: true setting.
There may be more to my solution that I should share but I have been at this for a couple of days and have gone in circles. I even remember getting what appeared as a hung process time and and killed the job thinking it was hung. I may have had some "success" early on but just was too quick to kill the execution.
I would still like to know why it is taking so long to create the H2 Entity Manager.
Thanks, Wes.
I have a spring based web-application that needs to get data from ActiveMQ via a JMX connection.
I am using MBeanServerConnectionFactoryBean (in Spring) to get various MBean attributes from ActiveMQ.
I have only one MBeanServerConnectionFactoryBean as a member variable, which is used to get the data. If multiple requests/threads come concurrently will there be any issues? Will there be any race conditions?
Please suggest the best way to keep the code thread-safe.
Spring FactoryBean objects are not intended to be used directly from your code, they're supposed to be used in your Spring config. As such, they are designed to be executed once and once only.
If you want to use them, including MBeanServerConnectionFactoryBean, then you need to create them, configure them, use them and discard them each and every time you want to get the object they create. They are most definitely not thread-safe.
Better yet, do it as the design intended and use them in your Spring config.