im developing a Project using Spring Batch and JavaConfig (no XML).
I am creating Jobs using an Autowired jobBuilderFactory.
is it somehow possible to set the Incrementer for the Factory globally ?
return jobBuilderFactory.get("jobName").incrementer(new RunIdIncrementer()).start(stepOne()).next(lastStep()).build();
sorry if this is a dump question but i am new to Spring Batch and did not find a working solution.
With XML config you would use bean definition inheritance, but you said you don't use XML.
Since there is no equivalent of XML bean definition inheritance with Java config (See details here: https://stackoverflow.com/a/23266686/5019386), you can create the RunIdIncrementer globally in your config and use it in job definitions:
public JobParametersIncrementer jobParametersIncrementer() {
return new RunIdIncrementer();
}
public JobBuilder getJobBuilder(String jobName) {
return jobBuilderFactory.get(jobName)
.incrementer(jobParametersIncrementer());
}
#Bean
public Job job1() {
return getJobBuilder("job1")
.start(step())
.build();
}
#Bean
public Job job2() {
return getJobBuilder("job2")
.start(step())
.build();
}
But again and as said in comments, you will end up having run.id values that are not consecutive for each job.
Related
I am developing Spring batch application on Spring Boot. Batch process is based on fire and forget, which means migration process is triggered by Http GET method and doesn't wait for result. There are 6 (six) csv files which containes migration data and these data must be processed and transferred to DB. Source files path is provided by application.yml file. Batch config (template) looks like as follows:
#Configuration
public class BeginCureBtachConfig {
#Value("$batch-files.begincure-source")
private String dataSourceFilePath;
#Autowired
IteamReader<BeginCure> itemReader;
#Autowired
ItemProcessor<BeginCure, BeginCure> itemProcessor;
#Bean Job beginCureJob(JobBuilderFactory jbFactory, StepBuilderDactory stp, ItemWriter<BeginCure> begnCure {
//// code goes here
}
#Bean
public FlatFileItemReader<BeginCure> beginCureItemReader() {
FlatFileItemReader<BeginCure> ffReader = new FlatFileItemReader<>();
ffReader.setResource(new FileSystemResource(dataSourceFilePath));
ffReader.setLineMapper(lineMapper());
return ffReader;
}
#Bean
public LineMapper<BeginCure> lineMapper () {
//..
}
}
The problem is when project is deployed on openshift, batch looks for files even migratin is not triggered.
In the code, I think this part of code giving some error ffReader.setResource(new FileSystemResource(dataSourceFilePath));
Is there any workaround or solution? Did anybody have this kind of problem?
Thank you in advance^^
So I want this to work
#Bean
#ConfigurationProperties("datasource.mybatis-factory")
public SqlSessionFactoryBean sqlSessionFactoryBean() {
SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
sqlSessionFactoryBean.setDataSource(dataSource());
return sqlSessionFactoryBean;
}
with property (among others)
datasource.mybatis-factory.mapper-locations=classpath*:sqlmap/*.xml
However, it fails even though the files are there:
Caused by: java.io.FileNotFoundException: class path resource [classpath*:sqlmap/*.xml] cannot be opened because it does not exist
Looking at setMapperLocations() I didn't do anything wrong, they clearly want me to use classpath*:...:
/**
* Set locations of MyBatis mapper files that are going to be merged into the {#code SqlSessionFactory} configuration
* at runtime.
*
* This is an alternative to specifying "<sqlmapper>" entries in an MyBatis config file. This property being
* based on Spring's resource abstraction also allows for specifying resource patterns here: e.g.
* "classpath*:sqlmap/*-mapper.xml".
*
* #param mapperLocations
* location of MyBatis mapper files
*/
public void setMapperLocations(Resource... mapperLocations) {
this.mapperLocations = mapperLocations;
}
Looking further down the code there's just this:
for (Resource mapperLocation : this.mapperLocations) {
if (mapperLocation == null) {
continue;
}
try {
XMLMapperBuilder xmlMapperBuilder = new XMLMapperBuilder(mapperLocation.getInputStream(),
targetConfiguration, mapperLocation.toString(), targetConfiguration.getSqlFragments());
xmlMapperBuilder.parse();
There is no code that would convert the classpath*:sqlmap/*.xml into openable resources or at least I don't see it. Or what am I missing here?
Work around:
What I have now and is working (note that I don't use datasource.mybatis-factory.mapper-locations as that would again overwrite what I set):
#Bean
#ConfigurationProperties("datasource.mybatis-factory")
public SqlSessionFactoryBean sqlSessionFactoryBean(
#Value("${datasource.mybatis-factory.mapper-location-pattern}") String mapperLocations) {
SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
sqlSessionFactoryBean.setDataSource(dataSource());
sqlSessionFactoryBean.setMapperLocations(findMapperLocations(mapperLocations));
return sqlSessionFactoryBean;
}
private Resource[] findMapperLocations(String resourcePaths) {
PathMatchingResourcePatternResolver patternResolver = new PathMatchingResourcePatternResolver();
return Stream.of(resourcePaths.split(","))
.map(LambdaExceptionUtilities.rethrowFunction(patternResolver::getResources))
.flatMap(Stream::of)
.toArray(Resource[]::new);
}
with property
datasource.mybatis-factory.mapper-location-pattern=classpath*:sqlmap/*.xml
So: what is missing here to make it work without the work around? How do XMLs on the classpath find the way into MyBatis? Maybe something Spring-Bootish missing?
I ran into the same issue recently. I believe this is what what you're looking for:
#Bean
#ConfigurationProperties("datasource.mybatis-factory")
public SqlSessionFactoryBean sqlSessionFactoryBean(
#Value("${datasource.mybatis-factory.mapper-location-pattern}") String mapperLocations) {
SqlSessionFactoryBean sqlSessionFactoryBean = new SqlSessionFactoryBean();
sqlSessionFactoryBean.setDataSource(dataSource());
sqlSessionFactoryBean.setMapperLocations(
new PathMatchingResourcePatternResolver().getResources("classpath*:sqlmap/*.xml")
);
return sqlSessionFactoryBean;
}
Basically what you need is this line of code in your #Bean definition above:
sqlSessionFactoryBean.setMapperLocations( new PathMatchingResourcePatternResolver().getResources("classpath*:sqlmap/*.xml")
);
Note: the method name is getResources (the plural) and not getResource
Feel free to replace the hard coded value of classpath*:sqlmap/*.xml with the #Value("datasource.mybatis-factory.mapper-location-pattern") injected value instead.
Because you're using MyBatis with Spring, the issue here is not so much a MyBatis issue, as much as it is a Spring issue. More specifically, the wildcard feature that you want to use to load multiple resources, namely, classpath*:sqlmap/*.xml is specific to Spring and not MyBatis.
I know, that the way it's documented in the MyBatis-Spring docs may lead you to believe that it's a MyBatis feature that let's you do this type of wildcard Resource loading, but it's not. Here's the relevant part of the MyBatis-Spring doc (source: https://mybatis.org/spring/factorybean.html#properties):
The mapperLocations property takes a list of resource locations. This property can be used to specify the location of MyBatis XML mapper files. The value can contain Ant-style patterns to load all files in a directory or to recursively search all paths from a base location.
However, sadly the docs only provide a Spring example based on XML and not Java configuration. If you read the Java Docs docs for SqlSessionFactoryBean, you'll find the following (source: https://mybatis.org/spring/apidocs/org/mybatis/spring/SqlSessionFactoryBean.html#setMapperLocations(org.springframework.core.io.Resource...)):
public void setMapperLocations(org.springframework.core.io.Resource... mapperLocations)
Set locations of MyBatis mapper files that are going to be merged into the
SqlSessionFactory configuration at runtime.
This is an alternative to specifying "<sqlmapper>" entries in an MyBatis config file.
This property being based on Spring's resource abstraction also allows for
specifying resource patterns here: e.g. "classpath*:sqlmap/*-mapper.xml".
Parameters:
mapperLocations - location of MyBatis mapper files
So, the setMapperLocations method needs one or more org.springframework.core.io.Resource object(s). So, using Spring ClassPathResource will not work here because ClassPathResource expects only a single resource. What you need to use instead is Spring's PathMatchingResourcePatternResolver class. See: https://docs.spring.io/spring-framework/docs/current/javadoc-api/org/springframework/core/io/support/PathMatchingResourcePatternResolver.html
You may also find this Stack Overflow post useful: How to use wildcards when searching for resources in Java-based Spring configuration?
I hope this helps!
your property should be like this.
if you use default configuration,
mybatis.mapper-locations: classpath*:sqlmap/**/*.xml
if you use your own as you mention above,
datasource.mybatis-factory.mapper-locations= classpath*:sqlmap/**/*.xml
this is a working example code, you can get an idea from this.
#Configuration
#MapperScans(
{
#MapperScan(
basePackages = {"com.example.seeker.repository"},
sqlSessionFactoryRef = "sqlSessionFactorySeeker",
sqlSessionTemplateRef = "sqlSessionTemplateSeeker"
)
}
)
public class SeekerDataSourceConfig {
#Autowired
Environment environment;
#Bean(value = "dsSeeker")
#ConfigurationProperties(prefix = "spring.datasource.seeker")
DataSource dsSeeker() {
return DataSourceBuilder.create().build();
}
#Qualifier("dsSeeker")
#Autowired
private DataSource dsSeeker;
#Bean(value = "sqlSessionFactorySeeker")
public SqlSessionFactory sqlSessionFactorySeeker() {
SqlSessionFactory sessionFactory = null;
try {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
PathMatchingResourcePatternResolver pathM3R = new PathMatchingResourcePatternResolver();
bean.setMapperLocations(pathM3R.getResources("classpath*:sqlmap/**/*.xml"));
bean.setDataSource(dsSeeker);
sessionFactory = bean.getObject();
} catch (Exception e) {
e.printStackTrace();
}
return sessionFactory;
}
#Bean(value = "sqlSessionTemplateSeeker")
public SqlSessionTemplate sqlSessionTemplateSeeker() {
return new SqlSessionTemplate(sqlSessionFactorySeeker());
}
#Bean(name = StaticResource.TxManager.TX_MANAGER_SEEKER)
public PlatformTransactionManager txManager() {
return new DataSourceTransactionManager(dsSeeker);
}
}
property file
spring.datasource.seeker.jdbc-url=jdbc:mysql://*********
spring.datasource.seeker.username=seeker***
spring.datasource.seeker.password=ewfky4eyrmggxbw6**
spring.datasource.seeker.driver-class-name=com.mysql.cj.jdbc.Driver
if you want to using with yml file you can add in the following path your application.yml file
mybatis:
mapperLocations: classpath:sql/*.xml
config-location: classpath:config/mybatis.xml
I have came across a situation where I need to fetch cron expression from database and then schedule it in Spring boot. I am fetching the data using JPA. Now the problem is in spring boot when I use #Scheduled annotation it does not allow me to use the db value directly as it is taken only constant value. So, what I am planning to do is to dynamically generate properties file and read cron expression from properties file. But here also I am facing one problem.The dynamically generated properties file created in target directory.
So I cant use it the time of program loading.
So can anyone assist me to read the dynamically generated file from the resource folder or how to schedule cron expression fetching from DB in spring boot?
If I placed all the details of corn expression in properties file I can schedule the job.
Latest try with dynamically generate properties file.
#Configuration
public class CronConfiguration {
#Autowired
private JobRepository jobRepository;
#Autowired
private ResourceLoader resourceLoader;
#PostConstruct
protected void initialize() {
updateConfiguration();
}
private void updateConfiguration() {
Properties properties = new Properties();
List<Job> morningJobList=new ArrayList<Job>();
List<String> morningJobCornExp=new ArrayList<String>();
// Map<String,String> map=new HashMap<>();
int num=1;
System.out.println("started");
morningJobList= jobRepository.findByDescriptionContaining("Morning Job");
for(Job job:morningJobList) {
//morningJobURL.add(job.getJobUrl());
morningJobCornExp.add(job.getCronExp());
}
for(String cron:morningJobCornExp ) {
properties.setProperty("cron.expression"+num+"=", cron);
num++;
}
Resource propertiesResource = resourceLoader.getResource("classpath:application1.properties");
try (OutputStream out = new BufferedOutputStream(new FileOutputStream(propertiesResource.getFile()))) {
properties.store(out, null);
} catch (Exception ex) {
// Handle error
ex.printStackTrace();
}
}
}
Still it is not able to write in properties file under resource folder.
Consider using Quartz Scheduler framework. It stores scheduler info in DB. No need to implement own DB communication, it is already provided.
Found this example: https://www.callicoder.com/spring-boot-quartz-scheduler-email-scheduling-example/
#ConfigurationProperties locations is deprecated in Spring Boot 1.4.x and option is now removed in 1.5.x
I was using it like this: BucketTestConfig.java
For now with deprecation, I'm trying to set the system property spring.config.location for both production code and test code as an alternative.
./gradlew clean test is still failing although I set the system property.
What is the best alternative for deprecated #ConfigurationProperties locations in this case?
UPDATE:
Using SpringApplicationBuilder.properties() doesn't work in the test (BucketTestRepositoryTests).
Using SpringApplicationBuilder.listeners() doesn't work in the test (BucketTestRepositoryTests), either.
UPDATE (2nd):
There was no reason to depend on #ConfigurationProperties in my case, so I went with Yaml instead as follows: https://github.com/izeye/spring-boot-throwaway-branches/commit/a1290672dceea98706b1a258f8a17e2628ea01ee
So this question's title is invalid and this question can be deleted.
Follow this thread for more information.
Basically, this thread suggests two options
First option is to set spring.config.name to a list of the files you want to load:
new SpringApplicationBuilder(Application.class)
.properties("spring.config.name=application,mine")
.run(args);
Second options is to add listeners
new SpringApplicationBuilder(SanityCheckApplication.class)
.listeners(new LoadAdditionalProperties())
.run(args);
Content of LoadAdditionalProperties
#Component
public class LoadAdditionalProperties implements ApplicationListener<ApplicationEnvironmentPreparedEvent> {
private ResourceLoader loader = new DefaultResourceLoader();
#Override
public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {
try {
Resource resource = loader.getResource("classpath:mine.properties");
PropertySource<?> propertySource = new PropertySourcesLoader().load(resource);
event.getEnvironment().getPropertySources().addLast(propertySource);
} catch (IOException ex) {
throw new IllegalStateException(ex);
}
}
}
I am new to the Spring ws.For the following things i need some clarification
1.I want to know how to customise the bindings,operations,port names etc..that is generated automatially by Spring.
2.How to specify multiple bindings,port types if we have multiple operations so that all should be generated in same wsdl.
You can customize the dynamic WSDL properties using DefaultWsdl11Definition bean
#Bean
public DefaultWsdl11Definition orders() {
DefaultWsdl11Definition definition = new DefaultWsdl11Definition();
definition.setPortTypeName("Orders");
definition.setLocationUri("http://localhost:8080/ordersService/");
definition.setSchema(new SimpleXsdSchema(new ClassPathResource("echo.xsd")));
return definition;
}
Ref : http://docs.spring.io/spring-ws/docs/current/reference/html/server.html
API Doc : http://docs.spring.io/spring-ws/sites/1.5/apidocs/org/springframework/ws/wsdl/wsdl11/DefaultWsdl11Definition.html