this is my batchLunch Code
package com.example.backend.batch.editor;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.springframework.batch.core.JobParameter;
import org.springframework.batch.core.JobParameters;
import org.springframework.batch.core.JobParametersInvalidException;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.batch.core.repository.JobExecutionAlreadyRunningException;
import org.springframework.batch.core.repository.JobInstanceAlreadyCompleteException;
import org.springframework.batch.core.repository.JobRestartException;
import org.springframework.scheduling.annotation.Scheduled;
import org.springframework.stereotype.Component;
import java.time.LocalDateTime;
import java.util.HashMap;
import java.util.Map;
#Slf4j
#Component
#RequiredArgsConstructor
public class EditorBatchJobLauncher {
private final JobLauncher jobLauncher;
private final EditorBatchJob editorBatchJob;
#Scheduled(cron = "0/10 * * * * *") //10초
// #Scheduled(cron = "0 0 0/3 * * *")
public void runJob() {
Map<String, JobParameter> confMap = new HashMap<>();
confMap.put("time", new JobParameter(String.valueOf(LocalDateTime.now())));
JobParameters jobParameters = new JobParameters(confMap);
try {
jobLauncher.run(editorBatchJob.jpaEditorWriterJob(), jobParameters);
} catch (JobExecutionAlreadyRunningException | JobInstanceAlreadyCompleteException
| JobParametersInvalidException | JobRestartException e) {
log.error(e.getMessage());
}
}
}
When I run the batch job, it runs normally, but the parameters are not saved in the database.
[2023-02-05 20:23:25:6756] INFO 73224 --- [ main]
o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob:
[name=jpaItemWriterJob]] launched with the following parameters:
[{}]
[2023-02-05 20:23:25:6792] INFO 73224 --- [ main]
o.s.batch.core.job.SimpleStepHandler : Step already complete or
not restartable, so no action to execute: StepExecution: id=1,
version=3, name=jpaItemWriterStep, status=COMPLETED,
exitStatus=COMPLETED, readCount=9, filterCount=0, writeCount=9
readSkipCount=0, writeSkipCount=0, processSkipCount=0, commitCount=1,
rollbackCount=0, exitDescription=
[2023-02-05 20:23:25:6802] INFO 73224 --- [ main]
o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob:
[name=jpaItemWriterJob]] completed with the following parameters:
[{}] and the following status: [COMPLETED] in 26ms
how can i add jobParameters to mysql???
and what did i wrong..
Related
I am learning Spring AOP and tried to call the #BeforeAdvise for two Different methods in different classes. But the log added in the first call is getting printed only. Please check the code below and let me know what is not correct here.
MainClass
package com.example.aspects;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.AnnotationConfigApplicationContext;
import com.example.aspects.AopClasses.DemoConfig;
#SpringBootApplication
public class AspectsApplication {
public static void main(String[] args) {
SpringApplication.run(AspectsApplication.class, args);
}
}
AspectClass
package com.example.aspects.Acpects;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.springframework.stereotype.Component;
#Aspect
#Component
public class MyLoggingDemo {
private static final Logger logger = LogManager.getLogger(MyLoggingDemo.class);
#Before("execution(public String test())")
public void beforeCheck() {
logger.info("Before Check !!!!!!!!!!!! "+System.currentTimeMillis() );
}
}
ConfigClass
package com.example.aspects.AopClasses;
import org.springframework.context.annotation.ComponentScan;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.EnableAspectJAutoProxy;
#Configuration
#EnableAspectJAutoProxy
#ComponentScan("com.example.aspects")
public class DemoConfig {
}
TestClass- Where the first call is made to the function
package com.example.aspects.AopClasses;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;
#RestController
#RequestMapping("/")
public class tesTClass {
private static final Logger logger = LogManager.getLogger(tesTClass.class);
MemberShipDAO dao = new MemberShipDAO();
#RequestMapping(value = "/")
public String test() {
logger.info("Hey I am here");
return "HEy There I am Working !!";
}
#RequestMapping(value ="/test")
public String testing() {
logger.info("MemberShipDAO Call");
return dao.test(); // method with same name called here again
}
}
MemberShipDAO - Declared the test method again here
package com.example.aspects.AopClasses;
import org.springframework.stereotype.Component;
#Component
public class MemberShipDAO {
public String test() {
return "HEy!!";
}
}
Console
2022-01-18 11:37:52.794 INFO 8032 --- [nio-8080-exec-1] o.a.c.c.C.[Tomcat].[localhost].[/] : Initializing Spring DispatcherServlet 'dispatcherServlet'
2022-01-18 11:37:52.796 INFO 8032 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Initializing Servlet 'dispatcherServlet'
2022-01-18 11:37:52.802 INFO 8032 --- [nio-8080-exec-1] o.s.web.servlet.DispatcherServlet : Completed initialization in 5 ms
2022-01-18 11:37:52.811 INFO 8032 --- [nio-8080-exec-1] c.example.aspects.Acpects.MyLoggingDemo : Before Check !!!!!!!!!!!! 1642486072811
2022-01-18 11:37:52.854 INFO 8032 --- [nio-8080-exec-1] c.example.aspects.AopClasses.tesTClass : Hey I am here
2022-01-18 11:37:57.383 INFO 8032 --- [nio-8080-exec-3] c.example.aspects.AopClasses.tesTClass : MemberShipDAO Call
Change this
MemberShipDAO dao = new MemberShipDAO();
to this
#Autowired MemberShipDAO dao;
Spring's magic for the AOP happens only when you use the instance created by Spring. Behind the scene, Spring instantiated your dao and wrapped it with a proxy that makes a call to your aspect.
I'm wasting a lot of time right now with AOP logging setup.
I don't know why AOP isn't working in my project.
I think I've done all the settings I can.
Please let me know if you guys have a solutions.
Thank you.
application.java
#EnableAspectJAutoProxy
#SpringBootApplication
#ComponentScan(basePackages = "com.demo.apiservice")
#MapperScan("com.demo.apiservice.mapper")
public class ApiServiceApplication extends SpringBootServletInitializer {
#Override
protected SpringApplicationBuilder configure(SpringApplicationBuilder builder) {
return builder.sources(ApiServiceApplication.class);
}
public static void main(String[] args) {
SpringApplication.run(ApiServiceApplication.class, args);
}
#Bean
public ModelMapper modelMapper() {
return new CustmizedModelMapper();
}
#Bean
public AopLoggingConfig loggingAspect(){
return new AopLoggingConfig();
}
}
build.gradle
configurations {
all {
exclude group: 'org.springframework.boot', module: 'spring-boot-starter-logging'
}
}
dependencies {
//implementation 'org.springframework.boot:spring-boot-starter-security:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-web:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-log4j2:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-mail:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-aop:2.5.5'
implementation 'org.mybatis.spring.boot:mybatis-spring-boot-starter:2.1.3'
testImplementation 'org.mybatis.spring.boot:mybatis-spring-boot-starter-test:2.1.3'
implementation 'org.springframework.boot:spring-boot-starter-data-rest:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-thymeleaf'
implementation 'org.projectlombok:lombok:1.18.8'
implementation 'org.modelmapper:modelmapper:2.3.8'
implementation 'org.springframework.boot:spring-boot-starter-data-jpa:2.5.5'
implementation 'org.springframework.boot:spring-boot-starter-jdbc:2.5.5'
implementation 'com.h2database:h2:1.4.200'
implementation 'org.springframework.boot:spring-boot-configuration-processor:2.5.5'
implementation 'org.springframework.security:spring-security-core:5.4.2'
implementation 'com.fasterxml.jackson.core:jackson-core:2.12.3'
implementation 'com.fasterxml.jackson.dataformat:jackson-dataformat-xml:2.12.3'
implementation 'com.fasterxml.jackson.core:jackson-databind:2.12.3'
implementation 'org.apache.httpcomponents:httpclient:4.5.13'
implementation 'com.nimbusds:nimbus-jose-jwt:9.7'
implementation 'io.springfox:springfox-swagger2:3.0.0'
implementation 'io.springfox:springfox-swagger-ui:2.9.2'
implementation 'joda-time:joda-time:2.10.10'
implementation 'io.jsonwebtoken:jjwt-api:0.11.1'
implementation 'javax.inject:javax.inject:1'
implementation 'com.googlecode.json-simple:json-simple:1.1'
implementation 'de.mkammerer:argon2-jvm:2.7'
implementation 'org.bouncycastle:bcprov-jdk15on:1.68'
implementation 'org.apache.maven.plugins:maven-surefire-plugin:2.22.0'
implementation 'javax.validation:validation-api:2.0.1.Final'
implementation 'org.postgresql:postgresql:42.1.4'
implementation 'org.hibernate:hibernate-gradle-plugin:5.6.1.Final'
implementation 'com.jayway.jsonpath:json-path:2.6.0'
compileOnly 'org.projectlombok:lombok:1.18.8'
testCompileOnly 'org.projectlombok:lombok:1.18.8'
runtimeOnly 'org.springframework.boot:spring-boot-devtools:2.5.5'
annotationProcessor 'org.projectlombok:lombok:1.18.8'
testAnnotationProcessor 'org.projectlombok:lombok:1.18.8'
testImplementation 'org.springframework.boot:spring-boot-starter-test:2.5.5'
testImplementation 'org.springframework.security:spring-security-test:5.5.2'
}
AopLoggingComponent.java
package com.demo.apiservice.config;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
#Component
#Aspect
public class AopLoggingConfig {
Logger logger = LoggerFactory.getLogger(AopLoggingConfig.class);
static String name = "";
static String type = "";
/**
* AspectJ is applied only to a specific class/method in package.
*/
#Around("execution(* com.demo.apiservice.customer.*Controller.*(..))")
public Object logPrint(ProceedingJoinPoint joinPoint) throws Throwable {
type = joinPoint.getSignature().getDeclaringTypeName();
if (type.indexOf("Controller") -1) {
name = "Controller \t: ";
}
else if (type.indexOf("Service") -1) {
name = "ServiceImpl \t: ";
}
else if (type.indexOf("DAO") -1) {
name = "DAO \t\t: ";
}
logger.debug(name + type + ".######################### " + joinPoint.getSignature().getName() + "()");
return joinPoint.proceed();
}
}
controller.java
package com.demo.apiservice.customer;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.PutMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.ResponseBody;
import org.springframework.web.bind.annotation.RestController;
import com.demo.apiservice.constant.Constants;
import com.demo.apiservice.customer.service.CustomerService;
import com.demo.apiservice.customer.service.impl.CustomerServiceImpl;
import com.demo.apiservice.request.CustomerRequest;
import com.demo.apiservice.request.LoginRequest;
import com.demo.apiservice.response.GetCustomerResponse;
import com.demo.apiservice.response.SuccessResponse;
import com.demo.apiservice.utils.ResponseHandler;
import io.swagger.annotations.Api;
import io.swagger.annotations.ApiOperation;
import io.swagger.annotations.ApiParam;
import io.swagger.annotations.ApiResponse;
import io.swagger.annotations.ApiResponses;
import lombok.extern.slf4j.Slf4j;
import java.util.List;
import javax.validation.Valid;
#Slf4j
#RestController
#Api(tags = "Customers APIs")
#RequestMapping("/apiservice/v1/customers")
public class CustomerController {
#Autowired
private CustomerService customerService;
#GetMapping
#ApiOperation(value = "List of Customers API")
#ApiResponses(value = {
#ApiResponse(code = 400, message = Constants.BAD_REQUEST),
#ApiResponse(code = 403, message = Constants.ACCESS_DENIED)})
public ResponseEntity<Object retrieveAll() {
log.info("Start of CustomerController::retrieveAll method");
return customerService.retrieveAll();
}
}
application.yml
logging:
level:
org:
springframework.web: DEBUG
hibernat: DEBUG
com:
atoz_develop:
mybatissample:
repository: TRACE
mybatis:
mapper-locations: classpath:/mappers/*.xml
type-aliases-package: com.demo.apiservice.entity
configuration:
map-underscore-to-camel-case: 'true'
debug: 'true'
spring:
datasource:
driver-class-name: org.postgresql.Driver
username: postgres
url: jdbc:postgresql://localhost:5432/postgres
platform: postgres
password: postgres
jpa:
generate-ddl: 'false'
properties:
hibernate:
dialect: org.hibernate.dialect.PostgreSQL10Dialect
format_sql: 'true'
hibernate:
ddl-auto: update
show-sql: 'true'
stack trace log
2021-11-17 16:05:19.992 DEBUG 23300 --- [nio-8080-exec-8] o.s.w.s.DispatcherServlet : GET /apiservice/v1/customers, parameters={} 2021-11-17 16:05:19.992 DEBUG 23300 --- [nio-8080-exec-8] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped to com.demo.apiservice.customer.CustomerController#retrieveAll()
2021-11-17 16:05:19.993 INFO 23300 --- [nio-8080-exec-8] c.l.a.c.CustomerController : Start of CustomerController::retrieveAll method
2021-11-17 16:05:19.993 INFO 23300 --- [nio-8080-exec-8] c.l.a.c.s.i.CustomerServiceImpl : Inside of the CustomerServiceImpl :: retrieveAll method
2021-11-17 16:05:19.996 INFO 23300 --- [nio-8080-exec-8] c.l.a.c.s.i.CustomerServiceImpl : End of the CustomerServiceImpl :: retrieveAll method
2021-11-17 16:05:19.996 DEBUG 23300 --- [nio-8080-exec-8] o.s.w.s.m.m.a.HttpEntityMethodProcessor : Using application/xhtml+xml, given [text/html, application/xhtml+xml, image/avif, image/webp, image/apng, application/xml;q=0.9, application/signed-exchange;v=b3;q=0.9, */*;q=0.8] and supported [application/json, application/*+json, application/json, application/*+json, application/xml;charset=UTF-8, text/xml;charset=UTF-8, application/*+xml;charset=UTF-8, application/xml;charset=UTF-8, text/xml;charset=UTF-8, application/*+xml;charset=UTF-8]
2021-11-17 16:05:19.996 DEBUG 23300 --- [nio-8080-exec-8] o.s.w.s.m.m.a.HttpEntityMethodProcessor : Writing [{data=[Customer(email=test1#test.com, password=null, customerId=null, storeName=eos, firstname=test1 (truncated)...]
2021-11-17 16:05:19.998 DEBUG 23300 --- [nio-8080-exec-8] o.s.w.s.DispatcherServlet : Completed 200 OK
Your aspect is triggered. I added an explicit controller method call in order to check:
public static void main(String[] args) {
ConfigurableApplicationContext context = SpringApplication.run(ApiServiceApplication.class, args);
context.getBean(CustomerController.class).retrieveAll();
}
Then I fixed some typos in your code like the one mentioned in my previous comment.
Probably your problem was that you simply did not see the log output because you forgot the log configuration for your application package com.demo.apiservice:
logging:
level:
org:
springframework.web: DEBUG
hibernate: DEBUG
com:
atoz_develop:
mybatissample:
repository: TRACE
demo.apiservice: DEBUG
BTW, I also corrected your typo hibernat to hibernate, but that is unrelated to the problem at hand.
Then I see this in the log:
[ restartedMain] o.s.b.w.e.t.TomcatWebServer : Tomcat started on port(s): 8080 (http) with context path ''
[ restartedMain] c.d.a.ApiServiceApplication : Started ApiServiceApplication in 5.101 seconds (JVM running for 6.117)
[ restartedMain] c.d.a.c.AopLoggingConfig : Controller : com.demo.apiservice.customer.CustomerController.######################### retrieveAll()
[ restartedMain] c.d.a.c.AopLoggingConfig : Controller : com.demo.apiservice.customer.CustomerController.######################### retrieveAll()
[ restartedMain] c.d.a.c.CustomerController : Start of CustomerController::retrieveAll method
Do you see the problem? You get duplicate logging, because the aspect is picked up once by component scan and instantiated one more time as a bean in your application configuration. So you need to remove this part from ApiServiceApplication:
#Bean
public AopLoggingConfig loggingAspect() {
return new AopLoggingConfig();
}
Now the duplicate logging is gone.
Next, maybe you want to simplify your aspect a bit and simply log joinPoint or joinPoint.getSignature(). You also want to make name and type local variables, because the static fields are not thread-safe. Instead, probably you want a static logger in the aspect.
#Component
#Aspect
public class AopLoggingConfig {
private static Logger logger = LoggerFactory.getLogger(AopLoggingConfig.class);
#Around("execution(* com.demo.apiservice.customer.*Controller.*(..))")
public Object logPrint(ProceedingJoinPoint joinPoint) throws Throwable {
String type = joinPoint.getSignature().getDeclaringTypeName();
String name = "";
if (type.contains("Controller")) {
name = "Controller \t: ";
}
else if (type.contains("Service")) {
name = "ServiceImpl \t: ";
}
else if (type.contains("DAO")) {
name = "DAO \t\t: ";
}
logger.debug(name + joinPoint.getSignature());
return joinPoint.proceed();
}
}
The log line becomes:
Controller : ResponseEntity com.demo.apiservice.customer.CustomerController.retrieveAll()
But actually, both the package name and the class name indicate that we are dealing with a controller, DAO or service. So why bother with the if-else stuff in the first place? Why make a simple matter complicated and the aspect slower? Besides, if you only want to log something and not influence the control flow, the method parameters or the return value, a simple #Before advice would do, the expensive #Around is unnecessary.
#Component
#Aspect
public class AopLoggingConfig {
private static Logger logger = LoggerFactory.getLogger(AopLoggingConfig.class);
#Before("execution(* com.demo.apiservice.customer.*Controller.*(..))")
public void logPrint(JoinPoint joinPoint) {
logger.debug(joinPoint.getSignature().toString());
}
}
The log line becomes:
ResponseEntity com.demo.apiservice.customer.CustomerController.retrieveAll()
Isn't that enough?
Or even simpler:
logger.debug(joinPoint.toString());
Log:
execution(ResponseEntity com.demo.apiservice.customer.CustomerController.retrieveAll())
Keep it simple!
The following should work:
package com.demo.apiservice.config;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
#Component
#Aspect
public class AopLoggingConfig {
Logger logger = LoggerFactory.getLogger(AopLoggingConfig.class);
static String name = "";
static String type = "";
#Pointcut("within(#org.springframework.web.bind.annotation.RestController *)")
public void controllerClassMethods() {}
#Around("controllerClassMethods()")
public Object logPrint(ProceedingJoinPoint joinPoint) throws Throwable {
type = joinPoint.getSignature().getDeclaringTypeName();
if (type.indexOf("Controller") -1) {
name = "Controller \t: ";
}
else if (type.indexOf("Service") -1) {
name = "ServiceImpl \t: ";
}
else if (type.indexOf("DAO") -1) {
name = "DAO \t\t: ";
}
logger.debug(name + type + ".######################### " + joinPoint.getSignature().getName() + "()");
return joinPoint.proceed();
}
}
This will match all the methods in all classes annotated with #RestController.
I am downloading file(s) from S3 bucket and then trying to read it using MultiResourceItemReader as there could be more then one csv file in the bucket.
I am able to download the files and place it on my local but I am not able to read the files, the files are not in the classpath. I am passing full filepath to read the files, but something is not working as expected. I am getting : No resources to read. Set strict=true if this should be an error condition.
I may not be passing the file path correctly.
Custom Log :
2021-04-21 00:32:09.690 INFO 106084 --- [ restartedMain] c.c.S.I.TaskletS3DownloadFiles : Started Downloading files from S3 bucket..
2021-04-21 00:32:10.086 INFO 106084 --- [ restartedMain] c.c.S.Config.S3BucketConfig : The input file : MI4275/input/MI4275sample.csv, has been successfully copied to : C:\Users\Desktop\data\in\MI4275sample.csv, and the file size is : 239505 bytes
2021-04-21 00:32:11.369 INFO 106084 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [step1MI4275S3ListCopyFiles] executed in 4s697ms
2021-04-21 00:32:16.475 INFO 106084 --- [ restartedMain] o.s.batch.core.job.SimpleStepHandler : Executing step: [step1ReadLoadMI4275CSV]
2021-04-21 00:32:17.477 WARN 106084 --- [ restartedMain] o.s.b.item.file.MultiResourceItemReader : No resources to read. Set strict=true if this should be an error condition.
2021-04-21 00:32:20.686 INFO 106084 --- [ restartedMain] o.s.batch.core.step.AbstractStep : Step: [step1ReadLoadMI4275CSV] executed in 4s210ms
2021-04-21 00:32:23.960 INFO 106084 --- [ restartedMain] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=jobCSProvMI4275]] completed with the following parameters: [{JobID=1618990319394}] and the following status: [COMPLETED] in 20s416ms
Code :
#Configuration
#EnableBatchProcessing
public class JobStepBuilderConfig {
#Autowired
JobBuilderFactory jobBuilderFactory;
#Autowired
StepBuilderFactory stepBuilderFactory;
#Autowired
DataSource datasource;
#Autowired
TaskletS3DownloadFiles taskletS3DownloadFiles;
#Value("${local.input.file.path.pattern}")
private Resource[] resources;
String currentDate ;
#Bean
#StepScope
// MultiResourceItemReader to read multiple files sequentially
public MultiResourceItemReader<Provider> providerMultiResourceItemReader() {
MultiResourceItemReader<Provider> multiResourceItemReader = new MultiResourceItemReader<>();
multiResourceItemReader.setResources(resources);
multiResourceItemReader.setDelegate(providerItemReader());
return multiResourceItemReader;
}
#Bean
#StepScope
// FlatfileItemReader to define file properties
public FlatFileItemReader<Provider> providerItemReader(){
// create FlatFileItemReader
FlatFileItemReader<Provider> reader = new FlatFileItemReader<>();
// skip header
reader.setLinesToSkip(1);
DefaultLineMapper<Provider> customerLineMapper = new DefaultLineMapper<>();
// tokenizer for delimited file
DelimitedLineTokenizer tokenizer = new DelimitedLineTokenizer();
customerLineMapper.setLineTokenizer(tokenizer);
customerLineMapper.setFieldSetMapper(new ProviderFieldSetMapper());
customerLineMapper.afterPropertiesSet();
reader.setLineMapper(customerLineMapper);
return reader ;
}
#Bean
// JdbcBatchItemWriter to write records into database
public JdbcBatchItemWriter<Provider> providerJdbcBatchItemWriter(){
JdbcBatchItemWriter<Provider> jdbcBatchItemWriter = new JdbcBatchItemWriter<>();
jdbcBatchItemWriter.setDataSource(this.datasource);
jdbcBatchItemWriter.setSql(insertQuery);
jdbcBatchItemWriter.setItemSqlParameterSourceProvider(new BeanPropertyItemSqlParameterSourceProvider<>());
jdbcBatchItemWriter.afterPropertiesSet();
return jdbcBatchItemWriter;
}
// Steps
#Bean
public Step step1(){
// step 1 : Read records from custom table and call stored procedure to update facets table
return stepBuilderFactory.get("step1S3ListCopyFiles")
.tasklet(taskletS3DownloadFiles)
.build();
}
#Bean
public Step step2(){
// step 2 : Read csv files and dump it into a custom table
return stepBuilderFactory.get("step2ReadLoadCSV")
.<Provider, Provider>chunk(1000)
.reader(providerMultiResourceItemReader())
.writer(providerJdbcBatchItemWriter())
.build();
}
// Job
#Bean
public Job job(){
return jobBuilderFactory.get("jobCSProvMI")
.start(step1())
.next(step2())
.incrementer(new RunIdIncrementer())
.build();
}
}
application.properties
local.input.file.path.pattern =file:C:/Users/Desktop/data/in/*.csv
You are declaring a bean definition method (the method providerMultiResourceItemReader() annotated with #Bean) in a class annotated with #Component. This is not correct. You should be declaring your beans in a class annotated with #Configuration instead.
I see no need for your #Component public class FileItemReader { .. } class, you could define the reader in the same class where you define your step, something like:
import org.springframework.batch.core.Job;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.Arrays;
#Configuration
#EnableBatchProcessing
public class MyJobConfiguration {
#Bean
public ItemReader<Integer> itemReader() {
return new ListItemReader<>(Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7, 8, 9));
}
#Bean
public ItemWriter<Integer> itemWriter() {
return items -> items.forEach(System.out::println);
}
#Bean
public Job job(JobBuilderFactory jobs, StepBuilderFactory steps) {
return jobs.get("job")
.start(steps.get("step")
.<Integer, Integer>chunk(5)
.reader(itemReader())
.writer(itemWriter())
.build())
.build();
}
}
If you really want to extract the reader's configuration in a separate class, then you need to annotate it with #Configuration:
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.support.ListItemReader;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.util.Arrays;
#Configuration
public class ItemReaderConfiguration {
#Bean
public ItemReader<Integer> itemReader() {
return new ListItemReader<>(Arrays.asList(0, 1, 2, 3, 4, 5, 6, 7, 8, 9));
}
}
Then import that class in your main job configuration:
import org.springframework.batch.core.Job;
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing;
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory;
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory;
import org.springframework.batch.item.ItemReader;
import org.springframework.batch.item.ItemWriter;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
#Configuration
#Import(ItemReaderConfiguration.class)
#EnableBatchProcessing
public class MyJobConfiguration {
#Bean
public ItemWriter<Integer> itemWriter() {
return items -> items.forEach(System.out::println);
}
#Bean
public Job job(JobBuilderFactory jobs, StepBuilderFactory steps, ItemReader<Integer> itemReader) {
return jobs.get("job")
.start(steps.get("step")
.<Integer, Integer>chunk(5)
.reader(itemReader)
.writer(itemWriter())
.build())
.build();
}
}
I was using SwaggerHub recently and tried to upgrade the spring-boot version (1.5.22.RELEASE to 2.3.3.RELEASE) in the generated Spring-Boot api, because of the access management (Keycloak).
Upgrading spring-boot version to 2.0.1.RELEASE does fix the issues using Keycloak, results in 404 error for every path of the api though. Summarily, no path can be accessed, despite being mapped according to the log.
Issues with the Versions:
1.5.22.RELEASE: API-Path can be accessed - Getting NoClassDefFoundError: WebServerFactoryCustomizer
2.0.0.RELEASE - 2.0.9.RELEASE: API-Paths are being mapped, but no access - No Error with Keycloak at startup.
2.1.0.RELEASE - 2.3.3.RELEASE: API-Paths don't get mapped - No Error with Keycloak at startup.
This is a section of the log using 2.0.1.RELEASE:
2020-10-05 10:46:47.012 INFO 25040 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getIcnByNumber],methods=[GET],produces=[application/json]}" onto public org.springframework.http.ResponseEntity<java.util.List<io.swagger.model.ModelConfiguration>> io.swagger.api.GetIcnByNumberApiController.getIcnByNumber()
2020-10-05 10:46:47.015 INFO 25040 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getMyConfigurations],methods=[GET],produces=[application/json]}" onto public org.springframework.http.ResponseEntity<java.util.List<io.swagger.model.ModelConfiguration>> io.swagger.api.GetMyConfigurationsApiController.callConfiguration()
2020-10-05 10:46:47.017 INFO 25040 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getMyProjects],methods=[GET],produces=[application/json]}" onto public org.springframework.http.ResponseEntity<java.util.List<io.swagger.model.ProjectModel>> io.swagger.api.GetMyProjectsApiController.getMyProjectsGet()
2020-10-05 10:46:47.020 INFO 25040 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getProjectConfiguration/{projectId}],methods=[GET],produces=[application/json]}" onto public org.springframework.http.ResponseEntity<io.swagger.model.ProjectConfiguration> io.swagger.api.GetProjectConfigurationApiController.getProjectConfigurationProjectIdGet(java.lang.String)
2020-10-05 10:46:47.021 INFO 25040 --- [ main] s.w.s.m.m.a.RequestMappingHandlerMapping : Mapped "{[/getRequestOverview],methods=[GET],produces=[application/json]}" onto public org.springframework.http.ResponseEntity<java.util.List<io.swagger.model.RequestOverview>> io.swagger.api.GetRequestOverviewApiController.getRequestOverview()
Code-Example:
/getMyConfigurations - Interface:
/**
* NOTE: This class is auto generated by the swagger code generator program (3.0.21).
* https://github.com/swagger-api/swagger-codegen
* Do not edit the class manually.
*/
package io.swagger.api;
import io.swagger.model.ModelConfiguration;
import io.swagger.annotations.*;
import org.springframework.http.ResponseEntity;
import org.springframework.validation.annotation.Validated;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.multipart.MultipartFile;
import org.springframework.web.bind.annotation.CookieValue;
import javax.validation.Valid;
import javax.validation.constraints.*;
import java.util.List;
import java.util.Map;
#javax.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2020-09-14T09:55:45.552Z[GMT]")
#Api(value = "getMyConfigurations", description = "the getMyConfigurations API")
public interface GetMyConfigurationsApi {
#ApiOperation(value = "searches inventory", nickname = "callConfiguration", notes = "By passing in the appropriate options, you can search for available inventory in the system ", response = ModelConfiguration.class, responseContainer = "List", tags={ "developers", })
#ApiResponses(value = {
#ApiResponse(code = 200, message = "search results matching criteria", response = ModelConfiguration.class, responseContainer = "List"),
#ApiResponse(code = 400, message = "bad input parameter") })
#RequestMapping(value = "/getMyConfigurations",
produces = { "application/json" },
method = RequestMethod.GET)
ResponseEntity<List<ModelConfiguration>> callConfiguration();
}
/getMyConfigurations - Controller:
package io.swagger.api;
import io.swagger.model.ModelConfiguration;
import io.swagger.model.Originator;
import com.fasterxml.jackson.databind.ObjectMapper;
import io.swagger.annotations.*;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.http.HttpStatus;
import org.springframework.http.ResponseEntity;
import org.springframework.stereotype.Controller;
import org.springframework.web.bind.annotation.CookieValue;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestHeader;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RequestPart;
import org.springframework.web.multipart.MultipartFile;
import javax.validation.constraints.*;
import javax.validation.Valid;
import javax.servlet.http.HttpServletRequest;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
#javax.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2020-08-28T09:14:46.542Z[GMT]")
#Controller
public class GetMyConfigurationsApiController implements GetMyConfigurationsApi {
private static final Logger log = LoggerFactory.getLogger(GetMyConfigurationsApiController.class);
private final ObjectMapper objectMapper;
private final HttpServletRequest request;
#org.springframework.beans.factory.annotation.Autowired
public GetMyConfigurationsApiController(ObjectMapper objectMapper, HttpServletRequest request) {
this.objectMapper = objectMapper;
this.request = request;
}
public ResponseEntity<List<ModelConfiguration>> callConfiguration() {
ArrayList<ModelConfiguration> mcList = new ArrayList<>();
return new ResponseEntity<List<ModelConfiguration>>(mcList,HttpStatus.OK);
}
}
EDIT (05.10.2020): Styling for better overview.
Tried to access the path on localhost:8080/TEST/API/1.0.0/getMyConfigurations which was functioning with version 1.5.33.RELEASE.
Seems like the server.contextPath attribute on application.properties is ignored in versions higher than 2.x.x.RELEASE. So it turned out that the path has to be called without server.contextPath (localhost:8080/getMyConfigurations).
Edit:
The property was moved from server.contextPath (or server.context-path) to server.servlet.context-path as described here
#martinspielmann thank you for the comment.
I tried to extend the Managing Transactions example in the spring boot guides to two datasources, but the #Transaction annotation seems to work for only one of the datasources.
In "Application.java" I added the beans for the two datasources and their JdbcTemplates. In "BookingService.java" I used the JdbcTemplate belonging to the second datasource.
Here is my "Application.java":
package hello;
import javax.sql.DataSource;
import org.junit.Assert;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.jdbc.DataSourceBuilder;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.context.ApplicationContext;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Primary;
import org.springframework.jdbc.core.JdbcTemplate;
#SpringBootApplication
public class Application {
private static final Logger log = LoggerFactory.getLogger(Application.class);
#Bean
BookingService bookingService() {
return new BookingService();
}
#Primary
#Bean(name="datasource1")
#ConfigurationProperties(prefix="datasource1")
DataSource datasource1() {
return DataSourceBuilder.create().build();
}
#Bean(name="jdbcTemplate1")
#Autowired
JdbcTemplate jdbcTemplate1(#Qualifier ("datasource1") DataSource datasource) {
return new JdbcTemplate(datasource);
}
#Bean(name="datasource2")
#ConfigurationProperties(prefix="datasource2")
DataSource datasource2() {
return DataSourceBuilder.create().build();
}
#Bean(name="jdbcTemplate2")
#Autowired
JdbcTemplate jdbcTemplate2(#Qualifier ("datasource2") DataSource dataSource) {
JdbcTemplate jdbcTemplate = new JdbcTemplate(dataSource);
log.info("Creating tables");
jdbcTemplate.execute("drop table BOOKINGS if exists");
jdbcTemplate.execute("create table BOOKINGS("
+ "ID serial, FIRST_NAME varchar(5) NOT NULL)");
return jdbcTemplate;
}
public static void main(String[] args) {
ApplicationContext ctx = SpringApplication.run(Application.class, args);
BookingService bookingService = ctx.getBean(BookingService.class);
bookingService.book("Alice", "Bob", "Carol");
Assert.assertEquals("First booking should work with no problem", 3,
bookingService.findAllBookings().size());
try {
bookingService.book("Chris", "Samuel");
}
catch (RuntimeException e) {
log.info("v--- The following exception is expect because 'Samuel' is too big for the DB ---v");
log.error(e.getMessage());
}
for (String person : bookingService.findAllBookings()) {
log.info("So far, " + person + " is booked.");
}
log.info("You shouldn't see Chris or Samuel. Samuel violated DB constraints, and Chris was rolled back in the same TX");
Assert.assertEquals("'Samuel' should have triggered a rollback", 3,
bookingService.findAllBookings().size());
try {
bookingService.book("Buddy", null);
}
catch (RuntimeException e) {
log.info("v--- The following exception is expect because null is not valid for the DB ---v");
log.error(e.getMessage());
}
for (String person : bookingService.findAllBookings()) {
log.info("So far, " + person + " is booked.");
}
log.info("You shouldn't see Buddy or null. null violated DB constraints, and Buddy was rolled back in the same TX");
Assert.assertEquals("'null' should have triggered a rollback", 3, bookingService
.findAllBookings().size());
}
}
And here is "BookingService.java":
package hello;
import java.sql.ResultSet;
import java.sql.SQLException;
import java.util.List;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.jdbc.core.JdbcTemplate;
import org.springframework.jdbc.core.RowMapper;
import org.springframework.transaction.annotation.Transactional;
public class BookingService {
private final static Logger log = LoggerFactory.getLogger(BookingService.class);
#Autowired
#Qualifier("jdbcTemplate2")
JdbcTemplate jdbcTemplate;
#Transactional
public void book(String... persons) {
for (String person : persons) {
log.info("Booking " + person + " in a seat...");
jdbcTemplate.update("insert into BOOKINGS(FIRST_NAME) values (?)", person);
}
};
public List<String> findAllBookings() {
return jdbcTemplate.query("select FIRST_NAME from BOOKINGS", new RowMapper<String>() {
#Override
public String mapRow(ResultSet rs, int rowNum) throws SQLException {
return rs.getString("FIRST_NAME");
}
});
}
}
These are the apllication properties in "application.yml":
datasource1:
url: "jdbc:h2:~/h2/ds1;DB_CLOSE_ON_EXIT=FALSE"
username: "sa"
datasource2:
url: "jdbc:h2:~/h2/ds2;DB_CLOSE_ON_EXIT=FALSE"
username: "sa"
The "pom.xml" here is the same as in Managing Transactions.
When the #Primary annotation is on the datasource2 bean, everything works as expected.
When the #Primary annotation is on the datasource1 bean, the write in datasource2 is not transactional and one gets the following output:
...
2016-05-27 16:01:23.775 INFO 884 --- [ main] hello.Application : So far, Alice is booked.
2016-05-27 16:01:23.775 INFO 884 --- [ main] hello.Application : So far, Bob is booked.
2016-05-27 16:01:23.775 INFO 884 --- [ main] hello.Application : So far, Carol is booked.
2016-05-27 16:01:23.775 INFO 884 --- [ main] hello.Application : So far, Chris is booked.
2016-05-27 16:01:23.775 INFO 884 --- [ main] hello.Application : You shouldn't see Chris or Samuel. Samuel violated DB constraints, and Chris was rolled back in the same TX
Exception in thread "main" 2016-05-27 16:01:23.776 INFO 884 --- [ Thread-2] s.c.a.AnnotationConfigApplicationContext : Closing org.springframework.context.annotation.AnnotationConfigApplicationContext#3901d134: startup date [Fri May 27 16:01:22 CEST 2016]; root of context hierarchy
java.lang.AssertionError: 'Samuel' should have triggered a rollback expected:<3> but was:<4>
at org.junit.Assert.fail(Assert.java:88)
at org.junit.Assert.failNotEquals(Assert.java:834)
at org.junit.Assert.assertEquals(Assert.java:645)
at hello.Application.main(Application.java:84)
2016-05-27 16:01:23.778 INFO 884 --- [ Thread-2] o.s.j.e.a.AnnotationMBeanExporter : Unregistering JMX-exposed beans on shutdown
So "Chris" wasn't rolled back.
I guess it has something to do with properly initializing both databases. Is this a bug, or am I missing something here?
Thanks!
I added two beans in "Application.java":
#Bean(name="tm1")
#Autowired
DataSourceTransactionManager tm1(#Qualifier ("datasource1") DataSource datasource) {
DataSourceTransactionManager txm = new DataSourceTransactionManager(datasource);
return txm;
}
#Bean(name="tm2")
#Autowired
DataSourceTransactionManager tm2(#Qualifier ("datasource2") DataSource datasource) {
DataSourceTransactionManager txm = new DataSourceTransactionManager(datasource);
return txm;
}
and changed the #Transactional in "BookingService.java" to:
#Transactional("tm2")
So now we have two resource-local transaction managers, one for each datasource, and it works as expected.
Many thanks to M.Deinum!