I am using spring batch applicationwith reader,writer and processor. File name is passed from batchjob to writer which is in stepscope.When bean is initialized I could see exception in BATCH_STEP_EXECUTION table as below
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'scopedTarget.resWriter' defined in class path resource : Bean instantiation via factory method failed; nested exception is org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.batch.item.file.FlatFileItemWriter]: Factory method 'resWriter' threw exception; nested exception is java.lang.IllegalArgumentException: Path must not be null
Spring batch code
#StepScope
#Bean
public FlatFileItemWriter<EntityObject> regulatedEntityWriter(#Value("#{jobParameters['fileName']}") String fileName){
/*
while bean is initialized fileName is empty and FlatFileItemWriter requries filename, then it throws Path must not be null exeption
*/
pretaFileName = fileName;
FlatFileItemWriter<EntityObject> csvFileWriter = new FlatFileItemWriter<>();
String exportFileHeader = "column1,column2,column3";
StringHeaderWriter headerWriter = new StringHeaderWriter(exportFileHeader);
csvFileWriter.setHeaderCallback(headerWriter);
csvFileWriter.setShouldDeleteIfEmpty(true);
CustomDelimitedLineAggregator<EntityObject> lineAggregator = new CustomDelimitedLineAggregator<>();
BeanWrapperFieldExtractor<EntityObject> fieldExtractor = new BeanWrapperFieldExtractor<>();
fieldExtractor.setNames(new String[]{"column1", "column2", "column3"});
lineAggregator.setFieldExtractor(fieldExtractor);
csvFileWriter.setLineAggregator(lineAggregator);
csvFileWriter.setEncoding(encodingType);
csvFileWriter.setResource(new FileSystemResource(fileName));
return csvFileWriter;
}
```
Above method is called using joblauncher
``` JobParameters params = new JobParametersBuilder()
.addString("JobID", String.valueOf(System.currentTimeMillis()))
.addString("fileName", "sample_file.txt")
.toJobParameters();
JobExecution jobExecution =jobLauncher.run(job, params);
I have tried #Lazy annotation, still when server is coming up it throws that exception.
I am using multi node cluster and it add entries for each node while server is coming up in BATCH_STEP_EXECUTION table.How to avoid this exception while server startup for the first time?
I have used below property in springboot application.properties to disable spring batch by default as I am triggering and passing parameter through cron trigger.
spring.batch.job.enabled=false
Related
i want to unit test an Kafka Stream Aggregate and i am totally confused which method to use.
I read about the TestSupportBinder but i do not think this is working in my case, therefore i use the KafkaEmbedded Method. This is how i initialize the embedded Kafka.
#Before
public void setUp() throws Exception{
Map<String, Object> consumerProps = KafkaTestUtils.consumerProps("group-id", "false", embeddedKafka);
consumerProps.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
consumerProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
consumerProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
DefaultKafkaConsumerFactory<Object, LoggerMessage> cf = new DefaultKafkaConsumerFactory<>(consumerProps);
consumer = cf.createConsumer();
embeddedKafka.consumeFromAnEmbeddedTopic(consumer, OUTPUT_TOPIC);
}
What i want to test is the following:
public interface Channels {
String LOGGER_IN_STREAM = "logger-topic-in-stream";
String LOGGER_IN = "logger-topic-in";
String LOGGERDATAVALIDATED_OUT = "loggerDataValidated-topic-out";
#Input(Channels.LOGGER_IN)
SubscribableChannel processMessage();
#Input(Channels.LOGGER_IN_STREAM)
KStream<Object, LoggerMessage> loggerKstreamIn();
#Output(Channels.LOGGERDATAVALIDATED_OUT)
MessageChannel validateLoggerData();
}
And i get following error message
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'some.domain.Channels': Invocation of init method failed; nested exception is java.lang.IllegalStateException: No factory found for binding target type: org.apache.kafka.streams.kstream.KStream among registered factories: channelFactory,messageSourceFactory
Caused by: java.lang.IllegalStateException: No factory found for binding target type: org.apache.kafka.streams.kstream.KStream among registered factories: channelFactory,messageSourceFactory
What am i doing wrong?
I missed to inject my Channels Interface as a MockBean. After i did that everything worked as expected.
I am using Apache Camel and I defined a route receiving inputs from 2 queues located on 2 distinct servers. I want ideally to consume from both server but I also want to be able to run the application when one of the 2 destinations is down.
Here's my route:
try {
from("mccdsJmsRequest1:queue:{{mccds.queues.in}}").to("direct:apolloMccdsRoute");
} catch (Exception e){
// debug line
System.out.println("Ms1 not reachable");
e.printStackTrace();
}
try {
from("mccdsJmsRequest2:queue:{{mccds.queues.in}}").to("direct:apolloMccdsRoute");
} catch (Exception e){
// debug line
System.out.println("Ms2 not reachable");
e.printStackTrace();
}
from("direct:apolloMccdsRoute").routeId("apolloMCCDSRoute")
// Main route starts here...
I declare my beans here:
#Bean
public JndiObjectFactoryBean mccdsJmsConnectionFactory1() {
JndiObjectFactoryBean cf = new JndiObjectFactoryBean();
cf.setJndiEnvironment(prodMccdsJndiProperties.getJndi1());
cf.setJndiName(jndiName1);
return cf;
}
#Bean
public JndiObjectFactoryBean mccdsJmsConnectionFactory2(){
JndiObjectFactoryBean cf = new JndiObjectFactoryBean();
cf.setJndiEnvironment(prodMccdsJndiProperties.getJndi2());
cf.setJndiName(jndiName2);
return cf;
}
#Inject
private CamelContext camelContext;
#Bean
public JmsComponent mccdsJmsRequest1() {
JmsComponent ac = new JmsComponent(camelContext);
ac.setConnectionFactory((ConnectionFactory) mccdsJmsConnectionFactory1().getObject());
ac.setConcurrentConsumers(5);
return ac;
}
#Bean
public JmsComponent mccdsJmsRequest2(){
JmsComponent ac = new JmsComponent(camelContext);
ac.setConnectionFactory((ConnectionFactory) mccdsJmsConnectionFactory2().getObject());
ac.setConcurrentConsumers(5);
return ac;
}
If one of the connection factory is not reachable the application doesn't start.
I would like to catch an ignore the exception:
o.s.b.f.s.DefaultListableBeanFactory : Bean creation exception on non-lazy FactoryBean type check: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'mccdsJmsConnectionFactory2' defined in class path resource [ca/bell/it/spa/uim/apollo/maximo/config/mccds/ProdMccdsJmsConfiguration.class]: Invocation of init method failed; nested exception is javax.naming.CommunicationException [Root exception is java.net.ConnectException: t3://someTestServerIP: Destination unreachable; nested exception is:
java.net.ConnectException: Connection refused (Connection refused); No available router to destination]
Try to set lookupOnStartup to false. (Add cf.setLookupOnStartup(false) to the mccdsJmsConnectionFactory1 and mccdsJmsConnectionFactory2 bean definitions.)
Also check this thread: http://forum.spring.io/forum/spring-projects/batch/95620-jndi-lookup
That won't work because your try/catch blocks are never executed during the execution of your route. In your code, you'd only get an exception during Camel's initial execution of this code upon application startup when it's building the route, when it doesn't care if your queues even exist or not.
Instead you need to determine when an error occurs in your route when it's executed after being constructed. There are a number of ways to do this, but a good start might be to look at Camel's Exception Clause
And alternative and perhaps even better choice is to make use of Camel's Load Balancer . It allows you to try any number of endpoints and failover to the next should one fail with an exception.
I have a requirement where I need to process files based on the rest call in which I get the name of the file, I am adding it to the job parameter and using it while creating the beans.
I am creating step scope Beans for (reader,writer) and using the job parameter.I am starting the job in a new thread as I am using asynchronus task exceutor to launch the job and my question is how will the beans be created by spring when we define #StepScope
jobParametersBuilder.addString("fileName", request.getFileName());
jobExecution = jobLauncher.run(job, jobParametersBuilder.toJobParameters());
#Bean
public JobLauncher jobLauncher() {
SimpleJobLauncher jobLauncher = new SimpleJobLauncher();
jobLauncher.setJobRepository(jobRepository());
jobLauncher.setTaskExecutor(asyncTaskExecutor());
return jobLauncher;
}
#Bean
#StepScope
public ItemWriter<Object> writer(#Value ("#{jobParameters['fileName']}"String fileName) {
JdbcBatchItemWriter<Object> writer = new JdbcBatchItemWriter<>();
writer.setItemSqlParameterSourceProvider(
new BeanPropertyItemSqlParameterSourceProvider<Object>());
writer.setSql(queryCollection.getquery());
writer.setDataSource(dataSource(fileName));
return writer;
}
A spring batch StepScope object is one which is unique to a specific step and not a singleton. As you probably know, the default bean scope in Spring is a singleton. But by specifying a spring batch component being StepScope means that Spring Batch will use the spring container to instantiate a new instance of that component for each step execution.
This is often useful for doing parameter late binding where a parameter may be specified either at the StepContext or the JobExecutionContext level and needs to be substituted for a placeholder, much like your example with the filename requirement.
Another useful reason to use StepScope is when you decide to reuse the same component in parallel steps. If the component manages any internal state, its important that it be StepScope based so that one thread does not impair the state managed by another thread (e.g, each thread of a given step has its own instance of the StepScope component).
I want to inject property values into the Spring context when it is started.
I am trying to do this using the new Environment and PropertySource features of Spring 3.1.
In the class in which loads the Spring context, I define my own PropertySource class as follows:
private static class CustomPropertySource extends PropertySource<String> {
public CustomPropertySource() {super("custom");}
#Override
public String getProperty(String name) {
if (name.equals("region")) {
return "LONDON";
}
return null;
}
Then, I add this property source to the application context:
ClassPathXmlApplicationContext springIntegrationContext =
new ClassPathXmlApplicationContext("classpath:META-INF/spring/ds-spring-intg-context.xml");
context.getEnvironment().getPropertySources().addLast( new CustomPropertySource());
context.refresh();
context.start();
In one of my beans, I try to access the property value:
#Value("${region}")
public void setRegion(String v){
...
}
bur recieve the folowing error:
java.lang.IllegalArgumentException: Caused by:
java.lang.IllegalArgumentException: Could not resolve placeholder
'region' in string value [${region}]
Any help is greatly appreciated
when you pass an XML file location as a constructor argument to ClassPathXmlApplicationContext(..) it straight away does the context.refresh()/context.start() methods. so by passing in your XML locations, you're essentially doing it all in one pass and the context is already started/loaded by the time you get to call context.getEnvironment().getPropertySources....
try this;
ClassPathXmlApplicationContext springIntegrationContext =
new ClassPathXmlApplicationContext();
context.getEnvironment().getPropertySources().addLast( new CustomPropertySource());
context.setLocations("classpath:META-INF/spring/ds-spring-intg-context.xml");
context.refresh();
it will set your sources, then your xml, then start the app context.
When using Distributed Mysql Database, I've Created multiple dataSources, sessionFactory using BeanDefinitionBuilder
But transactional annotation doesn't seem to work when I execute Insert SQL
using getBean('bean name') method below
( (SqlSessionTemplate)context.getBean('bean name') ).insert("xxxx",params)
Would you explain what I've missed?
Private GenericApplicationContext context = new GenericApplicationContext();
BeanDefinitionBuilder sessionFactoryBuilder = BeanDefinitionBuilder.rootBeanDefinition(org.mybatis.spring.SqlSessionFactoryBean.class);
sessionFactoryBuilder.addPropertyReference("dataSource", "dataSource" + beanName);
sessionFactoryBuilder.addPropertyValue("configLocation", "/sqlmap.xml");
context.registerBeanDefinition("sqlSessionFactory" + beanName, sessionFactoryBuilder.getBeanDefinition());
BeanDefinitionBuilder transactionManagerBuilder = BeanDefinitionBuilder.rootBeanDefinition(org.springframework.jdbc.datasource.DataSourceTransactionManager.class);
transactionManagerBuilder.addPropertyReference("dataSource", "dataSource" + beanName);
context.registerBeanDefinition("transactionManager",transactionManagerBuilder.getBeanDefinition());
ctx.refresh();
Please try to add the following codes below and see if it works.
ConfigurableListableBeanFactory factory = context.getBeanFactory();
AspectJAwareAdvisorAutoProxyCreator aspectJPostProcessor = new AspectJAwareAdvisorAutoProxyCreator();
aspectJPostProcessor.setBeanFactory( factory );
aspectJPostProcessor.setProxyClassLoader( context.getClassLoader() );
factory.addBeanPostProcessor(aspectJPostProcessor);