Why is it getting a endless loop? always return the ten records and doesn't continue with the next ones
<bean id="pagingItemReader1" class="org.springframework.batch.item.database.JdbcPagingItemReader" scope="step">
<property name="dataSource" ref="dataSource1" />
<property name="queryProvider">
<bean class="org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean">
<property name="dataSource" ref="dataSource1" />
<property name="selectClause" value="select id, image" />
<property name="fromClause" value="from squares" />
<property name="whereClause" value="where image like :value1 or image like :value2" />
<property name="sortKey" value="id" />
</bean>
</property>
<property name="parameterValues">
<map>
<entry key="value1" value="%/images/%" />
<entry key="value2" value="%/temp/%" />
</map>
</property>
<property name="pageSize" value="10" />
<property name="rowMapper">
<bean class="test.batch.ImagesRowMapper" />
</property>
</bean>
Using MySQL 5.1
Parentheses are missing.
Please change
"whereClause" value="where image like :value1 or image like :value2"
to
"whereClause" value="where (image like :value1 or image like :value2)"
I don't know why but I can share my solution with you.
Increase your pageSize over the total number of rows (of the current query of course).
This appears when I am using a 'OR' in whereClause
Someone can investigate ?
Related
I use quartz for scheduling my job (working on a maven project using Spring).
I updated quartz to the 2.3.0 version and I changed the CronTriggerBean and JobDetailBean in CronTriggerFactoryBean and JobDetailFactoryBean but with this configuration it doesn't instantiate the job at every request time like (cronexpression) it worked with the first configuration (CronTriggerBean).
Do I have to do some implementation?
quartz-context.xml
<bean id="jobImportFi01QuartzTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="group" value="xxx" />
<property name="jobDetail" ref="jobImportFi01Quartz" />
<property name="cronExpression" value="${jobImportFi01.cron.expression}" />
<property name="misfireInstructionName"
value="MISFIRE_INSTRUCTION_DO_NOTHING" />
</bean>
<bean id="jobImportFi01Quartz"
class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="group" value="xxx" />
<property name="jobClass"
value="com.batch.job.timdataimport.quartz.ImportJobDetail" />
<property name="description" value="Fi01Import" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="jobImportFi01" />
</map>
</property>
</bean>
This is what we use with Quartz 2.3.0 and it works OK:
<bean id="job1" class="org.quartz.impl.JobDetailImpl">
<property name="jobClass" value="com.quartzdesk.test.quartz.v2.TestJob"/>
<property name="group" value="quartzdesk-test"/>
<property name="name" value="Job1"/>
<property name="description"
value="Simple test job."/>
<property name="durability" value="true"/>
<property name="jobDataMap">
<bean class="org.quartz.JobDataMap">
<constructor-arg>
<util:map>
<entry key="jobKey01" value="value01"/>
</util:map>
</constructor-arg>
</bean>
</property>
</bean>
<bean id="job1Trigger"
class="org.quartz.impl.triggers.CronTriggerImpl">
<property name="name" value="Job1Trigger"/>
<property name="group" value="quartzdesk-test"/>
<property name="jobName" value="Job1"/>
<property name="jobGroup" value="quartzdesk-test"/>
<property name="description" value="Cron trigger that fires every 15 minutes."/>
<property name="cronExpression" value="0 1/15 * * * ?"/>
<property name="startTime" value="2016-01-01"/>
<property name="calendarName" value="annualCalendar"/>
<property name="jobDataMap">
<bean class="org.quartz.JobDataMap">
<constructor-arg>
<util:map>
<entry key="jobTriggerKey01" value="value01"/>
</util:map>
</constructor-arg>
</bean>
</property>
</bean>
can someone please let me know how to use oracle sequence in spring batch writer?.
I tried using custseq.nextval in the insert statement, but its failing.
<bean id="testSSRReader"
class="org.springframework.batch.item.database.JdbcCursorItemReader">
<property name="dataSource" ref="bconnectedDataSource" />
<property name="sql"
value="select CUST_USA_ID , CUST_FIRST_NAME , CUST_LAST_NAME from BL_CUSTOMER fetch first 100 rows only" />
<property name="rowMapper">
<bean class="com.macys.batch.rowmapper.TestSSRRowMapper" />
</property>
</bean>
<bean id="testSSRProcessor" class="com.macys.batch.processor.TestSSRProcessor" />
<bean id="testSSRWriter"
class="org.springframework.batch.item.database.JdbcBatchItemWriter">
<property name="dataSource" ref="ocDataSource" />
<property name="sql">
<value>
<![CDATA[
insert into TESTTABLESSR(CUSTOMER_ID,CUSTOMER_NAME,CITY)
VALUES (custseq.nextval,:firstName,:lastName)
]]>
</value>
</property>
<property name="itemSqlParameterSourceProvider" ref="itemSqlParameterSourceProvider" />
</bean>
I have two file types to insert in database.
Format are : aa;bb;cc and aa;bb;cc;dd;ee
This is my FlatFileItemReader :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=";"/>
<property name="names" value="aa,bb,cc,dd,ee" />
</bean>
</property>
</bean>
</property>
</bean>
I want a setup that works for both types of files.
For the moment, I have this :
org.springframework.batch.item.file.transform.IncorrectTokenCountException:
Incorrect number of tokens found in record: expected 3 actual 5
Do you have any ideas?
Thank you.
Edit : After correction :
<bean name="readerContractToAddIntoPRV" class="org.springframework.batch.item.file.FlatFileItemReader">
<property name="comments" value="#" />
<property name="linesToSkip" value="1" />
<property name="strict" value="false" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" p:lineTokenizer-ref="multilineFileTokenizer">
<property name="fieldSetMapper">
<bean class="net.wl.batchs.fieldSetMapper.LineToCreateIntoPrvFieldSetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="multilineFileTokenizer" class="org.springframework.batch.item.file.transform.PatternMatchingCompositeLineTokenizer">
<property name="tokenizers">
<map>
<entry key="*;*;*;*;*" value-ref="NSCE_ICCID_MSISDN_LOGIN_PWD"/>
<entry key="*;*;*" value-ref="NSCE_ICCID_MSISDN"/>
<entry key="*" value-ref="headerDefault"/>
</map>
</property>
</bean>
<bean id="parentLineTokenizer" class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer" abstract="true">
<property name="delimiter" value=";"/>
</bean>
<bean id="NSCE_ICCID_MSISDN_LOGIN_PWD" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn,login,pwd" />
</bean>
<bean id="NSCE_ICCID_MSISDN" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>
<bean id="headerDefault" parent="parentLineTokenizer">
<property name="names" value="nsce,iccid,msisdn" />
</bean>
The issue isn't your tokenizer. What you'll have to do is use the PatternMatchingCompositeLineMapper (http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/item/file/mapping/PatternMatchingCompositeLineMapper.html). This will allow you to create a pattern for each line type you have and associate it with the appropriate LineTokenizer.
You can see this LineMapper in action in our samples here: https://github.com/spring-projects/spring-batch/blob/master/spring-batch-samples/src/main/resources/jobs/multilineOrderInputTokenizers.xml
I have a spring mvc webapp with spring batch built into it. I am having some issues getting my spring batch jobs to be launchable in the spring batch admin console. This is what I see when I go to the jobs page...
All of my jobs are coming up as launchable=false. I was wondering how I can fix this. I read some documentation about why this would be so and it said that I need to use a AutomaticJobRegistrar.
I tried this but it didn't change anything. I've put my spring batch job configuration below. Would appreciate it someone could tell me what is missing.
thanks
<beans profile="pre,prod">
<bean id="jobLauncher"
class="org.springframework.batch.core.launch.support.SimpleJobLauncher">
<property name="jobRepository" ref="jobRepository" />
</bean>
<bean id="jobRepository"
class="org.springframework.batch.core.repository.support.JobRepositoryFactoryBean"
parent="abstractCustDbJdbcDao">
<property name="transactionManager" ref="custDbTransactionManager" />
<property name="databaseType" value="db2" />
<property name="tablePrefix" value="REPMAN.BATCH_" />
</bean>
<bean id="jobExplorer"
class="org.springframework.batch.core.explore.support.JobExplorerFactoryBean"
parent="abstractCustDbJdbcDao" />
<bean class="org.springframework.batch.core.configuration.support.JobRegistryBeanPostProcessor">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
<bean id="jobLoader" class="org.springframework.batch.core.configuration.support.AutomaticJobRegistrar">
<property name="applicationContextFactories">
<bean class="org.springframework.batch.core.configuration.support.ClasspathXmlApplicationContextsFactoryBean">
<property name="resources" value="classpath*:/META-INF/spring/jobs/*.xml" />
</bean>
</property>
<property name="jobLoader">
<bean class="org.springframework.batch.core.configuration.support.DefaultJobLoader">
<property name="jobRegistry" ref="jobRegistry" />
</bean>
</property>
</bean>
<bean id="jobRegistry"
class="org.springframework.batch.core.configuration.support.MapJobRegistry" />
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="jobDetails">
<list>
<ref bean="dailyTranCountJobDetail" />
<ref bean="bulletinBarMsgUpdateJobDetail" />
<ref bean="updateLovCacheJobDetail" />
</list>
</property>
<property name="triggers">
<list>
<ref bean="dailyTranCountCronTrigger" />
<ref bean="bulletinBarMsgUpdateCronTrigger" />
<ref bean="updateLovCacheCronTrigger" />
</list>
</property>
</bean>
<!-- scheduling properties -->
<util:properties id="batchProps" location="classpath:batch.properties" />
<context:property-placeholder properties-ref="batchProps" />
<!-- triggers -->
<bean id="dailyTranCountCronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="dailyTranCountJobDetail" />
<property name="cronExpression" value="#{batchProps['cron.dailyTranCounts']}" />
</bean>
<bean id="bulletinBarMsgUpdateCronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="bulletinBarMsgUpdateJobDetail" />
<property name="cronExpression" value="#{batchProps['cron.bulletinBarUpdateMsg']}" />
</bean>
<bean id="updateLovCacheCronTrigger" class="org.springframework.scheduling.quartz.CronTriggerBean">
<property name="jobDetail" ref="updateLovCacheJobDetail" />
<property name="cronExpression" value="#{batchProps['cron.updateLovCache']}" />
</bean>
<!-- job detail -->
<bean id="dailyTranCountJobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.myer.reporting.batch.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="job-daily-tran-counts" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
</property>
</bean>
<bean id="bulletinBarMsgUpdateJobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.myer.reporting.batch.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="job-bulletin-bar-msg-update" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
</property>
</bean>
<bean id="updateLovCacheJobDetail" class="org.springframework.scheduling.quartz.JobDetailBean">
<property name="jobClass" value="com.myer.reporting.batch.JobLauncherDetails" />
<property name="group" value="quartz-batch" />
<property name="jobDataAsMap">
<map>
<entry key="jobName" value="job-update-lov-cache" />
<entry key="jobLocator" value-ref="jobRegistry" />
<entry key="jobLauncher" value-ref="jobLauncher" />
</map>
</property>
</bean>
</beans>
There are a few things this could be:
Where is the XML file you reference above located? It needs to be the META-INF/spring/batch/jobs directory in your WAR file (that's where Spring Batch Admin will look).
Don't configure common components in your XML file. That includes the jobLauncher, jobRepository, jobExplorer, jobLoader, or jobRegistry. That being said, I don't see an actual job defined in your XML file. The XML file needs one of those ;)
You can read more about adding your own job definitions to Spring Batch Admin: http://docs.spring.io/spring-batch-admin/reference/jobs.html#Add_your_Own_Jobs_For_Launching_in_the_UI
Is there any way in Spring to set it up such that an authoritiespopulator will look in more than one location?
<bean id="authoritiesPopulator" class="org.springframework.security.ldap.populator.DefaultLdapAuthoritiesPopulator">
<constructor-arg ref="contextSource" />
<constructor-arg value="CN=Users" />
<property name="groupRoleAttribute" value="CN" />
<property name="searchSubtree" value="true" />
<property name="rolePrefix" value="" />
<property name="convertToUpperCase" value="false" />
</bean>
This is the general idea, but there are also groups in CN=OtherGroups, and right now they don't get loaded (obviously). However, setting the groupsearchbase (the 2nd constructor arg) to value="" results in an error:
Unprocessed Continuation Reference(s); remaining name ''
Any ideas?
Answer: set referral to follow.
<bean id="contextSource" class="org.springframework.security.ldap.DefaultSpringSecurityContextSource">
....
<property name="baseEnvironmentProperties">
<map>
<entry key="java.naming.referral" value="follow" />
</map>
</property>
</bean>