How to make GridCacheStore-backed get() store its result in the cache - caching

When I use a GridCacheStore-backed cache, the first get() may take a long time, because internally the GridCacheStore will perform calculations / slow search etc.
The second get() on that node will be quick because the result has been cached by that node.
However, the get() on other node is still slow because the result is not being replicated. How to make this replicated? I've set cacheMode=REPLICATED.
My config:
<bean parent="cache-template">
<property name="name" value="yagoEntityByLabel" />
<property name="cacheMode" value="REPLICATED" />
<property name="atomicityMode" value="ATOMIC" />
<property name="distributionMode" value="NEAR_PARTITIONED" />
<property name="backups" value="1" />
<property name="store">
<bean class="id.ac.itb.ee.lskk.lumen.core.yago.YagoEntityByLabelCacheStore" />
</property>
<property name="swapEnabled" value="false" />
<property name="evictionPolicy">
<bean class="org.gridgain.grid.cache.eviction.lru.GridCacheLruEvictionPolicy">
<property name="maxSize" value="10000" />
</bean>
</property>
</bean>
The workaround is to not use GridCacheStore-backed but to use put() instead, but there's a lot of typing and it's not atomic, since the logic will be:
#Nullable value = cache.get(key);
if (value == null) {
value = calculateHeavily(key);
cache.put(key, value);
}

GridGain (as most other data grids) does not replicate on Get operation - it simply loads data into cache. Your workaround using Put operation is good.
To make it atomic (if you need to do so), you can wrap your code into transaction, like so:
try (GridCacheTx tx = cache.txStart()) {
#Nullable value = cache.get(key);
if (value == null) {
value = calculateHeavily(key);
cache.put(key, value);
}
tx.commit();
}

Related

Spring Batch - Last item from the reader alone is getting updated

I have to read from a file (FlatFile) and update a column if that ID present in the file matches the id in the column.The file is being read properly but only the last id value is getting updated here . Please find the snippet
Job-Config.xml
<bean id="abcitemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="file:datafile/outputs/ibdData.txt" />
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="names" value="ID,NAM,TYPE" />
<property name="delimiter" value="|"/>
</bean>
</property>
<property name="fieldSetMapper">
<bean class="com.pershing.intraware.springbatch.mapper.abcFieldsetMapper" />
</property>
</bean>
</property>
</bean>
<bean id="abcitemWriter" class="org.springframework.batch.item.database.JdbcBatchItemWriter" scope="step">
<property name="dataSource" ref="dataSource" />
<property name="sql"><value>UPDATE TEST_abc SET BIZ_ARNG_CD = CASE WHEN ID IN (SELECT ID FROM TEST_abc WHERE ID= ? and MONTH=(to_char(sysdate, 'MM')) AND YR =(to_char(sysdate, 'YY'))) THEN 'Y' ELSE 'N' END</value></property>
<!-- It will take care matching between object property and sql name parameter -->
<property name="itemPreparedStatementSetter" ref="testPrepStatementSetter" />
</bean>
</beans>
Setter.java
public class IDItemPreparedStatementSetter implements ItemPreparedStatementSetter<Test> {
#Override
public void setValues(Test item, PreparedStatement ps) throws SQLException {
// TODO Auto-generated method stub
ps.setString(1, item.getID());
}
}
Your query is updating each row of database every time it is fired. You need to restrict that. Currently; it must be setting the BIZ_ARNG_CD to 'Y' for records with ID equal to the ID of the last record passed to the writer.
You can fix this in 2 ways -
Default the database column to 'N' and don't set it to 'N' in the update statement
Add where clause in update script ( BIZ_ARNG_CD != 'Y')

Reading in UTF-8 encoding in Java results in issues at only some places

I am reading a UTF_8 encoded file byte by byte in Java Spring Batch and saving the content in mongodb. The reading is delegated to FlatFileItemReader and the encoding is injected into the bean as UTF-8.
What I do see is that only at one or two specific places, the umlaut character is getting stored with an encoding problem in mongodb. I verified that the content in the original file is okay. What is interesting is that everywhere else, the umlaut character in the original file is interpreted correctly and stored correctly in mongodb.
Could you help with this?
<bean id="recordReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="resource" value="file:#{jobExecutionContext['inputfile']}" />
<property name="linesToSkip" value = "1" />
<property name= "encoding" value ="UTF-8"/>
<property name="lineMapper">
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper" >
<property name = "lineTokenizer">
<bean class = "com.xyz.NoQuoteDelimitedLineTokenizer">
<property name = "strict" value = "false" />
<property name = "names" value = "Col1,Col2,Col3,Col4,Col5,Col6,Col7,Col8" />
<property name="delimiter">
<util:constant static-field="org.springframework.batch.item.file.transform.DelimitedLineTokenizer.DELIMITER_TAB" />
</property>
</bean>
</property>
<property name = "fieldSetMapper">
<bean class = "com.xyz.MyFieldSetMapper" >
<property name = "ctx" value="#{jobExecutionContext['contextBean']}"/>
</bean>
</property>
</bean>
</property>
</bean>
FieldSetMapper
private void validateFields (String id, ConsumerDO ci, FieldSet fieldset_p) throws Exception
{try {
ci.setCaption(fieldset_p.readString("newCaption"));
}
catch (Exception e) {
}
}

spring quartz doesn't work

I've got some controller, which performs some task and Tomcat. It works fine.
I want some job that will work every 5 seconds during its work.
I've created xml file quartz-config.xml with data
<bean class="org.springframework.scheduling.quartz.SchedulerFactoryBean">
<property name="triggers">
<list>
<ref bean="someTrigger"/>
</list>
</property>
<property name="jobDetails">
<list>
<ref bean="someJob" />
</list>
</property>
</bean>
<bean id="someTrigger"
class="org.springframework.scheduling.quartz.CronTriggerFactoryBean">
<property name="jobDetail" ref="someJob"/>
<property name="cronExpression" value="0 0/5 * * * ?" />
</bean>
<bean id="someJob" class="org.springframework.scheduling.quartz.JobDetailFactoryBean">
<property name="name" value="someJob" />
<property name="jobClass" value="com.service.scheduler.SomeJob" />
</bean>
and added path to xml file in my spring-context.xml.
I have created class
public class SomeJob extends QuartzJobBean {
public void execute(JobDataMap jobDataMap) throws JobExecutionException {
executeInternal(null);
}
#Override
protected void executeInternal(JobExecutionContext context) throws JobExecutionException {
System.out.println("SCHEDULER WORKS!");
}
}
But it doesn't do anything in console. I am only getting messages batch acquisition of 0 triggers.
Why? What's the problem?
Check quart web page.
org.quartz.scheduler.batchTriggerAcquisitionMaxCount
The maximum number of triggers that a scheduler node is allowed to acquire (for firing) at once. Default value is 1. The larger the number, the more efficient firing is (in situations where there are very many triggers needing to be fired all at once) - but at the cost of possible imbalanced load between cluster nodes.
http://quartz-scheduler.org/documentation/quartz-2.x/configuration/ConfigMain][1]

Spring 3.2 with MVC, ContentNegotation, REST and PDF Generator

Let's say, I have a REST styled controller mapping
#RequestMapping(value="users", produces = {MediaType.APPLICATION_JSON_VALUE})
public List<User> listUsers(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
return service.loadUsers(query, offset, limit);
}
Serving JSON (or even XML) is not an issue, this is easy using ContentNegotation and MessageConverters
<bean id="contentNegotiationManager" class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
<property name="favorPathExtension" value="true" />
<property name="favorParameter" value="false" />
<property name="ignoreAcceptHeader" value="false" />
<property name="mediaTypes" >
<value>
html=text/html
json=application/json
xml=application/xml
</value>
</property>
</bean>
Now, I need to add support for PDF. Naturally, I want to use (Spring) MVC + REST as much as possible. Most examples I have found implement this with an explicit definition not using REST style, e.g.
#RequestMapping(value="users", produces = {"application/pdf"})
public ModelAndView listUsersAsPdf(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
List<User> users = listUsers(offset, limit, query); // delegated
return new ModelAndView("pdfView", users);
}
That works, but is not very comfortable because for every alternate output (PDF, Excel, ...) I would add a request mapping.
I have already added application/pdf to the content negotation resolver; unfortunately any request with a suffix .pdf or the Accept-Header application/pdf were be responded with 406.
What is the ideal setup for a REST/MVC style pattern to integrate alternate output like PDF?
You can create a WEB-INF/spring/pdf-beans.xml like below.
<bean id="listofusers" class="YourPDFBasedView"/>
And your controller method will return view name as listofusers.
#RequestMapping(value="users")
public ModelAndView listUsersAsPdf(#ReqestParams Integer offset, #ReqestParams Integer limit, #ReqestParams String query) {
List<User> users = listUsers(offset, limit, query); // delegated
return new ModelAndView("listofusers", users);
}
And you can use contentNegotiationViewResolver in this way:
<bean class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="order" value="1"/>
<property name="location" value="WEB-INF/spring/pdf-views.xml"/>
</bean>
<!--
View resolver that delegates to other view resolvers based on the content type
-->
<bean class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<!-- All configuration is now done by the manager - since Spring V3.2 -->
<property name="contentNegotiationManager" ref="cnManager"/>
</bean>
<!--
Setup a simple strategy:
1. Only path extension is taken into account, Accept headers are ignored.
2. Return HTML by default when not sure.
-->
<bean id="cnManager" class="org.springframework.web.accept.ContentNegotiationManagerFactoryBean">
<property name="ignoreAcceptHeader" value="true"/>
<property name="defaultContentType" value="text/html" />
</bean>
For JSON: Create a generic JSON view resolver like below and register it as bean in context file.
public class JsonViewResolver implements ViewResolver {
/**
* Get the view to use.
*
* #return Always returns an instance of {#link MappingJacksonJsonView}.
*/
#Override
public View resolveViewName(String viewName, Locale locale) throws Exception {
MappingJacksonJsonView view = new MappingJacksonJsonView();
view.setPrettyPrint(true); // Lay the JSON out to be nicely readable
return view;
}
}
Same for XML:
public class MarshallingXmlViewResolver implements ViewResolver {
private Marshaller marshaller;
#Autowired
public MarshallingXmlViewResolver(Marshaller marshaller) {
this.marshaller = marshaller;
}
/**
* Get the view to use.
*
* #return Always returns an instance of {#link MappingJacksonJsonView}.
*/
#Override
public View resolveViewName(String viewName, Locale locale)
throws Exception {
MarshallingView view = new MarshallingView();
view.setMarshaller(marshaller);
return view;
}
}
and register above xml view resolver in context file like this:
<oxm:jaxb2-marshaller id="marshaller" >
<oxm:class-to-be-bound name="some.package.Account"/>
<oxm:class-to-be-bound name="some.package.Customer"/>
<oxm:class-to-be-bound name="some.package.Transaction"/>
</oxm:jaxb2-marshaller>
<!-- View resolver that returns an XML Marshalling view. -->
<bean class="some.package.MarshallingXmlViewResolver" >
<constructor-arg ref="marshaller"/>
</bean>
You can find more information at this link:
http://spring.io/blog/2013/06/03/content-negotiation-using-views/
Using all view resolver techniques, you can avoid writing duplicate methods in controller, such as one for xml/json, other for excel, other for pdf, another for doc, rss and all.
Knalli, if you replace #ResponseBody with ModelAndView(), you can achieve both the features.
Is there any reason you want to keep #ResponseBody ? I just want to know if I am missing anything, just want to learn.
Other option is to write HttpMessageConverters then:
Some samples are here.
Custom HttpMessageConverter with #ResponseBody to do Json things
http://www.javacodegeeks.com/2013/07/spring-mvc-requestbody-and-responsebody-demystified.html
This is working sample. I have configured contentnegotiationviewresolver for this, and give highest order. After that I have ResourceBundleViewResolver for JSTL and Tiles View, then XmlViewResolver for excelResolver, pdfResolver, rtfResolver. excelResolver, pdfResolver, rtfResolver. XmlViewResolver and ResourceBundleViewResolver works only with MAV only, but MappingJacksonJsonView and MarshallingView takes care for both MAV and #ResponseBody return value.
<bean id="contentNegotiatingResolver" class="org.springframework.web.servlet.view.ContentNegotiatingViewResolver">
<property name="order"
value="#{T(org.springframework.core.Ordered).HIGHEST_PRECEDENCE}" />
<property name="mediaTypes">
<map>
<entry key="json" value="application/json" />
<entry key="xml" value="application/xml" />
<entry key="pdf" value="application/pdf" />
<entry key="xlsx" value="application/vnd.openxmlformats-officedocument.spreadsheetml.sheet" />
<entry key="doc" value="application/msword" />
</map>
</property>
<property name="defaultViews">
<list>
<!-- JSON View -->
<bean class="org.springframework.web.servlet.view.json.MappingJacksonJsonView" />
<!-- XML View -->
<bean class="org.springframework.web.servlet.view.xml.MarshallingView">
<constructor-arg>
<bean id="jaxbMarshaller" class="org.springframework.oxm.jaxb.Jaxb2Marshaller">
<property name="classesToBeBound">
<list>
<value>Employee</value>
<value>EmployeeList</value>
</list>
</property>
</bean>
</constructor-arg>
</bean>
</list>
</property>
<property name="ignoreAcceptHeader" value="true" />
</bean>
<bean class="org.springframework.web.servlet.view.ResourceBundleViewResolver"
id="resourceBundleResolver">
<property name="order" value="#{contentNegotiatingResolver.order+1}" />
</bean>
<bean id="excelResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-excel-views.xml</value>
</property>
<property name="order" value="#{resourceBundleResolver.order+1}" />
</bean>
<bean id="pdfResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-pdf-views.xml</value>
</property>
<property name="order" value="#{excelResolver.order+1}" />
</bean>
<bean id="rtfResolver" class="org.springframework.web.servlet.view.XmlViewResolver">
<property name="location">
<value>/WEB-INF/tiles/spring-rtf-views.xml</value>
</property>
<property name="order" value="#{excelResolver.order+1}" />
</bean>
And our XMLViewResolver spring-pdf-views.xml looks like this.
<bean id="employees"
class="EmployeePDFView"/>
And EmployeePDFView will have code for generating pdf and writing pdf byte stream on Response object. This will resolve to rest url that will end with .pdf extension, and when you return MAV with "employees" id.

How can I get the number of line and the filename in itemProcessor - spring batch

I am using spring batch to parse my files. In ItemProcessor I validate if the incoming fields are correct. If they are not I want to throw a ValidationException and log to a file the corresponding row which has the incorrect fields. So, how can I find the number of line and the filename in ItemProcessor?
Without seeing you ItemReader config I can't really be sure but if you are using something like FlatFileItemReader to parse a csv, if in strict mode it will validate the number of columns.
Assuming you reader looks like this, that is:
<bean id="iItemReader" class="org.springframework.batch.item.file.FlatFileItemReader" scope="step">
<property name="linesToSkip" value="1"/>
<property name="comments" value="#" />
<property name="encoding" value="UTF-8"/>
<property name="lineMapper" >
<bean class="org.springframework.batch.item.file.mapping.DefaultLineMapper">
<property name="lineTokenizer">
<bean class="org.springframework.batch.item.file.transform.DelimitedLineTokenizer">
<property name="delimiter" value=","/>
<property name="names">
<list >
<value>First_Field</value>
<value>Second_Field</value>
</list>
</property>
<property name="strict" value="true"/>
</bean>
</property>
<property name="fieldSetMapper">
<bean class="uk.co.package.FieldSetMapper">
<property name="dateFormat" value="yyyy-MM-dd HH:mm:ss"/>
</bean>
</property>
</bean>
</property>
</bean>
It will throw a FlatFileParseException for any lines that can't be processed. This includes the line number and can be handled in a listener.
As for the line number, you might build your own LineMapper and then store the line-number in your business object. An example in which I store the line unprocessed (as-is) together with the line number:
DefaultLineMapper<OneRow> lineMapper = new DefaultLineMapper<OneRow>() {
#Override
public OneRow mapLine(String line, int lineNumber) throws Exception {
return new OneRow(lineNumber, line);
}
};
Of course you can already map your Object, I had the need to have the whole line unprocessed as input to my Processors.
As a reference with the same idea: https://stackoverflow.com/a/23770421/5658642

Resources