Spring Batch BeanWrapperFieldExtractor for large number of fields - spring

I am in process of writing a Spring Batch application that reads a CSV file, does some transforming and writes a modified CSV to be sent to another batch process.
My writer configuration looks like this:
<beans:property name="lineAggregator">
<beans:bean class="org.springframework.batch.item.file.transform.FormatterLineAggregator">
<beans:property name="fieldExtractor">
<beans:bean class="org.springframework.batch.item.file.transform.BeanWrapperFieldExtractor">
<beans:property name="names" value="column1, column2, column3, column4 ------ 322 fields " />
</beans:bean>
</beans:property>
<beans:property name="format" value="%-8s%-12s%-11s%-16s" ----322 fields />
</beans:bean>
</beans:property>
I have to write around 322 fields. I am unable to get any FormatterLineAggregator to do my work. If I write the format like
<property name="format" value="%s;%s;%s;%s;%s;%s;%s;%f;%f;%s;%f;%f;%td.%tm.%​tY;%td.%tm.%<‌​tY;%s;%td.%tm.%&‌​lt;tY;%s;%s;%s;%s;%t‌​d.%tm.%tY" /> ,
its getting really messy and its tough to make sure all fields are correct.
I thought of 3 different solutions:
Either go with the approach above.
Write a CustomEditorFieldsExtractor but don't know what to write in the class and how to format the fields (preferred).
Use a "non-standard" BeanIO framework jar but I fear my client won't agree to this solution.
Can someone please provide some inputs. Appreciate your help!

You can proceed with solution #2 in this way:
Externalize how to format every property of you bean class (eg, in XML or text file)
Write a custom LineAggregator and make it works coupled with directive at point 1
class Aggregator implements LineAggregator<T> {
private Map<String, String> propertyFormat;
public String aggregate(T item) {
final StringBuilder sb = new StringBuilder();
for(final String property : propertyFormat.keySet()) {
final String format = propertyFormat.get(property);
final Object propertyValue = /* Extract property from item using Spring beans */;
sb.append(String.format(format, propertyValue));
}
return sb.toString();
}
}

Related

Change configuration on demand without restarting container

Spring MVC + Java 8 + Tomcat 8 stack
I am maintaining my configuration in yaml and flattening the properties using Spring's PropertyPlaceholderConfigurer and maintaining the configuration in a bean.
Today, it has a inherent problem as I am required to restart the server whenever there is a change to the YML files.
I believe there are ways to refresh the bean without restart, but my main concern is how to do in fail safe manner.
Lets assume, there was a request and that time the config was A, and then we refresh the configuration so now its B, but if any subsequent user request was dependent on the configuration, then it will blow up.
Add this configuration to your servlet-context.xml to catch property changes on the fly:
<context:property-placeholder
location="file:${A_CONFIG_LOCATION}/configuration.properties" />
<beans:bean id="propertiesLoader"
class="org.springframework.context.support.ReloadableResourceBundleMessageSource">
<beans:property name="cacheSeconds" value="1" />
<beans:property name="basenames">
<beans:list>
<beans:value>file:${A_CONFIG_LOCATION}/configuration
</beans:value>
</beans:list>
</beans:property>
</beans:bean>
And then you can read property values like this:
#Component
public class PropertiesReader {
private String value = "some_default_value";
#Autowired
MessageSource propertiesLoader;
public String getValue() {
value = propertiesLoader.getMessage("configuration.value", null, null);
return value;
}
}

How to use MappingJackson2HttpMessageConverter to convert Maps and List correctly?

In my Spring MVC project MappingJackson2HttpMessageConverter configured as follows:
<bean id="jsonMessageConverter" class="org.springframework.http.converter.json.MappingJackson2HttpMessageConverter"></bean>
<bean class="org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter">
<property name="messageConverters">
<list>
<ref bean="jsonMessageConverter" />
</list>
</property>
</bean>
For plain simple Java POJO Beans, serialization works well. All good. But, i also have Beans that include Maps that needed to be serialized as well, and it's fails (JsonMappingException).
From Jackson instructions, I know that to solve this what needed is to indicate the actual type for the object mapper. It looks like that:
Map<String, ResultValue> results = mapper.readValue(jsonSource,
new TypeReference<Map<String, ResultValue>>() { } );
How can be the same configuration be done to MappingJackson2HttpMessageConverter (with based on Jackson2 object mapper)?
It's not clear from your question what doesn't work, but I'm guessing that you have a bean that contains java.util.Map or java.util.List property and you're getting com.fasterxml.jackson.databind.JsonMappingException when deserializing that bean.
In that case you can give hints on fields with #JsonDeserialize annotation. So for instance if you have a java.util.Map<String, ResultValue> field, you can annotate it like:
#JsonDeserialize(keyAs = String.class, contentAs = ResultValue.class)
public Map<String, ResultValue> map;

Can I combine #controller and XML bean mapping in spring?

I currently have a #Controller declared in spring and have a bunch of mappings done like so:
#RequestMapping(value = "foo", method = RequestMethod.GET)
public ModelAndView foo() {
ModelAndView mav = new ModelAndView(
"myjsp");
return mav;
}
However every time I want to add a simple JSP mapping I need to recompile and build a new war and deploy.
This isnt so bad except sometimes other members of the team have requests and it would be easier if they can just go into the test env and create the mapping themselves without having to recompile.
I know that you can do similar mapping using xml but can I do this at the same time that I have the #Controller defined?
Like in the example above how could I define that mapping in XML rather than in java?
or say I needed foo2 to map to myjsp2.jsp
I am using spring MVC 3.2
Look into BeanNameUrlHandlerMapping which allows you specify url patterns for controllers in your configuration. Documentation
Example
<beans>
<bean id="handlerMapping" class="org.springframework.web.servlet.handler.BeanNameUrlHandlerMapping"/>
<bean name="/editaccount.form" class="org.springframework.web.servlet.mvc.SimpleFormController">
<property name="formView" value="account"/>
<property name="successView" value="account-created"/>
<property name="commandName" value="account"/>
<property name="commandClass" value="samples.Account"/>
</bean>
<beans>

Using encoded password for the datasource used in spring applicationContext.xml

I want to keep encoded password in my below mentioned springApplicationContext.xml
Is there any way to achieve this?
presently I have configured all properties using property-placeholder
as shown below but the raw password is still open in my database.properties
springApplicationContext.xml
<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<beans:property name="driverClassName"><beans:value>${db.driverClassName}</beans:value></beans:property>
<beans:property name="url"><beans:value>${db.url}</beans:value></beans:property>
<beans:property name="username"><beans:value>${db.username}</beans:value></beans:property>
<beans:property name="password"><beans:value>${db.password}</beans:value></beans:property>
</beans:bean>
but actual values are present in my database.properties
db.driverClassName=com.mysql.jdbc.Driver
db.url=jdbc:mysql://localhost/myDB
db.username=root
db.password=root
I want something like below:
springApplicationContext.xml (same as above)
<beans:bean id="dataSource" class="org.apache.commons.dbcp.BasicDataSource" destroy-method="close">
<beans:property name="driverClassName"><beans:value>${db.driverClassName}</beans:value></beans:property>
<beans:property name="url"><beans:value>${db.url}</beans:value></beans:property>
<beans:property name="username"><beans:value>${db.username}</beans:value></beans:property>
<beans:property name="password"><beans:value>${db.password}</beans:value></beans:property>
</beans:bean>
But password property value should be in encripted format in my database.properties
db.driverClassName=com.mysql.jdbc.Driver
db.url=jdbc:mysql://localhost/myDB
db.username=root
db.password=3g6n72ef8x (using any encription method).
and my dataSource internally decrypt the password before making new DB connection.
Highly appreciate for any help/suggestion in this.
Its might be funny that I am answering to my own question. but still I just wanted to tell my solution, others who might have faced same kind of issue..
for simplicity I have used BASE64Encoder & BASE64Decoder. later I will modify my code to use a secure/better encryption/decryption algorithm.
I have encoded my database password(ex: root for my case) by using the below code:
private String encode(String str) {
BASE64Encoder encoder = new BASE64Encoder();
str = new String(encoder.encodeBuffer(str.getBytes()));
return str;
}
and placed the encoded password in my database.properties file like below:
before
db.driverClassName=com.mysql.jdbc.Driver
db.url=jdbc:mysql://localhost/myDB
db.username=root
db.password=root
after
db.driverClassName=com.mysql.jdbc.Driver
db.url=jdbc:mysql://localhost/myDB
db.username=root
db.password=cm9vdA== (Note: encoded 'root' by using BASE64Encoder)
Now I have written a wrapper class for org.apache.commons.dbcp.BasicDataSource
and overridden setPassword() method:
import java.io.IOException;
import org.apache.commons.dbcp.BasicDataSource;
import sun.misc.BASE64Decoder;
public class MyCustomBasicDataSource extends BasicDataSource{
public CustomBasicDataSource() {
super();
}
public synchronized void setPassword(String encodedPassword){
this.password = decode(encodedPassword);
}
private String decode(String password) {
BASE64Decoder decoder = new BASE64Decoder();
String decodedPassword = null;
try {
decodedPassword = new String(decoder.decodeBuffer(password));
} catch (IOException e) {
e.printStackTrace();
}
return decodedPassword;
}
}
This way I am decoding(BASE64Decoder) the encoded password provided in database.properties
and also modified the class attribute of my dataSource bean mentioned in springApplicationContext.xml file.
<beans:bean id="dataSource" class="edu.config.db.datasource.custom.MyCustomBasicDataSource" destroy-method="close">
<beans:property name="driverClassName"><beans:value>${db.driverClassName}</beans:value></beans:property>
<beans:property name="url"><beans:value>${db.url}</beans:value></beans:property>
<beans:property name="username"><beans:value>${db.username}</beans:value></beans:property>
<beans:property name="password"><beans:value>${db.password}</beans:value></beans:property>
Thanks.
Create customized PropertyPlaceHolderConfigurer extending Spring PropertyPlaceHolderConfigurer
public class PropertyPlaceholderConfigurer extends
org.springframework.beans.factory.config.PropertyPlaceholderConfigurer {
#Override
protected String convertPropertyValue(final String originalValue) {
if (originalValue.startwith("SomeText:")) {
//Apply the decryption logic
...
}
}
}
You can encrypt the properties and append SomeText:. Use this customized PropertyPlaceHolderConfigurer to load the properties
I'd like to look at the larger picture here: why do you want to encrypt values in your properties file? What is your scenario where unauthorized people have access to your properties file?
A usual technique to deal with this larger problem of storing production credentials is to make credentials a part of your environment as opposed to part of your source code. Here are some ways to do this:
Placing the properties file (with plaintext passwords) on the classpath of the web server in production, this way access to that password is controlled by access to the production machine.
Store properties in web.xml (context-param with param-name), again this file is part of the environment in which you run your code and not distributed with your code - access to that file is controlled by access to the machine.
Use JNDI and configure that resource in your application server.
Create a wrapper class implementing the Datasource interface which delegates it's method calls to the underlying datasource but decrypts the password before doing so.
If you are using tomcat connection pool as data source, here is an implementation
http://www.jdev.it/encrypting-passwords-in-tomcat/
Create a class which extends org.apache.tomcat.jdbc.pool.DataSourceFactory and configure it in the server.xml
Update:
The newer way is to use Jasypt: http://www.jasypt.org/encrypting-texts.html

Spring MVC custom errors and internationalization

In my web application, I handle errors with annotations. Everything works fine and I can use custom messages via the "message" parameter.
#Digits(fraction = 0, integer = 3, message="my custom error message...")
private String price;
Now I'm trying to internationalize this message with a .properties files, but I certainly miss something and I can't get it to work.
My spring config :
<beans:bean id="messageSource" class="org.springframework.context.support.ReloadableResourceBundleMessageSource">
<beans:property name="basenames" value="classpath:i18n/messages, classpath:i18n/errors" />
<beans:property name="defaultEncoding" value="UTF-8" />
</beans:bean>
<beans:bean name="validator" class="org.springframework.validation.beanvalidation.LocalValidatorFactoryBean">
<beans:property name="validationMessageSource">
<beans:ref bean="messageSource" />
</beans:property>
</beans:bean>
<beans:bean id="localeResolver" class="org.springframework.web.servlet.i18n.FixedLocaleResolver">
<beans:property name="defaultLocale" value="fr" />
</beans:bean>
My new bean :
#Digits(fraction = 0, integer = 3)
private String price;
My "errors_fr.properties" file. I've already tried everything :
Digits.myBean.myNestedBean.price = my custom error message...
Digits.myNestedBean.price = my custom error message...
javax.validation.constraints.Digits.myNestedBean.price = my custom error message...
I always get the same generic message from spring, it's like as spring doesn't detect my .properties file. By the way, the message keys above can be found in the BindingResult object when debugging.
What am I missing here ?
Notice that I already have internationalized messages in my jsp's (in the "messages_fr.properties" file) and they work fine.
I had a similar problem in my application, and I hope that this can help you.
As discussed in this thread, http://forum.springsource.org/showthread.php?73240-Roo-JSR-303-Validations-and-Localization, you need to:
define the error messages referred by the annotation in file ValidationMessages.properties
in your annotation, refer to the error message key enclosed in curly brackets:
#Digits(fraction = 0, integer = 3, message="{message.key}")
Hope this helps.

Resources