I'm using a customReader that implements ItemReader. My reader take information from xls and treat it row by row. My constructor take Iterator value that will be read on each read() iteration. I'm trying to find a suitable way to manage exception. I look through SkipListener and ReaderListener onReadError. But Ican't use either because my exception will be thrown in the constructor before attempting read() method.
Is there any way to do that in order to allow me to manage properly action1/2/3 respectively to exceptions ?
#Component
public class CustomReaderFile implements ItemReader<Row> {
private final Iterator<Row> data;
private static final String FILE_NAME = "";
public CustomReaderFile() throws Exception {
this.data = iteratorFromXls();
}
#Override
public Row read() throws Exception {
if (this.data.hasNext()) {
return this.data.next();
} else {
return null;
}
}
public static Iterator<Row> iteratorFromXls() throws Exception {
Iterator<Row> iterator = null;
try {
FileInputStream excelFile = new FileInputStream(new File(FILE_NAME));
Workbook workbook = new XSSFWorkbook(excelFile);
Sheet dataTypeSheet = workbook.getSheetAt(0);
iterator = dataTypeSheet.iterator();
} catch (FileNotFoundException e) {
//action1
} catch (IOException e) {
//action2
} catch (NotOfficeXmlFileException e){
//action3
}
return iterator;
}
This is actually the reader's initialization code. The reading part is nothing more than calling .next on the iterator.
So I would make the reader implement ItemStreamReader and put the initialization code in the open method, in which you can throw an exception to signal to Spring Batch that the reader's initialization has failed.
Related
Java 8
import java.util.zip.GZIPOutputStream;
import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
private void createFile(final String json) throws IOException {
final String fileName = getConfigFileName(this.getSomePath());
GZIPOutputStream out = null;
try {
out = new GZIPOutputStream(new FileOutputStream(fileName + ".gz"));
out.write(json.getBytes());
} catch (IOException e) {
throw e;
} finally {
try {
if (out != null) {
out.finish();
out.close();
}
} catch (IOException e) {
LOGGER.error("createFile: IOException while closing resources", e);
}
}
}
Nice. This work fine.
Now I want to use try-with-resource
private void createFile(final String json) throws IOException {
final String fileName = getConfigFileName(this.getSomeFile());
try (GZIPOutputStream out = new GZIPOutputStream(new FileOutputStream(fileName + ".gz"))) {
out.write(json.getBytes());
} catch (IOException e) {
throw e;
} finally {
try {
if (out != null) {
out.finish();
}
} catch (IOException e) {
LOGGER.error("createFile: IOException while closing resources", e);
}
}
}
But now I get error in this line:
if (out != null) {
Error is:
out cannot be resolved
I know this error is rise because variable out is on finally section.
But how I can use try-with-resources and execute method out.finish ?
From a technical perspective - a variable declared in the try argument isn't available in the finally clause, as you've seen. The good news here is that from a function perspective - finish() shouldn't be in the finally block anyway. finish is part of the positive (a.k.a "happy") flow, and should only be called when you're done writing to the stream. In other words, if the write operation failed and an exception was thrown, you shouldn't call finish anyway.
To make a long story short - move the finish call inside the try block:
Side note: Since your method throws an IOException, there's no reason to catch the exception and rethrow it. You can clean up the code by allowing it to be thrown from the method call directly:
private void createFile(final String json) throws IOException {
final String fileName = getConfigFileName(this.getSomeFile());
try (GZIPOutputStream out = new GZIPOutputStream(new FileOutputStream(fileName + ".gz"))) {
out.write(json.getBytes());
out.finish();
}
}
MultiResourceItemReader reads all files sequentially.
I want once one file read completely, it should call processor/writer.it should not read next file.
Since file content is not constant, i can't go with chunk size.
Any idea on chunk policy to decide end of file content?
I think you should write a step which read/process/write only one file with a "single file item reader" (like FlatFileItemReader). And repeat the step while there are files remainig.
Spring batch gives you a feature to do so : conditional flows and in particular the programmatic flow decision which gives you a smart way to decide when to stop a loop between steps (when there is not file any more)
And since you will not be able to give a constant input file name to your reader, you should also have a look at Late binding section.
Hope this will be enough to help you. Please, make comments if you need more details.
Using MultiResourceItemReader, assigning multiple file reasources.
Using custom file reader as delegate, reading a file completely
For reading file completely, come up with a logic
#Bean
public MultiResourceItemReader<SimpleFileBean> simpleReader()
{
Resource[] resourceList = getFileResources();
if(resourceList == null) {
System.out.println("No input files available");
}
MultiResourceItemReader<SimpleFileBean> resourceItemReader = new MultiResourceItemReader<SimpleFileBean>();
resourceItemReader.setResources(resourceList);
resourceItemReader.setDelegate(simpleFileReader());
return resourceItemReader;
}
#Bean
SimpleInboundReader simpleFileReader() {
return new SimpleInboundReader(customSimpleFileReader());
}
#Bean
public FlatFileItemReader customSimpleFileReader() {
return new FlatFileItemReaderBuilder()
.name("customFileItemReader")
.lineMapper(new PassThroughLineMapper())
.build();
}
public class SimpleInboundReader implements ResourceAwareItemReaderItemStream{
private Object currentItem = null;
private FileModel fileModel = null;
private String fileName = null;
private boolean fileRead = false;
private ResourceAwareItemReaderItemStream<String> delegate;
public SimpleInboundReader(ResourceAwareItemReaderItemStream<String> delegate) {
this.delegate = delegate;
}
#Override
public void open(ExecutionContext executionContext) throws ItemStreamException {
delegate.open(executionContext);
}
#Override
public void update(ExecutionContext executionContext) throws ItemStreamException {
delegate.update(executionContext);
}
#Override
public void close() throws ItemStreamException {
delegate.close();
}
#Override
public void setResource(Resource resource) {
fileName = resource.getFilename();
this.delegate.setResource(resource);
}
String getNextLine() throws UnexpectedInputException, ParseException, NonTransientResourceException, Exception {
return delegate.read();
}
#Override
public SimpleFileBean read() throws Exception, UnexpectedInputException, ParseException, NonTransientResourceException {
SimpleFileBean simpleFileBean = null;
String currentLine = null;
currentLine=delegate.read();
if(currentLine != null) {
simpleFileBean = new SimpleFileBean();
simpleFileBean.getLines().add(currentLine);
while ((currentLine = getNextLine()) != null) {
simpleFileBean.getLines().add(currentLine);
}
return simpleFileBean;
}
return null;
}
}
I was wondering if anyone can help me with Apache FreeMarker? I'm trying to use a custom model but I can't figure it out.
Imagine I want to dump the result of a query (java ResultSet in a FreeMarker template). What is the best approach?
I have found on Google the class: ResultSetTemplateModel
import java.sql.ResultSet;
import freemarker.template.SimpleScalar;
import freemarker.template.TemplateHashModel;
import freemarker.template.TemplateModel;
import freemarker.template.TemplateModelException;
import freemarker.template.TemplateSequenceModel;
public class ResultSetTemplateModel implements TemplateSequenceModel {
private ResultSet rs = null;
public ResultSetTemplateModel(ResultSet rs) {
this.rs = rs;
}
public TemplateModel get(int i) throws TemplateModelException {
try {
rs.next();
} catch(Exception e) {
throw new TemplateModelException(e.toString());
}
TemplateModel model = new Row(rs);
return model;
}
public int size() throws TemplateModelException {
int size=0;
try {
rs.last();
size = rs.getRow();
rs.beforeFirst();
} catch (Exception e ) {
throw new TemplateModelException( e.toString());
}
return size;
}
class Row implements TemplateHashModel {
private ResultSet rs = null;
public Row(ResultSet rs) {
this.rs = rs;
}
public TemplateModel get(String s) throws TemplateModelException {
TemplateModel model = null;
try {
model = new SimpleScalar( rs.getString(s) );
} catch (Exception e) { e.printStackTrace(); }
return model;
}
public boolean isEmpty() throws TemplateModelException {
boolean isEmpty = false;
if ( rs == null ) { isEmpty = true; }
return isEmpty;
}
}
}
And I have a very simple class (I even made it easier than previous):
public static void main(String[] args) {
try {
Configuration cfg = new Configuration(Configuration.VERSION_2_3_27);
cfg.setTemplateExceptionHandler(TemplateExceptionHandler.RETHROW_HANDLER);
cfg.setClassForTemplateLoading(MyCLASS.class, "/");
StringWriter out = new StringWriter();
Map<String, Object> parameters = new TreeMap<>();
ResultSet rs = getResultSet("Select foo, bar FROM my_table");
parameters.put("hello", "World");
parameters.put("result", rs);
Template temp = cfg.getTemplate("template.txt");
temp.process(parameters, out);
System.out.println("out = " + out);
} catch (IOException | TemplateException e) {
e.printStackTrace();
}
}
My template
Hello ${hello}
<#-- how do I specify ResultSet columns here ?? -->
How can I use the custom template?? Any advice?? I know how to load the template file. But I don't know how to specify that it is a custom model in the template.
THank you guys for the support :)
There are two ways of using ResultSetTemplateModel for wrapping ResultSet-s:
Either extend DefaultObjectWrapper by overriding handleUnknownType, where you return new ResultSetTemplateModel((ResultSet) obj) if obj is a ResultSet, otherwise call super. Then use Configuration.setObjectWrapper to actually use it.
Or, add new ResultSetTemplate(rs) to parameters instead of rs; if something is already a TempalteModel, it will not be wrapped again. Note that if you get a ResultSet from somewhere else in the template, this approach will not work as it avoids your manual wrapping, so extending the DefaultObjectWrapper is what you want generally.
Note that the ResultSetTemplateModel implementation shown is quite limited. The ObjectWrapper should be passed to the constructor as well, and stored in a final field. Then, instead of new SimpleScalar( rs.getString(s) ) it should do objectWrapper.wrap(rs.getObject(s)).
I have this code,
SimpleJdbcCall sql = new SimpleJdbcCall(dataSource).withProcedureName(procName);
sql.execute(parameters);
And I believe that under the hood this uses a JDBC Statement. How can I get to that object from here? (I need to call the .getWarnings() method on the statement).
In other words how can I get SQLWarnings AND named parameters?
It took a lot of digging, but here is how you can get SQLWarnings (or Print statements) AND named parameters. I extended JdbcTemplate and overrode the handleWarnings() method, and then passed that into my SimpleJdbcCall.
public class JdbcTemplateLoggable extends JdbcTemplate{
List<String> warnings;
public JdbcTemplateLoggable(DataSource dataSource){
super(dataSource);
warnings = new ArrayList<String>();
}
protected void handleWarnings(Statement stmt){
try {
SQLWarning warning = stmt.getWarnings();
while(warning != null){
warnings.add(warning.getMessage());
warning = warning.getNextWarning();
}
} catch (SQLException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
public List<String> getWarnings(){
return warnings;
}
}
Then in my main program
JdbcTemplateLoggable template = new JdbcTemplateLoggable(dataSource);
SimpleJdbcCall sql = new SimpleJdbcCall(template).withProcedureName(procName);
sql.execute(parameters);
for(String s : template.getWarnings()){
log.info(s);
}
You should perhaps use JdbcTemplate directly, or subclass it for usage with your SimpleJdbcCall (instead of a DataSource). JdbcTemplate has a method execute(CallableStatementCreator, CallableStatementCallback), where a callback can be passed which gets the used Statement object.
You could override that method and wrap the passed callback with an own which stores the statement for later use.
public class CustomJdbcTemplate extends JdbcTemplate {
private CallableStatement lastStatement;
public CustomJdbcTemplate(DataSource dataSource) {
super(dataSource);
}
public CallableStatement getLastStatement() {
return lastStatement;
}
#Override
public <T> T execute(CallableStatementCreator csc, CallableStatementCallback<T> action) throws DataAccessException {
StoringCallableStatementCallback<T> callback = new StoringCallableStatementCallback<T>(action);
try {
return super.execute(csc, callback);
}
finally {
this.lastStatement = callback.statement;
}
}
private static class StoringCallableStatementCallback<T> implements CallableStatementCallback<T> {
private CallableStatementCallback<T> delegate;
private CallableStatement statement;
private StoringCallableStatementCallback(CallableStatementCallback<T> delegate) {
this.delegate = delegate;
}
#Override
public T doInCallableStatement(CallableStatement cs) throws SQLException, DataAccessException {
this.statement = cs;
return delegate.doInCallableStatement(cs);
}
}
}
Note that the statement will most probably be closed when you retrieve it later, so getWarnings() may cause errors, depending on used JDBC driver. So maybe you should store the warnings instead of the statement itself.
I am using Freemarker and DCEVM+HotSwapManager agent. This basically allows me to hotswap classes even when adding/removing methods.
Everything works like charm until Freemarker uses hotswapped class as model. It's throwing freemarker.ext.beans.InvalidPropertyException: No such bean property on me even though reflection shows that the method is there (checked during debug session).
I am using
final Method clearInfoMethod = beanWrapper.getClass().getDeclaredMethod("removeIntrospectionInfo", Class.class);
clearInfoMethod.setAccessible(true);
clearInfoMethod.invoke(clazz);
to clear the cache, but it does not work. I even tried to obtain classCache member field and clear it using reflection but it does not work too.
What am I doing wrong?
I just need to force freemarker to throw away any introspection on model class/classes he has already obtained.
Is there any way?
UPDATE
Example code
Application.java
// Application.java
public class Application
{
public static final String TEMPLATE_PATH = "TemplatePath";
public static final String DEFAULT_TEMPLATE_PATH = "./";
private static Application INSTANCE;
private Configuration freemarkerConfiguration;
private BeansWrapper beanWrapper;
public static void main(String[] args)
{
final Application application = new Application();
INSTANCE = application;
try
{
application.run(args);
}
catch (InterruptedException e)
{
System.out.println("Exiting");
}
catch (IOException e)
{
System.out.println("IO Error");
e.printStackTrace();
}
}
public Configuration getFreemarkerConfiguration()
{
return freemarkerConfiguration;
}
public static Application getInstance()
{
return INSTANCE;
}
private void run(String[] args) throws InterruptedException, IOException
{
final String templatePath = System.getProperty(TEMPLATE_PATH) != null
? System.getProperty(TEMPLATE_PATH)
: DEFAULT_TEMPLATE_PATH;
final Configuration configuration = new Configuration();
freemarkerConfiguration = configuration;
beanWrapper = new BeansWrapper();
beanWrapper.setUseCache(false);
configuration.setObjectWrapper(beanWrapper);
try
{
final File templateDir = new File(templatePath);
configuration.setTemplateLoader(new FileTemplateLoader(templateDir));
}
catch (IOException e)
{
throw new RuntimeException(e);
}
final RunnerImpl runner = new RunnerImpl();
try
{
runner.run(args);
}
catch (RuntimeException e)
{
e.printStackTrace();
}
}
public BeansWrapper getBeanWrapper()
{
return beanWrapper;
}
}
RunnerImpl.java
// RunnerImpl.java
public class RunnerImpl implements Runner
{
#Override
public void run(String[] args) throws InterruptedException
{
long counter = 0;
while(true)
{
++counter;
System.out.printf("Run %d\n", counter);
// Application.getInstance().getFreemarkerConfiguration().setObjectWrapper(new BeansWrapper());
Application.getInstance().getBeanWrapper().clearClassIntrospecitonCache();
final Worker worker = new Worker();
worker.doWork();
Thread.sleep(1000);
}
}
Worker.java
// Worker.java
public class Worker
{
void doWork()
{
final Application application = Application.getInstance();
final Configuration freemarkerConfiguration = application.getFreemarkerConfiguration();
try
{
final Template template = freemarkerConfiguration.getTemplate("test.ftl");
final Model model = new Model();
final PrintWriter printWriter = new PrintWriter(System.out);
printObjectInto(model);
System.out.println("-----TEMPLATE MACRO PROCESSING-----");
template.process(model, printWriter);
System.out.println();
System.out.println("-----END OF PROCESSING------");
System.out.println();
}
catch (IOException e)
{
e.printStackTrace();
}
catch (TemplateException e)
{
e.printStackTrace();
}
}
private void printObjectInto(Object o)
{
final Class<?> aClass = o.getClass();
final Method[] methods = aClass.getDeclaredMethods();
for (final Method method : methods)
{
System.out.println(String.format("Method name: %s, public: %s", method.getName(), Modifier.isPublic(method.getModifiers())));
}
}
}
Model.java
// Model.java
public class Model
{
public String getMessage()
{
return "Hello";
}
public String getAnotherMessage()
{
return "Hello World!";
}
}
This example does not work at all. Even changing BeansWrapper during runtime won't have any effect.
BeansWrapper (and DefaultObjectWrapper's, etc.) introspection cache relies on java.beans.Introspector.getBeanInfo(aClass), not on reflection. (That's because it treats objects as JavaBeans.) java.beans.Introspector has its own internal cache, so it can return stale information, and in that case BeansWrapper will just recreate its own class introspection data based on that stale information. As of java.beans.Introspector's caching, it's in fact correct, as it builds on the assumption that classes in Java are immutable. If something breaks that basic rule, it should ensure that java.beans.Introspector's cache is cleared (and many other caches...), or else it's not just FreeMarker that will break. At JRebel for example they made a lot of effort to clear all kind of caches. I guess DCEVM doesn't have the resources for that. So then, it seems you have to call Introspector.flushCaches() yourself.
Update: For a while (Java 7, maybe 6) java.beans.Introspector has one cache per thread group, so you have call flushCaches() from all thread groups. And this all is actually implementation detail that, in principle, can change any time. And sadly, the JavaDoc of Introspector.flushCaches() doesn't warn you...