Based on some properties set in the properties files if(prop1=false) Then how to remove the flow processingFlow1 in the FlowBuilder ..
Flow processingFlow1 = new FlowBuilder<Flow>("ProcessingFlow1").start(ProcessingFlow1Step).build();
Flow processingFlow2 = new FlowBuilder<Flow>("ProcessingFlow2").start(ProcessingFlow2Step).build();
New Flow
Flow rulesFlow = new FlowBuilder<Flow>("rulesFlow").build();
Adding flow to the flowbuilder
rulesFlow = new FlowBuilder<Flow>("rulesFlow").split(new SimpleAsyncTaskExecutor())
.add(ProcessingFlow1, ProcessingFlow2, ProcessingFlow3, )
.build();
My question based on some properties set in the properties files
if(prop1=false)
Then how to remove the flow processingFlow1 in the FlowBuilder ..
rulesFlow = new FlowBuilder<Flow>("rulesFlow").split(new SimpleAsyncTaskExecutor())
.add(, processingFlow2, processingFlow3, )
.build();
You can extract the builder as a variable and add flows dynamically based on your condition, something like:
Flow processingFlow1 = ...
Flow processingFlow2 = ...
Flow processingFlow3 = ...
FlowBuilder.SplitBuilder<Flow> rulesFlowBuilder = new FlowBuilder<Flow>("rulesFlow")
.split(new SimpleAsyncTaskExecutor());
if (shouldAddFlow()) { // check prop1=false in this method
rulesFlowBuilder.add(processingFlow1);
}
Flow rulesflow = rulesFlowBuilder.add(processingFlow2, processingFlow3).build();
Related
I tried the solution applied in the post here : Spring Batch - create a new file each time instead of overriding it for transferring data from CSV to XML, but it didn't worked for the annotation based approached I used.
fileItemWriter.setResource(new FileSystemResource("csv/employees-#{new java.text.SimpleDateFormat("Mddyyyyhhmmss").format(new java.util.GregorianCalendar().getTime())}.csv"));
My Batch job is scheduled to run in every 1 hours, this batch jobs reads table and write data to CSV file. When data writes I need to create new file altogether..will be good if file name is unique, so I was looking to implement the date etc as per post.
Could anyone guide what's wrong going on ?
#Bean(destroyMethod="")
public FlatFileItemWriter<Employees> employeesWriter(){
FlatFileItemWriter<Employees> fileItemWriter = new FlatFileItemWriter<>();
//fileItemWriter.setResource(new FileSystemResource("csv/employees.csv"));
fileItemWriter.setResource(new FileSystemResource("csv/employees-#{new java.text.SimpleDateFormat("Mddyyyyhhmmss").format(new java.util.GregorianCalendar().getTime())}.csv"));
fileItemWriter.setHeaderCallback(headerCallback());
BeanWrapperFieldExtractor<Employees> fieldExtractor = new BeanWrapperFieldExtractor<>();
fieldExtractor.setNames(new String[] {"employeeNumber", "lastName", "firstName", "extension", "email", "officeCode", "reportsTo", "jobTitle"});
DelimitedLineAggregator<Employees> lineAggregator = new DelimitedLineAggregator<>();
lineAggregator.setDelimiter(",");
lineAggregator.setFieldExtractor(fieldExtractor);
fileItemWriter.setLineAggregator(lineAggregator);
fileItemWriter.setShouldDeleteIfEmpty(true);
return fileItemWriter;
}
Could anyone guide what's wrong going on ?
Three things:
SpEL expressions are not interpreted when used like you do
The " copied from the xml sample will not work in Java config
The / in csv/... is not a valid character in a file name
You need to declare your writer as follows:
#Bean
public FlatFileItemWriter itemWriter(#Value("employees-#{new java.text.SimpleDateFormat('Mddyyyyhhmmss').format(new java.util.GregorianCalendar().getTime())}.csv") String filename) {
FlatFileItemWriter<Employees> fileItemWriter = new FlatFileItemWriter<>();
fileItemWriter.setResource(new FileSystemResource(filename));
...
return fileItemWriter;
}
But I would recommend using a step scoped item writer and pass the file name as a job parameter rather than using a SpEL expression.
i'm doing a simple batch job with Spring Batch and Spring Boot.
I need to read a flat file, separate the header data (first line) from the body data (rest of lines) for individual business logic processing and then write everything into a single file.
As you can see, the header has 5 params that have to be mapped to one class, and the body has 12 which have to be mapped to a different one.
I first thought of using FlatFileItemReader and skip the header. Then use the skippedLinesCallback to handle that line, but i couldn't figure out how to do it.
I'm new to Spring Batch and Java Config. If someone can help me writing a solution for my problem i would really aprecciate it!
I leave here the input file:
01.01.2017|SUBDCOBR|12:21:23|01/12/2016|31/12/2016
01.01.2017|12345678231234|0002342434|BORGIA RUBEN|27-32548987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,01
01.01.2017|12345673201234|2342434|ALVAREZ ESTHER|27-32533987-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,02
01.01.2017|12345673201234|0002342434|LOPEZ LUCRECIA|27-32553387-9|FA|A|2062-
00010443/444/445|142,12|30/08/2017|142,12
01.01.2017|12345672301234|0002342434|SILVA JESUS|27-32558657-9|NC|A|2062-
00010443|142,12|30/08/2017|142,12
Cheers!
EDIT 1:
This would be my first attepmt . My "body" POJO is called DetalleFacturacion and my "header" POJO is CabeceraFacturacion. The reader I thought to do it with DetalleFacturacion pojo, so i can skip the header and treat it later... however i'm not sure how to assign header's data into CabeceraFacturacion.
public FlatFileItemReader<DetalleFacturacion> readerDetalleFacturacion(){
FlatFileItemReader<DetalleFacturacion> reader = new FlatFileItemReader<>();
reader.setLinesToSkip(1);
reader.setResource(new ClassPathResource("/inputFiles/GLEO-MN170100-PROCESO01-SUBDFACT-000001.txt"));
DefaultLineMapper<DetalleFacturacion> detalleLineMapper = new DefaultLineMapper<>();
DelimitedLineTokenizer tokenizerDet = new DelimitedLineTokenizer("|");
tokenizerDet.setNames(new String[] {"fechaEmision", "tipoDocumento", "letra", "nroComprobante",
"nroCliente", "razonSocial", "cuit", "montoNetoGP", "montoNetoG3",
"montoExento", "impuestos", "montoTotal"});
LineCallbackHandler skippedLineCallback = new LineCallbackHandler() {
#Override
public void handleLine(String line) {
String[] headerSeparado = line.split("|");
String printDate = headerSeparado[0];
String reportIdentifier = headerSeparado[1];
String tituloReporte = headerSeparado[2];
String fechaDesde = headerSeparado[3];
String fechaHasta = headerSeparado[4];
CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
cabeceraFacturacion.setPrintDate(printDate);
cabeceraFacturacion.setReportIdentifier(reportIdentifier);
cabeceraFacturacion.setTituloReporte(tituloReporte);
cabeceraFacturacion.setFechaDesde(fechaDesde);
cabeceraFacturacion.setFechaHasta(fechaHasta);
}
};
reader.setSkippedLinesCallback(skippedLineCallback);
detalleLineMapper.setLineTokenizer(tokenizerDet);
detalleLineMapper.setFieldSetMapper(new DetalleFieldSetMapper());
detalleLineMapper.afterPropertiesSet();
reader.setLineMapper(detalleLineMapper);
// Test to check if it is saving correctly data in CabeceraFacturacion
CabeceraFacturacion cabeceraFacturacion = new CabeceraFacturacion();
System.out.println("Print Date:"+cabeceraFacturacion.getPrintDate());
System.out.println("Report Identif:
"+cabeceraFacturacion.getReportIdentifier());
return reader;
}
You are correct . You need to use skippedLinesCallback to handle skip lines.
You need to implement LineCallbackHandler interface and add you processing in handleLine method.
LineCallbackHandler Interface passes the raw line content of the lines in the file to be skipped. If linesToSkip is set to 2, then this interface is called twice.
This is how you can define Reader for the same.
Java Config - Spring Batch 4
#Bean
public FlatFileItemReader<POJO> myReader() {
return FlatFileItemReader<pojo>().
.setResource(new FileSystemResource("resources/players.csv"));
.name("myReader")
.delimited()
.delimiter(",")
.names("pro1,pro2,pro3")
.targetType(POJO.class)
.skippedLinesCallback(skippedLinesCallback)
.build();
}
How can I pass in an 'asset' POJO to the S3MessageHandler using Spring Integration and change the bucket?
I would like to be able to change the bucket, based on a folder in the asset, to something like 'root-bucket/a/b/c.'
Each asset can potentially have a different sub-path.
#Bean
IntegrationFlow flowUploadAssetToS3()
{
return flow -> flow
.channel(this.channelUploadAsset)
.channel(c -> c.queue(10))
.publishSubscribeChannel(c -> c
.subscribe(s -> s
// Publish the asset?
.<File>handle((p, h) -> this.publishS3Asset(
p,
(Asset)h.get("asset")))
// Set the action
.enrichHeaders(e -> e
.header("s3Command", Command.UPLOAD.name()))
// Upload the file
.handle(this.s3MessageHandler())));
}
MessageHandler s3MessageHandler()
{
return new S3MessageHandler(amazonS3, "root-bucket");
}
Thanks to the answer below, I got this working like so:
final String bucket = "'my-root";
final Expression bucketExpression = PARSER.parseExpression(bucket);
final Expression keyExpression = PARSER.parseExpression("headers['asset'].bucket");
final S3MessageHandler handler = new S3MessageHandler(amazonS3, bucketExpression);
handler.setKeyExpression(keyExpression);
return handler;
There is bucketExpression option which is SpEL to let to resolve a bucket from the requestMessage:
private static SpelExpressionParser PARSER = new SpelExpressionParser();
MessageHandler s3MessageHandler() {
Expression bucketExpression =
PARSER.parseExpression("payload.bucketProperty");
return new S3MessageHandler(amazonS3, bucketExpression);
}
Also be aware that root-bucket/a/b/c. isn't bucket name. You should consider to distinguish to bucket and the key to build that complex sub-folder path. For that purpose there is keyExpression option with similar functionality to resolve the key in bucket against requestMessage.
I need to generate BPEL XML code in runtime. The only way I can do it now is to create XML document with "bare hands" using DOM API. But there must be a framework that could ease such work incorporating some kind of object model.
I guess it should look something like this:
BPELProcessFactory.CreateProcess().addSequence
Do you know any?
The Eclipse BPEL designer project provides an EMF model for BPEL 2.0. The generated code can be used to programmatically create BPEL code with a convenient API.
In case anyone stumbles upon this.
Yes this can be done using the BPEL Model.
Here is a sample piece of code which generates a quite trivial BPEL file:
public Process createBPEL()
{
Process process = null;
BPELFactory factory = BPELFactory.eINSTANCE;
try
{
ResourceSet rSet = new ResourceSetImpl();
rSet.getResourceFactoryRegistry().getExtensionToFactoryMap()
.put("bpel", new BPELResourceFactoryImpl());
File file = new File("myfile.bpel");
file.createNewFile();
String filePath = file.getAbsolutePath();
System.out.println(filePath);
AdapterRegistry.INSTANCE.registerAdapterFactory( BPELPackage.eINSTANCE, BasicBPELAdapterFactory.INSTANCE );
Resource resource = rSet.createResource(URI.createFileURI(filePath));
process = factory.createProcess();
process.setName("FirstBPEL");
Sequence seq = factory.createSequence();
seq.setName("MainSequence");
Receive recieve = factory.createReceive();
PortType portType = new PortTypeProxy(URI.createURI("http://baseuri"), new QName("qname"));
Operation operation = new OperationProxy(URI.createURI("http://localhost"), portType , "operation_name");
recieve.setOperation(operation);
Invoke invoke = factory.createInvoke();
invoke.setOperation(operation);
While whiles = factory.createWhile();
If if_st = factory.createIf();
List<Activity> activs = new ArrayList<Activity>();
activs.add(recieve);
activs.add(invoke);
activs.add(if_st);
activs.add(whiles);
seq.getActivities().addAll(activs);
process.setActivity(seq);
resource.getContents().add(process);
Map<String,String> map = new HashMap<String, String>();
map.put("bpel", "http://docs.oasis-open.org/wsbpel/2.0/process/executable");
map.put("xsd", "http://www.w3.org/2001/XMLSchema");
resource.save(map);
}
catch(Exception e)
{
e.printStackTrace();
}
return process;
}
The dependencies require that you add the following jars to the project's build path from the plugins folder in eclipse installation directory:
org.eclipse.bpel.model_*.jar
org.eclipse.wst.wsdl_*.jar
org.eclipse.emf.common_*.jar
org.eclipse.emf.ecore_*.jar
org.eclipse.emf.ecore.xmi_*.jar
javax.wsdl_*.jar
org.apache.xerces_*.jar
org.eclipse.bpel.common.model_*.jar
org.eclipse.xsd_*.jar
org.eclipse.core.resources_*.jar
org.eclipse.osgi_*.jar
org.eclipse.core.runtime_*.jar
org.eclipse.equinox.common_*.jar
org.eclipse.core.jobs_*.jar
org.eclipse.core.runtime.compatibility_*.jar
I'm attempting to wire a queue to the front of a MessageChannel, and I need to do so programatically so it can be done at run time in response to an osgi:listener being triggered. So far I've got:
public void addService(MessageChannel mc, Map<String,Object> properties)
{
//Create the queue and the QueueChannel
BlockingQueue<Message<?>> q = new LinkedBlockingQueue<Message<?>>();
QueueChannel qc = new QueueChannel(q);
//Create the Bridge and set the output to the input parameter channel
BridgeHandler b = new BridgeHandler();
b.setOutputChannel(mc);
//Presumably, I need something here to poll the QueueChannel
//and drop it onto the bridge. This is where I get lost
}
Looking through the various relevant classes, I came up with:
PollerMetadata pm = new PollerMetadata();
pm.setTrigger(new IntervalTrigger(10));
PollingConsumer pc = new PollingConsumer(qc, b);
but I'm not able to put it all together. What am I missing?
So, the solution that ended up working for me was:
public void addEngineService(MessageChannel mc, Map<String,Object> properties)
{
//Create the queue and the QueueChannel
BlockingQueue<Message<?>> q = new LinkedBlockingQueue<Message<?>>();
QueueChannel qc = new QueueChannel(q);
//Create the Bridge and set the output to the input parameter channel
BridgeHandler b = new BridgeHandler();
b.setOutputChannel(mc);
//Setup a Polling Consumer to poll the queue channel and
//retrieve 1 thing at a time
PollingConsumer pc = new PollingConsumer(qc, b);
pc.setMaxMessagesPerPoll(1);
//Now use an interval trigger to poll every 10 ms and attach it
IntervalTrigger trig = new IntervalTrigger(10, TimeUnit.MILLISECONDS);
trig.setInitialDelay(0);
trig.setFixedRate(true);
pc.setTrigger(trig);
//Now set a task scheduler and start it
pc.setTaskScheduler(taskSched);
pc.setAutoStartup(true);
pc.start();
}
I'm not terribly clear if all the above is explicitly needed, but neither the trigger or the task scheduler alone worked, I did appear to need both. I should also note the taskSched used was the default taskScheduler dependency injected from spring via
<property name="taskSched" ref="taskScheduler"/>