Spring based Camel Aggregator to read multiple files - spring

Problem Statement
I have two files File1 and File2 which needs to be parsed using camel Bindy.
The data from File1 and File2 needs to be aggregated and then inserted into database.
what is the best way to define the agregator is there a can somebody provide a sample implementation
Here is what i want
<!--route1 -->
<route>
<from ref="file1Endpoint" />
<unmarshal ref="pojo1" />
</route>
<!--route2 -->
<route>
<from ref="file2Endpoint" />
<unmarshal ref="pojo2" />
</route>

Use the Content Enricher providing an Aggregation Strategy, something like the below:
#Component
MyAggregationStrategy implements AggregationStrategy {
public static final String PRE_ENRICH_KEY = "PRE_ENRICH_KEY";
//this is meant to be called *before* the actual enrich call
//to store the state before the enrichment
public void init(Exchange exchange) {
final String originalBody = exchange.getIn().getBody(String.class);
exchange.setProperty(PRE_ENRICH_KEY, originalBody);
}
#Override
public Exchange aggregate(Exchange original, Exchange enrichmentResponse) {
final String originalBody = original.getProperty(PRE_ENRICH_KEY, String.class);
//enrichmentResponse is the Exchange coming from the 2nd file endpoint
//the enricher returns
//your processing logic to merge the 2 happens here
String result = //..
original.getIn().setBody(result);
return original;
}
}
And finally the route will look something like:
#Component
public class MyRouteBuilder extends RouteBuilder {
#Autowired
private MyAggregationStrategy myAggregateStrategy;
#Override
public void configure() throws Exception {
from("fileEndpoint1")
.bean(myAggregateStrategy, "init")
.enrich("fileEndpoint2", myMerger);
//continue from here one to Bindy
}
//..
}

Related

Spring Integration AOP for Logging outbound Http requests

I was looking at a post from 2014 about using Spring AOP for logging HTTP requests/replies:
Spring integration + logging response time for http adapters(or any endpoint)
To this end, I tried this AOP configuration:
<aop:config >
<aop:aspect id="myAspect" ref="inboundOutboundHttpLogging">
<aop:pointcut id="handleRequestMessageMethod"
expression="execution(* org.springframework.integration.handler.AbstractReplyProducingMessageHandler.handleRequestMessage(*))
and
args(message))" />
<aop:before method="requestMessageSent" pointcut-ref="handleRequestMessageMethod" arg-names="message"/>
</aop:aspect>
</aop:config>
Is there perhaps a newer way of using AOP for logging HTTP requests? I want to avoid having to put per-request logging (i.e. outbound-gateway advice on each gateway).
Thanks for any pointers.
The handleRequestMessage() is essentially an input message to this gateway and output. So, if you don't like implementing an AbstractRequestHandlerAdvice and adding it into each your gateway via their <request-handler-advice-chain>, then consider to use a <wire-tap> for input and output channels of those gateway.
You may implement, though, a BeanPostProcessor.postProcessBeforeInitialization() to add your custom AbstractRequestHandlerAdvice into those HTTP gateways you are interested in.
My point is that <aop:aspect> you are presenting us really might lead to some unexpected behavior, like that final method concern you have edit out from your question...
Based upon the suggestions made by #artem-bilan, I was able to find a solution similar to AOP for injecting logging AbstractRequestHandlerAdvice into HTTP outbound request processing. I'm contributing this as a way of showing a possible solution for anyone else who comes across this question.
As #artem-bilan mentions, there is a mechanism for injecting AbstractRequestHandlerAdvice into a AbstractReplyProducingMessageHandler such as an HttpRequestExecutingMessageHandler. In my case, I'm wanting to log the message contents (header and payload) prior to the HTTP call and also log the return message (header and payload). This works nicely.
#artem-bilan suggests that the BeanPostProcessor mechanism can allow to inject the advice without having to add that declaration to each http outbound bean. The BeanPostProcessor looks like this:
public class AddHttpOutboundAdvicePostProcessor implements BeanPostProcessor {
final List<Advice> adviceList;
final AddHttpOutboundAdvicePostProcessor(List<Advice> adviceList) {
this.adviceList = adviceList;
}
#Override
public Object postProcessAfterInitialization(#NonNull Object bean,
#NonNull String beanName)
throws BeansException {
if (bean instanceof AbstractHttpRequestExecutingMessageHandler) {
((AbstractHttpRequestExecutingMessageHandler) bean).setAdviceChain(adviceList);
}
return bean;
}
}
We need to set up this bean into our context. (I'm a die-hard declarative fan hence this is in XML.)
<bean id = "addHttpLoggingPostProcessor"
class = "com.my.package.AddHttpOutboundAdvicePostProcessor" >
<constructor-arg name="adviceList>
<util:list>
<ref bean="outboundLogger" />
</util:list>
</constructor-arg>
</bean>
Here, the outboundLogger is a bean that managers the request-handler-advice. In my choice of implementation, I'm sending a copy of the outbound message to a channel for logging beforehand, and a copy of the response message down another channel for logging the response. The XML declaration of the bean takes the two channel names as constructors:
<bean id="outboundLogger" class="com.my.package.HttpRequestProcessorLogger" >
<constructor-arg name="requestLoggingChannelName" value="XXX" />
<constructor-arg name="responseLoggingChannelName" value="YYY" />
</bean>
where XXX and YYY are the names of channels to the components that perform the logging. I've set these channels to be ExecutorChannels so that the logging is performed asynchronously.
The HttpRequestProcessorLogger bean manages the call to handleRequestMessage():
public class HttpRequestProcessorLogger extends AbstractRequestHandlerAdvice {
private MessageChannel requestLoggingChannel;
private MessageChannel responseLoggingChannel;
private String requestLoggingChannelName;
private String responseLoggingChannelName;
private BeanFactory beanFactory;
public HttpRequestProcessorLogger(String requestLoggingChannelName, String responseLoggingChannelName) {
this.requestLoggingChannelName = requestLoggingChannelName;
this.responseLoggingChannelName = responseLoggingChannelName;
}
#Override
protected Object doInvoke(ExecutionCallback callback, Object target, Message<?> message) {
getChannels();
requestLoggingChannel.send(message);
final Object result = callback.execute();
final message<?> outputMessage =
(MessageBuilder.class.isInstance(result) ? ((MessageBuilder<?>) result).build()
: (Message<?>) result;
responseLoggingChannel.send(outputMessage);
return outputMessage;
}
private synchronized void getChannels() {
if (requestLoggingChannelName != null) {
final DestinationResolver<MessageChannel>
channelResolver = ChannelResolverUtils.getChannelResolver(this.beanFactory);
requestLoggingChannel = channelResolver.resolverDestination(requestLoggingChannelName);
responseLoggingChannel = channelResolver.resolverDestination(responseLoggingChannelName);
requestLoggingChannelName = null;
responseLoggingChannelName = null;
}
}
#Override
public void setBeanFactory(#NonNull BeanFactory beanFactory) throws BeanException {
this.beanFactory = beanFactory;
}
}

MockMvc tests always returns status code 200 for get requests

I I am very new to spring boot and only have been working with it for a couple of days, so I am also very confused about this project (which is not my own). I am supposed to write tests with MockMvc for the rest controller, however each test with MockMvc only returns the status code 200 although it should be a 404.
Here is one of the tests:
#WebMvcTest(ObjectController.class)
#SpringJUnitConfig(TestConfig.class)
public class MvcTest {
#Autowired
MockMvc mockMvc;
#Test
public void shouldReturn404() throws Exception {
mockMvc.perform(MockMvcRequestBuilders.get("/obj/123"))
.andExpect(MockMvcResultMatchers.status().is(HttpStatus.NOT_FOUND.value()));
}
}
This is my rest controller I would like to test.
#RestController
#RequestMapping("obj")
public class ObjectController {
#Autowired
MyClass myClass;
#GetMapping()
public List<MyObject> findAll() {
List<MyObject> objList;
objList = myClass.getObjects();
return objList;
}
#GetMapping("/{id}")
public MyObject findById(#PathVariable String id) {
for (MyObject obj : myClass.getObjects()) {
if (obj.getId().equals(id))
return obj;
}
return null;
}
}
Then there is this class:
public class MyClass {
private List<MyObject> objList = new ArrayList<MyObject>();
public MyObject addObject(MyObject obj) {
objList.add(obj);
return obj;
}
public void setObjects(List<MyObject> objList) {
this.objList = objList;
}
public synchronized List<MyObject> getObjects() {
return objList;
}
}
There is an xml file that belongs to the class looking like this which can be found in the resource folder:
<?xml version='1.0' encoding='ISO-8859-1'?>
<beans xmlns='http://www.springframework.org/schema/beans'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xmlns:context='http://www.springframework.org/schema/context'
xsi:schemaLocation='http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd'>
<bean id='MyClass' class='it.is.some.path.MyClass'>
<property name='objList'>
<list>
<ref bean='[R01] Object1' />
</list>
</property>
</bean>
</beans>
The referenced beans can be found in separate files under resources too, in a sub folder called 'objList' and look like this:
<?xml version='1.0' encoding='ISO-8859-1'?>
<beans xmlns='http://www.springframework.org/schema/beans'
xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'
xmlns:context='http://www.springframework.org/schema/context'
xsi:schemaLocation='http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context.xsd'>
<bean id='[R01] Object1' class='this.is.some.path.MyObject'>
<property name='id' value='123' />
</bean>
</beans>
The myclass.xml as well as the folder with all xmls of the objects are imported via #ImportResource in the Application class.
And then there is
public class MyObject {
public String id = RandomStringUtils.randomAlphanumeric(5);
public String getId() {
return id;
}
public void setId(String id) {
this.id = id;
}
}
am sorry for the lengthy explanation but I have been spending two days now trying to figure out how to get MockMvc running and I feel like I am completely stuck. I would be very, very grateful if anybody could help out.
I assume 200 is the correct response as you have a mapping for /obj/{id} where {id} is a placeholder for any path variable like 123 in /obj/123.
Even though your controller returns null from a code perspective, Spring won't map this automatically to a 404 not found if this is what you would expect.
So you can literally try it with every path variable, you'll always get HTTP 200 because there wasn't any exception and the DisptacherServlet could properly route the HTTP request to a controller mapping. HTTP 200 is the default response code if you don't specify anything.
If you want your controller endpoint to return 404 when your MyObject is null, you have to be more explicit about this, e.g.:
#GetMapping("/{id}")
public ResponseEntity<MyObject> findById(#PathVariable String id) {
for (MyObject obj : myClass.getObjects()) {
if (obj.getId().equals(id))
return ResponseEntity.ok(obj);
}
return ResponseEntity.status(HttpStatus.NOT_FOUND).body(null);
}

camel with twitter simple example

I am novice in apache-camel and I started with simple apache camel-twitter example using spring. I am using 2.20 version for camel and 2.19 version for camel-twitter. Below is my router code,
public class TwitterRouter extends RouteBuilder {
public void configure() throws Exception {
System.out.println("Test");
String twitter = "twitter://streaming/filter?type=event&keywords="+ URLEncoder.encode("london", "utf8")+"&consumerKey=key&consumerSecret=secretkey&accessToken=accesstoken&accessTokenSecret=accesstokensecret";
from(twitter).process(new Processor() {
public void process(Exchange exchange) throws Exception {
Status status = exchange.getIn().getBody(Status.class);
ProducerTemplate template = exchange.getContext().createProducerTemplate();
User user = status.getUser();
String name = user.getName();
System.out.println("name "+name);
template.sendBody("twitter","name "+name);
String screenName = user.getScreenName();
String text = status.getText();
}
});
System.out.println("Test1");
}
And my spring context file as below,
<bean id="routeBuilder" class="com.xyz.route.TwitterRouter" />
<camelContext xmlns="http://camel.apache.org/schema/spring">
<routeBuilder ref="routeBuilder" />
</camelContext>
So below are my queries,
After running program only Test and Test1 messages are printed on console. But whatever inside process method is not printed on console. I tried with producttemplate also but its not printing. So can anybody please help.
Also I tried to search for simpe camel-twitter example but not found. Does anybody have such example.

What are the other ways to specify code for Insight to analyze?

Spring Insight documentation states:
A trace represents a thread of execution. It is usually started by an HTTP request but can also be started by a background job
My application architecture style is one of queues running in the background that I'd like to instrument as well. However, I can't figure out how to get Spring Insight to instrument these calls initiated by queued message. I.e. I'd like to instrument the trace after a message is read off of the queue.
How can I ensure Insight instruments these background jobs?
I ended up creating an aspect that targets all of the Command Handlers. It extends the AbstractOperationCollectionAspect, implements the collectionPoint aspect passing in the Handler as an argument to use when it implements the createOperation method.
I.e.
public aspect CommandHandlerOperationCollectionAspect extends AbstractOperationCollectionAspect
{
public pointcut collectionPoint():
execution(* com.xtrac.common.core.handler.ThreadedHandler.HandlerRunnable.executeActorHandler(com.xtrac.common.core.handler.Handler,java.lang.Object));
protected Operation createOperation(JoinPoint jp)
{
Object[] args = jp.getArgs();
com.xtrac.common.core.handler.Handler handler = (Handler) args[0];
Operation operation = new Operation()
.type(XTRACOperationType.COMMAND_HANDLER)
.label(handler.getClass().getSimpleName())
.sourceCodeLocation(getSourceCodeLocation(jp));
return operation;
}
#Override
public String getPluginName()
{
return HandlerPluginRuntimeDescriptor.PLUGIN_NAME;
}
#Override
public boolean isMetricsGenerator()
{
return true;
}
}
I also implemented an AbstractSingleTypeEndpointAnalyzer to fill out the analyzer:
public class HandlerEndPointAnalyzer extends AbstractSingleTypeEndpointAnalyzer
{
private static final HandlerEndPointAnalyzer INSTANCE=new HandlerEndPointAnalyzer();
private HandlerEndPointAnalyzer() {
super(XTRACOperationType.COMMAND_HANDLER);
}
public static final HandlerEndPointAnalyzer getInstance() {
return INSTANCE;
}
#Override
protected EndPointAnalysis makeEndPoint(Frame handlerFrame, int depth) {
Operation operation = handlerFrame.getOperation();
String resourceLabel = operation.getLabel();
String exampleRequest = EndPointAnalysis.getHttpExampleRequest(handlerFrame);
return new EndPointAnalysis(EndPointName.valueOf(resourceLabel),
resourceLabel,
exampleRequest,
getOperationScore(operation, depth),
operation);
}
being sure to add it as a descriptor:
public class HandlerPluginRuntimeDescriptor extends PluginRuntimeDescriptor {
public static final String PLUGIN_NAME = "handler";
private static final HandlerPluginRuntimeDescriptor INSTANCE=new HandlerPluginRuntimeDescriptor();
private static final List<? extends EndPointAnalyzer> epAnalyzers=
ArrayUtil.asUnmodifiableList(HandlerEndPointAnalyzer.getInstance());
private HandlerPluginRuntimeDescriptor() {
super();
}
public static final HandlerPluginRuntimeDescriptor getInstance() {
return INSTANCE;
}
#Override
public Collection<? extends EndPointAnalyzer> getEndPointAnalyzers() {
return epAnalyzers;
}
#Override
public String getPluginName() {
return PLUGIN_NAME;
}
}
All noted in the spring xml file:
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:insight="http://www.springframework.org/schema/insight-idk"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
http://www.springframework.org/schema/insight-idk http://www.springframework.org/schema/insight-idk/insight-idk-1.0.xsd">
<insight:plugin name="handler" version="${project.version}" publisher="XTRAC Solutions LLC" />
<insight:operation-group group="XTRAC Handlers" operation="command_handler_operation" />
<insight:operation-group group="XTRAC Handlers" operation="event_handler_operation" />
<insight:operation-group group="XTRAC Classic" operation="xtrac_workflow_operation" />
<insight:operation-view operation="command_handler_operation"
template="com/xtrac/insight/command_handler_operation.ftl" />
<insight:operation-view operation="event_handler_operation"
template="com/xtrac/insight/event_handler_operation.ftl" />
<insight:operation-view operation="xtrac_workflow_operation"
template="com/xtrac/insight/xtrac_workflow_operation.ftl" />
<bean id="handlerPluginEndPointAnalyzer"
class="com.xtrac.insight.HandlerEndPointAnalyzer"
factory-method="getInstance"
lazy-init="true"
/>
<bean id="handlerPluginRuntimeDescriptor"
class="com.xtrac.insight.HandlerPluginRuntimeDescriptor"
factory-method="getInstance"
lazy-init="true"
/>
</beans>
along with some ftls.
I also created a MethodOperationCollectionAspect to collect some of the web service calls that occur in these handers. This sets it up for a nice display that tells me a lot about what is going on during the hander operation, and how much time it takes. E.g.
This set up a framework for maintaining a monitor on the health of the application if I set up the base line Thresholds for the named handlers
This is very useful because I can then tell if the application is healthy. Otherwise, the endpoints default to <200 ms for healthy.

Unable to move from controller to view in Spring MVC

I am using Spring MVC framework for my project.
I am unable to get my code running from controller to view.
Sharing the important chunk of code here.....
Inside AdminController.java controller
System.out.println("controller returning");
return new ModelAndView("dataFrame_","frameData",dataString);
Inside dispatcher-servlet.xml
<bean name="/dataFrame.htm"
class="com.organization.dept.spec.proj.module.controller.DataFrameController" >
</bean>
<bean id="dataFrameViewResolver"
class="com.organization.dept.spec.proj.module.view.DataFrameViewResolver">
<property name="dataFrameView">
<bean class="com.organization.dept.spec.proj.module.view.DataFrameView" />
</property>
<property name="dataFramePrefix" value="dataFrame_"></property>
</bean>
inside DataFrameViewResolver.java
public class DataFrameViewResolver extends AbstractCachingViewResolver {
private String dataFramePrefix;
private View dataFrameView;
#Override
protected View loadView (String viewName, Locale locale) throws Exception {
View view = null;
if(viewName.startsWith(this.dataFramePrefix)){
view = dataFrameView;
}
return view;
}
and
public String getDataFramePrefix() {
return dataFramePrefix;
}
public void setDataFramePrefix(String dataFramePrefix) {
this.dataFramePrefix = dataFramePrefix;
}
public View getDataFrameView() {
return dataFrameView;
}
public void setDataFrameView(View dataFrameView) {
this.dataFrameView = dataFrameView;
}
}
inside DataFrameView.java ...
public class DataFrameView extends AbstractView {
#Override
protected void renderMergedOutputModel(Map map, HttpServletRequest request,HttpServletResponse response) throws Exception {
System.out.println("RenderMergeoutputModel"); //line 99
I was unable to get the above system.out.println i.e. was unable to execute my code till that line 99.
The localhost log files of tomcat revealed some exception java.lang.ClassNotFoundException: javax.servlet.jsp.jstl.core.Config
I put the jstl-1.2.jar in lib and this could just get me rid of exception however still was unable get sysout of DataFrameView of line 99.

Resources