I am trying to implement routing slip (EI Pattern) using Spring integration. The configuration I have done is
#Bean
#Transformer(inputChannel = "routingServiceChannel")
public HeaderEnricher headerEnricher() {
return new HeaderEnricher(Collections.singletonMap(IntegrationMessageHeaderAccessor.ROUTING_SLIP,
new RoutingSlipHeaderValueMessageProcessor("routingChannel2",
"routingChannel1")));
}
So when it reaches routingServiceChannel, it goes to routingChannel2 and then routingChannel1. But in my case it always throws an exception that it does not have a reply channel. when I set output channel, like
#Transformer(inputChannel = "routingServiceChannel", outputChannel=""xyzChannel)
Then then routing happens to xyzChannel instead of going to routingChannel2, and routingChannel1.
When I debugged the code in spring integration core, I stumbled with this code,
In
class AbstractReplyProducingMessageHandler
protected final void handleMessageInternal(Message<?> message) {
Object result;
if (this.advisedRequestHandler == null) {
result = handleRequestMessage(message);
} else {
result = doInvokeAdvisedRequestHandler(message);
}
if (result != null) {
sendOutputs(result, message);
} else if (this.requiresReply) {
throw new ReplyRequiredException(message, "No reply produced by handler '" + getComponentName()
+ "', and its 'requiresReply' property is set to true.");
} else if (logger.isDebugEnabled()) {
logger.debug("handler '" + this + "' produced no reply for request Message: " + message);
}
}
Here in handle message method they obtain the routingSlip map and assigning it to the result.
And
protected void produceOutput(Object reply, Message<?> requestMessage) {
MessageHeaders requestHeaders = requestMessage.getHeaders();
Object replyChannel = null;
if (getOutputChannel() == null) {
Map<?, ?> routingSlipHeader = requestHeaders.get(IntegrationMessageHeaderAccessor.ROUTING_SLIP, Map.class);
if (routingSlipHeader != null) {
Assert.isTrue(routingSlipHeader.size() == 1, "The RoutingSlip header value must be a SingletonMap");
Object key = routingSlipHeader.keySet().iterator().next();
Object value = routingSlipHeader.values().iterator().next();
Assert.isInstanceOf(List.class, key, "The RoutingSlip key must be List");
Assert.isInstanceOf(Integer.class, value, "The RoutingSlip value must be Integer");
List<?> routingSlip = (List<?>) key;
AtomicInteger routingSlipIndex = new AtomicInteger((Integer) value);
replyChannel = getOutputChannelFromRoutingSlip(reply, requestMessage, routingSlip, routingSlipIndex);
if (replyChannel != null) {
// TODO Migrate to the SF MessageBuilder
AbstractIntegrationMessageBuilder<?> builder = null;
if (reply instanceof Message) {
builder = this.getMessageBuilderFactory().fromMessage((Message<?>) reply);
} else if (reply instanceof AbstractIntegrationMessageBuilder) {
builder = (AbstractIntegrationMessageBuilder<?>) reply;
} else {
builder = this.getMessageBuilderFactory().withPayload(reply);
}
builder.setHeader(IntegrationMessageHeaderAccessor.ROUTING_SLIP,
Collections.singletonMap(routingSlip, routingSlipIndex.get()));
reply = builder;
}
}
if (replyChannel == null) {
replyChannel = requestHeaders.getReplyChannel();
}
}
Message<?> replyMessage = createOutputMessage(reply, requestHeaders);
sendOutput(replyMessage, replyChannel);
}
Instead of getting the routingSlip configuration from the reply, they are trying to get from the requestMessage.
Am I missing something here? Are there any additional configuration that I need to set?
Thank you for the attention to the subject, BTW! :)
Everything looks good, but you have missed the point of the Routing Slip.
First of all you should configure it for the message. And since Routing Slip is a header you should use HeaderEnricher to add it to the headers of the message.
The routing is really caused in the downstream from and exactly for the requestMessage, not the reply. The Routing Slip is out of HeaderEnricher.
Although it may happen for the HeaderEnricher's requestMessage, of course.
If you would like to consult Routing Slip exactly after the HeaderEnricher, you should configure something like:
#BridgeTo
#Bean
public MessageChannel xyzChannel() {
return new DirectChannel();
}
Note: no one new header is available during HeaderEnricher logic. Only in the downstream. And Routing Slip is one of them.
Related
I am currently new on Apache Nifi and still exploring it.
I made a custom processor where I will fetch data from server with pagination.
I pass the input file which will contains the attribute "url".
Finally transfer the response in output flow file, as I fetch data with pagination, so I made a new output flow file for each page and transferred it to Successful relationship.
Below is the code part:
#Override
public void onTrigger(final ProcessContext context, final ProcessSession session) throws ProcessException {
FlowFile incomingFlowFile = session.get();
String api = null;
if (incomingFlowFile == null) {
logger.info("empty input flow file");
session.commit();
return;
} else {
api=incomingFlowFile.getAttribute("url");
}
session.remove(incomingFlowFile);
if(api == null) {
logger.warn("API url is null");
session.commit();
return;
}
int page = Integer.parseInt(context.getProperty(PAGE).getValue());
while(page < 3) {
try {
String url = api + "&curpg=" + page;
logger.info("input url is: {}", url);
HttpResponse response = httpGetApiCall(url, 10000);
if(response == null || response.getEntity() == null) {
logger.warn("response null");
session.commit();
return;
}
String resp = EntityUtils.toString(response.getEntity());
InputStream is = new ByteArrayInputStream(StandardCharsets.UTF_16.encode(resp).array());
FlowFile outFlowFile = session.create();
outFlowFile = session.importFrom(is, outFlowFile);
session.transfer(outFlowFile, SUCCESSFUL);
} catch (IOException e) {
logger.warn("IOException :{}", e.getMessage());
return;
}
++page;
}
session.commit();
}
I am facing issue that for a single Input flow file, this processor get triggered twice and so it generates 4 flow files for a single input flow file.
I am not able to figure out this where I have done wrong.
Please help in this issue.
Thanks in advance.
======================================================================
processor group 1(Nifi_Parvin)
processor group 2 (News_Point_custom)
I have a Jersey server running locally, it exposes a SSE resource just like the examples here: https://jersey.github.io/documentation/latest/sse.html. I have a local webpack Angular app, that binds to the exposed GET endpoint and listens for data.
On the GET, I start up a thread to send notifications at regular intervals over 6-8 seconds. I don't see anything on the client UNTIL the EventOutput object is closed.
What am I doing wrong, and how can I fix this?
The server code WORKS with just a simple curl, i.e.:
curl http://localhost:8002/api/v1/notify
But on both Chrome and Safari the following code exhibits the behavior
Client (TypeScript):
this.evSource = new EventSource('http://localhost:8002/api/v1/notify');
this.evSource.addEventListener(
'event',
(x => console.log('we have ', x))
);
this.evSource.onmessage = (data => console.log(data));
this.evSource.onopen = (data => console.log(data));
this.evSource.onerror = (data => {
console.log(data);
this.evSource.close();
});
Server (Java):
// cache callback
public void eventCallback(Iterable<CacheEntryEvent<? extends Integer, ? extends Integer>> events) {
for (CacheEntryEvent<? extends Integer, ? extends Integer> x : events) {
LOGGER.info("{} Sending the following value: " + x.getValue(), Thread.currentThread().getId());
final OutboundEvent sseEvent = new OutboundEvent.Builder().name("event")
.data(Integer.class, x.getValue()).build();
this.broadcaster.broadcast(sseEvent);
}
}
#GET
#Produces(SseFeature.SERVER_SENT_EVENTS)
#ApiOperation(value = "Setup SSE pipeline", notes = "Sets up the notification pipeline for clients to access")
#ApiResponses(value = {
#ApiResponse(code = HttpURLConnection.HTTP_UNAUTHORIZED,
message = "Missing, bad or untrusted cookie"),
#ApiResponse(code = HttpURLConnection.HTTP_OK,
message = "Events streamed successfully")
})
#Timed
#ResponseMetered
public EventOutput registerNotificationEvents(
#HeaderParam(SseFeature.LAST_EVENT_ID_HEADER) String lastEventId,
#QueryParam(SseFeature.LAST_EVENT_ID_HEADER) String lastEventIdQuery) {
if (!Strings.isNullOrEmpty(lastEventId) || !Strings.isNullOrEmpty(lastEventIdQuery)) {
LOGGER.info("Found Last-Event-ID header: {}", !Strings.isNullOrEmpty(lastEventId) ? lastEventId : lastEventIdQuery );
}
LOGGER.info("{} Received request", Thread.currentThread().getId());
this.continuation = true;
final EventOutput output = new EventOutput();
broadcaster.add(output);
Random rand = new Random();
IntStream rndStream = IntStream.generate(() -> rand.nextInt(90));
List<Integer> lottery = rndStream.limit(15).boxed().collect(Collectors.toList());
IgniteCache<Integer, Integer> cache = this.ignite.cache(topic_name);
executorService.execute(() -> {
try {
lottery.forEach(value -> {
try {
TimeUnit.MILLISECONDS.sleep(500);
LOGGER.info("{} Sending the following value to Ignite: " + value + " : " + count++, Thread.currentThread().getId());
if (!cache.isClosed()) {
cache.put(1, value);
}
} catch (InterruptedException ex) {
ex.printStackTrace();
}
});
TimeUnit.MILLISECONDS.sleep(500);
continuation = false;
TimeUnit.MILLISECONDS.sleep(500);
if (!output.isClosed()) {
// THIS is where the client sees ALL the data broadcast
// in one shot
output.close();
}
} catch (InterruptedException ex) {
ex.printStackTrace();
} catch (IOException ex) {
ex.printStackTrace();
}
});
LOGGER.info("{} Completing request", Thread.currentThread().getId());
return output;
}
}
Looks like http://github.com/dropwizard/dropwizard/issues/1673 captures the problem. GZip default won't flush even if upper levels ask for it. Solution is something like
((AbstractServerFactory)configuration.getServerFactory()).getGzipFilterFactory().setSyncFlush(true);
will enable flushing to synchronize with GZip if disabling GZip all up is not an option
In my new project, I am going to include google recaptcha.
my question is fairly simple even if we do client side validation that user is not a robot even though it is suggested to do server side validation.
I want to know why it is necessary to do server side validation for google recaptcha? how does it add the extra layer of security? and how to do in spring boot with spring security?
Server side validation is MUST !! reCAPTCHA is designed in a way that client side just generates the 'g-captcha-response' which along with secret key (stored at server-side) is sent to https://www.google.com/recaptcha/api/siteverify for validation. The response is a JSON which states sucesss true or false and it is further pushed to client side. Validating only at the client side is technically possible, but it defeats the purpose. Moreover, you may get CORS (Cross-Origin Resource Sharing) policy error in console if you do only client side validation. I can share steps to do simple java based server side validation in servlet. Let me know if you need that.
Here is the code. Few points to be noted:
The parameter userResponse = request.getParameter("recaptchaResponse") is the way by which i am getting the 'g-recaptcha-response' generated by the user when he clicked reCAPTCHA widget on UI. On your javascript, capture the value of field 'g-recaptcha-response' and pass it appended to request. Then in servlet, we can get it from request.getParameter.
Sample code:
var recaptchaResponse = document.getElementById("g-recaptcha-response").value;
//alert("g-recaptcha-response= "+recaptchaResponse);
if (recaptchaResponse.length > 0)
{
var xmlhttp1;
if (window.XMLHttpRequest)
{
xmlhttp1=new XMLHttpRequest();
}
else
{
xmlhttp1=new ActiveXObject("Microsoft.XMLHTTP");
}
var query1 = "?recaptchaResponse=" + recaptchaResponse;
xmlhttp1.open("POST","captchaVerificationServlet" + query1, false);
xmlhttp1.send(null);
var resp1 = xmlhttp1.responseText;
alert("resp1= "+resp1);
if(resp1=='matched'){
return true;
}
else{
alert("resp1 did not match");
return false;
}
}
else{
alert("error: recaptcha response is blank");
return false;
}
For simplicity i am checking presence of "success:true" in returned JSON response. As you know, returned JSON contains two parameters : success and error-codes. You may use a JSONReader to read and parse JSON and obtain all parameters fully. Sample code will be like
JsonReader rdr = Json.createReader(your_inputstream);
JsonObject jsonObject = rdr.readObject();
Needless to say, remove all alerts and sop statements in production!
public class CaptchaVerificationServlet extends HttpServlet {
private static final String sec = YOUR_SECRET_KEY;
public void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {}
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
String userResponse = request.getParameter("recaptchaResponse");
response.setCharacterEncoding("UTF-8");
System.out.println("userResponse= "+userResponse);
//verify user response with Google ReCaptcha API
String ipAddress = request.getRemoteAddr(); //get client's ip address
System.out.println("ipAddress= "+ipAddress);
try{
String s = validateCaptcha(sec, userResponse, ipAddress);
Boolean success = (s.contains("\"success\": true"));
if(success)
response.getWriter().write("matched");
}
catch(Exception ioe){
ioe.printStackTrace();
ioe.printStackTrace(response.getWriter());
}
}
private String validateCaptcha(String secret, String response, String remoteip) throws IOException
{
URLConnection connection = null;
InputStream is = null;
String output = "";
String proxyHost = "YOUR_PROXY_NAME";
int proxyPort = 80; //proxy server port, generally 80 or 443 (confirm from sys-admin)
SocketAddress addr = new InetSocketAddress(proxyHost, proxyPort);
Proxy httpProxy = new Proxy(Proxy.Type.HTTP, addr);
String filename = System.getProperty("java.home") + "/lib/security/cacerts".replace('/', File.separatorChar);
String password = "changeit";
System.setProperty("javax.net.ssl.trustStore",filename);
System.setProperty("javax.net.ssl.trustAnchors",filename);
System.setProperty("javax.net.ssl.trustStorePassword",password);
String charset = Charset.forName("UTF-8").name();
String url = "https://www.google.com/recaptcha/api/siteverify";
try {
String query = String.format("secret=%s&response=%s&remoteip=%s",
URLEncoder.encode(secret, charset),
URLEncoder.encode(response, charset),
URLEncoder.encode(remoteip, charset));
URL fullURL = new URL(url + "?" + query);
connection = fullURL.openConnection(httpProxy);
connection.addRequestProperty("User-Agent", "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0");
is = connection.getInputStream();
System.out.println("connection InputStream");
BufferedReader reader = null;
String responseXXX = "";
reader = new BufferedReader(new InputStreamReader(is));
responseXXX = reader.readLine();
while (responseXXX!=null) {
output+= responseXXX;
responseXXX = reader.readLine();
}
System.out.println("Output: " + output);
}
finally {
if (is != null) {
try {
is.close();
} catch (IOException e) {
//cannot do anything here
}
}
}
return output;
}
}
I have a Web API, When the incoming request is not valid then the API sends back a HttpStatusCode.BadRequest and API would also add a CorrelationId into Response's HttpHeader. Something like below
public class ValidateRequestAttribute : ActionFilterAttribute
{
public ValidateRequestAttribute()
{
}
public override void OnActionExecuting(ActionExecutingContext context)
{
if (context.ModelState.IsValid == false)
{
context.HttpContext.Response.StatusCode = (int)HttpStatusCode.BadRequest;
context.HttpContext.Response.Headers.Add("x-correlationid", "someid");
context.Result = new ContentResult()
{
Content = "bad request."
};
}
}
}
On client side im using HttpClient to access the API. I am not sure how client would retrieve HttpStatusCode and HttpHeader here. Here is my client code
public bool Process(url)
{
bool result = false;
try
{
Task.Run(async () => await _httpClient.GetStringAsync(url).ConfigureAwait(false)).Result;
}
catch (Exception ex)
{
if(ex is AggregateException)
{
var aggregateException = ex as AggregateException;
foreach(var innerException in aggregateException.InnerExceptions)
{
if (innerException is HttpRequestException)
{
var httpRequestException = innerException as HttpRequestException;
// how do i get StatusCode and HttpHeader values here??
}
}
}
}
return result;
}
I have already gone through SO post here and MSDN article here and also Stephen Cleary's article here
Even though its recommended to make async all the way down, I this case Client and API are both disconnected from each other and client is synchronous. Note that Client's Process method is synchronous method.
Like this:
public bool Process(string url)
{
var result = _httpClient.GetAsync(url).ConfigureAwait(false).GetAwaiter().GetResult();
if (result.StatusCode == HttpStatusCode.BadRequest)
{
IEnumerable<string> values;
if (result.Headers.TryGetValues("x-correlationid", out values))
{
// Should print out "someid"
Console.WriteLine(values.First());
}
}
return result.IsSuccessStatusCode;
}
Also note that doing .GetAwaiter().GetResult(); vs .Result; is recommended since it makes the code easier to work with because it does not throw an AggregateException.
If you want to read the response content as a string just do:
var content = result.Content.ReadAsStringAsync().ConfigureAwait(false).GetAwaiter().GetResult();
If you want to make your code async though you should use the async/await keyword and skip the .GetAwaiter().GetResult();.
I have sent data from mobile to wear as
private static final String IMAGE_PATH = "/image";
private static final String IMAGE_TITLE = "imageTitle";
private static final String IMAGE_TO_SEND = "image";
PutDataMapRequest putDataMapRequest = PutDataMapRequest.create(IMAGE_PATH);
putDataMapRequest.getDataMap().putString(IMAGE_TITLE, "hi this handheld");
PutDataRequest putDataRequest = putDataMapRequest.asPutDataRequest();
putDataRequest.setUrgent();
Wearable.DataApi.putDataItem(googleApiClient, putDataRequest).setResultCallback(new ResultCallback<DataApi.DataItemResult>() {
#Override
public void onResult(#NonNull DataApi.DataItemResult dataItemResult) {
if (dataItemResult.getStatus().isSuccess()) {
} else {
}
}
});
On wear side I want to first check if data has been already sent or not. If data has been sent already then I want to use that data otherwise I want to request data for mobile:
I have done it but it always fails. My code is as follow:
Wearable.NodeApi.getLocalNode(googleApiClient).setResultCallback(new ResultCallback<NodeApi.GetLocalNodeResult>() {
#Override
public void onResult(#NonNull NodeApi.GetLocalNodeResult getLocalNodeResult) {
Uri uri = new Uri.Builder()
.scheme(PutDataRequest.WEAR_URI_SCHEME)
.path(IMAGE_PATH)
.authority(getLocalNodeResult.getNode().getId())
.build();
Wearable.DataApi.getDataItem(googleApiClient, uri).setResultCallback(new ResultCallback<DataApi.DataItemResult>() {
#Override
public void onResult(#NonNull DataApi.DataItemResult dataItemResult) {
if (dataItemResult.getStatus().isSuccess() && dataItemResult.getDataItem() != null) {
Log.d(TAG, "onResult: success result");
DataMap dataMap = DataMap.fromByteArray(dataItemResult.getDataItem().getData());
} else {
}
}
});
}
});
I found solution. I was checking data by getting local nodes id ,so that it was providing null data.Local Node id i-e id of my watch, Node Id must be id of node which has sent data using Data Layer Api. In my case at first my handheld sends data through the Data Layer Api and my watch checks data exist or not.If data found gets data otherwise send request to Handheld.
Wearable.NodeApi.getConnectedNodes(googleApiClient).setResultCallback(new ResultCallback<NodeApi.GetConnectedNodesResult>() {
#Override
public void onResult(NodeApi.GetConnectedNodesResult nodes) {
for (Node node : nodes.getNodes()) {
connectedNode = node;
}
Uri uri = new Uri.Builder()
.scheme(PutDataRequest.WEAR_URI_SCHEME)
.path(IMAGE_PATH)
.authority(connectedNode.getId()) //id which has sent data
.build();
Wearable.DataApi.getDataItem(googleApiClient, uri).setResultCallback(new ResultCallback<DataApi.DataItemResult>() {
#Override
public void onResult(#NonNull DataApi.DataItemResult dataItemResult) {
if (dataItemResult.getStatus().isSuccess() && dataItemResult.getDataItem() != null) {
Log.d(TAG, "onResult: successfully got previous data");
} else {
makeRequestToSendData();
Log.d(TAG, "onResult: failed to got previous data");//request handheld to get data using **Message API**
}
}
});
}
});
We have to change getLocalNode to getConnectedNodes .
Your code looks fine, is the node id non null ?
Does it fail at the getDataItem result callback ?
To get the DataMap from the DataItemResult, use this code :
DataItem dataItem = dataItemResult.getDataItem();
if (dataItem != null) {
DataMap dataMap = DataMapItem.fromDataItem(dataItem).getDataMap();
}