When I use curl to post a SOAP message into message flow, it takes 9 seconds to respond. Debug won't stop at breakpoints and User Trace doesn't report anything. Meanwhile when I do the same request from Postman or SoapUI (the first message takes the same amount of time, later all messages take around 70 - 200ms) debugger and user trace work as intended. What is cause of this behavior?
IBM App Connect Enterprise 11.0.0.8
curl --trace-time output:
03:29:07.484000 * Trying <host>...
03:29:07.484000 * TCP_NODELAY set
03:29:07.531000 * Connected to <host_name> (<host>) port 7800 (#0)
03:29:07.531000 > POST /service HTTP/1.1
03:29:07.531000 > Host: <host_name>:7800
03:29:07.531000 > User-Agent: curl/7.55.1
03:29:07.531000 > Accept: */*
03:29:07.531000 > Content-Length: 508
03:29:07.531000 > Content-Type: application/x-www-form-urlencoded
03:29:07.531000 >
03:29:07.546000 * upload completely sent off: 508 out of 508 bytes
03:29:16.671000 < HTTP/1.1 200 OK
03:29:16.671000 < Cache-Control: no-cache
03:29:16.671000 < Pragma: no-cache
03:29:16.687000 < Expires: -1
03:29:16.687000 < X-AspNet-Version: 4.0.30319
03:29:16.687000 < X-Powered-By: ASP.NET
03:29:16.687000 < Date: Fri, 22 May 2020 01:29:14 GMT
03:29:16.687000 < Content-Type: text/xml; charset=utf-8
03:29:16.703000 < Server: IBM App Connect Enterprise
03:29:16.703000 < Content-Length: 243
EDIT: I'm still trying to resolve the problem - this time with help of Wireshark and user trace:
curl:
02:56:37.781000 > POST /service HTTP/1.1
after few milliseconds Wireshark detects POST message from "curl machine" - that means there are no problems with the connection
after around 10s delay SoapInput receives data. Why it takes so long?
2020-05-23 02:56:37.257076 6220 UserTrace BIP11304I: The Parser of type 'MQROOT' has been deleted from address '0x131f1312190'. This thread now has '0' cached parsers.
2020-05-23 02:56:40.591580 3684 UserTrace BIP11303I: A Parser of type 'MQROOT' has been created at address '0x131f13144a0'. This thread now has '36' cached parsers.
2020-05-23 02:56:45.143380 3684 UserTrace BIP11501I: Received data from input node 'SOAP Input'.
The input node 'SOAP Input' has received data and has propagated it to the message flow 'link'.
2020-05-23 02:56:45.143880 3684 UserTrace BIP6060I: Node 'link.SOAP Input' used parser type 'Properties' to process a portion of the incoming message of length '0' bytes beginning at offset '0'.
Fixed by restarting the machine. Any clue what exactly caused this problem?
Related
I'm trying to recreate a request that works in Postman using a Spring RestTemplate. I've tried to make the requests identical, down to the fake "User-Agent" header, but for whatever reason the one from Postman returns 200 OK and the one from RestTemplate returns 301 Moved.
What could I possibly be doing wrong?
Both requests are below, "anonymized" by removing the urls and session cookie, both of which are identical - confirmed by copying one and searching the other.
This is the working request from Postman.
Frame 1541: 638 bytes on wire (5104 bits), 638 bytes captured (5104 bits) on interface en0, id 0
Transmission Control Protocol, Src Port: 51102, Dst Port: 80, Seq: 573, Ack: 840, Len: 572
Hypertext Transfer Protocol
POST <removed> HTTP/1.1\r\n
User-Agent: Java/17.0.2\r\n
Accept: text/plain, application/json, application/*+json, */*\r\n
Content-Type: application/x-www-form-urlencoded; charset=UTF-8\r\n
Cookie: JSESSIONID="<removed>"\r\n
Host: <removed>\r\n
Connection: keep-alive\r\n
Content-Length: 63\r\n
\r\n
[Full request URI: <removed>
[HTTP request 2/2]
[Prev request in frame: 1373]
[Response in frame: 1596]
File Data: 63 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
Form item: "<removed>" = "<removed>"
Key: <removed>
Value: <removed>
This is the failing request using RestTemplate.
Frame 3365: 129 bytes on wire (1032 bits), 129 bytes captured (1032 bits) on interface en0, id 0
Transmission Control Protocol, Src Port: 51110, Dst Port: 80, Seq: 509, Ack: 1, Len: 63
[2 Reassembled TCP Segments (571 bytes): #3363(508), #3365(63)]
Hypertext Transfer Protocol
POST <removed> HTTP/1.1\r\n
User-Agent: Java/17.0.2\r\n
Accept: text/plain, application/json, application/*+json, */*\r\n
Content-Type: application/x-www-form-urlencoded;charset=UTF-8\r\n
Cookie: JSESSIONID="<removed>"\r\n
Host: <removed>\r\n
Connection: keep-alive\r\n
Content-Length: 63\r\n
\r\n
[Full request URI: <removed>
[HTTP request 1/1]
[Response in frame: 3377]
File Data: 63 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
Form item: "<removed>" = "<removed>"
Key: <removed>
Value: <removed>
In my website, only one request,(ie. the root request) is taking more time. Especially the "wait" time is high. The HTTP2.0 Header for the request that is taking more time is posted below. Do you see any issue that might cause this request to take more "wait" time?
I have just removed the domain name. Please post your thoughts. Your help would be greatly appreciated.
cache-control : max-age=0
cf-cache-status : DYNAMIC
cf-ray : 5a1adf4d010925-SEA
cf-request-id : c4e5021110925303a9200000001
content-encoding : br
content-type : text/html; charset=UTF-8
date : Thu, 11 Jun 2020 12:26:39 GMT
expect-ct : max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
expires : Thu, 11 Jun 2020 12:26:37 GMT
host-header : b7440e60b07ee7b80454eff108fab2118
link : <https://www.<domainname>.com/wp-json/>; rel="https://api.w.org/", <https://www.<domainname>.com/>; rel=shortlink
server : cloudflare
set-cookie : __cfduid=d775816f66f9fe58d133f15c6546544654078397; expires=Sat, 11-Jul-20 12:26:37 GMT; path=/; domain=.<domainname>.com; HttpOnly; SameSite=Lax
status :200
vary : Accept-Encoding
x-cache-enabled : False
x-proxy-cache : MISS
Thread Name: Thread Group 1-1
Sample Start: 2019-02-08 15:51:46 IST
Load time: 1412
Connect Time: 525
Latency: 1412
Size in bytes: 1508
Sent bytes:603
Headers size in bytes: 843
Body size in bytes: 665
Sample Count: 1
Error Count: 1
Data type ("text"|"bin"|""): text
Response code: 401
Response message: Unauthorized
Response headers:
HTTP/1.1 401 Unauthorized
Public-Key-Pins-Report-Only: pin-sha256="9n0izTnSRF+W4W4JTq51avSXkWhQB8duS2bxVLfzXsY="; pin-sha256="5kJvNEMw0KjrCAu7eXY5HZdvyCS13BbA0VJG1RSP91w="; pin-sha256="njN4rRG+22dNXAi+yb8e3UMypgzPUPHlv4+foULwl1g="; max-age=86400; includeSubDomains; report-uri="https://a.forcesslreports.com/hpkp-report/00Dq0000000DFMbm";
Expect-CT: max-age=0; report-uri="https://a.forcesslreports.com/Expect-CT-report/00Dq0000000DFMbm";
X-Content-Type-Options: nosniff
X-XSS-Protection: 1; mode=block
Content-Security-Policy: upgrade-insecure-requests
Referrer-Policy: origin-when-cross-origin
Cache-Control: no-cache,must-revalidate,max-age=0,no-store,private
Content-Type: text/html;charset=UTF-8
Expires: Thu, 01 Jan 1970 00:00:00 GMT
X-Powered-By: Salesforce.com ApexPages
P3P: CP="CUR OTR STA"
Transfer-Encoding: chunked
Looking into JMeter – Logging Into Salesforce for Automated Testing article you need to extract session id dynamic parameter from the very first request (Log In). The process is known as correlation
The Session ID parameter can be extracted using XPath Extractor, the relevant configuration would be something like:
The server rejects the request with a “401 Unauthorized” error because your app needs to be authorized. Usually Authentication request returns this value. So, you need to extract this token from the answer.
Add Cookie Manager to your Test Plan.
Add Regular Expressions Extractor to your auth request to Extract auth token to variable and send its value to the next request (where you get error) with
This guide also could be helpful how to test authentification with JMeter
Hi I am using Apache camel http4 component to send a https request with keepAlive=true but when i see the netstat after sending multiple requests I see each request opens a new TCP port to the peer.
I feel this should not be the usual behavior of keepAlive transactions, why the same TCP port is not being reused for communicating with the server and how can that be achieved if at all it can be.
Turns out this is not a keep alive problem; connections are in fact kept alive properly. The problem is that the connections as part of the pool managed by default PoolingHttpClientConnectionManager were not being reused. Both things could've been easily seen by enabling the logging for Apache's HttpClient (that's used under the hood):
Keep-Alive is being used:
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "HTTP/1.1 200 OK[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Keep-Alive: timeout=5, max=300[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Server: Apache-Coyote/1.1[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Content-Encoding: gzip[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Vary: Accept-Encoding[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Cluster-Id: A[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Date: Wed, 19 Dec 2018 06:59:17 GMT[\r][\n]"
2018/12/19 07:59:17:470 CET [DEBUG] wire - http-outgoing-7 << "Content-Type: text/xml[\r][\n]"
2018/12/19 07:59:17:471 CET [DEBUG] wire - http-outgoing-7 << "Content-Length: 239[\r][\n]"
2018/12/19 07:59:17:471 CET [DEBUG] wire - http-outgoing-7 << "[\r][\n]"
Connections not being reused:
2018/12/19 08:00:08:240 CET [DEBUG] PoolingHttpClientConnectionManager - Connection request: [route: {s}->https://someurl.com:443][total kept alive: 1; route allocated: 1 of 1; total allocated: 1 of 1]
2018/12/19 08:00:08:240 CET [DEBUG] DefaultManagedHttpClientConnection - http-outgoing-7: Close connection
2018/12/19 08:00:08:242 CET [DEBUG] PoolingHttpClientConnectionManager - Connection leased: [id: 8][route: {s}->https://someurl.com:443][total kept alive: 0; route allocated: 1 of 1; total allocated: 1 of 1]
Note that one can easily enable logging for HttpClient by passing some arguments to JVM at startup.
So, why are connections not being reused then? This is due to the fact that SSL is being used and the PoolingHttpClientConnectionManager used by Apache's HttpClient not allowing reuse of a connection in case the user principal from the existing connection differs from the requested connection (facilitated by DefaultUserTokenHandler). See also e.g. this Stackoverflow post. Solution is to implement a custom UserTokenHandler (or use the NullTokenHandler if that's sufficient) and configure the HttpClientBuilder accordingly.
Did you already analyze the HTTP response headers in order to check whether KeepAlive is being taken into account and with which timeout value.
Example of expected response:
HTTP/1.1 200 OK
Connection: Keep-Alive
Keep-Alive: timeout=10, max=20
Content-Type: text/html; charset=UTF-8
Date: ...
Content-Length: ...
I've encountered the same issue as in this question, using Spring Boot 1.3.0 and not having my controllers annotated with #RestController, just #Path and #Service. As the OP in that question says,
this is, to me, anything but sensible
I also can't understand why would they have it redirect to /error. And it is very likely that I'm missing something, because I can only give back 404s or 200s to the client.
My problem is that his solution doesn't seem to work with 1.3.0, so I have the following request flow: let's say my code throws a NullPointerException. It'll be handled by one of my ExceptionMappers
#Provider
public class GeneralExceptionMapper implements ExceptionMapper<Throwable> {
private static final Logger LOGGER = LoggerFactory.getLogger(GeneralExceptionMapper.class);
#Override
public Response toResponse(Throwable exception) {
LOGGER.error(exception.getLocalizedMessage());
return Response.status(Response.Status.INTERNAL_SERVER_ERROR).build();
}
}
And my code returns a 500, but instead of sending it back to the client, it tries to redirect it to /error. If I don't have another resource for that, it'll send back a 404.
2015-12-16 18:33:21.268 INFO 9708 --- [nio-8080-exec-1] o.glassfish.jersey.filter.LoggingFilter : 1 * Server has received a request on thread http-nio-8080-exec-1
1 > GET http://localhost:8080/nullpointerexception
1 > accept: */*
1 > host: localhost:8080
1 > user-agent: curl/7.45.0
2015-12-16 18:33:29.492 INFO 9708 --- [nio-8080-exec-1] o.glassfish.jersey.filter.LoggingFilter : 1 * Server responded with a response on thread http-nio-8080-exec-1
1 < 500
2015-12-16 18:33:29.540 INFO 9708 --- [nio-8080-exec-1] o.glassfish.jersey.filter.LoggingFilter : 2 * Server has received a request on thread http-nio-8080-exec-1
2 > GET http://localhost:8080/error
2 > accept: */*
2 > host: localhost:8080
2 > user-agent: curl/7.45.0
2015-12-16 18:33:37.249 INFO 9708 --- [nio-8080-exec-1] o.glassfish.jersey.filter.LoggingFilter : 2 * Server responded with a response on thread http-nio-8080-exec-1
2 < 404
And client's side (curl):
$ curl -v http://localhost:8080/nullpointerexception
* STATE: INIT => CONNECT handle 0x6000572d0; line 1090 (connection #-5000)
* Added connection 0. The cache now contains 1 members
* Trying ::1...
* STATE: CONNECT => WAITCONNECT handle 0x6000572d0; line 1143 (connection #0)
* Connected to localhost (::1) port 8080 (#0)
* STATE: WAITCONNECT => SENDPROTOCONNECT handle 0x6000572d0; line 1240 (connection #0)
* STATE: SENDPROTOCONNECT => DO handle 0x6000572d0; line 1258 (connection #0)
> GET /nullpointerexception HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.45.0
> Accept: */*
>
* STATE: DO => DO_DONE handle 0x6000572d0; line 1337 (connection #0)
* STATE: DO_DONE => WAITPERFORM handle 0x6000572d0; line 1464 (connection #0)
* STATE: WAITPERFORM => PERFORM handle 0x6000572d0; line 1474 (connection #0)
* HTTP 1.1 or later with persistent connection, pipelining supported
< HTTP/1.1 404 Not Found
* Server Apache-Coyote/1.1 is not blacklisted
< Server: Apache-Coyote/1.1
< Content-Length: 0
< Date: Wed, 16 Dec 2015 17:33:37 GMT
<
* STATE: PERFORM => DONE handle 0x6000572d0; line 1632 (connection #0)
* Curl_done
* Connection #0 to host localhost left intact
So it's always a 404. Unless I do have such an /error resource, then what? what am I supposed to return? All I have at that point is a GET request to /error. And I don't want those extra requests consuming resources and polluting my logs.
What am I missing? And if nothing, what should I do with my exception handling?
You can set the Jersey property ServerProperties.RESPONSE_SET_STATUS_OVER_SEND_ERROR to true.
Whenever response status is 4xx or 5xx it is possible to choose between sendError or setStatus on container specific Response implementation. E.g. on servlet container Jersey can call HttpServletResponse.setStatus(...) or HttpServletResponse.sendError(...).
Calling sendError(...) method usually resets entity, response headers and provide error page for specified status code (e.g. servlet error-page configuration). However if you want to post-process response (e.g. by servlet filter) the only way to do it is calling setStatus(...) on container Response object.
If property value is true the method Response.setStatus(...) is used over default Response.sendError(...).
Type of the property value is boolean. The default value is false.
You can set Jersey property simply by calling property(key, value) in your ResourceConfig subclass constructor.