Has anyone already found this issue at capturing traffic of snapchat?
Every https data deriving from other sites via application (of ios, android) are captured successfully by Fiddler but a few apps (appstore, snapchat) don't show anything just that request:
CONNECT app.snapchat.com:443 HTTP/1.1
Host: app.snapchat.com
User-Agent: Snapchat/10.8.1.0 (iPhone8,1; iOS 10.2.1; gzip)
Connection: keep-alive
Connection: keep-alive
A SSLv3-compatible ClientHello handshake was found. Fiddler extracted the parameters below.
Version: 3.3 (TLS/1.2)
Random: 59 23 9E E1 1C 23 49 F1 A1 21 6E 60 C5 94 AB E2 9F 09 10 C3 E0 C3 99 9B 78 9B 97 1F 74 69 5F 1C
"Time": 2089.12.12. 15:48:57
SessionID: empty
Extensions:
server_name app.snapchat.com
elliptic_curves secp256r1 [0x17], secp384r1 [0x18], secp521r1 [0x19]
ec_point_formats uncompressed [0x0]
signature_algs sha256_rsa, sha1_rsa, sha384_rsa, sha512_rsa, sha256_ecdsa, sha1_ecdsa, sha384_ecdsa, sha512_ecdsa
NextProtocolNego empty
ALPN http/1.1, http/1.0
status_request OCSP - Implicit Responder
SignedCertTimestamp (RFC6962) empty
extended_master_secret empty
Ciphers:
[00FF] TLS_EMPTY_RENEGOTIATION_INFO_SCSV
[C02C] TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384
[C02B] TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256
[C024] TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384
[C023] TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256
[C00A] TLS1_CK_ECDHE_ECDSA_WITH_AES_256_CBC_SHA
[C009] TLS1_CK_ECDHE_ECDSA_WITH_AES_128_CBC_SHA
[C030] TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384
[C02F] TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256
[C028] TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384
[C027] TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256
[C014] TLS1_CK_ECDHE_RSA_WITH_AES_256_CBC_SHA
[C013] TLS1_CK_ECDHE_RSA_WITH_AES_128_CBC_SHA
Compression:
[00] NO_COMPRESSION
What would be done for the unimpeded working?
Extending this issue more what's the reason the fiddler (or other interceptors) isn't able to capture all https data but their Connect handshakes?
Probably this is due to certificate pinning. Snapchat is know to use this to prevent MITM attacks.
Take a look at this answer here:
https://stackoverflow.com/a/40543302/1353689
and the links related to it as well.
Related
I'm trying to recreate a request that works in Postman using a Spring RestTemplate. I've tried to make the requests identical, down to the fake "User-Agent" header, but for whatever reason the one from Postman returns 200 OK and the one from RestTemplate returns 301 Moved.
What could I possibly be doing wrong?
Both requests are below, "anonymized" by removing the urls and session cookie, both of which are identical - confirmed by copying one and searching the other.
This is the working request from Postman.
Frame 1541: 638 bytes on wire (5104 bits), 638 bytes captured (5104 bits) on interface en0, id 0
Transmission Control Protocol, Src Port: 51102, Dst Port: 80, Seq: 573, Ack: 840, Len: 572
Hypertext Transfer Protocol
POST <removed> HTTP/1.1\r\n
User-Agent: Java/17.0.2\r\n
Accept: text/plain, application/json, application/*+json, */*\r\n
Content-Type: application/x-www-form-urlencoded; charset=UTF-8\r\n
Cookie: JSESSIONID="<removed>"\r\n
Host: <removed>\r\n
Connection: keep-alive\r\n
Content-Length: 63\r\n
\r\n
[Full request URI: <removed>
[HTTP request 2/2]
[Prev request in frame: 1373]
[Response in frame: 1596]
File Data: 63 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
Form item: "<removed>" = "<removed>"
Key: <removed>
Value: <removed>
This is the failing request using RestTemplate.
Frame 3365: 129 bytes on wire (1032 bits), 129 bytes captured (1032 bits) on interface en0, id 0
Transmission Control Protocol, Src Port: 51110, Dst Port: 80, Seq: 509, Ack: 1, Len: 63
[2 Reassembled TCP Segments (571 bytes): #3363(508), #3365(63)]
Hypertext Transfer Protocol
POST <removed> HTTP/1.1\r\n
User-Agent: Java/17.0.2\r\n
Accept: text/plain, application/json, application/*+json, */*\r\n
Content-Type: application/x-www-form-urlencoded;charset=UTF-8\r\n
Cookie: JSESSIONID="<removed>"\r\n
Host: <removed>\r\n
Connection: keep-alive\r\n
Content-Length: 63\r\n
\r\n
[Full request URI: <removed>
[HTTP request 1/1]
[Response in frame: 3377]
File Data: 63 bytes
HTML Form URL Encoded: application/x-www-form-urlencoded
Form item: "<removed>" = "<removed>"
Key: <removed>
Value: <removed>
When I use curl to post a SOAP message into message flow, it takes 9 seconds to respond. Debug won't stop at breakpoints and User Trace doesn't report anything. Meanwhile when I do the same request from Postman or SoapUI (the first message takes the same amount of time, later all messages take around 70 - 200ms) debugger and user trace work as intended. What is cause of this behavior?
IBM App Connect Enterprise 11.0.0.8
curl --trace-time output:
03:29:07.484000 * Trying <host>...
03:29:07.484000 * TCP_NODELAY set
03:29:07.531000 * Connected to <host_name> (<host>) port 7800 (#0)
03:29:07.531000 > POST /service HTTP/1.1
03:29:07.531000 > Host: <host_name>:7800
03:29:07.531000 > User-Agent: curl/7.55.1
03:29:07.531000 > Accept: */*
03:29:07.531000 > Content-Length: 508
03:29:07.531000 > Content-Type: application/x-www-form-urlencoded
03:29:07.531000 >
03:29:07.546000 * upload completely sent off: 508 out of 508 bytes
03:29:16.671000 < HTTP/1.1 200 OK
03:29:16.671000 < Cache-Control: no-cache
03:29:16.671000 < Pragma: no-cache
03:29:16.687000 < Expires: -1
03:29:16.687000 < X-AspNet-Version: 4.0.30319
03:29:16.687000 < X-Powered-By: ASP.NET
03:29:16.687000 < Date: Fri, 22 May 2020 01:29:14 GMT
03:29:16.687000 < Content-Type: text/xml; charset=utf-8
03:29:16.703000 < Server: IBM App Connect Enterprise
03:29:16.703000 < Content-Length: 243
EDIT: I'm still trying to resolve the problem - this time with help of Wireshark and user trace:
curl:
02:56:37.781000 > POST /service HTTP/1.1
after few milliseconds Wireshark detects POST message from "curl machine" - that means there are no problems with the connection
after around 10s delay SoapInput receives data. Why it takes so long?
2020-05-23 02:56:37.257076 6220 UserTrace BIP11304I: The Parser of type 'MQROOT' has been deleted from address '0x131f1312190'. This thread now has '0' cached parsers.
2020-05-23 02:56:40.591580 3684 UserTrace BIP11303I: A Parser of type 'MQROOT' has been created at address '0x131f13144a0'. This thread now has '36' cached parsers.
2020-05-23 02:56:45.143380 3684 UserTrace BIP11501I: Received data from input node 'SOAP Input'.
The input node 'SOAP Input' has received data and has propagated it to the message flow 'link'.
2020-05-23 02:56:45.143880 3684 UserTrace BIP6060I: Node 'link.SOAP Input' used parser type 'Properties' to process a portion of the incoming message of length '0' bytes beginning at offset '0'.
Fixed by restarting the machine. Any clue what exactly caused this problem?
I'm seeing a bunch of messages on the dead letter queue and I don't understand what causes this.
I'm using MQ Explorer to browse such messages. Here's what I see in the Dead-Letter Header:
This doesn't tell me what the real cause of the problem is. How can I find out ?
I've read this article from IBM and it tells that the reason is likely a badly formatted message. In what way badly formatted?
(note: I'm in control of both producer and consumer)
you miss the forest for the trees :-)
The cause is in the field 'Reason' .
MQRC_BACKOUT_THRESHOLD_REACHED
This is described here in Knowledge Center
MQRC_BACKOUT_THRESHOLD_REACHED (0x93A; 2362)
Cause
The message has reached the Backout Threshold defined on the QLOCAL, but no Backout
Queue is defined. On platforms where you cannot define the Backout
Queue, the message has reached the JMS-defined backout threshold of
20.
Action
If this is not wanted, define the Backout Queue for the relevant QLOCAL. Also look for the cause of the multiple backouts.
As I expected the MQRC_BACKOUT_THRESHOLD_REACHED reason is really just a knock-on effect. To find the real reason you would need to look into the logs on either the consumer-side or the producer-side depending on what you see in the Put application name field in the screenshot from the dead-letter header (above).
I've now learned that MQ Classes for JMS produces log files in current dir named mqjms.log.x. By looking at this I can see the true reason for the problem:
July 19, 2016 4:48:33 PM CEST[Queue Service thread] com.ibm.msg.client.wmq.common.internal.messages.WMQReceiveMarshal
A received message could not be correctly parsed.
EXPLANATION:
The message with messageID = '414D512064657620202020202020202012048D5720064E04' and correlationID = '310000000000000000000000000000000000000000000000' could not be correctly parsed. The last successful data read from the message was at position '192' in buffer '0000: 5246 4820 0000 0002 0000 00c0 0000 0111 RFH ............
0010: 0000 04b8 4d51 5354 5220 2020 0000 0000 ....MQSTR ....
0020: 0000 04b8 0000 0020 3c6d 6364 3e3c 4d73 ....... <mcd><Ms
0030: 643e 6a6d 735f 7465 7874 3c2f 4d73 643e d>jms_text</Msd>
0040: 3c2f 6d63 643e 2020 0000 0074 3c6a 6d73 </mcd> ...t<jms
0050: 3e3c 4473 743e 7175 6575 653a 2f2f 2f6d ><Dst>queue:///m
0060: 7971 7565 7565 3c2f 4473 743e 3c54 6d73 yqueue</Dst><Tms
0070: 3e31 3436 3839 3339 3731 3338 3234 3c2f >1468939713824</
0080: 546d 733e 3c45 7870 3e31 3436 3839 3339 Tms><Exp>1468939
0090: 3734 3338 3234 3c2f 4578 703e 3c43 6964 743824</Exp><Cid
00a0: 3e49 443a 3331 3c2f 4369 643e 3c44 6c76 >ID:31</Cid><Dlv
00b0: 3e31 3c2f 446c 763e 3c2f 6a6d 733e 2020 >1</Dlv></jms>
00c0: 3c64 6174 614d 7367 2073 656e 7454 696d <mymessage .....
.................
' with exception '
Message : java.lang.Exception
Class : class java.lang.Exception
Stack : com.ibm.msg.client.wmq.internal.WMQConsumerShadow.getMsg(WMQConsumerShadow.java:1900)
: com.ibm.msg.client.wmq.internal.WMQSyncConsumerShadow.receiveInternal(WMQSyncConsumerShadow.java:231)
: com.ibm.msg.client.wmq.internal.WMQConsumerShadow.receive(WMQConsumerShadow.java:1471)
: com.ibm.msg.client.wmq.internal.WMQMessageConsumer.receive(WMQMessageConsumer.java:659)
: com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.receiveInboundMessage(JmsMessageConsumerImpl.java:1036)
: com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.receive(JmsMessageConsumerImpl.java:461)
: com.ibm.msg.client.jms.internal.JmsMessageConsumerImpl.receive(JmsMessageConsumerImpl.java:495)
: com.ibm.mq.jms.MQMessageConsumer.receive(MQMessageConsumer.java:209)
: org.mycorp.client.base.connection.QueueService.recvMessages(QueueService.java:129)
: org.mycorp.client.base.connection.QueueService.run(QueueService.java:92)
: java.lang.Thread.run(Thread.java:745)
' with MQMD 'version:2(0x2) report:0(0x0) msgType:8(0x8) expiry:300(0x12c) feedback:0(0x0) encoding:273(0x111) codedCharSetId:1208(0x4b8) format:'MQHRF2 ' priority:4(0x4) persistence:0(0x0) msgId:414D512064657620202020202020202012048D5720064E04 correlId:310000000000000000000000000000000000000000000000 backoutCount:0(0x0) replyToQ:' ' replyToQMgr:'dev ' userIdentifier:'peter ' accountingToken:160105150000008D3439C9CC13CC025B66F34BE903000000000000000000000B applIdentityData:' ' putApplType:28(0x1c) putApplName:'Carrefour Server ' putDate:'20160719' putTime:'14483382' applOriginData:' ' groupId:000000000000000000000000000000000000000000000000 msgSeqNumber:1(0x1) physicalMsgOffset:0(0x0) msgFlags:0(0x0) originalLength:-1(0xffffffff) '
EXPLANATION:
null
ACTION:
null
So there you have it. Somehow I've managed to create a JMS message which is invalid and without the producer-side noticing that there's a problem.
I'll need to figure that out but that'll be a topic for another post.
In short the answer to the question is: The backout is just a knock-on effect. The real reason is - as the doc from IBM says - a badly formatted message. The only method to figure that out is to look into whatever logs are dumped by either the message producer or (more likely) the message consumer. In my case - since no remote queues are involved - my message consumer is not an intermediary Queue Manager but my actual destination application. That's where I found the log.
I have one application WAR. I have created two profiles AppSrv01, AppSrv02 on Websphere 8.0.0.0.
My application is having MQ connectivity, J2ee application on struts 1.1.
I have deployed WAR on both the server AppSrv01 as a context A and AppSrv02 as a context B.
I have session listener in my application which will take note of total available session.
The scenario is as given below....
Going and login to url localhost:9082/A, go to some page in it.
Going and login to url localhost:9082/B, go to some page in it in another tab of same browser.
Going to url localhost:9082/A, clicking on some of link or url from the existing page.
on remote debug from server when it come to SessionListener ... it shows below given stack
Daemon Thread [WebContainer : 10] (Suspended (breakpoint at line 24 in SessionCounterListener))
SessionCounterListener.sessionCreated(HttpSessionEvent) line: 24
WasHttpSessionObserver(HttpSessionObserver).sessionCreated(ISession) line: 111
SessionEventDispatcher.sessionCreated(ISession) line: 98
SessionManager.createISession(String, int, boolean) line: 268
SessionManager.createSession(ServletRequest, ServletResponse, SessionAffinityContext, boolean) line: 640
WsSessionContext(SessionContext).getIHttpSession(HttpServletRequest, HttpServletResponse, boolean, boolean) line: 485
WsSessionContext(SessionContext).getIHttpSession(HttpServletRequest, HttpServletResponse, boolean) line: 419
SRTRequestContext.getSession(boolean, WebApp) line: 104
SRTServletRequest.getSession(boolean) line: 2099
SRTServletRequest.getSession() line: 2083
RequestProcessor.processLocale(HttpServletRequest, HttpServletResponse) line: 631
RequestProcessor.process(HttpServletRequest, HttpServletResponse) line: 230
ActionServlet.process(HttpServletRequest, HttpServletResponse) line: 1482
ActionServlet.doPost(HttpServletRequest, HttpServletResponse) line: 525
ActionServlet(HttpServlet).service(HttpServletRequest, HttpServletResponse) line: 595
ActionServlet(HttpServlet).service(ServletRequest, ServletResponse) line: 668
ServletWrapperImpl(ServletWrapper).service(ServletRequest, ServletResponse, WebAppServletInvocationEvent) line: 1147
ServletWrapperImpl(ServletWrapper).handleRequest(ServletRequest, ServletResponse, WebAppDispatcherContext) line: 722
ServletWrapperImpl(ServletWrapper).handleRequest(ServletRequest, ServletResponse) line: 449
ServletWrapperImpl.handleRequest(ServletRequest, ServletResponse) line: 178
WebAppFilterManagerImpl(WebAppFilterManager).invokeFilters(ServletRequest, ServletResponse, IServletContext, RequestProcessor, EnumSet<CollaboratorInvocationEnum>) line: 1020
CacheServletWrapper.handleRequest(ServletRequest, ServletResponse) line: 87
WSWebContainer(WebContainer).handleRequest(IRequest, IResponse) line: 883
WSWebContainer.handleRequest(IRequest, IResponse) line: 1659
WCChannelLink.ready(VirtualConnection) line: 195
HttpInboundLink.handleDiscrimination() line: 452
HttpInboundLink.handleNewRequest() line: 511
HttpInboundLink.processRequest() line: 305
HttpInboundLink.ready(VirtualConnection) line: 276
NewConnectionInitialReadCallback.sendToDiscriminators(VirtualConnection, TCPReadRequestContext, boolean) line: 214
NewConnectionInitialReadCallback.complete(VirtualConnection, TCPReadRequestContext) line: 113
AioReadCompletionListener.futureCompleted(IAbstractAsyncFuture, Object) line: 165
AsyncFuture(AbstractAsyncFuture).invokeCallback(ICompletionListener, AbstractAsyncFuture, Object) line: 217
AsyncFuture(AsyncChannelFuture).fireCompletionActions() line: 161
AsyncFuture.completed(long) line: 138
ResultHandler.complete(AsyncFuture, int, int) line: 204
ResultHandler.runEventProcessingLoop(boolean) line: 775
ResultHandler$2.run() line: 905
ThreadPool$Worker.run() line: 1648
It is creating another session instance. need to understand why it is creating another instance.
When you open the tabs in IE, for IE its same session. So you will have only one cookie JSESSIONID per domain.
when you go to localhost:9082/A - you have JSESSIONID like SESSIONID-A.
when you go to localhost:9082/B - you have JSESSIONID like SESSIONID-B.
So when you access application on AppSrv01, JSESSIONID goes to AppSrv01 is SESSIONID-B. Appserver doesn't know about this session id, so it thinks its a new user and creates a session.
Try opening a IE-> File -> New Session and access the application on AppSrv02 you won't face the above issue.
If you have something like IEWATCH or Fidler, you will be able to see the JSESSIONID going from client/browser.
There's also a setting HttpSessionIdReuse that might help with this as well. I know it works for multiple Servers in the same Cell, but I'm not sure whether it works across Profiles.
http://www-01.ibm.com/support/docview.wss?uid=swg21210881
I use G-wan download files from directory 'www' ,it can download the files size less than 16M,if the files size is 16M, the G-wan is wrong:
Signal : 11:Unknown SIGSEGV problem
Signal src : 128:.
errno : 0
Thread : 1
Code Pointer: 0000004081a8 (module:gwan, function:??, line:0)
Access Address: 000000000000
Registers : EAX=7f1dbc0afa4e CS=00000033 EIP=0000004081a8 EFLGS=000000010287
EBX=7f1dc24af000 SS=d5ab0400 ESP=7f1dd59acd30 EBP=7f1dbc0afa4e
ECX=ffe3e5a1746a4230 DS=d5ab0400 ESI=1c997c47a33a4e FS=00000033
EDX=1c1a5e8b984000 ES=d5ab0400 EDI=7f1dbc0afa3e CS=00000033
Module :Function :Line # PgrmCntr(EIP) RetAddress FramePtr(EBP)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Segmentation fault
thanks for Gil.
Os: debian 7,rhel6.4
dwonload the file size is 80M ,file:0.0.0.0_8080/#0.0.0.0/www/MFC-7340-inst-B2-zh.EXE.this is trace content:
Wed, 14 Aug 2013 02:57:36 GMT: start
Wed, 14 Aug 2013 02:58:00 GMT 12 127.0.0.1
GET /MFC-7340-inst-B2-zh.EXE HTTP/1.1^M
Host: localhost:8080^M
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:17.0) Gecko/20130806 Firefox/17.0 Iceweasel/17.0.8^M
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8^M
Accept-Language: en-US,en;q=0.5^M
Accept-Encoding: gzip, deflate^M
Connection: keep-alive^M
Cookie: SESS49960de5880e8c687434170f6476605b=iOQ_BgGEgAWbhB9zA0U5jZbEOXzmu3nkzhyBW00GJGM; Drupal.tableDrag.showWeight=0; Drupal.toolbar.collapsed=1; MOIN_SESSION_800_ROOT=8a77c4a58fefc1ad6b6830b5bc51ca680a480bae^M
^M
Wed, 14 Aug 2013 02:58:02 GMT: signal 11: Unknown SIGSEGV problem
source : 128:Kernel
thread : 0/2
state : SEND
client : 127.0.0.1:58472
request : /MFC-7340-inst-B2-zh.EXE
----------------
----------------
Signal : 11:Unknown SIGSEGV problem
Signal src : 128:.
errno : 0
Thread : 0
Code Pointer: 0000004081a8 (module:gwan, function:??, line:0)
Access Address: 000000000000
Registers : EAX=7f85482fd0c0 CS=00000033 EIP=0000004081a8 EFLGS=000000010293
EBX=7f8565501000 SS=6cd00400 ESP=7f856cbfcd30 EBP=7f85482fd0c0
ECX=ffeea9eeb46863d5 DS=6cd00400 ESI=11d59693c9ef4b FS=00000033
EDX=1156114b9a1e8b ES=6cd00400 EDI=7f85482fd0b0 CS=00000033
Module :Function :Line # PgrmCntr(EIP) RetAddress FramePtr(EBP)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
G-WAN has been used to serve much larger file sizes. Could you specify the exact file size you have used so we can try to duplicate this issue?
Generating a graceful crash report (your trace) is as easy as using the crash script examples given with the download archive so we need a bit more information to start answering your question.