I wanted to know is there any way programmatically in win32, where I can get the URL being fetched from browser.
Like for example as we have MIB table, which shows data sent and received from Ethernet card, can we get the URL being fetched from system
Thanks in advance.
This is an IE-only solution, but if you write a browser helper object, it will be notified before IE navigates to a new URL.
There is no simple way to do this. The main problem you will encounter is that each browser on your system will independently connect to a webserver. That's just a straightforward HTTP connection, usually on port 80. The browser will send the URL in an HTTP request, possibly in multiple TCP packets. So, unless you are going to inspect and reassemble those TCP packets, you're not going to get this information. Even if you did, you'd miss out on the URLs of HTTPS fetches (by design).
An easier solution is to set up a proxy, and hope that the webbrowser doesn't bypass it.
You could try using WinPCap, which is what's utilized by Wireshark. What this would allow you to do is put the network interface into "promiscuous mode," and from there you could just look for HTTP traffic. From there, you could extract the URLs that are being requested, no matter which browser's being used.
Related
I am trying to find (or write) a caching proxy tool that accepts all traffic from a specific container in my localhost (using Iptables). What I want to do with this traffic is to save it and cache the response, and later, if I see that a request was already sent to a server, return the cached response to the requesting party (and not sending the request to the server again, because a previous similar request was already sent).
Here's a diagram to demonstrate what I'm trying to do:
I'm not sure exactly how big is the problem I'm trying to deal with here. I want to do it for all traffic, including HTTP, TLS and other TCP based traffic (database connections and such). I tried to check mitmproxy, and it seems to deal pretty good with HTTP and the TLS part, but intercepting raw TCP traffic (for databases etc.) is not possible.
Any advices or resources I can use to accomplish that? (Not necessarily in Python). How complex do you think this problem is? Do you think I can find a generic solution?
Thanks in advance!
Context: I have an application which communicates with a server of the owner. This application requests certain information from the server by accessing some URL's. I don't know these URL's, except for a few, but once I know them, I can manually visit them in the browser and obtain said information.
Goal: Figuring out the URLs of those requests, i.e., what are the requests being made by the application when I'm using it, so that I can, in the future, make them manually myself.
Progress:
Since the communications are in HTTPS, reading the packets with Wireshark while using the application was unsuccessfull since they are encrypted.
However, I was able to find where in the binary of the application is the URL of the server located. Thus, I can theoretically redirect the requests of the application to any other server. Hence, I thought a good idea to be able to receive the unencrypted requests would be to set up a proxy server, redirect the application to it, and then execute the application and obtain the results.
Problem: I don't know how to implement this idea in practice though, and it is here where I'd appreciate your help. I suppose that, ideally, I would be able to both receive the requests made by the application (and thus read them), as well as redirect them to the server and read the received information.
I have a program that I'm trying to reverse engineer.
It gets a specific key by using HTTP GET on some URLs.
I need to figure out the details on how this works.
The good news is that there's the option to preform these requests over an HTTP proxy.
Would anybody know of a program to monitor a specific application's network traffic?
I've tried Wireshark, but its no giving me enough information (Headers, URL path).
After Wireshark, I tried FreeProxy. The problem with FreeProxy is that it only gives headers for around 1/3 of the requests and it doesn't give the full path either.
Could anyone suggest a better alternative for monitoring the internet activity of my application?
I thought Wireshark was able to capture the full packet with all its content? If so, how can it not give you enough information? Maybe you need to revise your traffic capture config?
It's been a while since I used Wireshark, but if you have trouble capturing full packets, what you can do is use tcpdump to capture and write to file, then view the capture file using Wireshark. tcpdump's -s option will allow you to set the packet size so as to capture full packets.
I use Fiddler for all my HTTP traffic monitoring. It is very powerful and displays data in the HTTP layer only. Wireshark will get all of your data, but it displays the details at a much lower layer. It even has capability to decrypt SSL traffic.
Fiddler installs itself as a proxy, and configures IE and FF automatically to use it when it is on. If you are having too much traffic mix in, then you can install Fiddler on a remote box, and point your proxy to that IP address.
I was recommemded another program called "mitmproxy" which worked perfectly for what I needed. Fiddler also worked, but SSL was giving me problems.
This might sound really naive but I would really find a descriptive answer helpful.
So, my question is this:
I can use Firebug to look at AJAX requests made from any website I visit. So, am I right in saying that I wouldn't be able to examine the same communication between the client and the server if the website choses to use Websockets? In other words, does this make it more secure?
No. Not at all. Just because the browser does not (yet) have a tool to show WebSocket traffic, doesn't make it any more secure. You can always run a packet sniffer to monitor the traffic, for example.
No, because there will be other ways beside the browser-build in tools to read your traffic.
Have a try: Install and run Wireshark and you will be able to see all packets you send and receive via Websockets.
Depends on the application. If you are fully Ajax without reloading the document for data then I would think websockets would provide a better authentication for data requests then a cookie session in regards to connection hijack. Given that you are using SSL of course.
Never rely on secrecy of algorithm cause it only gives you false sense of security. Wiki: Security by obscurity
Remember that browser is a program on my computer and I am the one who have a full control over what is send to you, not my browser.
I guess it's only matter of time (up to few months IMO) when developer tools such as Firebug will provide some fancy tool for browsing data send/received by WebSockets.
WebSockets has both an unencrypted (ws://) and encrypted mode (wss://). This is analogous to HTTP and HTTPS. WebSockets protocol payload is simply UTF-8 encoded. From a network sniffing perspective there is no advantage to using WebSockets (use wss and HTTPS for everything at all sensitive). From the browser perspective there is no benefit to using WebSockets for security. Anything running in the browser can be examined (and modified) by a sufficiently knowledgeable user. The tools for examining HTTP/AJAX requests just happen to be better right now.
I want to set something up on my computer where if something (program, browser, whatever) makes a request (to a certain domain, I guess) I can send my own response to the request. I once saw a program that did that, and I was wondering how it did it. How would it be done?
How about just editing hosts file? Popular operating systems have it. Just use it to redirect to whereever you want. To me it looks like the simplest solution to this question.
You can build a HTTP proxy that will intercept all requests from the browser. But you have to configure the browser to use the proxy.
What mikerobi said, but you can also set up your computer or gateway machine to act as a transparent proxy. (You need this if you either don't want to configure your browser, or you're working with an application that uses HTTP but doesn't allow for configuration of a proxy.)
On a FreeBSD or Linux machine (I'm sure there are other OSes that can do this; I mention the ones I use), you can set up your own firewall rules to intercept particular traffic (say, all port 80 traffic, or all port 80 traffic to a particular domain, or whatever) and forward the traffic to your own special proxy, which can return whatever you want.