Take this scenario:
I send an HTML email with images embedded in it - such as a newsletter with images placed inside using an tag.
I open the email
An image is requested
During that image request, what is the full set of information sent to the server? Can someone paste it in?
Would love to see the full request & header when the email is opened in a client like Outlook and at least one webmail platform like Gmail.
As to the comment about which server -- I guess HTTP? If I'm viewing an email in Gmail, and an tag is served, that is being served over HTTP, right?
Thanks!
As far as the client computer is concerned, mail client applications are completely different from a web-based mail front-end.
Typically, mail client applications (formally Mail User Agents or MUAs) use a mail-specific protocol such as POP3 or IMAP to access a mailbox on a remote server.
Depending on the capabilities of the protocol, the MUA may have to download a whole message before displaying anything (POP3) or it may be able to request each MIME attachment separately (IMAP). Usually images are included as MIME attachments and are encoded in a printable form (i.e. restricted to 7-bit ASCII) using something like Base64, in order to ensure delivery without corruption.
Web-based mail front-ends, on the other hand, are accessed through the generic HTTP protocol - the message decoding happens at the remote server. Images are retrieved through normal GET requests like every other image.
A few years back, you might have been able to see both cases at work using a network sniffer tool such as Wireshark. These days most decent mail providers prefer the encrypted versions of these protocols (POP3S, IMAPS and HTTPS) which cannot be usefully studied using a network sniffer.
If you really wish to have a look at the protocols, you might still be able to find a POP3/IMAP service that allows unencrypted connections, although it's probably not going to be a big provider. You could also set-up your own server for experimentation.
If you use Firefox, on the other hand, there is at least one add-on that allows you to see the requests made by the browser, by intercepting the requests before they are encrypted. It will allow you to see what happens as you use e.g. Gmail.
Related
Context: I have an application which communicates with a server of the owner. This application requests certain information from the server by accessing some URL's. I don't know these URL's, except for a few, but once I know them, I can manually visit them in the browser and obtain said information.
Goal: Figuring out the URLs of those requests, i.e., what are the requests being made by the application when I'm using it, so that I can, in the future, make them manually myself.
Progress:
Since the communications are in HTTPS, reading the packets with Wireshark while using the application was unsuccessfull since they are encrypted.
However, I was able to find where in the binary of the application is the URL of the server located. Thus, I can theoretically redirect the requests of the application to any other server. Hence, I thought a good idea to be able to receive the unencrypted requests would be to set up a proxy server, redirect the application to it, and then execute the application and obtain the results.
Problem: I don't know how to implement this idea in practice though, and it is here where I'd appreciate your help. I suppose that, ideally, I would be able to both receive the requests made by the application (and thus read them), as well as redirect them to the server and read the received information.
i'm too beginner in squid. i want a way to remain anonymous over the net. i also want to be able to access the contents of the internet which are filtered. my Windows computer is beyond firewall (filtered). my server (CentOS 5) is not. for example, when i enter http://facebook.com in the browser url, it redirects to an intranet ip which tells me to avoid going to this site!
now i've installed squid on server and traffic is propagated through this server. but this redirection occurs. so still i can't open filtered sites.
what can i do? a friend of mine told that the only way is to use https. ie. the connection between browser (Firefox) and the server must use this protocol. is it right? and how can i do that?
what's your suggestion? i don't want necessarily to use squid. besides, https protocol gets banned or decreased in speed in my country sometimes. so i prefer the protocol remain http. i thought also about writing a code in client and server to transform, compress/decompress and packetize as hoax binary http packets to be sent as much speed and success as possible. but i'm not an expert in this context and now i prefer more straightforward ways.
i respect any help/info.
I assume you are located in Iran. I would suggest using TOR if you mainly access websites. The latest release works reasonably well in Iran. It also includes an option to obfuscate traffic so it is not easily detectable that you are using TOR.
See also this question: https://tor.stackexchange.com/questions/1639/using-tor-in-iran-for-the-first-time-user-guide
A easy way to get the TOR package is using the autoresponder: https://www.torproject.org/projects/gettor.html
In case the website is blocked, it works as follows:
Users can communicate with GetTor robot by sending messages via email.
Currently, the best known GetTor email address is gettor#torproject.org.
This should be the most current stable GetTor robot as
it is operated by Tor Project.
To ask for Tor Browser a user should send an email to GetTor robot
with one of the following options in the message body:
windows: If the user needs Tor Browser for Windows.
linux: If the user needs Tor Browser for Linux.
osx: If the user needs Tor Browser for Mac OSX.
I have an app, and it makes an https connection to a server. Is it possible to use something like wireshark or charlesproxy to just see the useragent that it's connecting with? I don't want to see any of the actual data, just the useragent - but I'm not sure if that is encrypted as well? (and if it's worth trying)
Thanks
Is it possible to...
No. Browser first establishes secure connection with server, then use it for transfer all data including requests' data, various headers etc.
Too late for the original inquirer, but the answer is that it may be possible in some cases, depending on application implementation.
You can use fiddler, and by turning on the 'decrypt https traffic' you also have visibility to the HTTPS content in some cases.
What fiddler does (on windows at least) is register itself within the wininet as system proxy. It can also add certificates (requires your approval when you select to decrypt https traffic) and generates on the fly certificates for the accessed domains, thus being MitM.
Applications using this infrastructure will be 'exposed' to this MitM. I ran fiddler and ran a few applications and was able to view https traffic related to office products (winword, powerpoint, outlook) other MS executables (Searchprotocolhost.exe) but also to some non-microsoft products such as apple software update, cisco jabber)
How do I get the HTTPS event from raw data?
If you are asking how to decrypt captured HTTPS network packets after the fact, that is not normally possible. You need at least the HTTPS session keys, which can only be retrieved by modifying the browser - but if you have that kind of access to the browser, you can intercept the unencrypted data anyway.
Things are easier if you have the private key of the HTTPS server, although there are encryption algorithms that use Diffie-Hellman key exchange to offer perfect forward secrecy, thus making the decryption of captured data impossible.
See also this Wikipedia article, if you would like more information on the TLS/SSL protocol that is used in HTTPS.
If you are only interested in monitoring your own browser, e.g. for debugging, you might be able to use a plugin, such as LiveHTTP Headers for Firefox, that will tap into the browser internals to show you what is being transmitted and received via an encrypted connection.
This question tries to look into whether doing HTTPS log in is very important for any website.
Is it true that for many websites, if the login is done through HTTP but not HTTPS, then anybody can pretty much see the userID and password easily along the internet highway (or by looking between a router and the internet connection in an Internet Cafe)?
If so... do popular frameworks actually use HTTPS by default (or at least as an option), such as Rails 2.3.5 or Django, CakePHP, or .Net?
Yes, any machine on the pathway (that the packets pass through) can just examine the contents of the those packets. All it takes is a capturing proxy or a promiscuous mode network card with something like WireShark. Assuming that the passwords aren't encrypted in some other way (at a higher level), they will be visible.
I can't answer the second part of your question since I have no knowledge of those particular products but I would say that the inability to use secure sockets would pretty much make them useless.
Pax is right about passwords that aren't otherwise encrypted being visible.
Still, most sites don't use SSL still, and it does put the users at a certain degree of risk when accessing sites from public wifi.
HTTPS isn't a framework level option, it would be something you'd do when you set up the webserver. If you were to use an apache configuration for instance, you would open it up to a properly configured https, close http and install a certification. The framework wouldn't have a direct influence on that portion of the release.
If the user credentials are submitted via an HTML webform without HTTPS, then it is unsecure, the data is submitted in plain text. However, if the website uses HTTP authentication instead, then the server can send back a 401 reply (or 407 for proxies) to any request that does not provide valid credentials. 401/407 is the server's way to ask for credentials, and the reply provides a list of authentication schemes (Digest, NTLM, Negotiate, etc) that the server supports, which are usually more secure by themselves. The client/browser sends the same request again with the necessariy credentials in one of the schemes, then the server either sends the requested data, or sends another 401/407 reply if the credentials are rejected.