I'm running a local web server written in Go and I can debug traffic going to it from my browser; but, I can't see the http request that it makes to external services.
Do I have to run some particular configuration of the web server in order to get the traffic to appear in fiddler? It is running as a background process.
Short answer: you can't...
...unless your web application is written to open a connection to a Proxy server and route requests through that connection (e.g. connect to a remote proxy, and then send requests through it).
Typically what developers do is just dump the Web Request/Response to a debug file to inspect during development (or to debug on a live server, by enabling it with a flag at runtime).
Fiddler is a "proxy" service/server. When you are using it normally to debug browser requests, your Browser is configured to connect to a Proxy server. That is, it will send all web requests through your fiddler's local server (I think it's localhost:8888 if i remember from my Windows days of using Fiddler) which in turn makes a connection to your local web server that you are debugging.
You can read more about Proxies at Wikipedia.
In that picture above, your local web server would be Alice. Meaning, Alice would need to be configured to connect to a proxy server and then make web requests through it.
EDIT:
(for the "I really need this" crowd)
If you really want to modify your web server to send requests through a proxy, there are a few Go packages already written to help you. GoProxy is one such package.
Related
I made a proxy server in python 3. It listens on the port 4444. It basically receives the request from clients and sends it to the server. I want to use it as a firewall to my Dvwa server. So added another functionality to the proxy. What it does is, before sending the request to the DVWA server, it validates the input.
But the problem is, the clients have to configure their proxy settings in the browser to use my proxy server. Is there any way to access the proxy without configuring the browser settings. Basically I want to host the proxy server instead of the original web server. So that all the traffic goes through the proxy before going to the webserver.
Thanks in advance...
You don't say whether your Python3 proxy is hosted on the same machine as the DVWA.
Assuming it is, the solution is simple: a reverse-proxy configuration. Your proxy transparently accepts and forwards requests to your server who then processes them and sends them back via the proxy to the client.
Have your proxy listen on port 80
Have the DVWA listen on a port other than 80 so it's not clashing (e.g. 8080)
Your proxy, which is now receiving requests for the IP/hostname which would otherwise go to the DVWA, then forwards them as usual.
The client/web browser is none the wiser that anything has changed. No settings need changing.
That's the best case scenario, given the information provided in your question. Unfortunately, I can't give any alternative solutions without knowing the network layout, where the machines reside, and the intent of the project. Some things to consider:
do you have a proper separation of concerns for this middleware you're building?
what is the purpose of the proxy?
is it for debugging/observing traffic?
are you actually trying to build a Web Application Firewall?
Our web application has a button that is supposed to send data to a server on the local network that in turn prints something on a printer.
So far it was easy: The button triggered an AJAX POST request to http://printerserver/print.php with a token, that page connected to the web application to verify the token and get the data to print and then printed.
However, we are now delivering our web application via HTTPs (and I would rather not go back to HTTP for this) and newer versions of Chrome and Firefox don't make the request to the HTTP address anymore, they don't even send the request to check CORS headers.
Now, what is a modern alternative to the cross-protocol XHR? Do Websockets suffer from the same problem? (A Google search did not make clear what is the current state here.) Can I use TCP Sockets already? I would rather not switch to GET requests either, because the action is not idempotent and it might have practical implications with preloading and caching.
I can change the application on the printerserver in any way (so I could replace it with NodeJS or something) but I cannot change the users' browsers (to trust a self-signed certificate for printerserver for example).
You could store the print requests on the webserver in a queue and make the printserver periodically poll for requests to print.
If that isn't possible I would setup a tunnel or VPN between the webserver and printserver networks. That way you can make the print request from the webserver on the server-side instead of the client. If you use curl, there are flags to ignore invalid SSL certificates etc. (I still suspect it's nicer to introduce a queue anyway, so the print requests aren't blocking).
If the webserver can make an ssh connection to something on the network where the printserver is on, you could do something like: ssh params user#host some curl command here.
Third option I can think of, if printserver can bind to for example a subdomain of the webserver domain, like: print.somedomain.com, you may be able to make it trusted by the somedomain.com certificate, IIRC you have to create a CSR (Certificate Signing Request) from the printserver certificate, and sign it with the somedomain.com certificate. Perhaps it doesn't even need to be a subdomain for this per se, but maybe that's a requirement for the browser to do it client-side.
The easiest way is to add a route to the webapp that does nothing more than relay the request to the print server. So make your AJAX POST request to https://myapp.com/print, and the server-side code powering that makes a request to http://printerserver/print.php, with the exact same POST content it received itself. As #dnozay said, this is commonly called a reverse proxy. Yes, to do that you'll have to reconfigure your printserver to accept (authenticated) requests from the webserver.
Alternatively, you could switch the printserver to https and directly call it from the client.
Note that an insecure (http) web-socket connection on a secure (https) page probably won't work either. And for good reason: generally it's a bad idea to mislead people by making insecure connections from what appears to them to be a secure page.
The server hosting the https webapp can reverse proxy the print server,
but since the printer is local to the user, this may not work.
The print server should have the correct CORS headers
Access-Control-Allow-Origin: *
or:
Access-Control-Allow-Origin: https://www.example.com
However there are pitfalls with using the wildcard.
From what I understand from the question, printserver is not accessible from the web application so the reverse proxy solution won't work here.
You are restricted from making requests from the browser to the printserver by cross-origin-policy.
If wish to communicate with the printserver from an HTTPS page you will need the printserver to expose print.php as HTTPS too.
You could create a DNS A record as a subdomain of your web application that resolves to the internal address of your printserver.
With those steps in place you should be able to update your printserver page to respond with permissive CORS headers which the browser should then respect. I don't think the browser will even issue CORS requests across different protocol schemes (HTTPS vs HTTP) or to internal domains, without a TLD.
Is it possible to use browser's internet connection for other application?
I'm in environment where browser is the only application that may access internet connection. Meanwhile, I need to run maven projects which needs internet access to download the required JARs.
I was thinking about creating Proxy Server in Chrome extensions or other browser. But, I hope I can find another easier way.
Sounds like Selenium WebDriver is what you need. You could try to redirect the download to browser or try to programmatically simulate browser.
Yes it is possible.
Recently I created a http_proxy server in Java which start 3 socket servers:
Http server to receive user request, and
WebSocket server to allow Chromium extension to create connection to the http_proxy server.
Http Server to receive response body from Chromium.
So, there will be 3 socket connections:
To receive http user request and send back the content, and
To send the URL requested to Chromium browser
When response received by browser (via Xhr), it will do HTTP POST to http_proxy server to send the content.
That's all!
I have Charles Proxy set up to look at outgoing https requests, and I need to re-route traffic from one server to a local http server.
I have a MacOSX machine that is set up this way: I have an ethernet connection that I hardwire, and share the internet connection via the airport interface. On a second machine, I've installed the Charles cert, and when I connect via the shared interface. I can see the traffic (unencrypted) in Charles, so I know the communications and certs are all working properly.
I need to intercept all the https traffic going to one server (https://www.foo.com) to a local http server (localhost:8001). I've tried using Map Remote, but it doesn't seem to unencrypt the traffic before forwarding it (or possibly it re-encrypts it).
How can I configure Charles to do this? (or, please point me to any other software package, if Charles isn't capable of this)
I figured out what was happening, there were two issues.
I had misconfigured the Map Remote entry, and my two different clients (MyApp and curl) were hitting two different servers - the app was hitting the correct server (locally) but the request was malformed.
Curl from the macOSX box where the proxy was running was NOT looping through the proxy, since I hadn't included the -x localhost:8888 flag.
I need a tool that I can set up between my client browser a web server that will allow me to add HTTP headers into the response coming back from the web server. I am envisioning that the tool would run, like fiddler, my client browser goes through fillder to connect to the webserver and I can configure fiddler to modify the HTTP respones coming back i.e. to add in an HTTP header. Are there tools that can do this?
Is this in production or for development only?
For production, use Apache and its many modules.
Would the tool be on yor PC (forward proxy) or server (reverse proxy)? Apache can be used for both.
If development only, fiddler does the job?