How can network requests be monitored and evaluated for their request URL, parameters, request, and response data?
Desired solution
I want to be notified or being given a custom exception, if specific content occurs in request or response.
Example
Assume a web application with many dynamic Ajax requests. A request or response might contain a broken value, e.g. undefined.
Request URL:
http://localhost:8080/app/?undefined=1
Response JSON data:
{"undefined":"1"}
Attempts
Filtering for request content in Dev Tools is not possible
PostMan tests seem not viable (e.g. no user interactions)
Not tried/found yet
Guesses of what might work ...
Software to intercepts requests and log/alert details
Proxying any URLs on an OS via some standalone application
If importing axios, you could leverage custom interceptors:
// Add a response interceptor
axios.interceptors.response.use(function (response) {
// Any status code that lie within the range of 2xx cause this function to trigger
// Do something with response data
return response;
}, function (error) {
// Any status codes that falls outside the range of 2xx cause this function to trigger
// Do something with response error
return Promise.reject(error);
});
// Similar functionality can be achieved for request
axios.interceptors.request.use(function (config) {
// Do something before request is sent
return config;
}, function (error) {
// Do something with request error
return Promise.reject(error);
});
Proxy software
For this case, any proxy that comes with logging ability, or can be extended:
https://mitmproxy.org
https://github.com/http-party/node-http-proxy
Exemplary for many variants under NodeJS
Tunneling
Expose localhost to a domain for testing:
https://ngrok.com
Use the logging capability to get request details:
https://ngrok.com/docs/ngrok-agent/api#list-requests
Localhost vs. 127.0.0.1
localhost should better become 127.0.0.1 or a custom dev domain. On MacOS, localhost did not work with HTTP interceptor browser plugins.
Related
In my ng-resource files, I enable the ajax header:
var app = angular.module('custom_resource', ['ngResource'])
app.config(['$httpProvider', function($httpProvider) {
//enable XMLHttpRequest, to indicate it's ajax request
//Note: this disables CORS
$httpProvider.defaults.headers.common["X-Requested-With"] = 'XMLHttpRequest';
}])
app.factory('Article', ['$resource', function($resource) {
return $resource('/article/api/:articleId', {articleId: '#_id'}, {
update: {method: 'PUT'},
query: {method: 'GET', isArray: true}
})
}])
So that I can separate ajax and non-ajax request and response accordingly (to send json data like res.json(data), or to send the entire html page like res.render('a.html')
for example, in my error handler, I need to decide to render error.html page or to just send a error message:
exports.finalHandler = function(err, req, res, next) {
res.status(err.status || 500)
var errorMessage = helper.isProduction() ? '' : (err.message || 'unknown error')
if (req.xhr) {
res.json({message: errorMessage})
}
else {
res.render(dir.error + '/error_page.ejs')
}
}
But now I need to do CORS request to other sites. Is it possible to do CORS request while keeping the ajax header? or other ways I can identify ajax and non-ajax request from server?
In case my question is not clear, heres a relevant article about angular and CORS
http://better-inter.net/enabling-cors-in-angular-js/
Basically, we need to delete xhr header to enable cors for other server, but I need the header for my own server
EDIT 2:
today I tried integrating google map and I got this error:
XMLHttpRequest cannot load http://maps.googleapis.com/maps/api/geocode/json?address=Singapore&sensor=false. Request header field X-Requested-With is not allowed by Access-Control-Allow-Headers.
Setting custom headers on XHR requests triggers a preflight request.
So, it doesn't disable CORS but your server is most likely not handling the preflight request.
Inspired from this post: https://remysharp.com/2011/04/21/getting-cors-working
The solution should be to use the cors module and add the following to your node.js code before your routes:
var corsOptions = {
origin: true,
methods: ['GET', 'PUT', 'POST'],
allowedHeaders: ['X-Requested-With','Content-Type', 'Authorization']
};
app.options('*', cors(corsOptions)); //You may also be just fine with the default options
You can read more at: https://github.com/expressjs/cors
you may try to use cors package
First, to address you primary concern is it possible to do CORS request while keeping the ajax header?: the answer is YES, provided the sites you are accessing allow requests from you or any other external clients at all.
You wrote:
//Note: this disables CORS
$httpProvider.defaults.headers.common["X-Requested-With"] = 'XMLHttpRequest';
But I don't understand what you mean by, it "disables CORS". The X-Requested-With header is not a standard header, and the known effect of adding a non-standard header to a request (made from a browser) is the triggering of a pre-flight request [3].
If the other sites you are interested in would set their servers to refuse processing of requests that do not originate from their own domain, then whether you set that header or not, your request should fail.
It seems everything is working fine for you, for requests sent to you own server. Otherwise you can solve the problem by appending the Access-Control-Allow-Origin header in your server responses as follows:
if you need to allow requests from specific domains
response.set("Access-Control-Allow-Origin", "one-host-domain, your-host-domain, some-other-host-domain"); // second argument is a comma-delimited list of allowed domains
(It may be better for you to actually check the request object for the origin, and if it's permitted based on presence in a pre-determined list, then send back the exact same origin).
If you need to permit all requests regardless of its origin
response.set("Access-Control-Allow-Origin", "*");
That should do, and I hope it clears your doubts for you.
More info on handling CORS when using AJAX: 0, 1 & 2.
EDIT
Following exchanges in the comment, I add the following points to support this answer further.
As it is today, the only side that needs disabling/enabling CORS in the client-server system is the server. All modern browsers allow cross origin requests by default and you don't need to do anything additional to support that capability. I understood that you're adding a custom header to distinguish AJAX requests from the rest?? AFAIK, that header changes nothing about how requests are made by browsers.
Here is how all cross-origin requests are handled by browsers today: for all request methods (but usually with the exception of GET), browsers send a pre-flight request with the OPTION method. If the destination server allows it, the actual request is then sent, otherwise the request fails. In the case where the servers, responds with a refusal there's nothing you nor whatever library you use can do about it. This is the fact from my own experience.
There are 3 solutions that come to my mind:
1. Ask site's admin to enable x-requested-with header in CORS.
2. Use proxy server.
3. Send request without x-requested-with header.
This article should make it clear how CORS works and how to make CORS requests.
Particularly "Simple requests" section and "Access-Control" section, especially access-control-allow-headers description is important in this case.
As it says: for simple requests access-control-allow-origin is enough. However if the request includes custom header (a header which is not included by default, such as x-requested-with header), the preflight request is triggered, and server's response to this request should enable this custom header in access-control-allow-headers by setting its value to either "*" or to the name of a custom header (x-requested-with).
Hope it makes it a little bit clearer.
first Ill comment my config in server
i have a apache with my html page with angular in 80 port
and other web server in port 4000 listening like a API
i am trying to make a $http request to my server but don't succes, just error message, this is my code
$http.post('http://localhost:4000/tareas', {msg:'asd'}).
success(function(data, status) {
alert('funciono');
}).
error(function(data, status) {
alert('error');
});
my server receive that request because i send a message when a controller is accesed, so the request connect with my API, but always execute the error my request in angular, even with $http.jsonp, that send me the error alert, my server return a simple json.
{data:'hurra.!', status:200}
i need a different response from my server to success?
even with F12 and neetwork, i see the request "canceled" with get and post, but my server send the message so i receive that request. with jsonp, chromium don't say a thin, everything is ok, but the $http.json still send me to error, is because my server response?
thanks to all.
well, searching more i find that cross domain request need "callback=JSON_CALLBACK" in the url and do a jsonp request, so changing my code to
$http.jsonp('http://localhost:4000/tareas?callback=JSON_CALLBACK'}).
success(function(data, status) {
alert('funciono');
}).
error(function(data, status) {
alert('error');
});
do the work and my ajax request now success... i find this in other tread, i put here the link for more info.
parsing JSONP $http.jsonp() response in angular.js
thanks
According to the Crossrider docs, there is currently a webRequest object with the onRequest event, which allows for accessing the request URI, etc., but I could not find any way to inspect the response data. I was wondering if there is a way to actually inspect the response of that request, such as the response headers and possibly the response data. Thanks for the help.
I'm going out on a limb here but I think your after an AJAX request, in which case you are after the appAPI.request API where you can indeed obtain response and header data. The appAPI.webRequest API is like Chrome's webRequest API used "to observe and analyze traffic and to intercept, block, or modify requests in-flight".
If this is not the case, please can you clarify your scenario.
[RESPONSE TO COMMENT]
Let me first clarify that the webRequest API runs before the requests are made (i.e. the browser is requesting the page but has not yet made the actual HTTP request) and hence it's not relevant to talk of response data.
However, since the URL is provided in the details of the webRequest, you can get the response data and headers yourself using the aforementioned appAPI.request API, as follows:
appAPI.ready(function() {
appAPI.webRequest.monitor.onBeforeNavigate.addListener({
callback: function(details) {
appAPI.request.get({
url: details.requestUrl,
onSuccess: function(response, additionalInfo) {
appAPI.db.async.set(details.requestUrl, {
response: response,
headers: additionalInfo.headers
});
}
});
}
});
});
NOTE: For this example I have used the new webRequest.monitor API designed to be lightweight specifically for monitoring. It's not currently documented (docs should be completed within the next week or so) but you can start using it already for Chrome, Firefox, and Internet Explorer.
[Disclosure: I am a Crossrider employee]
I'm picking my way through the dartiverse_search example from the welcome page in dart editor. I see that it uses a path route to decide whether to transform a request into a websocket:
// The client will connect using a WebSocket. Upgrade requests to '/ws' and
// forward them to 'handleWebSocket'.
router.serve('/ws')
.transform(new WebSocketTransformer())
.listen(handleWebSocket);
Is it possible to turn a request into a websocket without using a routing path, for example using a query string to the root url?
You can specify any condition for upgrading to a WebSocket connection. You can even upgrade any connection request to a WebSocket connection without specifying a condition like this:
WebSocketTransformer.upgrade(request).then((WebSocket websocket) {
websocket.listen((String text) {
// process sent data
});
websocket.add(JSON.encode("Hello"));
});
If the request is not a valid web socket upgrade request a HTTP response with status code 500 will be returned. Otherwise the returned future will complete with the [WebSocket] when the upgrade process is complete.
We are trying to create a RESTful API that will be hosted on server x.foo.com. Client html applications (built in jquery) will be hosted on y.foo.com.
I am dealing with cross-domain issues by setting the Access-Control-Allow-Origin header as described here http://www.w3.org/TR/cors/.
So far so good, and I can now successfully make AJAX calls from host y to host x.
However, I ran into a gotcha with POST requests. The typical response to a post request is a redirect. However, the XMLHttpRequest object will not follow cross domain redirects, thus resulting in a failed call.
// Hosted on y.foo.com
$.ajax({
type: "POST",
url : http://x.foo.com/myapp/",
success: function(data) {
alert("success!");
}
});
// Return status: 302
// (Which errors out in firebug)
Anyone know of any techniques to handle the redirect (to a resource on server x) that I get from this post for a client hosted on y?
How about the client sends a special header for AJAX requests, and depending on whether it's an AJAX request or not, you can change the response instead of doing a redirect.