Proxy to validate xml with xsd - proxy

I've been searching for a while for something that can act as a proxy that I can feed it XSDs to validate xml content within the request body of HTTP traffic.
Does anyone know if such a thing exists on Windows or Linux?
I checked out ModSecurity but I am bit lost on it - I believe SecRule XML maybe able to do it but I can't find how I would go about checking the xml against multiple xsd files to see if it passed any particular one?
validateSchema
Description: This operator requires the request body to be processed as XML.
Example:
SecDefaultAction log,deny,status:403,phase:2
SecRule REQUEST_HEADERS:Content-Type ^text/xml$ \
phase:1,t:lowercase,nolog,pass,ctl:requestBodyProcessor=XML
SecRule REQBODY_PROCESSOR "!^XML$" nolog,pass,skip:1
SecRule XML "#validateSchema /path/to/apache2/conf/xml.xsd"
This operator requires request body to be processed as XML.
https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#validateSchema
I am thinking there is some way of creating Chained rule evaluation and it allows messages through IF any of the rules are met but denies the message through when none are met.
Thank you!

XML / XSD checking in ModSecurity works, but it's not used by a lot of people (unlike the core functionality of ModSec), so I would not be surprised if you found rough edges.
So use with caution and think about alternatives (-> mod_lua, use XML library, etc.).
What stops you from checking them
SecRule XML "#validateSchema /path/to/apache2/conf/xml.xsd" "phase:2,id:1,log,deny"
SecRule XML "#validateSchema /path/to/apache2/conf/xml2.xsd" "phase:2,id:2,log,deny"
If done this way, you can also branch and skip rules depending on the context. Like Header-X, then pass via id:1, if Header-Not-X, then pass via id:2.
If you lack the ModSecurity knowledge to pull this off, then you might want to buy the ModSec Handbook or look at the tutorials at https://netnea.com.

Related

What HTTP Protocol can I use if I need to GET something from the server but I also need to send a requestbody?

I am using SpringBoot...
I can not use GET protocol and include a body, but I am not going to create or update anything on the server so I do not want to use POST or PUT, any other protocol that acts like a GET with body?
if you wonder what I need to send in that body it is an url parameter, like for example http://somewebsite.com/stuff/etc and I feel that putting this inside a request body is better than putting it as a requestparam
I can not use GET protocol and include a body, but I am not going to create or update anything on the server so I do not want to use POST or PUT, any other protocol that acts like a GET with body?
Your best bet, where suitable, would be to mimic how HTML forms work; which is to say having a family of resources with identifiers that are filled in by the client (in general, via URI templates -- often via query parameters as would happen with an HTML form).
When that's not appropriate: as of 2022-11, your best bet is POST. It's not a great answer (in particular, general purpose HTTP components won't know that the semantics of the request are safe), but it is the best option available of the registered methods.
POST serves many useful purposes in HTTP, including the general purpose of "
"this action isn’t worth standardizing." -- Roy Fielding, 2009
Eventually, the HTTPbis-wg will finalize the safe-method-with-a-body proposal, and at that point that will become a much better option than POST (for the cases that match the new semantics).

JMeter: Script to compare response kept in an external parameterised file

I have following requirement
1. Keep responses in an external xml file.
2. Hit the API and compare the response with external response (Kept in xml file. )
3. Also while comparison, I have to ignore dynamic components like , etc.
4. Also I have to ignore sequence of parameters.
Can you please if any such utility/program to do so in JMeter
Thanks in advance
Regards
Vishal Pachpute
I believe it makes more sense to use XML Schema Assertion. This way you will validate your XML response syntax and structure, elements and attributes, number and order of attributes, data types, etc. but this assertion won't care in the slightest about the content.
You can ask the .xsd schema from the developers, most likely they have it, if not the majority of IDEs can do this, there are even online services.
References:
XML Schema Tutorial
How to Use JMeter Assertions in Three Easy Steps

How to validate request against XSD and return an error object?

My task is to implement a webservice that:
consumes an XML file on a POST endpoint
in happy flow, it returns a DTO as JSON + HTTP 2xx
the incoming XML file is validated against a XSD; if the validation fails, a JSON with a list of all validation errors is returned (including the line, column, error) with HTTP Bad request
the application exposes two endpoints, only one of them should be validated
I have started the implementation with Spring Boot + web, using regular #PostMapping which has "consumes" and "produces" set to application/xml and application/json, respectively. The usual flow works perfectly fine. Now, I stumbled upon the issue of validating the incoming payload. What I figured out:
1) I have to validate the payload before it is converted (marshalled) to an object.
2) Once validated, I have to either:
allow further processing
stop any further processing, write the error object to the response and set the status code to 400 Bad request
My approaches were:
1) using a RequestBodyAdvice, more specifically the beforeBodyRead method implementation. I had the following issue here: I don't know how to write anything to the output in case the validation fails.
2) using a Filter (I've extended OncePerRequestFilter) - fortunately, I can read the request (request.getInputStream()) and write to the response (response.getOutputStream()).
However, how can I do the selective filtering (as mentioned, I only want to validate one single endpoint)?
Are there any other alternatives for placing the incoming request XSD validation? Is spring-web the appropriate choice here? Would you recommend some other library / framework?
To validate xml against xsd schema, my preference is XML Beans. It is very easy to use. Other options are JABX, Castor. Take a look at Java to XML conversions?.
You will need to jar using xsd schmema and will need to put it in the classpath of your application so that it's classes are available for you for validation. Please take a look at this blog.
You can use validation API as mentioned here.
I would prefer to write validation code in the aspect so that it can be reused with other APIs.
If validation fails, throw valid exception from the aspect itself.
If validation is passed, process your input string that you receive.
Please let us know if you need any more information.

Validating Xml Chef template

Google hasn't helped me with this:
We just started using Chef deployment. (And I'm embarrassingly new to it)
I have a web.config template that will have variables set by a data bag.
Using some free online tools, I was able to find and fix syntax problems in my data bag. JSONLint
Is there a way to check the template itself?
Online Xml validation tools like this one fail once a Ruby tag <% is found... and this makes sense because it's not valid Xml. Is there a tool/way to validate the Xml in a template so that the Ruby tags are valid?
All I want to know is if syntactically speaking, the file is valid. Logic errors would be dealt with another way... if that makes sense.
You could substitute the values that are supposed to be set by your <%=...%> tags and validate that since your final .xml file won't contain any of those tags if you did your template correct.

What does a Ajax call response like 'for (;;); { json data }' mean? [duplicate]

This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Why do people put code like “throw 1; <dont be evil>” and “for(;;);” in front of json responses?
I found this kind of syntax being used on Facebook for Ajax calls. I'm confused on the for (;;); part in the beginning of response. What is it used for?
This is the call and response:
GET http://0.131.channel.facebook.com/x/1476579705/51033089/false/p_1524926084=0
Response:
for (;;);{"t":"continue"}
I suspect the primary reason it's there is control. It forces you to retrieve the data via Ajax, not via JSON-P or similar (which uses script tags, and so would fail because that for loop is infinite), and thus ensures that the Same Origin Policy kicks in. This lets them control what documents can issue calls to the API — specifically, only documents that have the same origin as that API call, or ones that Facebook specifically grants access to via CORS (on browsers that support CORS). So you have to request the data via a mechanism where the browser will enforce the SOP, and you have to know about that preface and remove it before deserializing the data.
So yeah, it's about controlling (useful) access to that data.
Facebook has a ton of developers working internally on a lot of projects, and it is very common for someone to make a minor mistake; whether it be something as simple and serious as failing to escape data inserted into an HTML or SQL template or something as intricate and subtle as using eval (sometimes inefficient and arguably insecure) or JSON.parse (a compliant but not universally implemented extension) instead of a "known good" JSON decoder, it is important to figure out ways to easily enforce best practices on this developer population.
To face this challenge, Facebook has recently been going "all out" with internal projects designed to gracefully enforce these best practices, and to be honest the only explanation that truly makes sense for this specific case is just that: someone internally decided that all JSON parsing should go through a single implementation in their core library, and the best way to enforce that is for every single API response to get for(;;); automatically tacked on the front.
In so doing, a developer can't be "lazy": they will notice immediately if they use eval(), wonder what is up, and then realize their mistake and use the approved JSON API.
The other answers being provided seem to all fall into one of two categories:
misunderstanding JSONP, or
misunderstanding "JSON hijacking".
Those in the first category rely on the idea that an attacker can somehow make a request "using JSONP" to an API that doesn't support it. JSONP is a protocol that must be supported on both the server and the client: it requires the server to return something akin to myFunction({"t":"continue"}) such that the result is passed to a local function. You can't just "use JSONP" by accident.
Those in the second category are citing a very real vulnerability that has been described allowing a cross-site request forgery via tags to APIs that do not use JSONP (such as this one), allowing a form of "JSON hijacking". This is done by changing the Array/Object constructor, which allows one to access the information being returned from the server without a wrapping function.
However, that is simply not possible in this case: the reason it works at all is that a bare array (one possible result of many JSON APIs, such as the famous Gmail example) is a valid expression statement, which is not true of a bare object.
In fact, the syntax for objects defined by JSON (which includes quotation marks around the field names, as seen in this example) conflicts with the syntax for blocks, and therefore cannot be used at the top-level of a script.
js> {"t":"continue"}
typein:2: SyntaxError: invalid label:
typein:2: {"t":"continue"}
typein:2: ....^
For this example to be exploitable by way of Object() constructor remapping, it would require the API to have instead returned the object inside of a set of parentheses, making it valid JavaScript (but then not valid JSON).
js> ({"t":"continue"})
[object Object]
Now, it could be that this for(;;); prefix trick is only "accidentally" showing up in this example, and is in fact being returned by other internal Facebook APIs that are returning arrays; but in this case that should really be noted, as that would then be the "real" cause for why for(;;); is appearing in this specific snippet.
Well the for(;;); is an infinite loop (you can use Chrome's JavaScript console to run that code in a tab if you want, and then watch the CPU-usage in the task manager go through the roof until the browser kills the tab).
So I suspect that maybe it is being put there to frustrate anyone attempting to parse the response using eval or any other technique that executes the returned data.
To explain further, it used to be fairly commonplace to parse a bit of JSON-formatted data using JavaScript's eval() function, by doing something like:
var parsedJson = eval('(' + jsonString + ')');
...this is considered unsafe, however, as if for some reason your JSON-formatted data contains executable JavaScript code instead of (or in addition to) JSON-formatted data then that code will be executed by the eval(). This means that if you are talking with an untrusted server, or if someone compromises a trusted server, then they can run arbitrary code on your page.
Because of this, using things like eval() to parse JSON-formatted data is generally frowned upon, and the for(;;); statement in the Facebook JSON will prevent people from parsing the data that way. Anyone that tries will get an infinite loop. So essentially, it's like Facebook is trying to enforce that people work with its API in a way that doesn't leave them vulnerable to future exploits that try to hijack the Facebook API to use as a vector.
I'm a bit late and T.J. has basically solved the mystery, but I thought I'd share a great paper on this particular topic that has good examples and provides deeper insight into this mechanism.
These infinite loops are a countermeasure against "Javascript hijacking", a type of attack that gained public attention with an attack on Gmail that was published by Jeremiah Grossman.
The idea is as simple as beautiful: A lot of users tend to be logged in permanently in Gmail or Facebook. So what you do is you set up a site and in your malicious site's Javascript you override the object or array constructor:
function Object() {
//Make an Ajax request to your malicious site exposing the object data
}
then you include a <script> tag in that site such as
<script src="http://www.example.com/object.json"></script>
And finally you can read all about the JSON objects in your malicious server's logs.
As promised, the link to the paper.
This looks like a hack to prevent a CSRF attack. There are browser-specific ways to hook into object creation, so a malicious website could use do that first, and then have the following:
<script src="http://0.131.channel.facebook.com/x/1476579705/51033089/false/p_1524926084=0" />
If there weren't an infinite loop before the JSON, an object would be created, since JSON can be eval()ed as javascript, and the hooks would detect it and sniff the object members.
Now if you visit that site from a browser, while logged into Facebook, it can get at your data as if it were you, and then send it back to its own server via e.g., an AJAX or javascript post.

Resources