I am using AJAX for validation. I am using servlet in server side.I am writing in the response as follows
response.setContentType("text/xml");
response.setCharacterEncoding("UTF-8");
response.setHeader("Cache-Control", "no-cache");
response.getWriter().write("123");
The response XMl is coming as invalid if i have any special charcters like ©
,® and soon in IE.
Please let me know how to get the response xml as valid by having the above special characters in it.
Download Apache Commons StringEscapeUtils. It has a function for escaping things like ©
Related
I have a Spring application that receives a request like http://localhost/foo?email=foo+bar#example.com. This triggers a controller that roughly looks like this:
#RestController
#RequestMapping("/foo")
public class FooController extends Controller {
#GetMapping
public void foo(#RequestParam("email") String email) {
System.out.println(email)
}
}
By the time I can access email, it's been converted to foo bar#example.com instead of the original foo+bar#example.com. According to When to encode space to plus (+) or %20? this should only happen in requests where the content is application/x-www-form-urlencoded. My request has a content type of application/json. The full MIME headers of the request look like this:
=== MimeHeaders ===
accept = application/json
content-type = application/json
user-agent = Dashman Configurator/0.0.0-dev
content-length = 0
host = localhost:8080
connection = keep-alive
Why is Spring then decoding the plus as a space? And if this is the way it should work, why isn't it encoding pluses as %2B when making requests?
I found this bug report about it: https://jira.spring.io/browse/SPR-6291 which may imply that this is fixed on version 3.0.5 and I'm using Spring > 5.0.0. It is possible that I may misinterpreting something about the bug report.
I also found this discussion about RestTemplate treatment of these values: https://jira.spring.io/browse/SPR-5516 (my client is using RestTemplate).
So, my questions are, why is Spring doing this? How can I disable it? Should I disable it or should I encode pluses on the client, even if the requests are json?
Just to clarify, I'm not using neither HTML nor JavaScript anywhere here. There's a Spring Rest Controller and the client is Spring's RestTemplate with UriTemplate or UriComponentsBuilder, neither of which encode the plus sign the way Spring decodes it.
Original Answer
You are mixing 2 things, a + in the body of the request would mean a space when header has application/x-www-form-urlencoded. The body or content of the request would be dependent on the headers but a request can just have a url and no headers and no body.
So the encoding of a URI cannot be controlled by any headers as such
See the URL Encoding section in https://en.wikipedia.org/wiki/Query_string
Some characters cannot be part of a URL (for example, the space) and some other characters have a special meaning in a URL: for example, the character # can be used to further specify a subsection (or fragment) of a document. In HTML forms, the character = is used to separate a name from a value. The URI generic syntax uses URL encoding to deal with this problem, while HTML forms make some additional substitutions rather than applying percent encoding for all such characters. SPACE is encoded as '+' or "%20".[10]
HTML 5 specifies the following transformation for submitting HTML forms with the "get" method to a web server.1 The following is a brief summary of the algorithm:
Characters that cannot be converted to the correct charset are replaced with HTML numeric character references[11]
SPACE is encoded as '+' or '%20'
Letters (A–Z and a–z), numbers (0–9) and the characters '*','-','.' and '_' are left as-is
All other characters are encoded as %HH hex representation with any non-ASCII characters first encoded as UTF-8 (or other specified encoding)
The octet corresponding to the tilde ("~") is permitted in query strings by RFC3986 but required to be percent-encoded in HTML forms to "%7E".
The encoding of SPACE as '+' and the selection of "as-is" characters distinguishes this encoding from RFC 3986.
And you can see the same behaviour on google.com as well from below screenshots
Also you can see the same behaviour in other frameworks as well. Below is an example of Python Flask
So what you are seeing is correct, you are just comparing it with a document which refers to the body content of a request and not the URL
Edit-1: 22nd May
After debugging it seems the decoding doesn't even happen in Spring. I happens in package org.apache.tomcat.util.buf; and the UDecoder class
/**
* URLDecode, will modify the source.
* #param mb The URL encoded bytes
* #param query <code>true</code> if this is a query string
* #throws IOException Invalid %xx URL encoding
*/
public void convert( ByteChunk mb, boolean query )
throws IOException
{
int start=mb.getOffset();
And below is where the conversion stuff actually happens
if( buff[ j ] == '+' && query) {
buff[idx]= (byte)' ' ;
} else if( buff[ j ] != '%' ) {
This means that it is an embedded tomcat server which does this translation and spring doesn't even participate in this. There is no config to change this behaviour as seen in the class code. So you have to live with it
SPR-6291 fixed this problem in v3.0.5 but this remains unresolved in some other cases like SPR-11047 is still unresolved. While SPR-6291's priority was Major, SPR-11047's priority is Minor.
I faced this problem when I was working on REST API in old Spring last year. There are multiple ways we can get data in Spring controller. So two of them are via #RequestParam or #PathVariable annotation
As others mentioned I think its spring's internal issue and does not specifically belong to URL encoding because I was sending data over POST request but it is somewhat encoding problem. But I also agree with others as now it remains problematic only in URL.
So there are two solutions I know:
You can use #PathVariable instead of #RequestParam because as of SPR-6291 this plus sign issue is fixed in #PathVariable and still remains open for #RequestParam as SPR-11047
My version of spring was not even accepting plus sign via #PathVariable annotation, so this is how I overcome the problem (I don't remember it step by step but it will give you hint).
In your case you can get the fields via JS and escape the plus sign before sending a request. Something like this:
var email = document.getElementById("emailField").value;
email = email.replace('+', '%2B');
If you have this request:
http://localhost/foo?email=foo+bar#example.com
then the original is foo bar#example.com. If you say the original should be foo+bar#example.com then the request should be:
http://localhost/foo?email=foo%2Bbar#example.com
So Spring is working as supposed to. Maybe on client you should check if the URI is properly encoded. The client-side URL encoding is responsible for building a correct HTTP request.
See encodeURI() if you generate the request in JavaScript or uriToString() if you generate the request in Spring.
Build your request string (the part after ?), without any encoding, with unencoded values like foo+bar#email.com, and only in the end, before actually using it in GET, encode all of it with whatever is available on the client platform. If you want to use POST then you should encode it according to the MIME type of your choice.
I have a client application which is dated and is sending the request headeer as Content-Type: json instead of Content-Type: application/json.
There is no way (in the near future) the client application can be changed.
I am implementing the services using Spring and I almost got the endpoint working but only when content-Type is application/json. if "json" is set s content-type, I get 415 unsupported MediaType error, which makes sense.
How can I work around this problem, a hack for a short term fix would be ideal
This works but requires the client to send the header as "application/json"
#RequestMapping(value="person", method = RequestMethod.POST, consumes="application/json")
#ResponseBody
public List<PersonProfile> getProfiles (#RequestBody Wrapper wrapper) {
This fails during container start-up
#RequestMapping(value="person", method = RequestMethod.POST, consumes="json")
#ResponseBody
public List<PersonProfile> getProfiles (#RequestBody Wrapper wrapper) {
415 unsupported MediaType error.
I don`t know your ajax code.
But, you read in jquery document. maybe solve
contentType
contentType (default: 'application/x-www-form-urlencoded; charset=UTF-8')
Type: Boolean or String
When sending data to the server, use this content type. Default is "application/x-www-form-urlencoded; charset=UTF-8", which is fine for most cases. If you explicitly pass in a content-type to $.ajax(), then it is always sent to the server (even if no data is sent). As of jQuery 1.6 you can pass false to tell jQuery to not set any content type header. Note: The W3C XMLHttpRequest specification dictates that the charset is always UTF-8; specifying another charset will not force the browser to change the encoding. Note: For cross-domain requests, setting the content type to anything other than application/x-www-form-urlencoded, multipart/form-data, or text/plain will trigger the browser to send a preflight OPTIONS request to the server.
dataType
dataType (default: Intelligent Guess (xml, json, script, or html))
Type: String
The type of data that you're expecting back from the server. If none is specified, jQuery will try to infer it based on the MIME type of the response (an XML MIME type will yield XML, in 1.4 JSON will yield a JavaScript object, in 1.4 script will execute the script, and anything else will be returned as a string). The available types (and the result passed as the first argument to your success callback) are:
"xml": Returns a XML document that can be processed via jQuery.
"html": Returns HTML as plain text; included script tags are evaluated when inserted in the DOM.
"script": Evaluates the response as JavaScript and returns it as plain text. Disables caching by appending a query string parameter, =[TIMESTAMP], to the URL unless the cache option is set to true. Note: This will turn POSTs into GETs for remote-domain requests.
"json": Evaluates the response as JSON and returns a JavaScript object. Cross-domain "json" requests are converted to "jsonp" unless the request includes jsonp: false in its request options. The JSON data is parsed in a strict manner; any malformed JSON is rejected and a parse error is thrown. As of jQuery 1.9, an empty response is also rejected; the server should return a response of null or {} instead. (See json.org for more information on proper JSON formatting.)
"jsonp": Loads in a JSON block using JSONP. Adds an extra "?callback=?" to the end of your URL to specify the callback. Disables caching by appending a query string parameter, "=[TIMESTAMP]", to the URL unless the cache option is set to true.
"text": A plain text string.
multiple, space-separated values: As of jQuery 1.5, jQuery can convert a dataType from what it received in the Content-Type header to what you require. For example, if you want a text response to be treated as XML, use "text xml" for the dataType. You can also make a JSONP request, have it received as text, and interpreted by jQuery as XML: "jsonp text xml". Similarly, a shorthand string such as "jsonp xml" will first attempt to convert from jsonp to xml, and, failing that, convert from jsonp to text, and then from text to xml.
And you need to search by consumes and produces.
You wrong using consumes..
I have a mirth instance (version 3.0.1) sending out using a POST method to a web api restfull service.
[POST("MessagesHl7/OML_O21")] public HttpResponseMessage
PostOmlo21([FromBody] sting receivedmessage) {..}
The problem is that the message hl7 that is sent to the service
in a outbound message is cut in the first characters. For example, in the message:
MSH|^~\&|CPSI^1.3.6.1.4.1.27248.1.17^ ISO|CGH|...
in the receivedmessage variable the text MSH|^~\ is received only.
How can I do in order that the message is not cut?
In the http channel, the configuration is:POST, not query parameters,
in
headers content-type application/x-www-form-urlencoded,
Content-Type value application/xml,
and the value that send is =${message.encodedData}.
Change your action method to not use binding and just read the request body as string.
[POST("MessagesHl7/OML_O21")]
public Task<HttpResponseMessage> PostOmlo21()
{
string receivedMessage = await Request.Content.ReadAsStringAsync();
}
I would suggest to use Base64 encoding for the HL7 piped message since
there are many special characters within the message which can be interpreted
in the wrong way during parsing. Especially during the parsing of xml.
Of course you have to decode the HL7 message on Server side.
But i think Mirth gives you all functionallity to do that.
I don't know which class to use in C#/ASP in Java appropriate classes and frameworks for
encoding an decoding Base64 exist. I believe the same is true for C# and ASP.
I have a form with a fileupload control in it, and I call form.submit with a success function.
My server side does the usual trick of setting the content type to text/html to get it to arrive in one piece.
In the success function, action.response.responseText does contain the JSON which I sent.
When it leaves the server, it looks like:
{
html: "<div>a div</div>"
}
When it arrives in the success function, the tags are missing. What's going on? Do I need to put some sort of html cdata wrapper around the entire response on the server to avoid this?
A string in a string in JSON. As long as it is well-formed you are allowed to put HTML in string values (making sure you escape quotes etc.).
It's probably the function you're using to insert the HTML that's stripping the tags.
Here's the situation. When you ask ExtJS or JQuery to do Ajax for a form with a file upload, it has to use an iframe. For the response to come back correctly, it has to be of content type text/html whatever is in it. So it has to have it's HTML characters escaped for HTML, which I accomplished with a function from CommonsLang.
To upload a file using inputType: 'file', and for a form prescribed fileUpload: true.
Everything works fine, the file is saved. But after saving the file block success (from ajax request) is not satisfied. Ie all the stops to waitMsg: 'Saving Data ...'.
What could be wrong?
Also, remember to set the Content-Type header to correct MIME type in your server response: "text/html". Anything else will result in ExtJS throwing an error when decoding your response.
In PHP, this can be done with
<?php
header('Content-type: text/html');
echo json_encode(array('success' => true));
?>
From ExtJS API docs:
If the server is using JSON to send the return object, then the Content-Type header must be > set to "text/html" in order to tell the browser to insert the text unchanged into the document body.
Characters which are significant to an HTML parser must be sent as HTML entities, so encode > "<" as "<", "&" as "&" etc.
Make sure you do escape special HTML characters as suggested. If you don't, ExtJS may still succeed in parsing the server response but with unexpected twists: single quotes in HTML-like strings turn into "', etc.
Maybe you should send back server result to extjs form by print this on server:
echo '{success:true, data: "save" }'