My configuration: Symfony 4.2.5 , JMSSerialiser 3.2. I cannot obtain UTF-8 response with serialization. I thougt solution was in config:
jms_serializer:
visitors:
json:
options: [JSON_UNESCAPED_UNICODE]
But it does'nt work, it gives me an UNICODE printing. So, I get this response:
`"aeiou\u00e0\u00e9\u00e8\u00f4\u00fb"`
Here is my code:
/**
* #Route("/route1",name="route1")
*/
public function route1(){
$serializer = SerializerBuilder::create()->build();
$chaine = "aeiouàéèôû";
$reponse = new Response(
$serializer->serialize($chaine, 'json'),
Response::HTTP_OK,
['content-type' => 'text/html']
);
$reponse->setCharset('UTF-8');
return $reponse;
}
I discover that the problem disappears when I launch the request from firefox which offers me two display modes: json, raw and an additional tab for request and response headers. The Chrome browser only offers me the raw display. Should I conclude that the solution lies in the configuration of Chrome?
Subject resolved. Chrome displays by default the raw presentation of the file where accented characters are in escaped form.
To get the json form in Chrome, you need to install the jsonView extension. Once this extension is activated the display of accented characters is done correctly
Related
Now I tried to make like a translator through Roblox Studio using Https service by sending a request to the translate.google.com link the thing is that anything I get in return does not give me the translated text.
I put what I received in a google doc and tried to find it by pressing ctrl + f to try to find it but no luck the only thing I could find is that text that was supposed to be translated. Here is the code in case you want to try it for yourself but I do warn you that running this might make Roblox unresponsive for a while since it is a lot of info they gave back.
I don't know if I am doing something wrong or not someone please help! I just want it to give me what 'Hello world' would be in french, there are also no error messages.
local http = game:GetService("HttpService")
local Message = "Hello world"
http:UrlEncode(Message) -- 'Hello world' -> 'Hello%20world'
local response = http:RequestAsync(
{
Url = "https://translate.google.com/?sl=en&tl=fr&text=" .. Message .. "!&op=translate";
Method = "GET"
}
)
if response.Success then
print(response.StatusMessage)
print(response.StatusCode)
print(response.Body)
--print(response.Headers)
else
print("The request failed: ", response.StatusCode, response.StatusMessage)
end
When visiting on your browser (for example) the url https://translate.google.com/?sl=en&tl=fr&text=Hello%20World!&op=translate, the translation you see is fetched using Javascript code executed by the browser after loading the page.
The browser retrieves the html body of the page (like you did in your code) and then executes the javascript in the html body which retrieves the translation and updates the page.
Unless you use a browser driver like Selenium I don't see how you can do what you want in a simple way.
Plus, I'm sure that Google has some protection against automatic bots, so after too many request your program will probably will be blocked by ReCaptcha.
The correct way to translate the text is to use the Google Cloud Translate API which I think is free up to 500k requests per month. There is also Azure Translator from Microsoft which also has a free tier.
Your issue is likely in how you are URL Encoding the string.
http:UrlEncode(Message)
HttpService.UrlEncode returns the encoded string as a new value. It doesn't mutate the existing value. So you just need to store the result of the function call.
Message = http:UrlEncode(Message)
EDIT : Just as #Mohamed AMAZIRH pointed out, hitting this URL will only return HTML.
I have 2 Laravel(5.8) apps. One is a user application, the other is more of an API.
The Api has pdfs stored in the storage directory, I need to be able to allow a user to download the pdfs in the other application.
Really got no clue how to send the file over from app to the other.
The user app makes an api to the api with relevant file ids and things (fine), just can't work out how to send the file back, and then download it on the other end.
Tried things like return response()->stream($pdf) on the api and return response()->download($responeFromApiCall) and loads of other things but really getting nowhere fast.
Any advice welcome.
The laravel code you posted is basically correct, you can use one of stream(), download() or file() helper to serve a file.
file()
Serve a file using a binary response, the most straightforward solution.
// optional headers
$headers = [];
return response()->file(storage_path('myfile.zip'), $optionalHeaders);
You can also use your own absolute file path instead of the storage_path helper.
download()
The download helper is similar to file(), but it also sets a Content-Disposition header which will result in a download-popup if you retrieve the file using your browser.
$location = storage_path('myfiles/invoice_document.pdf');
// Optional: serve the file under a different filename:
$filename = 'mycustomfilename.pdf';
// optional headers
$headers = [];
return response()->download($location, $filename, $headers);
stream()
The stream() helper is a bit more complex and results in reading and serving a file in chunks. This can be used for situations where the filesize is unknown (like passing through another incoming stream from an API). It results in a Transfer-Encoding: chunked header which indicates that the data stream is divided into a series of non-overlapping chunks:
// optional status code
$status = 200;
// optional headers
$headers = [];
// The stream helper requires a callback:
return response()->stream(function() {
// Load a file from storage.
$stream = Storage::readStream('somebigfile.zip');
fpassthru($stream);
if(is_resource($stream)) {
fclose($stream);
}
}, $status, $headers);
Note: Storage::readStream takes the default storage disk as defined in config/filesystems.php. You can change disks using Storage::disk('aws')->readStream(...).
Retrieving your served file
Say your file is served under GET example.com/file, then another application can retrieve it with curl (assuming PHP). A popular wrapper for this would be Guzzle:
$client = new \GuzzleHttp\Client();
$file_path = __DIR__ . '/received_file.pdf';
$response = $client->get('http://example.com/file', ['sink' => $file_path]);
You can derive the filename and extension from the request itself by the way.
If your frontend is javascript, then you can retrieve the file as well but this another component which I dont have one simple example for.
So if I understand correctly you have two systems: user app and api. You want to serve a pdf from user app to the actual user. The user app does a call to api, which works fine. Your issue is converting the response of the api to a response which you can serve from your user app, correct?
In that case I'd say you want to proxy the response from the api via the user app to the user. I don't know what the connection between the user app and the api is, but if you use Guzzle you could look at this answer I posted before.
Here are the steps you should follow to get the PDF:
Make an API call using AJAX ( you are probably already doing it ) from your public site ( User site ) to the API server, with file ID.
API server fetches the PDF, copy it to the public/users/ directory, generate the URL of that PDF file.
Send the URL as a response back to the User site. Using JS add a button/ link in the DOM, to that PDF file.
Example:
jQuery.post(ajaxurl, data, function(response) {
var responseData = JSON.parse(response);
var result = responseData.result;
var pdf_path = responseData.pdf_path;
$(".nametag-loader").remove();
$(".pdf-download-wrapper").append("<a class='download-link' href='"+ pdf_path +"' target='_blank'>Download</a>");
});
I have a Spring application that receives a request like http://localhost/foo?email=foo+bar#example.com. This triggers a controller that roughly looks like this:
#RestController
#RequestMapping("/foo")
public class FooController extends Controller {
#GetMapping
public void foo(#RequestParam("email") String email) {
System.out.println(email)
}
}
By the time I can access email, it's been converted to foo bar#example.com instead of the original foo+bar#example.com. According to When to encode space to plus (+) or %20? this should only happen in requests where the content is application/x-www-form-urlencoded. My request has a content type of application/json. The full MIME headers of the request look like this:
=== MimeHeaders ===
accept = application/json
content-type = application/json
user-agent = Dashman Configurator/0.0.0-dev
content-length = 0
host = localhost:8080
connection = keep-alive
Why is Spring then decoding the plus as a space? And if this is the way it should work, why isn't it encoding pluses as %2B when making requests?
I found this bug report about it: https://jira.spring.io/browse/SPR-6291 which may imply that this is fixed on version 3.0.5 and I'm using Spring > 5.0.0. It is possible that I may misinterpreting something about the bug report.
I also found this discussion about RestTemplate treatment of these values: https://jira.spring.io/browse/SPR-5516 (my client is using RestTemplate).
So, my questions are, why is Spring doing this? How can I disable it? Should I disable it or should I encode pluses on the client, even if the requests are json?
Just to clarify, I'm not using neither HTML nor JavaScript anywhere here. There's a Spring Rest Controller and the client is Spring's RestTemplate with UriTemplate or UriComponentsBuilder, neither of which encode the plus sign the way Spring decodes it.
Original Answer
You are mixing 2 things, a + in the body of the request would mean a space when header has application/x-www-form-urlencoded. The body or content of the request would be dependent on the headers but a request can just have a url and no headers and no body.
So the encoding of a URI cannot be controlled by any headers as such
See the URL Encoding section in https://en.wikipedia.org/wiki/Query_string
Some characters cannot be part of a URL (for example, the space) and some other characters have a special meaning in a URL: for example, the character # can be used to further specify a subsection (or fragment) of a document. In HTML forms, the character = is used to separate a name from a value. The URI generic syntax uses URL encoding to deal with this problem, while HTML forms make some additional substitutions rather than applying percent encoding for all such characters. SPACE is encoded as '+' or "%20".[10]
HTML 5 specifies the following transformation for submitting HTML forms with the "get" method to a web server.1 The following is a brief summary of the algorithm:
Characters that cannot be converted to the correct charset are replaced with HTML numeric character references[11]
SPACE is encoded as '+' or '%20'
Letters (A–Z and a–z), numbers (0–9) and the characters '*','-','.' and '_' are left as-is
All other characters are encoded as %HH hex representation with any non-ASCII characters first encoded as UTF-8 (or other specified encoding)
The octet corresponding to the tilde ("~") is permitted in query strings by RFC3986 but required to be percent-encoded in HTML forms to "%7E".
The encoding of SPACE as '+' and the selection of "as-is" characters distinguishes this encoding from RFC 3986.
And you can see the same behaviour on google.com as well from below screenshots
Also you can see the same behaviour in other frameworks as well. Below is an example of Python Flask
So what you are seeing is correct, you are just comparing it with a document which refers to the body content of a request and not the URL
Edit-1: 22nd May
After debugging it seems the decoding doesn't even happen in Spring. I happens in package org.apache.tomcat.util.buf; and the UDecoder class
/**
* URLDecode, will modify the source.
* #param mb The URL encoded bytes
* #param query <code>true</code> if this is a query string
* #throws IOException Invalid %xx URL encoding
*/
public void convert( ByteChunk mb, boolean query )
throws IOException
{
int start=mb.getOffset();
And below is where the conversion stuff actually happens
if( buff[ j ] == '+' && query) {
buff[idx]= (byte)' ' ;
} else if( buff[ j ] != '%' ) {
This means that it is an embedded tomcat server which does this translation and spring doesn't even participate in this. There is no config to change this behaviour as seen in the class code. So you have to live with it
SPR-6291 fixed this problem in v3.0.5 but this remains unresolved in some other cases like SPR-11047 is still unresolved. While SPR-6291's priority was Major, SPR-11047's priority is Minor.
I faced this problem when I was working on REST API in old Spring last year. There are multiple ways we can get data in Spring controller. So two of them are via #RequestParam or #PathVariable annotation
As others mentioned I think its spring's internal issue and does not specifically belong to URL encoding because I was sending data over POST request but it is somewhat encoding problem. But I also agree with others as now it remains problematic only in URL.
So there are two solutions I know:
You can use #PathVariable instead of #RequestParam because as of SPR-6291 this plus sign issue is fixed in #PathVariable and still remains open for #RequestParam as SPR-11047
My version of spring was not even accepting plus sign via #PathVariable annotation, so this is how I overcome the problem (I don't remember it step by step but it will give you hint).
In your case you can get the fields via JS and escape the plus sign before sending a request. Something like this:
var email = document.getElementById("emailField").value;
email = email.replace('+', '%2B');
If you have this request:
http://localhost/foo?email=foo+bar#example.com
then the original is foo bar#example.com. If you say the original should be foo+bar#example.com then the request should be:
http://localhost/foo?email=foo%2Bbar#example.com
So Spring is working as supposed to. Maybe on client you should check if the URI is properly encoded. The client-side URL encoding is responsible for building a correct HTTP request.
See encodeURI() if you generate the request in JavaScript or uriToString() if you generate the request in Spring.
Build your request string (the part after ?), without any encoding, with unencoded values like foo+bar#email.com, and only in the end, before actually using it in GET, encode all of it with whatever is available on the client platform. If you want to use POST then you should encode it according to the MIME type of your choice.
Using the recaptcha javascript client ( http://www.google.com/recaptcha/api.js?onload=vcRecaptchaApiLoaded&render=explicit ) the automatic request (GET https://www.google.com/recaptcha/api2/userverify?k=XXXXXXXXX) after you select the correct images returns an invalid json which afterwards is sent to back to google and their json validation fails. This worked as expected for months now. Not anymore :(
Server side library used reCaptcha PHP (1.1)
Response
)]}'
["uvresp","03AHJ_Vuup5SJJ583dSIfezFl80dp2AlJ_rZpz3vqGWlOTmbjZqH8izwjruJASNhQI0tOOnmj2Pzg14xMw7dryeqVfTGhx6dg_x2i0PRA1ZeDyrBNn8DX-w5S262Zb3_ZWKj5JDBiqPVnXtNLGbyBxjd97VHanspJWU_-qKWLSWVKxLK6n3lm9Biw33oUCEiGA39GNa09Z6TSAEtnolQCf_LPPRWKoE_e50f2s5ZpUVG5GNVdX7qGBRwphTgcUhwOjA8uYTzmA9co3Jwk2KR5UQ0zzVRJzRZZTBuK9km3PE1WV05ACAwrJi29niDpVaRmpooAnIkHNgGyGBu7u3W7gU6YAHtwya8PYhdF__G_MMG8XpVFDTBa196hKD6hxw-E2PsxmoIQJrU1K89mmzNIh-xLNQ7KJvrBMzVf8A5FHyUQgL5UNDWVwSkWCdC_3swxBzi7R3p8VIrUtkIqJFA_GSAxy0cBRJ8J55Pfs5rzhfR8j-x1hGCzi_6vJrbrwfNesoLEB7GWJtElcljhBYvcDNzU_B_VJ7Sck-6i1Nd0qdmtSiCRZYNyaZ8uGLoDdNgCY-0Oi4802AlI26H7TjGBcKnr4gmaHXTNRf1W7x_3FV05DxVsTqeAlo8zGqmiVqcgmX64BbLK4fD9Xoait1_Lp5vK26fCaOQmGKF7CJaYPuxnX-zXgSkfZDCG6rs6xv1CfZQnIKD0W3Yz522VD4YdNfATb3FywhFWbZuxoBIt6vslZDlPXh2MYOkAmYfIPIo8WoWazMoLI_8iNBZPiMlRL0PS5aQiLSrvbf-sknMHhfM2MJYsfrQjC52aDRaHYdcZbY6Wxlhw0tQEknX8B47_DAQzCKkpoFsecO1eMHuInIykZ1l7TOdZMytI-NzGg21KeKAE8dK6ZWee0UEqDJvCkj5aH5TQcBA--ygbOS186bAptUP5n6WvORx1Nb2ZU_AF9fB23PJWH1xvB4gZoNDvhLmdpkE1Po9Lyim1P61E2rrgYjWgPRwT4jUo",1,120]
Any ideas?
Thanks
I think it's a duplicate of https://stackoverflow.com/a/36862268/2140017
And here is my answer from that link:
Actually the value returned is not valid json but well parsed by the
Google's API.
Is it a protection ? I don't know, but if you look at the javascript,
you can find that:
var jm=function(a,b,c,d,e,g,h,l,r){this.xl=a;this.$c=c||"GET";this.Ka=d;this.Gg=e||null;this.Td=m(h)?h:1;this.ye=0;this.xh=this.Nh=!1;this.uh=b;this.Mh=g;this.md=l||"";this.Zb=!!r;this.Wf=null};f=jm.prototype;f.getUrl=function(){return this.xl};f.ug=function(){return this.$c};f.Ca=function(){return this.Ka};f.fi=function(){return this.Zb};f.bi=function(){return this.md};var nm=function(){G.call(this);this.nj=new hm(0,mm,1,10,5E3);H(this,this.nj);this.ad=0};x(nm,G);var mm=new Nh;nm.prototype.send=function(a){return new Lc(function(b,c){var d=String(this.ad++);this.nj.send(d,a.Uf.toString(),a.ug(),a.Ca(),mm,void 0,u(function(a,d){var h=d.target;if(Xk(h)){var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;b(new l(h))}else c(new om(a))},this,a))},this)};var om=function(a){y.call(this);this.request=a};x(om,y);
especially take a look at:
var l=a.ml;h.B?(h=h.B.responseText,0==h.indexOf(")]}'\n")&&(h=h.substring(5)),h=Hg(h)):h=void 0;`
The parser explicitly checks that the value begins by )]} and strips
it.
I suggest you to just apply the same substring on the "json" string
To upload a file using inputType: 'file', and for a form prescribed fileUpload: true.
Everything works fine, the file is saved. But after saving the file block success (from ajax request) is not satisfied. Ie all the stops to waitMsg: 'Saving Data ...'.
What could be wrong?
Also, remember to set the Content-Type header to correct MIME type in your server response: "text/html". Anything else will result in ExtJS throwing an error when decoding your response.
In PHP, this can be done with
<?php
header('Content-type: text/html');
echo json_encode(array('success' => true));
?>
From ExtJS API docs:
If the server is using JSON to send the return object, then the Content-Type header must be > set to "text/html" in order to tell the browser to insert the text unchanged into the document body.
Characters which are significant to an HTML parser must be sent as HTML entities, so encode > "<" as "<", "&" as "&" etc.
Make sure you do escape special HTML characters as suggested. If you don't, ExtJS may still succeed in parsing the server response but with unexpected twists: single quotes in HTML-like strings turn into "', etc.
Maybe you should send back server result to extjs form by print this on server:
echo '{success:true, data: "save" }'