It looks like golang reverseproxy double escapes url, when making a http request,
server receives:
/id/EbnfwIoiiXbtr6Ec44sfedeEsjrf0RcXkJneYukTXa%252BIFVla4ZdfRiMzfh%252FEGs7f
expected:
/id/EbnfwIoiiXbtr6Ec44sfedeEsjrf0RcXkJneYukTXa%2BIFVla4ZdfRiMzfh%2FEGs7f
Is there a way to avoid double escaping?
Go isn't double-escaping the URL. It's constructing the URL given the values that you gave it (which means escaping the path). You've already escaped the path. Don't do that. If you want to send +, then use +. Go will escape it correctly.
If you have an escaped path already for some reason, then unescape it with url.PathUnescape() before constructing the URL.
Related
This is what happens when I try to use a colon in th:text:
and a backslash doesn't seem to fix it:
How can I use the colon symbol in th:text?
If you want to place a literal into th:text, you have to use single quotes: th:text="'7:00AM'". See documentation here.
(By contrast, something like this th:text="7_00AM" is valid - because it is a literal token. Such strings can only use a subset of characters, but do not need enclosing 's.)
I've got a REST WebService and a PLSQL Package, from which i want to call the WebService.
The parameters for the call are located within the URI of the WebService.
htttp://myservice:8080/some/path/action?value1=123456&value2=some chars&value3=aGermanSonderzeichenCalledÄ
As you can see, there are 2 problems with the URI. First the whitespace for value2 and second the special character for value3.
That said, it is clear to me that the URI has to be encoded to a more friendly format.
The WebService desires UTF-8, so the URI is encoded with:
UTL_URL.ESCAPE(url,false,'UTF-8').
This results in the following URI:
htttp://myservice:8080/some/path/action?value1=123456&value2=some%20chars&value3=aGermanSonderzeichenCalled%C3%84
So far, so good. This encoded URI is passed to UTL_HTTP.BEGIN_REQUEST(url,'GET').
When I execute this request, and intercept it with Wireshark, i can see that the actual URI that got called is:
htttp://myservice:8080/some/path/action?value1=123456&value2=some%2520chars&value3=aGermanSonderzeichenCalled%25C3%2584
What we can see is, that UTL_HTTP escapes the reserved character '%' to %25.
So in my case the whitespace first got converted to %20 and after that to %2520.
What I'm looking for is a way to stop UTL_HTTP from escaping the reserved characters.
As an alternative, a way in which UTL_HTTP deals with the whitespace and special character, without me calling UTL_URL, would also work for me.
I want to send a curl post request with query params. One of these params is an array of strings. An example for such a string (using Postman chrome extension):
http://www.example.com/my_rest_endpoint?start=123456&duration=11&array_param=["activity level"]
Now, how can I translate this request to a curl command? no matter what I do, I can't seem to send the last parameter, array_param to curl. I've tried escaping the quotes and/or brackets with \, tried using -i flag and other various options, all with no success.
Solution was very close to what benaryorg suggested: Brackets and spaces needs to be placed with %5B, %5D and + according to the url encoding. The quotes were escaped using \". The final result is:
http://www.example.com/my_rest_endpoint?start=123456&duration=11&array_param=%5B\"Activity+Level\"%5D".
Have you tried placing ' around the parameters instead of escaping?
Also if that does not work, you could try to escape them like this example from Wikipedia.
You can find the codes for encoding here.
I am using Net::HTTP to get a request from a Google API with a custom header:
req = Net::HTTP::Get.new(uri.request_uri, {'Authorization' => 'GoogleLogin auth=#{auth}'})
The #{auth} is a variable that changes each time I run the program, so I made a variable with it, but the single quotes don't expand it. I can't change the single quotes to double quotes, because Google only accepts the header with single quotes.
Is there any way to expand the variable but keep the single quotes?
However, I can't change the single quotes to double quotes, because google only accepts the header with single quotes.
So hard to believe it.
Anyway, try Kernel%sprintf or its shorter version just str % [arguments..]. It will help.
Why doesn't URI.escape escape single quotes?
URI.escape("foo'bar\" baz")
=> "foo'bar%22%20baz"
For the same reason it doesn't escape ? or / or :, and so forth. URI.escape() only escapes characters that cannot be used in URLs at all, not characters that have a special meaning.
What you're looking for is CGI.escape():
require "cgi"
CGI.escape("foo'bar\" baz")
=> "foo%27bar%22+baz"
This is an old question, but the answer hasn't been updated in a long time. I thought I'd update this for others who are having the same problem. The solution I found was posted here: use ERB::Util.url_encode if you have the erb module available. This took care of single quotes & * for me as well.
CGI::escape doesn't escape spaces correctly (%20) versus plus signs.
According to the docs, URI.escape(str [, unsafe]) uses a regexp that matches all symbols that must be replaced with codes. By default the method uses REGEXP::UNSAFE. When this argument is a String, it represents a character set.
In your case, to modify URI.escape to escape even the single quotes you can do something like this ...
reserved_characters = /[^a-zA-Z0-9\-\.\_\~]/
URI.escape(YOUR_STRING, reserved_characters)
Explanation: Some info on the spec ...
All parameter names and values are escaped using the [rfc3986]
percent- encoding (%xx) mechanism. Characters not in the unreserved
character set ([rfc3986] section 2.3) must be encoded. characters in
the unreserved character set must not be encoded. hexadecimal
characters in encodings must be upper case. text names and values must
be encoded as utf-8 octets before percent-encoding them per [rfc3629].
I know this has been answered, but what I wanted was something slightly different, and I thought I might as well post it up: I wanted to keep the "/" in the url, but escape all the other non-standard characters. I did it thus:
#public filename is a *nix filepath,
#like `"/images/isn't/this a /horrible filepath/hello.png"`
public_filename.split("/").collect{|s| ERB::Util.url_encode(s)}.join("/")
=> "/images/isn%27t/this%20a%20/horrible%20filepath/hello.png"
I needed to escape the single quote as I was writing a cache invalidation for AWS Cloudfront, which didn't like the single quotes and expected them to be escaped. The above should make a uri which is more safe than the standard URI.escape but which still looks like a URI (CGI Escape breaks the uri format by escaping "/").