In XQuery 3.1 (under eXist-db 4.4) I receive search requests to the a controller where I create a parameter docset from the URL's query string text:
else if (starts-with(lower-case($exist:path), "/search")) then
<dispatch xmlns="http://exist.sourceforge.net/NS/exist">
<forward url="{$exist:controller}/search.html"/>
<view>
<forward url="{$exist:controller}/modules/view.xql">
<add-parameter name="docset"
value="{search:search-term-cleaner(request:get-parameter("text","norequest"))}"/>
<add-parameter name="pagetype" value="search"/>
</forward>
</view>
</dispatch>
I clean any incoming such requests to /search?text="" to permit only certain characters into the search query:
declare function search:search-term-cleaner($text as xs:string?) as xs:string?
{
let $cleanterm := replace($text,'[^A-Za-z\+*0-9]', '')
return $cleanterm
};
There are two problems, under two slightly different scenarios:
If the request comes in /search?text=some%+text the site complains with
org.eclipse.jetty.http.BadMessageException: 400: Unable to parse URI query
java.lang.IllegalArgumentException: Not valid encoding '%+t'
If the request comes in /search?text=some+text, the controller passes through sometext without the permitted + sign
Googling this has not lead me to a solution, but I am not experienced in managing HTTP parsing and may not understand the problem enough to search for the solution.
This is via local host http://localhost:8081/exist/apps/.
When getting parameters via request:get-parameter() you don’t need to unescape parameters that are URI-encoded. %20 and + are automatically handed to you as space characters.
functions such as util:unescape-uri and escape-uri are your friends.
Since the string you are working with gets send over http it will undergo escaping. You can find out more about available escaping functions by searching for escape in the function documentation
for more elaborate operations consider normalize-unicode
Related
I have a JMeter (v 5.4.1 ) script that login's to an SSO Service, using the following steps which are a result of using BlazeMeter Recorder.
GET on https://stage-aa.example.com/user/login
GET on https://stage-aa.example.com/saml_login
use this to declare SAMLRequest and SAMLResponse, Regular Expression Extractors
SAMLRequest : Regex: SAMLRequest=(.*)RelayState=(.*)
POST on https://sso.example.com/idp/EFUUU/resumeSAML20/idp/SSO.ping
This sample uses parameters:
username: user_1
password: pass_1
The parameters are named as:
pf.username
pf.pass
POST on https://stage-aa.example.com/simplesaml/module.php/saml/sp/saml2-acs.php/default-sp
with Parameter:
RelayState: https://stage-aa.example.com/saml_login
SAMLResponse: PHN8730Kkascnn...uyituNN (AlphaNumeric String )
with the Parameters:
RelayState: https://stage-aa.example.com/saml_login
SAMLResponse : Long String, which i am trying to Dynamically replace using ${SAMLResponse}
SAMLResponse is not getting value for the Regex Extractor !!!!!!!!, used SAMLResponse=(.*)
GET on https://stage-aa.example.com/admin/content
This gets error: 403 Forbidden
Since the user is not logged in, using SSO
Now, without having all results as rendered HTML, I am not sure, how I can get name/value for web elements
to be able to create a regex, which can be used in correlation
I added Debug Sampler, and I am getting values for SAMLRequest but not for SAMLResponse.
here is how SAMLResponse is exhibited in the browser:
RelayState=https%3A%2F%2Fstage-aa.example.com%2Fsaml_login&SAMLResponse=PHNhbWxwOlJlc3B....
here is the Regex Extractor for SAMLResponse
Adding the complete JMeter Test plan ( jmx file ) had to obfuscate the URLs, but this will give you a pretty good idea. I can add the results of the execution if needed.
jmx file gist
Also, getting error in step 5, in the View Results Tree ( Response Body )
<br />
<b>Warning</b>: DOMDocument::loadXML(): Start tag expected, '<' not found in Entity, line: 1 in <b>/var/www/vendor/simplesamlphp/saml2/src/SAML2/HTTPPost.php</b> on line <b>79</b><br />
<br />
<b>Warning</b>: Cannot modify header information - headers already sent by (output started at /var/www/vendor/simplesamlphp/saml2/src/SAML2/HTTPPost.php:79) in <b>/var/www/vendor/simplesamlphp/simplesamlphp/modules/redirecterrors/lib/RedirectErrors.php</b> on line <b>18</b><br />
here is a screenshot of SAMLResponse as Payload
Unfortunately we cannot help you with your regular expression without seeing the response data (at least partial) and your Regular Expression Extractor configuration. If you cannot come up with a good regular expression you can consider switching to
Particular your SAML implementation might be different, however my expectation is that SAMLResponse comes in a hidden input field so it makes more sense to consider switching to CSS Selector Extractor configured like:
You may also find How to Load Test SAML SSO Secured Websites with JMeter article useful.
i have identified the dynamic values in my website which i am handling through "regular expression extractor" still when i run the website the login request fails and throw error as token not found. please help,i m struggling...[this image contain the regEx extractor part and the failed login request also][1]
We cannot help you without seeing the response (at least partial) which you're getting for the /-7 request containing this hidden input with the token, you can try locating it using "RegExp Tester" mode of the View Results Tree listener and see whether your regular expression matches something or not
One thing is obvious: using regular expressions for extracting data from HTML is not the best idea, I would recommend considering switching to CSS Selector Extractor instead:
I need to load test one Python api in below URL format:
//IP:PORT/Sub_Part/?where={"KEY1":"VALUE1","KEY2":"VALUE2","KEY3":"VALUE3"}
I tried to pass the key value pair through csv as well as directly in http request but getting error message.
java.net.URISyntaxException: Illegal character in query at index 47:
http://IP:PORT/Sub_Part/?where={"KEY1":"VALUE1","KEY2":"VALUE2","KEY3":"VALUE3"}
Here key and value are dummy data I have placed here for easy understanding.
Please help me with correct syntax for this URL.
Thanks in advance for all your help.
In REST APIs, JSON objects are typically sent (POST) or returned in the body of the request. They are not typically encoded as part of the URL.
For a GET request, you can either pass the information as segments in the url or as querystring parameters.
For more details refer here, How to send a GET request with a "/" in the query
As per HTML URL Encoding Reference:
URLs can only be sent over the Internet using the ASCII character-set.
so you need to define the request in JMeter's HTTP Request sampler as follows:
Pay attention to URL Encode? checkbox, it has to be ticked otherwise the parameter will be sent "as is"
Another option is using __urlencode() function like:
http://IP:PORT/Sub_Part/?where=${__urlencode({"KEY1":"VALUE1"\,"KEY2":"VALUE2"\,"KEY3":"VALUE3"})}
which will generate an encoded sequence which you could use in the URL path:
%7B%22KEY1%22%3A%22VALUE1%22%2C%22KEY2%22%3A%22VALUE2%22%2C%22KEY3%22%3A%22VALUE3%22%7D
as you can see, all non-ASCII characters where removed.
Check out Apache JMeter Functions - An Introduction to learn more about JMeter Functions concept.
I'm trying to use Spring's ServletUriComponentsBuilder to create paging next and prev links from the current request.
The problem I'm having is that the ServletUriComponentsBuilder.fromCurrentRequest() is not unencoding percent-encoded values like:
http://example.com/articles?page%5Bnumber%5D=2
The problem is uses could have called the page with unencoded square brackets like http://example.com/articles?page[number]=2 without any problems.
Spring Data is accepting both variants (both unencoded square brackets and encoded square brackets) in it's pageable argument resolver.
This to the fact that under water the Coyote web request get parameter is used which contains the unencoded param names.
Also Spring's #RequestParam("page[number]") accepts without any problem the encoded request like http://example.com/articles?page%5Bnumber%5D=2.
From the server side I always want to return percent encoded url's as per RFC 3986.
But there does not seem a way to this as the UriComponents query params might contain both encoded en uncoded names. Because to that, if I would call encode() on the builder the already encoded query params get encoded another time, but if would contain unencoded names toURI() will fail as an unencoded [ is not allowed.
Note that the url's might contain multiple query params besides paging, e.g. for filtering.
A request could come in like:
http://example.com/articles?filter[category]=food
And would return a response with a encoded next link like:
http://example.com/articles?page%5Bnumber%5D=2&filter%5Bcategory%5D=food
My workaround it to ignore ServletUriComponentsBuilder and simply get the request url and do so custom regexp replacing.
I know this is an older question and you also found a workaround. But did you try to use ServletUriComponentsBuilder's build() method?
Some kind of the following:
ServletUriComponentsBuilder.fromCurrentRequest().build().toUriString();
I had some issues when handling JSON Strings and this helped.
I am currently developing a Ruby API based on Sinatra. This API mostly receives GET requests from an existing social platform which supports external API integration.
The social platform fires off GET requests in the following format (only relevant parameters shown):
GET /{command}
Parameters: command and text
Where text is a string that the user has entered.
In my case, params[:text] is in fact a series of commands, delimited by a space. What I want to achieve is, for example: If params[:text]="corporate finance"
Then I want my API to interpret the request as a GET request to
/{command}/corporate/finance
instead of requesting /{command} with a string as a parameter containing the rest of the request.
Can this be achieved on my side? Nothing can be changed in terms of the initial request from the social platform.
EDIT: I think a better way of explaining what I am trying to achieve is the following:
GET /list?text=corporate finance
Should hit the same endpoint/route as
GET /list/corporate/finance
This must not affect the initial GET request from the social platform as it expects a response containing text to display to the user. Is there a neat, best practice way of doing this?
get "/" do {
text = params[:text].split.join "/"
redirect "#{params[:command]}/#{text}"
end
might do the trick. Didn't check though.
EDIT: ok, the before filter was stupid. Basically you could also route to "/" and then redirect. Or, even better:
get "/:command" do {
text = params[:text].split.join "/"
redirect "#{params[:command]}/#{text}"
}
There a many possible ways of achieving this. You should check the routes section of the sinatra docs (https://github.com/sinatra/sinatra)
The answer by three should do the trick, and to get around the fact that the filter will be invoked with every request, a conditional like this should do:
before do
if params[:text]
sub_commands = params[:text].split.join "/"
redirect "#{params[:command]}/#{sub_commands}"
end
end
I have tested it in a demo application and it seems to work fine.
The solution was to use the call! method.
I used a regular expression to intercept calls which match /something with no further parameters (i.e. /something/something else). I think this step can be done more elegantly.
From there, I split up my commands:
get %r{^\/\w+$} do
params[:text] ? sub_commands="/"+params[:text].split.join("/") : sub_commands=""
status, headers, body = call! env.merge("PATH_INFO" => "/#{params[:command]}#{sub_commands}")
[status, headers, body]
end
This achieves exactly what I needed, as it activates the correct endpoint, as if the URL was typed it the usual format i.e. /command/subcommand1/subcommand2 etc.
Sorry, I completely misunderstood your question, so I replace my answer with this:
require 'sinatra'
get '/list/?*' do
"yep"
end
like this, the following routes all lead to the same
You need to add a routine for each command or replace the command with a * and depend your output based on a case when.
The params entered by the user can be referred by the params hash.
http://localhost:4567/list
http://localhost:4567/list/corporate/finance
http://localhost:4567/list?text=corporate/finance