The situation
Our Google Search Appliance (Software Version: 7.2.0.G.112) is setup to expand queries using a custom synonyms file containing for example this entry: {men, mens}
The problem
The search appliance appears to use these synonyms when responding to a GET request but not when responding to a POST request. See the table below
+-----------------+-------------+----------------+-----------+
| Request Type | Query | Result Count | Good? |
+-----------------+-------------+----------------+-----------+
| GET | mens | 705 | yes |
| POST | mens | 691 | yes |
| GET | men | 706 | yes |
| POST | men | 88 | no |
+-----------------+-------------+----------------+-----------+
The Question
How can I enable the query expansion/ synonyms for the POST request so it will return (approximately) the same results.
The Requests in Detail
Get Request
GET /search?q=men&output=xml_no_dtd&client=default_frontend&
getfields=*&filter=0&start=0&num=25&site=some_value&
requiredfields=(-core__isblocked.core__brandid:brand.
(core__catalog:163%252D2101|(inv__0104|inv__3301))) HTTP/1.1
Host: xxx.xxx.xxx.xxx:80
Cache-Control: no-cache
Post Request
POST /search HTTP/1.1
Host: xxx.xxx.xxx.xxx
Content-Type: application/x-www-form-urlencoded
Content-Length: 242
Cache-Control: no-cache
q=men&output=xml_no_dtd&client=default_frontend&
getfields=*&filter=0&start=0&num=2&site=some_value&
requiredfields=(-core__isblocked.core__brandid:brand.
(core__catalog:163%2D2101|(inv__0104|inv__3301)))
Bonus question: why is the result for GET and POST for "mens" also different.
You can set "Query Expansion Policy" in the frontend. Are you sure you are using the the same frontend for both the queries. In my knowledge, the GET/POST method should not affect the search result.
Why isn't stackflow allowing me to add a comment to the question. Gosh..
UPDATE
Also, core__catalog:163 -- THis value is different in the GET and POST. Might be something to do with the character encoding/decoding? Can you remove all those requiredfields and just supply 'q' and look for the count?
Related
We have a web-application hosted on Azure and it sends Telemetry to App Insights and the Dev team is asking if it is ok to Turn off sending the SESSION/KEEPALIVE data thats being posted from web-application. Will this affect any functionality like User Flows etc in Application Insights?
Any guidance on this?
Following is sample data:-
timestamp | id | source | name | url | success | resultCode | duration | performanceBucket
-- | -- | -- | -- | -- | -- | -- | -- | --
2019-09-25T16:00:31.8191577Z | \|Ac34D.9fIx+.4c3e0b35_ | POST session/keepalive | http://XXXXXXXXXXXXXX.com/session/keepalive | TRUE | 200 | 15.8274 | <250ms
2019-09-25T16:00:42.7423811Z | \|Ac34D.FqSNy.83ee6e0d_ | POST session/keepalive | http://XXXXXXXXXXXXXX.com/session/keepalive | TRUE | 200 | 38.3679 | <250ms
2019-09-25T16:00:48.716939Z | \|Ac34D.h8kwN.34c0b012_ | POST session/keepalive | http://XXXXXXXXXXXXXX.com/session/keepalive | TRUE | 200 | 16.0359 | <250ms
2019-09-25T16:00:54.1607213Z | \|Ac34D.v2qfF.4c3e0b36_ | POST session/keepalive | http://XXXXXXXXXXXXXX.com/session/keepalive | TRUE | 200 | 15.2518 | <250ms
Views in Applications Insights typically target a specific set of telemetry item types.
For instance, user flows UI leverages PageView and CustomEvent telemetry types. Therefore, if keep alive is reported as one of those types it will be displayed in that UI.
However, if the example above is Dependency telemetry, then that view won't be affected.
In general, if you'd like to drop some of the telemetry before it reaches AI and is processed for storage, you'd use TelemetryProcessor (in case of Java Script SDK, TelemetryInitializer) to filter it out:
var telemetryInitializer = (envelope) => {
if (envelope.data.someField == 'keepalive') return false;
};
appInsights.addTelemetryInitializer(telemetryInitializer);
I have an application which is authorized with keycloak (using openid).When i used to only application URL then login page came. In address bar 'state' & 'nonce' value is coming. But while recording in Jmeter 5.1 I am not able to find these values in response of which request. If i can find the exact request so i can create Regex in that request and store that values in variable for further use. Please help me for this case. Thank you in advanced.
App url - http://{ServerDNS}/{tagname}/#
Redirected URL once used App URL - https://{ServerDNS:Port}/auth/realms/{keycloakNmae}/protocol/openid-connect/auth?response_type=id_token%20token&client_id=oauth2&state=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B&redirect_uri={APP_URL}&scope=openid%20profile%20email%20voucher&nonce=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B
Here state & nonce values are -
state=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B
nonce=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B
I didn't find these values in response of any request. these values are coming in redirect url before providing user credentials.
Again Thank you.
Here state & nonce values are -
state=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B
nonce=yaFY1QJATZxm4LHJoobGu9YEN9aS95zlGWbuTG5B
I didn't find these values in the response of any request. these values are coming in redirect URL before providing user credentials.
Looking into OpenID Connect Core 1.0 incorporating errata set 1
+--------+ +--------+
| | | |
| |---------(1) AuthN Request-------->| |
| | | |
| | +--------+ | |
| | | | | |
| | | End- |<--(2) AuthN & AuthZ-->| |
| | | User | | |
| RP | | | | OP |
| | +--------+ | |
| | | |
| |<--------(3) AuthN Response--------| |
| | | |
| |---------(4) UserInfo Request----->| |
| | | |
| |<--------(5) UserInfo Response-----| |
| | | |
+--------+ +--------+
and into ID Token
nonce
String value used to associate a Client session with an ID Token, and to mitigate replay attacks. The value is passed through unmodified from the Authentication Request to the ID Token. If present in the ID Token, Clients MUST verify that the nonce Claim Value is equal to the value of the nonce parameter sent in the Authentication Request. If present in the Authentication Request, Authorization Servers MUST include a nonce Claim in the ID Token with the Claim Value being the nonce value sent in the Authentication Request. Authorization Servers SHOULD perform no other processing on nonce values used. The nonce value is a case sensitive string.
and into Nonce Implementation Notes
The nonce parameter value needs to include per-session state and be unguessable to attackers. One method to achieve this for Web Server Clients is to store a cryptographically random value as an HttpOnly session cookie and use a cryptographic hash of the value as the nonce parameter. In that case, the nonce in the returned ID Token is compared to the hash of the session cookie to detect ID Token replay by third parties. A related method applicable to JavaScript Clients is to store the cryptographically random value in HTML5 local storage and use a cryptographic hash of this value.
So you could obtain this nonce value either from your browser cookies or from its local storage
Just in case check out OpenID Connect - How to Load Test with JMeter for better understanding of the concept.
When defining HTTP Request, there's a checkbox for each parameter: Include Equals
This checkbox can't be unchecked even when choosing different method or parameter.
I don't see any reference in HTTP Request for using it.
Why is this checkbox shown? Is there any usage for it?
Also it seems that Content-Type value per parameter is ignored,in GET it isn't sent:
GET http://www.google.com/?token=0Bfdsa
GET data:
In POST it send the regular www-form-urlencoded:
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
I've also stumbled upon what does it mean, and I think I've found it. It gives you the option to include = (equals) sign or not for parameters with no value: foo= vs. foo. If the parameter has a value you cannot uncheck "Include Equals?":
| Name: | Value | Include Equals? |
|-------|-------|:---------------:|
| foo | | [x] |
| bar | | [ ] |
| baz | qux | [x] |
The above configuration generates the following url-encoded form:
foo=&bar&baz=qux
The "Content-Type" appears used with the "Use multipart/form-data" option checked – every parameter is sent as a separate part and its own Content-Type:
[x] Use multipart/form-data
| Name: | Value | Content-Type |
|-------|-------|--------------|
| foo | | text/x-foo |
| bar | | text/x-bar |
| baz | qux | text/x-baz |
The generated request looks like:
Content-Type: multipart/form-data; boundary=zIVpNBG_m1irxcTtk7ByTwBgDHbsjB1UjTdRTS
--zIVpNBG_m1irxcTtk7ByTwBgDHbsjB1UjTdRTS
Content-Disposition: form-data; name="foo"
Content-Type: text/x-foo; charset=US-ASCII
Content-Transfer-Encoding: 8bit
--zIVpNBG_m1irxcTtk7ByTwBgDHbsjB1UjTdRTS
Content-Disposition: form-data; name="bar"
Content-Type: text/x-bar; charset=US-ASCII
Content-Transfer-Encoding: 8bit
--zIVpNBG_m1irxcTtk7ByTwBgDHbsjB1UjTdRTS
Content-Disposition: form-data; name="baz"
Content-Type: text/x-baz; charset=US-ASCII
Content-Transfer-Encoding: 8bit
qux
--zIVpNBG_m1irxcTtk7ByTwBgDHbsjB1UjTdRTS--
Here it worked for me,
I unchecked 'use multipart/form-data' and from header pass 'Content-Type application/x-www-form-urlencoded'
I am not sure it this is intended to be so, but I am confused by the behavior.
When I have the following Scenario Outline:
Scenario Outline: outline1
Given url
And query parameters <query_params>
When method
Then status is
Examples:
| method | endpoint | query_params | status |
| GET | /endpoint1 | ?a=1&b=1 | 200 |
| GET | /endpoint1 | ?a=1&b=1&c=3 | 200 |
I see the following snippet generated.
func FeatureContext(s *godog.Suite) {
s.Step(^method GET$, methodGET)
s.Step(^query parameters \?a=(\d+)&b=(\d+)$, queryParametersAB)
s.Step(^query parameters \?a=(\d+)&b=(\d+)&c=(\d+)$, queryParametersABC)
}
As you can see 2 lines of "query parameters" produces 2 different functions. Why is godog parsing this text? This is a little different from cucumber gherkin parsing.
One side effect of this is that if I have 100 lines in the data table, I am forced to implement all of them.
Is there a way I can ask godog to not do this parsing?
The solution to the problem is to use double quotes around as given below.
Scenario Outline: outline1
Given url
And query parameters "<query_params>"
When method
Then status is
Examples:
| method | endpoint | query_params | status |
| GET | /endpoint1 | ?a=1&b=1 | 200 |
| GET | /endpoint1 | ?a=1&b=1&c=3 | 200 |
Then the following will be generated:
s.Step(`^query parameters "([^"]*)"$`, queryParameters)
I have a cucumber feature that checks a website has processed payment files correctly (BACS,SEPA, FPS etc). The first stage of the process is to create the payment files which in-turn create expected result data in a database. This data is then used to validate against the payment processing website.
If I process one file, my feature works perfectly validating the expected results. Where I'm stuck is how I get the feature to run (n) number of times depending on the number of records/files that were originally processed.
I've tried an 'Around' hook using a record count iteration with no joy, can't see how I can fit it into an outline scenario and now think that perhaps a rake task to call the feature might work.
Any ideas would be greatly appreciated.
Here's a sample of the feature:
Feature: Processing SEPA Credit Transfer Files. Same Day Value Payments.
Background:
Given we want to test the "SEPA_Regression" scenario suite
And that we have processed a "SEPA" file from the "LDN" branch
And we plan to use the "ITA1" environment
Then we log in to "OPF" as a "SEPA Department" user
#feature #find_and_check_sepa_interchange #all_rows
Scenario: Receive SEPA Credit Transfer Files for branch
Given that we are on the "Payment Management > Interchanges" page
When I search for our Interchange with the following search parameters:
| Field Name |
| Transport Date From |
| Bank |
| Interchange Reference |
Then I can check the following fields for the given file in the "Interchanges" table:
| Field Name|
| Interchange Reference |
| Transport Date |
| File Name |
| File Format |
| Clearing Participant |
| Status |
| Direction |
| Bank |
When I select the associated "Interchange Id" link
Then the "Interchange Details" page is displayed
Update I've implemented nested steps for the feature so that I can call the database records first and feed each set of records (or at least the row id) into the main feature like so:
Feature
#trial_feature
Scenario: Validate multiple Files
Given we have one or more records in the database to process for the "SEPA_Regression" scenario
Then we can validate each file against the system
Feature steps:
Then(/^we can validate each file against the system$/) do
x = 0
while x <= $interchangeHash.count - 1
$db_row = x
# Get the other sets of data using the file name in the query
id = $interchangeHash[x]['id']
file_name = $interchangeHash[x]['CMS_Unique_Reference_Id']
Background.get_data_for_scenario(scenario, file_name)
steps %{
Given that we are on the "Payment Management > Interchanges" page
When I search for our Interchange with the following search parameters:
| Field Name |
| Transport Date From |
| Bank |
| Interchange Reference |
Then I can check the following fields for the given file in the "Interchanges" table:
| Field Name|
| Interchange Reference |
| Transport Date |
| File Name |
| File Format |
| Clearing Participant |
| Status |
| Direction |
| Bank |
When I select the associated "Interchange Id" link
Then the "Interchange Details" page is displayed
Seems a bit of a 'hack' but it works.
If you have batch processing software, then you should have several Given (setup) steps, 1 When (trigger) step, several Then (criteria) steps.
Given I have these SEPA bills
| sepa bill 1 |
| sepa bill 2 |
And I have these BAC bills
| bac bill 1 |
| bac bill 2 |
When the payments are processed
Then these sepa bills are completed
| sepa bill 1 |
| sepa bill 2 |
And I these bac bills are completed
| bac bill 1 |
| bac bill 2 |
It's simpler, easier to read what is supposed to be done, and can be expanded to more. The works should be done in the step definitions of setting up and verifying.