LogQL - label_format conditionally format a label - grafana-loki

I have a log stream from where I am extracting a set of fields to be set as either labels or metric values. The stream is not in a standard format so I am extracting the fields with regexp pipeline command, as below.
(...)
| regexp "(?P<api>\\w+)\\sAPI"
| regexp "\\[performed\\.(?P<action>\\w+)"
| regexp "duration\\s\\[(?P<duration_ms>\\d+)"
| regexp "response \\[(?P<response>.*?)\\]"
The problem is that the api captured field, on some interaction, is not being populated, and I wanted to update those cases so that a default value was set - For presentation purposes.
I've tried using the native LogLQ's contains and hasPrefix template commands as the documentation suggests they can be used with if else blocks. The documentation is not clear on how to build those blocks inside the label_format or the line_format pipeline commands. But depending on the approach it either returns a format error or does not do anything.
An working example would be appreciated.
Thank you.
Note: Tried to tag this as a LogQL topic but not enough reputation to do so.

I believe I have gotten something working with line_format:
{environment=~"$environment",level=~"$level"} | json | line_format "{{ if hasSuffix `Exception` .thrown_name }} Exception occurred! {{end}} {{.message}}" |~ "(?i)$grep"
The documentation for back ticks was difficult to find, and the order for method arguments is also a bit different. Hopefully this helps?

With #AnthonyA's reply I was able to, after extracting a field, modify its value using the label_format template function.
(..)
| regexp "response \\[(?P<response>.*?)\\]"
(..)
| label_format api=`{{ if hasPrefix "Error" .response }}ERROR{{else}}{{.response}}{{end}}`
(..)
This way the field's value is replaced by ERROR if it starts with "Error" and keeping its original value otherwise.

Related

My flow fails for no reason: Invalid Template Language? This is what I do

Team,
Occasionally my flow fails and its enough test it manually to running again. However, I want to avoid that this error ocurrs again to stay in calm.
The error that appears is this:
Unable to process template language expressions in action 'Periodo' inputs at line '0' and column '0': 'The template language function 'split' expects its first parameter to be of type string. The provided value is of type 'Null'. Please see https://aka.ms/logicexpressions#split for usage details.'.
And it appears in 2 of the 4 variables that I create:
Client and Periodo
The variable Clientlooks this:
The same scenario to "Periodo".
The variables are build in the same way:
His formula:
trim(first(split(first(skip(split(outputs('Compos'),'client = '),1)),'indicator')))
His formula:
trim(first(split(first(skip(split(outputs('Compos'),'period = '),1)),'DATA_REPORT_DELIVERY')))
The same scenario to the 4 variables. 4 of them strings (numbers).
Also I attached email example where I extract the info:
CO NIV ICE REFRESCOS DE SOYA has finished successfully.CO NIV ICE REFRESCOS DE SOYA
User
binary.struggle#mail.com
Parameters
output = 7
country = 170
period = 202204012
DATA_REPORT_DELIVERY = NO
read_persistance = YES
write_persistance = YES
client = 18277
indicator_group = SALES
Could you give some help? I reach some attepmpts succeded but it fails for no apparent reason:
Thank you.
I'm not sure if you're interested but I'd do it a slightly different way. It's a little more verbose but it will work and it makes your expressions a lot simpler.
I've just taken two of your desired outputs and provided a solution for those, one being client and the other being country. You can apply the other two as need be given it's the same pattern.
If I take client for example, this is the concept.
Initialize Data
This is your string that you provided in your question.
Initialize Split Lines
This will split up your string for each new line. The expression for this step is ...
split(variables('Data'), '\n')
However, you can't just enter that expression into the editor, you need to do it and then edit in in code view and change it from \\n to \n.
Filter For 'client'
This will filter the array created from the split line step and find the item that contains the word client.
`contains(item(), 'client')`
On the other parallel branches, you'd change out the word to whatever you're searching for, e.g. country.
This should give us a single item array with a string.
Initialize 'client'
Finally, we want to extract the value on the right hand side of the equals sign. The expression for this is ...
trim(split(body('Filter_For_''client''')[0], '=')[1])
Again, just change out the body name for the other action in each case.
I need to put body('Filter_For_''client''')[0] and specify the first item in an array because the filter step returns an array. We're going to assume the length is always 1.
Result
You can see from all of that, you have the value as need be. Like I said, it's a little more verbose but (I think) easier to follow and troubleshoot if something goes wrong.

Combine two string variables into one in Jmeter

I have a JSON API request that contains these lines (and many others)
"vatNumber": "${variableC},
I already have a variable called variableA, which generates a random number string. I also have a variable called variableB which is always going to be "01" as a string, i.e. variableC = variableA + variableB.
I've tried this in a preprocessor, but it doesn't work.
String var1 = vars.get("customerOrgNr");
String var2 = vars.get("vatSuffix");
vars.put("customerVatNr", var1+var2);
Also tried
"vatNumber": ${variableA}${variableB},
but that doesn't work either. The json request is not accepted.
Your "doesn't work" statement "doesn't tell" anything to us.
If you just need to concatenate 2 values - go for ${customerOrgNr}${vatSuffix}, it will generate combined string
If you have variable like foo01 with the value of bar and want to use this value - go for __V() function like ${__V(${customerOrgNr}${vatSuffix},)}
Demo:
More information: Here’s What to Do to Combine Multiple JMeter Variables
I also think you need to surround your value with quotation marks:
"vatNumber": "${customerVatNr}",
Check the request in View Results Tree listener and check the generated JSON again i.e. online JSON Validator

ModX getResources &tvFilters not working

I have output from getResourses that I need to sort out by template variable value, but filter is showing resources that have only one value.
For example, this code is showing resources that have only Manhattan TV value.
Template variable input type is Tag, output type is default. I have tried to change the input type to Auto-Tag.
[[!getPage#Series?
&element=`getResources`
&parents=`[[*id]]`
&tpl=`New-series-catalog`
&limit=`0`
&debug=`1`
&includeTVs=`1`
&processTVs=`1`
&tvFilters=`door-color==Manhattan`
&tvPrefix=``
]]
Screenshot of TV input options
Output with intentionally broken tpl name
TV value of resource that doesn't output, but should because it has Manhattan tag
I have tried to use only English value names and changing search value to Russian value name but the result is same.
I found the answer.
To filter by TV that has multiple values you need to add % % to the value.
For example, my code should look like
[[!getPage#Series?
&element=`getResources`
&parents=`[[*id]]`
&tpl=`New-series-catalog`
&limit=`0`
&debug=`1`
&includeTVs=`1`
&processTVs=`1`
&tvFilters=`door-color==%Manhattan%`
&tvPrefix=``
]]
Instead of
[[!getPage#Series?
&element=`getResources`
&parents=`[[*id]]`
&tpl=`New-series-catalog`
&limit=`0`
&debug=`1`
&includeTVs=`1`
&processTVs=`1`
&tvFilters=`door-color==Manhattan`
&tvPrefix=``
]]
I hope that will help you so you don't waste time trying to figure out whats wrong.

Primitive type as data structure for API Blueprint

I want to use primitive type for describe data structure. Like so:
# Data Structures
## Video Delete (enum[number])
+ `0` - Successful deletion.
+ `1` - Error occured.
And the output is.
{
"enum": [
1,
0
],
"$schema": "http://json-schema.org/draft-04/schema#"
}
So description is missing. I've tried to put description in different places. I did a lot of things (do not wanna talk about them). Also I've tried to add info to enum values like so:
+ `0` (number) - Successful deletion.
I do not know whether this problem deals with MSON syntax or Aglio generator.
The syntax above is supported by MSON as far as I can tell. The problem is that Aglio doesn't do anything with the description, and when I went to look into adding it I realized that it isn't really supported in JSON Schema. There seem to be two methods people use to get around that fact:
Add the enumerated value descriptions to the main description, the Olio theme 1.6.2 has support for this but the C++ parser seems to still have some bugs around this feature:
## Video Delete (enum[number]) - 0 for success, 1 for error
Use a weird oneOf syntax where you create sets of single enums with a description. I don't recommend this.
Unfortunately the first option requires work on your part and can't easily be done in Aglio. Does anyone else have a better description and some samples of MSON input -> JSON Schema output?

JMeter does not replace ${SUBPANEL_RELATE_MODULE_g1} with its value

It's a long story, but I will try make it simple:
I generated MeterMaid XML files with SugarMMM; I chose only the Accounts module:
I converted above files to JMeter format with MeterMaid (I consolidated the tests into one file and named it "filename.xml"):
ruby GenMeter.rb --inputfile=filename.xml --outputfile=filename.jmx
I did the necessary CSV setup. All the CSV file contains is the login details (usr,pwd) for testing concurrent user logins. This part works well indeed.
When I run the test, I can see that a whole bunch of ${} variables are converted into corresponding values. Sadly, ${SUBPANEL_RELATE_MODULE_g1} doesn't get resolved... here's what the GET url (from View Results Tree Listener component) looks like:
http://localhost/sugarcrm/index.php?module=${SUBPANEL_RELATE_MODULE_g1}&action=Popup&hide_clear_button=true&mode=MultiSelect&create=true&metadata=undefined
Note that it's not the only variable that isn't resolved. The following screenshot shows the other tests that fail, all also caused by other variables not replaced by their corresponding values:
Here's how this variable is set up (which is well before the time it's used):
Here's what Debug Sampler says:
JMeterVariables:
CAMPAIGN_ID=CAMPAIGN_ID_ERROR
CAMPAIGN_NAME=CAMPAIGN_NAME_ERROR
CONTACT_ID=997a3171-aa60-b2d6-a457-4e0ba8b0052b
CONTACT_ID_g=4
CONTACT_ID_g0=onclick="send_back('Contacts','997a3171-aa60-b2d6-a457-4e0ba8b0052b');">Prof
CONTACT_ID_g1=onclick="send_back('Contacts','
CONTACT_ID_g2=997a3171-aa60-b2d6-a457-4e0ba8b0052b
CONTACT_ID_g3=');">
CONTACT_ID_g4=Prof
CONTACT_NAME=Prof
CONTACT_NAME_g=4
CONTACT_NAME_g0=onclick="send_back('Contacts','997a3171-aa60-b2d6-a457-4e0ba8b0052b');">Prof
CONTACT_NAME_g1=onclick="send_back('Contacts','
CONTACT_NAME_g2=997a3171-aa60-b2d6-a457-4e0ba8b0052b
CONTACT_NAME_g3=');">
CONTACT_NAME_g4=Prof
FOUND_ID=1
JMeterThread.last_sample_ok=true
JMeterThread.pack=org.apache.jmeter.threads.SamplePackage#3c1635
MEMBER_OF_ID=d7c26344-cad8-0503-b02a-4e0cb4db3985
MEMBER_OF_ID_g=4
MEMBER_OF_ID_g0=onclick="send_back('Accounts','d7c26344-cad8-0503-b02a-4e0cb4db3985');">searchSearchForm
MEMBER_OF_ID_g1=onclick="send_back('Accounts','
MEMBER_OF_ID_g2=d7c26344-cad8-0503-b02a-4e0cb4db3985
MEMBER_OF_ID_g3=');">
MEMBER_OF_ID_g4=searchSearchForm
MEMBER_OF_NAME=searchSearchForm
MEMBER_OF_NAME_g=4
MEMBER_OF_NAME_g0=onclick="send_back('Accounts','d7c26344-cad8-0503-b02a-4e0cb4db3985');">searchSearchForm
MEMBER_OF_NAME_g1=onclick="send_back('Accounts','
MEMBER_OF_NAME_g2=d7c26344-cad8-0503-b02a-4e0cb4db3985
MEMBER_OF_NAME_g3=');">
MEMBER_OF_NAME_g4=searchSearchForm
OPPORTUNITY_ID=864e402f-0d76-ab6e-b54f-4e0cb42f0249
OPPORTUNITY_ID_g=4
OPPORTUNITY_ID_g0=onclick="send_back('Opportunities','864e402f-0d76-ab6e-b54f-4e0cb42f0249');">value
OPPORTUNITY_ID_g1=onclick="send_back('Opportunities','
OPPORTUNITY_ID_g2=864e402f-0d76-ab6e-b54f-4e0cb42f0249
OPPORTUNITY_ID_g3=');">
OPPORTUNITY_ID_g4=value
OPPORTUNITY_NAME=value
OPPORTUNITY_NAME_g=4
OPPORTUNITY_NAME_g0=onclick="send_back('Opportunities','864e402f-0d76-ab6e-b54f-4e0cb42f0249');">value
OPPORTUNITY_NAME_g1=onclick="send_back('Opportunities','
OPPORTUNITY_NAME_g2=864e402f-0d76-ab6e-b54f-4e0cb42f0249
OPPORTUNITY_NAME_g3=');">
OPPORTUNITY_NAME_g4=value
RANDOM_CHAR=o
RANDOM_CHAR_g=1
RANDOM_CHAR_g0=o
RANDOM_CHAR_g1=o
RANDOM_STRING=value
RANDOM_STRING_g=1
RANDOM_STRING_g0=value
RANDOM_STRING_g1=value
RECORD_NAME=NOT_FOUND
RECORD_NUMBER=3250317d-6c79-b20d-5e36-4e0cb4746e84
RECORD_NUMBER_g=2
RECORD_NUMBER_g0=javascript:lvg_nav('Accounts', '3250317d-6c79-b20d-5e36-4e0cb4746e84
RECORD_NUMBER_g1=javascript:lvg_nav('Accounts', '
RECORD_NUMBER_g2=3250317d-6c79-b20d-5e36-4e0cb4746e84
SEARCH_FIELD=SEARCH_FIELD_ERROR
START.HMS=190308
START.MS=1309453388621
START.YMD=20110630
SUBPANEL_RELATE_MODULE=Accounts
TEAM_ID=seed-Teams8
TEAM_ID_g=4
TEAM_ID_g0=onclick="send_team_to_form('Teams','seed-Teams8');">Ball
TEAM_ID_g1=onclick="send_team_to_form('Teams','
TEAM_ID_g2=seed-Teams8
TEAM_ID_g3=');">
TEAM_ID_g4=Ball
TEAM_NAME=Ball
TEAM_NAME_g=4
TEAM_NAME_g0=onclick="send_team_to_form('Teams','seed-Teams8');">Ball
TEAM_NAME_g1=onclick="send_team_to_form('Teams','
TEAM_NAME_g2=seed-Teams8
TEAM_NAME_g3=');">
TEAM_NAME_g4=Ball
TESTSTART.MS=1309455500088
pwd=user1
usr=user1
UPDATE:
Here's after changing Template to $1$$2$$3$$4$:
And here's the Debug output (the Sampler is put just after regex Controller):
JMeterVariables:
JMeterThread.last_sample_ok=true
JMeterThread.pack=org.apache.jmeter.threads.SamplePackage#18fde89
RANDOM_CHAR=t
RANDOM_CHAR_g=1
RANDOM_CHAR_g0=t
RANDOM_CHAR_g1=t
RANDOM_STRING=Tanzania
RANDOM_STRING_g=1
RANDOM_STRING_g0=Tanzania
RANDOM_STRING_g1=Tanzania
RECORD_NAME=NOT_FOUND
RECORD_NUMBER=DOCTYPE
RECORD_NUMBER_g=1
RECORD_NUMBER_g0=DOCTYPE
RECORD_NUMBER_g1=DOCTYPE
START.HMS=100932
START.MS=1312531772599
START.YMD=20110805
SUBPANEL_RELATE_MODULE=Accounts
TESTSTART.MS=1312542237235
pwd=user1
usr=user1
Verify in your CSV dataset config, all variables are declared correctly (no typos, no omissions, no spaces before variable names, etc.)
I would also suggest putting a debug sampler high in your tree, as it will show you every variable and its value and can save a lot of time.
Edit:
It looks like your regex is setup incorrectly for creating multiple groups. If you look in your Debug sampler, you have SUBPANEL_RELATE_MODULE=Accounts but not SUBPANEL_RELATE_MODULE_g1= . This implies you don't have GROUPS setup.
In looking at your regex, the line template: $0$ is saying "only give me one group", whereas it looks like you want 4. Thus, you should try template: $1$$2$$3$$4$ the first match should be _g1 the second _g2 and so on. Check out the manual for details.
make judicious use of the ${SUBPANEL_RELATE_MODULE_g1} variable throughout the script after it gets defined by the Regular Expression Extractor. Verify it exists. Use it in Controller titles, test headings, at the beginning of tests, and at the end of tests.
That will help narrow it down
This way you can follow it along in the script to make sure it exists just before the moment it is used, and find where it's breaking down. Basically print statement debugging.
I don't know about sugar CRM or metermaid but for jmeter I had a similar problem with variable that were not resolve.
Check if this variable is declared somewhere or maybe you need to write a reg ex to extract that value from the previous request.
The problem I got It was that I extracted the variable from a CSV files
the variable name was CONTRACTNO, CLIENTNO
my URL : /SomeURL/bla?eventId=contractSelected&contractNoSelected=${CONTRACTNO}&applicationID=BLa
And it wasn't working so I changed the variable name in the csv file to :
CONTRACTNO,CLIENTNO (look that I removed a space)
Jmeter tend to be very picky with space in name and variable definition.
You are expecting the following values
${SUBPANEL_RELATE_MODULE_g1}
but you have configured the Regular Expression Extractor reference name as
"SUBPANEL_RELATE_MODULE"
There two options you can get the values
You have to change the ${SUBPANEL_RELATE_MODULE_g1} into ${SUBPANEL_RELATE_MODULE}
You have to change the Regular Expression Extractor reference name as "SUBPANEL_RELATE_MODULE _g1" to "SUBPANEL_RELATE_MODULE"
After this your values will be replaced properly.
It looks like your regular expression isn't picking up a match, your default value is set to "Accounts" and in your debug output that's the value of the variable. You'll only get match groups if the regular expression matches.
The View Results Tree listener now has a RegExp tester, so you can go to the request that you're trying to extract the variable SUBPANEL_RELATE_MODULE from the result in the results tree and choose RegExp Tester from the dropdown where it says "Text".
You can then run your regular expression on the response data in the RegExp tester and probably find that it doesn't match and then hone your regular expression so that it matches and update it in your Regular Expression Extractor.

Resources