In Google's Cloud Logging query language, is it possible to query for the existence of a particular key in the jsonPayload dict?
E.g., suppose I know the jsonPayload will either be
{'keyA':'<some string>'}
or
{'keyB':'<some string'}
But I don't know what the <some string> will be. I want all logs that have the keyB key. I suppose I could test that for that key having a regex that includes everything, but is that the best/only way?
I want all logs that have the keyB key. I suppose I could test that for that key having a regex that includes everything, but is that the best/only way?
I could think of using Regex for this use case:
Example:
jsonPayload.message =~"Job status: *"
In your case keyB
jsonPayload.keyB =~"regex-query"
For my example, when the expression matches, getting output as below:
Use the "has" operator with a *, like
jsonPayload.keyB:*
See the end of https://cloud.google.com/logging/docs/view/logging-query-language#comparisons
Related
Is there a way to get list of objects which starts and ends with specific criteria
as example:
a/b/c/id1.json
a/b/c/id2.json
a/b/c/id3.json
a/c/id1.json
and we wanna query for
Prefix: "a/",
EndOffset: "id1.json"
expected output should be:
a/b/c/id1.json
and we wanna filter out other options and we don't know what the b folder name would be.
So:
a is always a
b is random uniq string
c is always c
and we always want the specific json.
As i am tying to achieve this with:
query := &storage.Query{
Prefix: "a/",
//StartOffset: "",
EndOffset: "id1.json",
//Delimiter:
}
query.SetAttrSelection([]string{"Name"})
or
Prefix: "c/id1.json",
Delimiter: "/",
IncludeTrailingDelimiter: true,
and for some reason i am getting in return all of those files.
And of course i would like to limit the results as much as possible for better performances.
Maybe there is a way to use some regex in Prefix definition ?
like a/*/c/id1.json
Thanks
----------------------- ========= Edited ========= -----------------------
Please note that this is already implemented by me storage_list_files_with_prefix-go and do not work as i would like to have it. So the main question is HOWTO make this filtering working with the example I am showing.
Key points:
Cloud Storage Buckets do not have directories.
The namespace is flat.
Object names are just strings.
The slash / character which is often used to separate directory names in file systems is just a character in an Bucket object name. The slash has no significance but can be used as a delimiter.
You can specify a prefix and a delimiter to reduce the returned object list.
Cloud Storage does not support regex expressions.
The asterisk * is a character and not a wildcard.
Summary:
You must implement additional filtering in your code.
List the objects in a bucket using a prefix filter
I'm trying to use regular expression extractor concept in Jmeter. By using regEx concept I'm able to get the required token id's. And for all I'm using regEx as (.*?). So this is working fine when we have constant prefix and suffix text/values.
But in this case, there is no suffix,
Ex: Key is = #bluerelay.com/a43a816dcdd14873bd5757b3a3709d5c,
ClickHereForImageForm
I want to take the key ID into a variable with using RegEx. I have tried to get it with (.*?) but it didn't work, it returns the full value, not the required part. It'd be excellent if you could give any suggestion.
The source value is:
https://navitus-internal-app.bluerelay.com/#/token/systemadministrator#bluerelay.com/a43a816dcdd14873bd5757b3a3709d5c
The expected result is to extract a43a816dcdd14873bd5757b3a3709d5c from the above URL and put it into a variable.
You can use regex to get last text after / sign
(.*)\/(\w+)
See demo
I am indexing documents that may contain any special/reserved characters in their fulltext body. For example
"PDF/A is an ISO-standardized version of the Portable Document Format..."
I would like to be able to search for pdf/a without having to escape the forward slash.
How should i analyze my query-string and what type of query should i use?
The default standard analyzer will tokenize a string like that so that "PDF" and "A" are separate tokens. The "A" token might get cut out by the stop token filter (See Standard Analyzer). So without any custom analyzers, you will typically get any documents with just "PDF".
You can try creating your own analyzer modeled off the standard analyzer that includes a Mapping Char Filter. The idea would that "PDF/A" might get transformed into something like "pdf_a" at index and query time. A simple match query will work just fine. But this is a very simplistic approach and you might want to consider how '/' characters are used in your content and use slightly more complex regex filters which are also not perfect solutions.
Sorry, I completely missed your point about having to escape the character. Can you elaborate on your use case if this turns out to not be helpful at all?
To support queries containing reserved characters i now use the Simple Query String Query (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html)
As of not using a query parser it is a bit limited (e.g. no field-queries like id:5), but it solves the purpose.
"<LDAP://DC=top,DC=example,DC=com>;(&(objectCategory=person)(objectClass=user)(sn=%lastname%));sAMAccountName,sn,givenName,distinguishedname,userAccountControl,cn"
Above is a snippet of my query that I will send to AD. When I type an EXACT last name, it works without issues. However when I enter a partial last name, I get no results.
Can anyone please tell me how to perform a search on a partial name?
% is not a valid wildcard in LDAP filters, and you also can't use environment variables (%variable%) in VBScript LDAP queries. A filter with a partial value can be defined like this:
(&(objectCategory=person)(objectClass=user)(sn=*something*))
If you want to use a variable in a filter you have to use string concatenation:
"(&(objectCategory=person)(objectClass=user)(sn=" & variable & "))"
The current question is not the same as this one.
I have an LDAP entry which the content "This is a SimpleTest indeed" in the "supName" field.
I need to write a filter so that when the user of my software introduces any substring of this content in any case (upper, lower or mixed case), it finds the entry. It must work even if the user does not input a complete word ("impletes", for example).
The supName field follows DirectoryString syntax. This means that the default matching rule is exact and case sensitive ("caseExactMatch"). But this syntax in theory, should allow also "caseIgnoreMatch" and "caseIgnoreSubstringsMatch" matching rules. I though I just needed to force to use the last one ("caseIgnoreSubstringsMatch"), so I tried this filter:
(supName:caseIgnoreSubstringsMatch:=*impletes*)
But this does not work. I make my tests using Apache Directory Studio, and that tool refuses to accept the above filter. It complains on the asterisks, and I don't understand why, since I am using a Substring match (and thus asterisks should be allowed). If I run the filter from command line (using ldapsearch), I get this error message:
ldap_search_ext: Bad search filter (-7)
Therefore this is not an issue with Apache Directory Studio.
So my question is: What is the correct way of defining a case-insensitive substring filter on a field that is case-sensitive by default?
Futher tests:
What follows are some other filters I have tested, and the reasons they do not suit me.
Test #1 filters:
(supName=*impleTes*)
This operator (=) returns my test entry, but it's not case-insensitive. If I replace "impleTes" for "impletes" it does not return anything.
Test #2 filter:
(supName~=simpletest)
This operator (~=) works, but:
It needs a complete word. If I replace "simpletest" for "impletes" it does not return anything.
As it is an "approximate" search operator, it may return unwanted results. For example, the filter above returns also a second entry: "This is a SimpleTast indeed" (notice the "a" instead of "e" in "SimpleTast"). I don't want approximate results.
Test #3 filter:
(supName:caseIgnoreMatch:=this is a simpletest indeed)
This returns the entry I was expecting, and only that entry. It is also case-insensitive. But it forces the user to write the whole content of the field: it is not a substring search, but a case-insensitive exact-match search.
Test #4 filter:
(supName:caseIgnoreMatch:=*impletes*)
This returns a "Bad search filter (-7)" error, with is expected since I am not allowed to use substring syntax in an exact matching rule.
And finally, the Test #5 filter:
(supName:caseIgnoreSubstringsMatch:=*impletes*)
Which I expected to work, but returns a "Bad search filter (-7)" error.
Additional info - Opposite example
I have found here (see the "Extensible Matching" section at the end) examples of the opposite case. In the example, the field "sn" uses the "caseIgnoreMatch" matching rule by default (making it case-insensitive). So what they want in the example is to do a case-sensitive substring search. This is the filter they use:
(sn:caseExactSubstringMatch:=*S*)
But I doubt if this example is correct, because if I try exactly the same filter on my side:
(supName:caseExactSubstringMatch:=*S*)
I get a "Bad search filter (-7)" error.
So maybe my issue is due to limitation on OpenLDAP 2.4.33 but would work with other LDAP servers, although the example comes from a guide that is supposed to cover OpenLDAP 2.x ... (?)
If I'm reading RFC 4515ยง3 correctly, an extensible match can only be done with an assertion value (read: a fixed string) and not with a substring filter. If that were permitted, I would expect your original example (supName:caseIgnoreSubstringsMatch:=*impletes*) to work.
Other option is
(Attribute~=value)
-- http://publib.boulder.ibm.com/tividd/td/IBMDS/IDSprogref52/en_US/HTML/progref.htm#HDRLDAPSRCH