I am trying to filter logs based on resource mapping using "protoPayload.resource" property. It works fine if I specify the exact resource name.
metadata.serviceName="appengine.googleapis.com"
metadata.labels."appengine.googleapis.com/module_id"={module}
metadata.labels."appengine.googleapis.com/version_id"={version}
log="appengine.googleapis.com/request_log"
protoPayload.resource = ***********
How to supply a regex for this property to filter out logs?
Maven dependency for logging api:
<dependency>
<groupId>com.google.apis</groupId>
<artifactId>google-api-services-logging</artifactId>
<version>v2beta1-rev9-1.21.0</version>
</dependency>
The Stackdriver Logging API does not currently support regular expressions.
It does, however, support a "has" operator through using a : instead of an = in your filter expression, e.g. path.to.field: "value". This matches substrings of the actual value in a case-insensitive fashion, which is most of what people typically used it for anyway.
See also: Write effective advanced filters
If the "has" operator doesn't accomplish your goal, consider filing feedback through the speech bubble button in the top right of the Cloud Console providing details of your use case and what you're ultimately trying to accomplish, and we'll incorporate that feedback as we plan the future direction of the product.
The advanced queries documentation page lists the regex operator =~
Further, there is a dedicated documentation page specifically for the regular expression use in the new log viewer. As of 2020-09-13, this page is marked as "new"
Related
I am debugging an API request in Firefox and I am looking to filter multiple domains in the developer tools' Network panel. I can filter by one domain with the domain:domainname.com, but how do I add additional domains?
Filtering by two domains is not directly possible because the filters in the Network panel is always accumulative, though one workaround is to use a regular expression filter.
This allows you to provide several domains separating them by pipes like this:
regexp:domain1.com|domain2.com
That should work in most cases, but note that this is not just filtering by domain but searching within all the data inside the requests. That means that when the domain name appears in one of the other columns, the request will also be listed.
Another way to achieve this is to use negative filtering by prepending the filter expression with a minus.
So in order to get the requests of two domains you have to write several -domain: expressions for all domains you want to exclude.
There's also the regexp keyword to use Regular Expressions for URL filtering.
https://developer.mozilla.org/en-US/docs/Tools/Network_Monitor/request_list#Filtering_by_properties
If you want to filter out domains that are cluttering up your requests pane, you simply use the - prefix and separate them with spaces.
-domain:domain.com -domain:domain2.com
The important thing here is that in Firefox you cannot use wildcards like you can in Chrome. So this won't work.
-domain:email.*.com
This is what kept tripping me up. Also, for awhile I thought you separated them with commas. Nope, use spaces.
Is it possible to enable wild card queries by default using query_string?
I'm having to manually append * to each of the terms. I had a look at the documentation but couldn't find anything.
No there is no way to enable it. You can enable/disable using wildcards "allow_leading_wildcard" the way how it works, that ES try to match tokens. So if you search for car it will match car until you search car* then it will match cars (sure it depends on analysis but further there is link for you to read).
I dont know case what you want to do but you should look to dealing with language. It should help also note that using leading wildcard could have performance issues that is why sometimes is better to disable it.
Can anyone provide some insight on the required syntax to use to search LOINC using FHIR for a specific string in the labs descriptive text portion of an Observation resource?
Is this even possible?
The documentation is all over the place and I can't find an example for this generic kind of search.
I found similar examples here: https://www.hl7.org/fhir/2015Sep/valueset-operations.html
Such as: GET "[base]/ValueSet/23/$validate-code?system=http://loinc.org&code=1963-8&display=test"
But none of them are providing a general enough case to do a global search of the LOINC system for a specific string in an Observation resource.
None of my attempts to use the FHIR UI here, http://polaris.i3l.gatech.edu:8080/gt-fhir-webapp/search?serverId=gatechreadonly&resource=Observation , have been successful. I keep getting a 500 Internal Server Error because I don't know the correct syntax to use for the value part of the search, and I can't find any documentation out of all the copious documents online that explains this very simple concept.
Can anyone provide some insight?
Totally frustrated at this point.
Observation?code=12345-6
or
Observation?code=http://loinc.org|12345-6
where 12345-6 is whatever LOINC code you want to look for (e.g. 39802-4)
The second ensures you'll only match on LOINC codes as opposed to codes from other systems, though given the relatively unique format of LOINC codes, you're mostly safe without including that.
If you want to search for a set of codes, then you can separate the codes or the tuples with commas: E.g.
Observation?code=12345-6,12345-7
or
Observation?code=http://loinc.org|12345-6,http://loinc.org|123456
If you expect to search by a really long list of codes frequently, you can define a value set that includes all the desired codes and then filter by value set:
Observation?code:in=http://somwhere.org/whatever/ValueSet/123
Note: for readability, I haven't escaped the URL contents, but you'll need to escape the URL values appropriately.
I am writing a complex application (a compiler analysis). To debug it I need to examine the application's execution trace to determine how its values and data structures evolve during its execution. It is quite common for me to generate megabytes of text output for a single run and sifting my way through all that is very labor-intensive. To help me manage these logs I've written my own library that formats them in HTML and makes it easy to color text from different code regions and indent code in called functions. An example of the output is here.
My question is: is there any better solution than my own home-spun library? I need some way to emit debug logs that may include arbitrary text and images and visually structure them and if possible, index them so that I can easily find the region of the output I'm most interested. Is there anything like this out there?
Regardless you didn't mentioned a language applied, I'd like to propose apache Log4XXX family: http://logging.apache.org/
It offers customizable details level as well as tag-driven loggers. GUI tool (chainsaw) can be combined with "old good" GREP approach (so you see only what you're interested in at the moment).
Colorizing, search and filtering using an expression syntax is available in the latest developer snapshot of Chainsaw. The expression syntax also supports regular expressions (using the 'like' keyword).
Chainsaw can parse any regular text log file, not just log files generated by log4j.
The latest developer snapshot of Chainsaw is available here:
http://people.apache.org/~sdeboy
The File, load Chainsaw configuration menu item is where you define the 'format' and location of the log file you want to process, and the expression syntax can be found in the tutorial, available from the help menu.
Feel free to email the log4j users list if you have additional questions.
I created a framework that might help you, https://github.com/pablito900/VisualLogs
I am sorry if it is not right to post a question on two forums.
We use Tivoli to monitor our logs files. The log4j log level is set to ERROR and Tivoli would raise tickets for these statements. But there are some known issues for which we would not want Tivoli to raise tickets. Is there a way to specify that some statements need to be ignored ?
Current regex : [/var/tmp/abc.log;ERROR(.*);error found: RegExp1]
This is very generic. We need to exclude certain framework errors (Hibernate / Mule) for a known issue. Is there a way to specify using a regex ?
Thanks,
Midhun
If you are using the LO Agent you can configure situation formula based on regular expression to fit your needs.
Below the LO Agent User Guide
http://pic.dhe.ibm.com/infocenter/tivihelp/v15r1/topic/com.ibm.itm.doc_6.2.3fp1/logfileagent623fp2_user.pdf
Take a look at the "Log File RegEx Statistics attribute group" section:
The Log File RegEx Statistics attribute group contains information that shows the statistics of the log file
regular expression search expressions. Regular expressions can be used to filter records or to define
records. This attribute group shows information about both types. When the Result Type attribute value
is INCLUDE or EXCLUDE, the filter is used to filter records;
Hope this helps
I don't have the reputation yet to post a comment but I'd have liked to ask if you are using the Tivoli Log File Agent (lo) of the Unix Log Agent (ul) before answering.
If your question is still actual...
Here is documentationof LogAgent - http://www-01.ibm.com/support/knowledgecenter/SS4EKN_7.2.0.2/com.ibm.itm.doc_6.3/logfile/klo_fileformat_specs.htm
You can specify new Regex as DISCARDED and all records mathced this regex will be not catched by ITM Events.
If you use the special predefined event class DISCARD as your event class, any log records matching the associated pattern are discarded, and no events are generated for them. For example:
REGEX DISCARD
As a pattern matched, nothing is written to the unmatch log. The log file status records matched include these discarded events.
BTW
[/var/tmp/abc.log;ERROR(.*);error found: RegExp1]
may be better as
[/var/tmp/abc.log;ERROR([^;]*);error found: RegExp1]
.* is greedy and best avoided when possible