Graphite storage-aggregation.conf regex scope - metrics

I have had a good read through the doc on this but am still none the wiser.
http://graphite.readthedocs.io/en/latest/config-carbon.html
If we have a metric like so:
/var/lib/graphite/whisper/p1/p2/account/count_num_events.wsp
Does anyone know for sure exactly what part of this path graphite is applying the storage-aggregation regex to?
I assume that it will be just
"count_num_events"
and as such I could use a regex "^count.*" to match it. Or will it be applied to all or part of the rest of the path?
Cheers.

You are correct. that will be the thing you need. BUT note the metric will be using dots not slashes
p1.p2.account.count_num_events
So what you'll need in storage-aggregation is any of the following
*count_num_events
p1.p2.*.count_num_events
p1.*.account.count_num_events
*.account.count_num_events
*.count_num_events
*count_num_events$
p1.p2.account.count_num_events

I have finally got round to doing some testing on this.
Thanks for the answer Fred S I wish I had seen the response before doing the testing, would have helped.
So the answer is that graphite matches on the full metric name which is . separated. Which for the example metric file:
/var/lib/graphite/whisper/p1/p2/account/count_num_events.wsp
Would be:
p1.p2.account.count_num_events
So the most strict regex you could do would be:
^p1\.p2\.account\.count_num_events$

Related

Suffix and Prefix Asterisk in combination with Wildcard in Datadog Metrics

It's quite simple: I have
a datadog-dashbaord
a template-variable named env, which can have following values ['prod', 'test']
And I want to display metrics based on the env:
from-resource for test is unified-importer-test-sqsimportdlq11419573-xl6dn7o5wqtj
from-resource for prod is unified-importer-prod-sqsimportdlq11419573-prmohksrvxxg
So naturally I'd use following syntax:
unified-importer-$env.value-sqsimportdlq*
But this does not display anything, nor shows it any error.
This, however, works as expected: unified-importer-test-sqsimportdlq* (or unified-importer-prod-sqsimportdlq* respectively).
It looks like asterisk in combination with wildcards is not working.
Additionally, DD seems to dislike using two asterisks (as prefix and suffix):
How can I leverage the template-var env easily in this situation?
Well, it turned out that following solution works:
sum:aws.sqs.approximate_number_of_messages_visible{service:unified-importer AND env:$env.value AND queuename IN (unified-importer-test-sqsimportdlq86419573-al6dn7o5wqtj,unified-importer-prod-sqsimportdlq86419573-prmohksrvmxg)}
There's no way to use the template variable in the middle of a string, it can only go at the end. That would be a feature request to the Datadog team

Pythons Elasticsearch-DSL filter for exactly one match from list of values

I saw some realted posts but none of them match my exact issue.
Using Python 2.7 with Elasticsearch-dsl (6.3, that is also my Elasticsearch version).
I want to do something like,
s = Search(using=elastic_conn, index='my_index').filter("match", service_name=['exmp_name1', 'exmp_name2'])
This syntax doesn't work though.
I wish to get back all documents with service_name == 'exmp_name1' OR service_name == 'exmp_name2'
I prefer to use the filter context rather then query context as from my understanding it's faster and scoring really isn't important to me, just an absolute match (or mismatch).
How can I achieve this behavior?
Thanks
Ok. All I needed is to filter by terms rather then match.
The terms syntax supports several values.
Working code:
s = Search(using=elastic_conn, index='audit').filter("terms", service_name=['exmp_name1', 'exmp_name2'])

Find HTML Tags in Properties

My current issue is to find HTML-Tags inside of property values. I thought it would be easy to search with a query like /jcr:root/content/xgermany//*[jcr:contains(., '<strong>')] order by #jcr:score
It looks like there is a problem with the chars < and > because this query finds everything which has strong in it's property. It finds <strong>Some Text</strong> but also This is a strong man.
Also the Query Builder API didn't helped me.
Is there a possibility to solve it with a XPath or SQL Query or do I have to iterate through the whole content?
I don't fully understand why it finds This is a strong man as a result for '<strong>', but it sounds like the unexpected behavior comes from the "simple search-engine syntax" for the second argument to jcr:contains(). Apparently the < > are just being ignored as "meaningless" punctuation.
You could try quoting the search term:
/jcr:root/content/xgermany//*[jcr:contains(., '"<strong>"')]
though you may have to tweak that if your whole XPath expression is enclosed in double quotes.
Of course this will not be very robust even if it works, since you're trying to find HTML elements by searching for fixed strings, instead of actually parsing the HTML.
If you have an specific jcr:primaryType and the targeted properties you can do something like this
select * from nt:unstructured where text like '%<strong>%'
I tested it , but you need to know the properties you are intererested in.
This is jcr-sql syntax
Start using predicates like a champ this way all of this will make sense to you!
HTML Encode <strong>
HTML Decimal <strong>
Query builder is your friend:
Predicates: (like a CHAMP!)
path=/content/geometrixx
type=nt:unstructured
property=text
property.operation=like
property.value=%<strong>%
Have go here:
http://localhost:4502/libs/cq/search/content/querydebug.html?charset=UTF-8&query=path%3D%2Fcontent%2Fgeometrixx%0D%0Atype%3Dnt%3Aunstructured%0D%0Aproperty%3Dtext%0D%0Aproperty.operation%3Dlike%0D%0Aproperty.value%3D%25%3Cstrong%3E%25
Predicates: (like a CHAMP!)
path=/content/geometrixx
type=nt:unstructured
property=text
property.operation=like
property.value=%<strong>%
Have a go here:
http://localhost:4502/libs/cq/search/content/querydebug.html?charset=UTF-8&query=path%3D%2Fcontent%2Fgeometrixx%0D%0Atype%3Dnt%3Aunstructured%0D%0Aproperty%3Dtext%0D%0Aproperty.operation%3Dlike%0D%0Aproperty.value%3D%25%26lt%3Bstrong%26gt%3B%25
XPath:
/jcr:root/content/geometrixx//element(*, nt:unstructured)
[
jcr:like(#text, '%<strong>%')
]
SQL2 (already covered... NASTY YUK..)
SELECT * FROM [nt:unstructured] AS s WHERE ISDESCENDANTNODE([/content/geometrixx]) and text like '%<strong>%'
Although I'm sure it's entirely possible with a string of predicates, it's possibly heading down the wrong route. Ideally it would be better to parse the HTML when it is stored or published.
The required information would be stored on simple properties on the node in question. The query will then be a lot simpler with just a property = value query, than lots of overly complex query syntax.
It will probably be faster too.
So if you read in your HTML with something like HTMLClient and then parse it with a OSGI service, that can accurately save these properties for you. Every time the HTML is changed the process would update these properties as necessary. Just some thoughts if your SQL is getting too much.

How to use substring() with Import.io?

I'm having some issues with XPath and import.io and I hope you'll be able to help me. :)
The html code:
<a href="page.php?var=12345">
For the moment, I manage to extract the content of the href ( page.php?var=12345 ) with this:
./td[3]/a[1]/#href
Though, I would like to just collect: 12345
substring might be the solution but it does not seem to work on import.io as I use it...
substring(./td[3]/a[1]/#href,13)
Any ideas of what the problem is?
Thank's a lot in advance!
Try using this for the xpath: (Have the field selected as Text)
.//*[#class='oeil']/a/#href
Then use this for your regex:
([^=]*)$
This will get you the ISBN number you are looking for.
import.io only support functions in XPath when they return a node list
Your path expression is fine, but perhaps it should be
substring(./td[3]/a[1]/#href,14)
"Does not seem to work" is not a very clear description of what is wrong. Do you get error messages? Is the output wrong? Do you have any code surrounding the path expression you could show?
You can use substring, but using substring-after() would be even better.
substring-after(/a/#href,'=')
assuming as input the tiny snippet you have shown:
<a href="page.php?var=12345"/>
will select
12345
and taking into account the structure of your input
substring-after(./td[3]/a[1]/#href,'=')
A leading . in a path expression selects only immediate child td nodes of the current context node. I trust you know what you are doing.

How do I define a SemgrexPattern in Stanford API, to match nodes' text without using {lemma:...}?

I am working with the edu.stanford.nlp.semgrex and edu.stanford.nlp.tress.semgraph packages and am looking for a way to match nodes with a text value other than the lemma: directive.
I couldn't find all possible attribute names in javadoc for SemgrexPattern, only those for lemma, tag, and relational operators - is there a comprehensive list available?
For example, in the following sentence
My take-home pay is $20.
extracting the 'take-home' node is not possible using
(SemgrexPattern.compile( "{lemma:take-home}"))
.matcher( "My take-home pay is $20.").find()
yields false, because take-home is deemed not to be a lemma.
What do I need to do to match nodes with non-lemma, arbitrary text?
Thanks for any advice or comment.
Sorry - I realize that {word:take-home} would work in the example above.
Thanks..

Resources