I have two searches that will return orderNumbers
search1
index=main "Failed insert" | table orderNumber
//returns small list
search2
index=main "Successful insert" | table orderNumber
//returns huge list
I want a list of "Failed insert" orderNumbers that have NOT had a "Successful insert" previously. How can I use the results of the second search to filter the results of the first search?
I tried:
index=main "Failed insert"
NOT [
index=main "Successful insert" | table orderNumber]
| dedup orderNumber
| table orderNumber
...but it returned orderNumbers that did have "Successful insert" events.
I believe this is what you're looking for:
index=main sourcetype=srctp orderNumber=* "failed insert" NOT
[search index=main sourcetype=srctp orderNumber=* "successful insert"
| stats count by orderNumber
| fields - count ]
| stats count by orderNumber
| fields - count
First, stats is going to be a lot more performant than dedup
Second, so long as your "successful insert" search is 10k items or fewer, it should complete
If it's longer than 10k items, you may need to do something like this:
index=main sourcetype=srctp orderNumber=* ("failed insert" OR "successful insert")
| rex field=_raw "(?<insert>\w+ )insert"
| stats values(insert) as inserts by orderNumber
| search inserts="*failed*"
| where mvcount(inserts)<2
What this should do is extract the type of insert ("failed" or "successful") into a new field named insert
Then stats values() all of the insert types each orderNumber had
Then ensure we're only looking at orderNumber entries that have a "failed" insert
Then check to make sure there is only one entry in the values()'d field (ie there is no "successful" entry present
Related
I have created an alert policy in GCP MOnitoring which will notify me when a certain kind of log message stops appearing (a dead man's switch). I have create a logs-based metric with a label, "client", which I use to group the metric and get a timeseries per client. I have been using "absence of data" as the trigger for the alert. This has all been working well, until...
After a recent change, the logs now also com from different resources, so there is a need to combine the metric across those resources. I can achieve this using QML
{ fetch gce_instance::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
| every 30m
; fetch global::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
| every 30m }
| union
Notice that I need to align the two series with the same bucket size (30m) to be able to join them, which makes sense. I notice that the value for a timeseries is "undefined" in those buckets where the metric data was absent (by downloading a CSV of the query).
To create an alert using this query, I tried something like this:
{ fetch gce_instance::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
| every 30m
; fetch global::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
| every 30m }
| union
| absent_for 1h
If I look at the CSV output for this query it doesn't reflect the absence of metric data for a timeseries, and this is presumably because a value of "undefined" doesn't qualify as absent data.
Is there a way to detect for absence of data for a "unioned" metric (and therefore aligned) across multiple resources?
Update 1
I have tried this, which seems to get me some of the way there. I'd really appreciate comments on this approach.
{
fetch gce_instance::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
;
fetch global::logging.googleapis.com/user/ping
| group_by [metric.client], sum(val())
}
| union
| absent_for 1h
I have settled on a solution as follows,
{
fetch gce_instance::logging.googleapis.com/user/ping
| group_by [metric.client]
;
fetch global::logging.googleapis.com/user/ping
| group_by [metric.client]
}
| union
| absent_for 1h
| every 30m
Note:
group_by [metric.client] conforms the tables from different resource, which allows the union to work
absent_for does align input timeseries using the default period or one specified by a following every
I found it really hard to debug these MQL queries, in particular to confirm that absent_for was going to trigger an alert. I realised that I could use value [active] to show a plot of the active column (which absent_for produces) and that gave me confidence that my alert was actually going to work.
{
fetch gce_instance::logging.googleapis.com/user/ping
| group_by [metric.client]
;
fetch global::logging.googleapis.com/user/ping
| group_by [metric.client]
}
| union
| absent_for 1h
| value [active]
I want to filter the same record with capital letters, like:
my.user#mydomain.com
My.USER#mydomain.com
I tried to filter with this
| where AccountName in~ ('my.user')
I want to filter to have only one record showing up.
Now I have two records for the same user, but only one is needed.
You can summarize by user and take just one value by using max(AccountName)
| summarize AccountName=max(AccountName) by user
Full example:
let T1=datatable(user:string, AccountName:string) [
"Paul", "my.user#mydomain.com",
"Paul", "My.USER#mydomain.com",
"Eric", "email1#mail.com",
"Petra", "email2#mail.com",
];
T1
| summarize AccountName=max(AccountName) by user
There are two related tables A and B. The relation is one to many - A may point to the multiple entities in B.
Example:
Table A
id | title
--------
1 | 'title 1'
2 | 'title 2'
Table B
id | id_from_a
--------
1 | 1
2 | 1
I want to find all entities in table A that do not have anything in table B using graphQL.
For time being I am using the following contraption:
query getData {
table_a(where: {_not: {table_b: {id_from_a: {_neq: -9999}}}}) {
id
title
}
}
It works, but it is ugly as hell, so I guess there should be some better graphql solution.
I noticed that LIMIT queries will return more than the expected number of rows when they are executed against tables that contain nested or repeated data. For example, the following query run against the persons sample data set from the developer guide produces the following results:
% bq query 'SELECT fullName, children.name FROM [persons.person] LIMIT 1'
+----------+---------------+
| fullName | children_name |
+----------+---------------+
| John Doe | Jane |
| John Doe | John |
+----------+---------------+
It looks like BQL is applying the LIMIT operator before flattening the results as opposed to the other way around (which I think would make more sense).
Is this a bug in the BQL implementation or is this the expected behavior? If this is the expected behavior can someone please provide an explanation for why this makes sense?
This is expected given the way BigQuery flattens query results. When you run the query, the LIMIT 1 applies to the repeated record. Then the results get flattened in the output, and you get two rows. A workaround is to use an explicit flatten operation. For example:
SELECT fullName, children.name
FROM (FLATTEN([persons.person], children.name) LIMIT 1
This will return only a single row.
I have been trying to come up with a birt report to print food tag to no avail. What i want to show on the report is:
foodtag1 | foodtag2 | foodtag3
foodtag4 | foodtag5 | foodtag6
foodtag7 | foodtag8 | foodtag9
Can this be done?
the data is taken from a MySql Query "select dishes.name
from dishes
where find_in_set (dishes.id,(select orders.dishes from orders where orders.id = ))"
** Note: FoodTags 1-9 are all unique name of dishes
** Also note that foodtag 1-9 are representatives of a dish name. FoodTag1 can be "Yang Zhou Fried Rice", it can be "italian Pasta". it can be "Mee Goreng". Data is taken out from a datasource in MYSQL server
The easiest way--
Drag a grid element to your report, set it at 3 columns and 3 rows
In property editor, bind the grid to the data set
Drag a dynamic text element to the first cell in the grid
Then use JavaScript simular to this to filter to the desired text.
if (row["FoodTagColumn"]=='foodtag1'){
row["FoodTagColumn"]
}else null