Grafana & Elastic - How to count sub array length - elasticsearch

So I have a document that has two nested arrays i.e.
foo.bars[].baz[]
I am trying to figure out how I can use graphana to group by bars and give me a count of bar's for each bar. So it would look something like:
| bars.id| count|
| 1 | 10 |
| 2 | 15 |
| 3 | 20 |
What I have tried is the following:
Group by bars.id
Add a Sum metric for bars.baz.id
Override the script value to return 1
While this does give me the count of the bars, it does so for all bars in the document and not grouped by the bars.id i.e.
| bars.id| count|
| 1 | 45 |
| 2 | 45 |
| 3 | 45 |
Any help to achieve this would be very helpful.
Now if this can be done I have another more complex problem. I have another collection let's call it bobs that is a child of the root document. Now bobs isn't nested under the bars array but it has a bar.id field. I would also like to sum this based on that i.e.
{
bobs: [
{bar_id: 1},
{bar_id: 2},
],
bars: [
{id: 1, bazes: []},
{id: 2, bazes: []}
]
}
In this case I would also like in the table:
| bars.id| bobs.count|
| 1 | 1 |
| 2 | 1 |
| 3 | 0 |
Is this possible?

Related

ArrayFormula - If cell contains match, combine other cells with TEXTJOIN

I have a Google Sheet that contains names of characters, together with corresponding values for the group name, "selected" and attack power. It looks like this:
Sheet1
| NAME | GROUP NAME | SELECTED | ATTACK POWER |
|:---------|:-----------|----------:|-------------:|
| guile | Team Red | 1 | 333 |
|----------|------------|-----------|--------------|
| blanka | Team Red | 1 | 50 |
|----------|------------|-----------|--------------|
| sagat | Team Red | | 500 |
|----------|------------|-----------|--------------|
| ruy | Team Blue | 1 | 450 |
|----------|------------|-----------|--------------|
| vega | Team Blue | 2 | 150 |
Sheet2
In my second sheet, I have two columns. Group name, which contains names of each team from Sheet1 and names, which contains my current ArrayFormula:
=ARRAYFORMULA(TEXTJOIN(CHAR(10); 1;
REPT('Sheet1'!A:A; 1*('Sheet1'!B:B=A2))))
Using this formula I can combine all characters into one cell (with textjoin, repeated with row breaks) based on the value in Group name. The result looks like the following:
| GROUP NAME | NAME |
|:-----------|:--------------------------|
| Team Red | guile |
| | blanka |
| | sagat |
|------------|---------------------------|
| Team Blue | ruy |
| | vega |
|------------|---------------------------|
The problem is that I only want to combine the characters with having a selected value of 1. End-result should instead look like this:
| GROUP NAME | NAME |
|:-----------|:--------------------------|
| Team Red | guile |
| | blanka |
|------------|---------------------------|
| Team Blue | ruy |
|------------|---------------------------|
I tried the following setup using a IF-statement, but it just returns a string of FALSE:
=ARRAYFORMULA(TEXTJOIN(CHAR(10); 1;
REPT(IF('Sheet1'!C:C="1";'Sheet1'!A:A); 1*('Sheet1'!B:B=A2))))
Can this be one?
paste in F2 cell:
=UNIQUE(FILTER(B:B, C:C=1))
paste in G2 cell and drag down:
=TEXTJOIN(CHAR(10), 1, FILTER(A:A, B:B=F2, C:C=1))
or G2 cell be like:
=ARRAYFORMULA(TEXTJOIN(CHAR(10), 1,
REPT(FILTER(Sheet1!A:A, Sheet1!C:C=1), 1*(FILTER(Sheet1!B:B, Sheet1!C:C=1)=F2))))

MDX - filter empty outside of selected range

Cube is populated with data divided into time dimension ( period ) which represents a month.
Following query:
select non empty {[Measures].[a], [Measures].[b], [Measures].[c]} on columns,
{[Period].[Period].ALLMEMEMBERS} on rows
from MyCube
returns:
+--------+----+---+--------+
| Period | a | b | c |
+--------+----+---+--------+
| 2 | 3 | 2 | (null) |
| 3 | 5 | 3 | 1 |
| 5 | 23 | 2 | 2 |
+--------+----+---+--------+
Removing non empty
select {[Measures].[a], [Measures].[b], [Measures].[c]} on columns,
{[Period].[Period].ALLMEMEMBERS} on rows
from MyCube
Renders:
+--------+--------+--------+--------+
| Period | a | b | c |
+--------+--------+--------+--------+
| 1 | (null) | (null) | (null) |
| 2 | 3 | 2 | (null) |
| 3 | 5 | 3 | 1 |
| 4 | (null) | (null) | (null) |
| 5 | 23 | 2 | 2 |
| 6 | (null) | (null) | (null) |
+--------+--------+--------+--------+
What i would like to get, is all records from period 2 to period 5, first occurance of values in measure "a" denotes start of range, last occurance - end of range.
This works - but i need this to be dynamically calculated during runtime by mdx:
select non empty {[Measures].[a], [Measures].[b], [Measures].[c]} on columns,
{[Period].[Period].&[2] :[Period].[Period].&[5]} on rows
from MyCube
desired output:
+--------+--------+--------+--------+
| Period | a | b | c |
+--------+--------+--------+--------+
| 2 | 3 | 2 | (null) |
| 3 | 5 | 3 | 1 |
| 4 | (null) | (null) | (null) |
| 5 | 23 | 2 | 2 |
+--------+--------+--------+--------+
I tried looking for first/last values but just couldn't compose them into the query properly. Anyone has this issue before ? This should be pretty common seeing as I want to get a continuous financial report without skipping months where nothing is going on. Thanks.
Maybe try playing with NonEmpty / Tail function in a WITH clause:
WITH
SET [First] AS
{HEAD(NONEMPTY([Period].[Period].MEMBERS, [Measures].[a]))}
SET [Last] AS
{TAIL(NONEMPTY([Period].[Period].MEMBERS, [Measures].[a]))}
SELECT
{
[Measures].[a]
, [Measures].[b]
, [Measures].[c]
} on columns,
[First].ITEM(0).ITEM(0)
:[Last].ITEM(0).ITEM(0) on rows
FROM MyCube;
to debug a custom set, to see what members it is returning you can do something like this:
WITH
SET [First] AS
{HEAD(NONEMPTY([Period].[Period].MEMBERS, [Measures].[a]))}
SELECT
{
[Measures].[a]
, [Measures].[b]
, [Measures].[c]
} on columns,
[First] on rows
FROM MyCube;
I think reading your comment about Children means that this is also an alternative - to add an extra [Period]:
WITH
SET [First] AS
{HEAD(NONEMPTY([Period].[Period].[Period].MEMBERS
, [Measures].[a]))}
SET [Last] AS
{TAIL(NONEMPTY([Period].[Period].[Period].MEMBERS
, [Measures].[a]))}
SELECT
{
[Measures].[a]
, [Measures].[b]
, [Measures].[c]
} on columns,
[First].ITEM(0).ITEM(0)
:[Last].ITEM(0).ITEM(0) on rows
FROM MyCube;

Elasticsearch index with jdbc driver

Sorry my english is bad
I am using elasticsearch and jdbc river. I have two table with many-to-many relations. For example:
product
+---+---------------+
| id| title |
+---+---------------+
| 1 | Product One |
| 2 | Product Two |
| 3 | Product Three |
| 4 | Product Four |
| 5 | Product Five |
+---+---------------+
product_category
+------------+-------------+
| product_id | category_id |
+------------+-------------+
| 1 | 1 |
| 1 | 2 |
| 1 | 3 |
| 2 | 4 |
| 2 | 5 |
+------------+-------------+
category
+---+---------------+
| id| name |
+---+---------------+
| 1 | Category One |
| 2 | Category Two |
| 3 | Category Three|
| 4 | Category Four |
| 5 | Category Five |
+---+---------------+
I want to use array type.
{
"id": 1,
"name": "Product one",
"categories": {"Category One", "Category Two", "Category Three"}
},
How should I write a sql?
Use elasticsearch-jdbc structured objects with sql, no need to group_concat:
SELECT
product.id AS _id,
product.id,
title,
name AS categories
FROM product
LEFT JOIN (
SELECT *
FROM product_category
LEFT JOIN category
ON product_category.category_id = category.id
) t
ON product.id = t.product_id
Since river has been deprecated since ES v1.5, maybe run a standalone importer is better.

Multiple Tables Group and substract sum of columns using linq sql

Here i have two tables
Table One
+---------------+----------+------------+
| Raw Material | Size | Qty |
+---------------+----------+------------+
| A | 1 | 5 |
| A | 2 | 2 |
| A | 1 | 2 |
| B | 0 | 5 |
| B | 0 | 1 |
+---------------+----------+------------+
Table Two
+---------------+----------+------------+
| Raw Material | Size | Qty |
+---------------+----------+------------+
| A | 1 | 2 |
| A | 2 | 1 |
| A | 1 | 1 |
+---------------+----------+------------+
I want out put like
+---------------+----------+------------+
| Raw Material | Size | Qty |
+---------------+----------+------------+
| A | 1 | 4 |
| A | 2 | 1 |
| B | 0 | 6 |
+---------------+----------+------------+
Want to get substract first two tables sum of qty by grouping Rawmaterial and Size
Something like this should do the job
var result = tableA.Select(e => new { Item = e, Factor = 1 })
.Concat(tableB.Select(e => new { Item = e, Factor = -1 }))
.GroupBy(e => new { e.Item.RawMaterial, e.Item.Size }, (key, elements) => new
{
RawMaterial = key.RawMaterial,
Size = key.Size,
Qty = elements.Sum(e => e.Item.Qty * e.Factor)
}).ToList();
First we create a union of the two tables using Concat, including the information which one is additive (in Factor field), and then just do the normal grouping.
If you want the result to be List<YourTableElementType>, just replace the final anonymous type projection (new { ... }) with new YourTableElementType { ... }.

Sum of the grouped distinct values

This is a bit hard to explain in words ... I'm trying to calculate a sum of grouped distinct values in a matrix. Let's say I have the following data returned by a SQL query:
------------------------------------------------
| Group | ParentID | ChildID | ParentProdCount |
| A | 1 | 1 | 2 |
| A | 1 | 2 | 2 |
| A | 1 | 3 | 2 |
| A | 1 | 4 | 2 |
| A | 2 | 5 | 3 |
| A | 2 | 6 | 3 |
| A | 2 | 7 | 3 |
| A | 2 | 8 | 3 |
| B | 3 | 9 | 1 |
| B | 3 | 10 | 1 |
| B | 3 | 11 | 1 |
------------------------------------------------
There's some other data in the query, but it's irrelevant. ParentProdCount is specific to the ParentID.
Now, I have a matrix in the MS Report Designer in which I'm trying to calculate a sum for ParentProdCount (grouped by "Group"). If I just add the expression
=Sum(Fields!ParentProdCount.Value)
I get a result 20 for Group A and 3 for Group B, which is incorrect. The correct values should be 5 for group A and 1 for group B. This wouldn't happen if there wasn't ChildID involved, but I have to use some other child-specific data in the same matrix.
I tried to nest FIRST() and SUM() aggregate functions but apparently it's not possible to have nested aggregation functions, even when they have scopes defined.
I'm pretty sure there is some way to calculate the grouped distinct sum without needing to create another SQL query. Anyone got an idea how to do that?
Ok I got this sorted out by adding a ROW_NUMBER() function my SQL query:
SELECT Group, ParentID, ROW_NUMBER() OVER (PARTITION BY ParentID ORDER BY ChildID ASC) AS Position, ChildID, ParentProdCount FROM Table
and then I replaced the SSRS SUM function with
=SUM(IIF(Position = 1, ParentProdCount.Value, 0))
Put a grouping over the ParentID and use a summation over that group,
eg:
if group over ParentID = "ParentIDGroup"
then
column sum of ParentPrdCount = SUM(Fields!ParentProdCount.Value,"ParentIDGroup")

Resources