How to aggregate data between two filters in Kibana? - elasticsearch

I have two filters transfer* and transfer_failures. Each filter is the output of a database query using ES DB queries. I need to visualize a graph that aggregate monthly the number of transfer_failures/total transfers in the month.
Both have timestamp created field with the same format, but I wanna know how to plot table that uses two filters!
So suppose I have 40 total transfers on May 'should be calculated using the transfer* filter' and 10 transfer_failures 'should be calculated by the transfer_failure* filter' so the graph should blot 10/40=.25 on May supposing building a Vertical Bar.
I've some screenshots to show, but I am not allowed to post these here.

You can use the filter ratio aggregation on Visualize -> TSVB/Time Series Visual Builder if you want to calculate percentages using two filters.

Related

Calculated fields counts aggregated by different time variables on quicksight

I have two event columns on a table with dates. I need to count the number of events in each column create a calculated field that is a ratio between them and create a filter by a period on the interface.
That is easy if I have separated plots as I use the date field for filtering.
separated plots
The problem occurs when I need to create a calculated field that is a ratio between them. I can't have two variables and two filters. Is that a way to solve this only using quicksigth capabilities? It is not possible to assess the filtered data to use as input for a calculated field ?

Excluding a Metric from a Visualisation Filter

I am a new user to MicroStrategy Developer and am stuck with a task that I want to do. I have created a table visualisation and applied two filters to the visualisation via the visualisation filters:
Top 10 Rank by Volume Filter
A current month Filter
My table has the following table headings:
Region
Sales Person Name
Sales Volume
Average Sales Volume To Date
I want the table to display the Current Months values for the 1st three columns and then the last column to display the average sales volume to date (based on 2 years worth of data). However clearly Average Sales Volume is not working correctly, I am assuming this is due to the Month filter applied to the visualisation.
Being a new user to MicroStrategy, I am not sure how best to get round this? Is there a way of making the Average Sales Volume To Date metric independent of the Date filter applied to the visualisation? Or a better way to build this without applying visualisation filters?
Any hints or tips would be much appreciated.
Thanks!
I am not very sure whether this works out but you can give it a try.
Create the "Average Sales Volume To Date" metric as a level metric and set the filtering as Ignore.

Kibana 5.5 visualization data aggregation percentage

I'm using Kibana 5.5 and would like to create a visualization that shows percentage of successful transactions made to my server.
In order to do that I have a field in the document named "httpCode" and I would like to get the percentage of "httpCode: 200" out of the total transactions made.
I managed to put this in visulalization that does separate lines, but I would like to get percentage over time and not count.
Thanks,
Moshe
Create a new bar or area chart and choose count as aggregation, set your field for Date Histogram. Next in buckets tab add Split Series and choose Filters as Sub Aggregation. Your first filter should look like that:
Filter 1
httpCode:200
Use http.keyword instead if is available.
Filter 2
NOT httpCode:200
Go to Metrics & Axes tab, expand LeftAxis-1 and change mode to percentage:
On my exmaple it should look like that:

How to sort in filter without using Dynamic Ranking in Endeca?

We are using Endeca to fetch and display records in frontend as a datagrid. In that datagrid, we have 10 columns and we display data sorted in table on the basis of 2 columns (say X and Y). For this, we use Endeca.stratify(collection()/record[not%20(X)])||X|1||*,Endeca.stratify(collection()/record[not%20(Y)])||Y|1.
We can also apply filter on the columns where we display data sorted asc/desc. We used Dynamic Ranking in Endeca and created dimensions for each field with selecting dynamic ranking and set maximum dimension value to return as 20 as per the requirement. Since we know that dynamic ranking is the relevancy ranking, it fetches most used records and does sorting on that data.
However, we need to select 20 unique values and sort them in asc/desc order. Example: if we have date as the column, then we need to fetch 20 unique dates with most recent at the top. i.e. in descending order.
Is there any other way to do sorting on filter apart from dynamic ranking? If we disable dynamic ranking, then we won't have option to set maximum dimension value as 20 from developer studio.
Please suggest for the ranking.
We finally found a solution!! I removed/unchecked "dynamic ranking" for the properties in dimensions from the pipeline using developer studio. I did not want it to remove since we had already selected an option as sort "alphabetically" instead of "dynamically" in dynamic ranking tab in dimensions.
Also, if we uncheck dynamic ranking then the option for giving maximum limit for displaying the dimensions (which was set 20 for us as per the requirement) was also gone.
So, I handled this in java to display only 20 values by putting a check on results obtained and created a counter which would add values only till the 20 are received. Now this is working as required!!!!!

elasticsearch query to display lag time between two datetime fields

I have setup a dashboard that displays incoming message rates in a graph, filtered terms in a pie graph and all with nothing really to do. I want to be able to display a table/graph or some other type of panel to display elapsed time between two datetime fields that appear in my message:
{"pubDate":"2014-02-27T07:11:44",
"fetchDate":"2014-02-27T07:11:55",
"client":"ME",
"query":"(work OR my job boring OR mundane OR stodgy OR interesting
OR sucks OR exciting OR fun)",
"lang":"en"}
What I am able to get now is a language filter graph, and a query filtered pie chart with the normal incoming rate graph as well. I want to create a graph/table that gives the difference between the pubdate and fetchdate. Is this possible?
Also related to same project is that the query field does not work correctly in that elasticsearch seems to break long queries into individual terms and I wonder if it interperates the OR itself or just based on length.
Any advice appreciated.

Resources