Kibana graph for anomaly detection - elasticsearch

With some data loaded into ES using LS, is there a way to mention in Kibana a threshold and display an indicator it breaches the threshold. For example number of transaction in an hour across the system lets set to 200. When gets refreshed the dashboard, it should indicate in red or something when it goes beyond 200 in a time line graph or some visualization.
So If we see for whole day some hours will be red or spikes and most will be green or lower lines. This will be useful for us to display capability of anamoly detection. We are still working to understand Prelert but this kind of detection in Kibana will help us to see in graphs.
Regards,
Raj

There's no such direct threshold mechanism which you could apply on the Kibana Visualizations as the issue's still open.
But then a workaround for this could be to go with a scripted field in Kibana so that you could add up a constant value and use it as a margin in your visualization. Note that The scripted field may work for line visualizations, but it isn't compatible with an area visualization using sub-aggregations.
OR you could use a filter and stacked lines with having one color for above threshold and another color for the ones which under the threshold.
You could still follow up on this ticket and this SO could be handy. Hope it helps!

Related

Kibana Visual: Show visual only when a field. is decreasing

I am trying to set up a simple Kibana line chart with these conditions -
I'm trying to set up a Power vs Timestamp chart. In elastic, I have the metrics Current and Voltage. I'll calculate the Power from that. Problem is, I need to show the visual ONLY if the current is decreasing.
So at time t, the current value should be less than the last reported current value.
Cannot figure out how to do this in Lens. If it can't be done in Lens, any help with Vega is appreciated.
Thanks
So it sounds like you want a graph that only shows the values of current for when the previous value is greater. This is not doable in lens, but definitely doable in Vega.
Since you haven't provided an example I will just point you in the right direction. You will need a series of transforms to get the data in a plottable form.
sort the data by timestamp (collect transform)
run a window transform that stores the previous value (window transform)
calculate the difference between these values (formula transform)
filter values with a negative/positive difference (filter transform)
From here you can build the marks etc. if Vega is new to you I would start by looking at the Vega Documentation, Vega Gallery and the Kibana-Vega Documentation (for data ingest).

Size of the circles in the Kibana Tile map visualization

In Kibana we have a visualization called Tile map which has has several options to customize, such as map type.
My question is about one of this types: Shaded Circle Markers which is basically about changing colors based on value.
Documentation says:
Displays the markers with different shades based on the metric aggregation’s value.
and that's correct, but I don't understand why some bubbles are bigger than others.
I assume that this may be due to the distortion caused by Mercator projection, which which is well presented by Tissot's indicatrix, but I didn't find any source which confirming my hypothesis.
Could you confirm my suspicion or explain in other way this behavior?
I've got an answer from Elastic Team Member who confirmed my suspiction. This distortion is caused by Mercator projection.
Link to Elastic Search forum: https://discuss.elastic.co/t/size-of-the-circles-in-kibana-tile-map-visualization/108395

how to add specific visualization to histogram

I use Kibana to create a date histogram of some web traffic, tagged by category. Using an algorithm running somewhere else, I know that a certain histogram value was an anomaly, and I can export that data into Elasticsearch.
But I can't figure out how to make Kibana visualize the combination of the two in one graph, something like I sketched in the picture below, where the red outline indicates the anomaly.
The anomaly data are just records with timestamp and count values. I can visualize those separate from the histogram easily:
Any hints on how to combine these two series ?
Thanks
Bart

DC.js Crossfilter on "nested" dimensions

I'm quite confused and might need help just formulating the question, so please give good comments...
I'm trying to crossfilter some data where each data point has its own sub-dataset that I want to chart and filter on as well. Each point represents a geographic region, and associated with each point is a time series which measures a certain metric over time.
Here's what I've got so far: http://michaeldougherty.info/dcjs/
The top bar chart shows a particular value for 10 regions, and the choropleth is linked with the same data. Now, below that are two composite line charts. Each line corresponds to a region -- there are 10 lines in each graph, and each graph is measuring a different metric over time. I would like the lines to be filtered as well, so if one bar is selected, only one line will show on the line chart.
Moreover, I want to be able to filter by time on the line charts (through brushing) in addition to some other filter, so I can make queries like "filter out all regions whose line value between 9 AM and 5 PM is less than 20,000", which would also update the bar and choropleth charts.
This is where I'm lost. I'm considering scrapping DC.js for this and using crossfilter and d3.js directly because it seems so complicated, but I would love it if I'm missing something and DC.js can actually handle this. I'd also love some ideas on where to start implementing this in straight crossfilter, because I haven't fully wrapped my head around that yet either.
How does one deal with datasets within datasets?
Screenshot of the link above included for convenience:

Visualization of river- animation via code

I am trying to visualize a river flow- basically, should be able to visualize river current direction and speed based on an user-defined external parameter. This is required to demonstrate vectors in two dimensions- given education needs, animation quality can be minimal- 'tolerable enough'.
I tried a simplistic approach by a blue background with lines indicating currents- comes out very weak and below my low standards!!
Can someone point out a good example/ approach for achieving the same? Thanks.
You can create an image filter that looks like water. Look at Jerry's image filters. Specifically look at the the caustic filter. You could animate it moving from one end of the river to the other end. You can also experiment with varying the time parameter. Since it's open source, you can translate it to other languages.
Here are some links to 3d visualizations.

Resources