ElasticSearch-Kibana-Vega Plugin - elasticsearch

I'm very new with elastic search and kibana . I'm using vega plugin in kibana visualization.
But not able to create Bar Chart using elastic search aggs.
I'm getting proper result when I'm using kibana dev tools.
I'am attaching the following details with the sample code after run this I'm getting a blank page
Visualization Section:
{
"$schema": "https://vega.github.io/schema/vega/v3.0.json",
"autosize": "fit",
"padding": 6,
"data": [
{
"name": "traffic-revenue",
"url": {
"index": "brnl_tms_plaza",
"body": {
"size": "0",
"aggs": {
"group_by_vehicle_subcat": {
"terms": {
"field": "VehicleSubCatCode.keyword"
}
}
}
},
"format": {
"property": "aggregations.group_by_vehicle_subcat.buckets"
}
}
}
],
"scales": [
{
"name": "xscale",
"type": "band",
"domain": {
"data": "traffic-revenue",
"field": "key"
},
"range": "width",
"padding": 0.05,
"round": true
},
{
"name": "yscale",
"domain": {
"data": "traffic-revenue",
"field": "doc_count"
},
"nice": true,
"range": "height"
}
],
"axes": [
{
"orient": "bottom",
"scale": "xscale"
},
{"orient": "left", "scale": "yscale"}
],
"marks": [
{
"type": "rect",
"from": {
"data": "traffic-revenue"
},
"encode": {
"enter": {
"x": {
"scale": "xscale",
"field": "key",
"axis": {"title": "Vehicle category"}
},
"width": {
"scale": "xscale",
"band": 1
},
"y": {
"scale": "yscale",
"field": "doc_count",
"axis": {"title": "Vehicle Rate Count"}
},
"y2": {
"scale": "yscale",
"value": 0
}
},
"update": {
"fill": {"value": "steelblue"}
},
"hover": {"fill": {"value": "red"}}
}
}
]
}
Data Set
{
"took": 7,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"skipped": 0,
"failed": 0
},
"hits": {
"total": 48,
"max_score": 0,
"hits": []
},
"aggregations": {
"group_by_vehicle_subcat": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "LMV",
"doc_count": 35
},
{
"key": "BUS",
"doc_count": 3
},
{
"key": "LCV",
"doc_count": 3
},
{
"key": "MAV-5",
"doc_count": 3
},
{
"key": "MAV-4 with trailer",
"doc_count": 2
},
{
"key": "MAV-3 without trailer",
"doc_count": 1
},
{
"key": "MINI-BUS",
"doc_count": 1
}
]
}
}
}

I would recommend debugging your vega code using static data to make sure it is defined properly.
I'm not sure why, but I was able to get your visualization to draw when I set the autosize property to none and set the height and width explicitly.
Here is a vega specification based off of the one you provided which should run in the online vega editor.
{
"$schema": "https://vega.github.io/schema/vega/v3.0.json",
"autosize": "none",
"width": 400,
"height": 500,
"padding": 20,
"data": [
{
"name": "traffic-revenue",
"values": [
{"key": "a", "doc_count": 5},
{"key": "b", "doc_count": 22},
{"key": "c", "doc_count": 1},
{"key": "d", "doc_count": 7},
{"key": "e", "doc_count": 12},
{"key": "f", "doc_count": 2}
]
}
],
"scales": [
{
"name": "xscale",
"type": "band",
"domain": {
"data": "traffic-revenue",
"field": "key"
},
"range": "width",
"padding": 0.05,
"round": true
},
{
"name": "yscale",
"domain": {
"data": "traffic-revenue",
"field": "doc_count"
},
"nice": true,
"range": "height"
}
],
"axes": [
{
"orient": "bottom",
"scale": "xscale"
},
{"orient": "left", "scale": "yscale"}
],
"marks": [
{
"type": "rect",
"from": {
"data": "traffic-revenue"
},
"encode": {
"enter": {
"x": {
"scale": "xscale",
"field": "key",
"axis": {"title": "Vehicle category"}
},
"width": {
"scale": "xscale",
"band": 1
},
"y": {
"scale": "yscale",
"field": "doc_count",
"axis": {"title": "Vehicle Rate Count"}
},
"y2": {
"scale": "yscale",
"value": 0
}
},
"update": {
"fill": {"value": "steelblue"}
},
"hover": {"fill": {"value": "red"}}
}
}
]
}
You may already know this since you have the format tag on your elasticsearch data, but if your visualization is working with statically defined data, and not when you pull data from an elasticsearch query, try looking at the data source directly using the vega debuggging functions described here https://vega.github.io/vega/docs/api/debugging/.
Running the following in the browser console should let you look at the data in the format vega is receiving it. VEGA_DEBUG.view.data("")

Related

Issue- Not displaying the custom visualization in vega

I am trying to make a custom word cloud visualization from the existing index using vega in kibana dashboard.
It is showing error as "Cannot read properties of undefined (reading 'datum')".
I have used type-formula in "transform property" to create custom visualization word cloud rather than these any types are there?
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"title": "A Wordcloud",
"width": 900,
"height": 500,
"padding": 100,
"autosize": "none",
"background": "pink",
"data": [
{
"name": "table",
"url": {
"index": "nupur2",
"body": {
"aggs": {
"2": {
"terms": {"field": "hashtags","order":{"_count": "asc"}, "size": 100},
"aggs": {
"_count": {
"avg": {"field": "vaderSentiment"}
}
}
}
}
}
},
"format": {"property": "aggregations.2.buckets"},
"transform": [
{
"type": "formula",
"as": "angle",
"expr": "datum.size >= 4 ? 0 : [-45,-30, -15, 0, 15, 30, 45][floor(random() * 7)]"
}
]
}
],
"scales": [
{
"name": "color",
"type": "ordinal",
"domain": {"data": "table", "field": "hashtags"},
"range": ["green", "orange", "red"]
}
],
"marks": [
{
"type": "group",
"from": {"data": "table"},
"encode": {
"enter": {
"text": {"field": "hashtags"},
"align": {"value": "center"},
"baseline": {"value": "alphabetic"},
"fill": {"scale": "color", "field": "hashtags"}
},
"update": {
"fillOpacity": {"value": 1}
},
"hover": {
"fillOpacity": {"value": 0.5}
}
},
"transform": [
{
"type": "wordcloud",
"size": [800, 400],
"text": {"field": "hashtags"},
"rotate": {"field": "datum.angle"},
"font": "Helvetica Neue, Arial",
"fontSize": {"field": "datum.size"},
"fontWeight": {"field": "datum.weight"},
"fontSizeRange": [12, 56],
"padding": 2
}
]
}
]
}
This is the image which I got output : https://i.stack.imgur.com/nbb8f.png

Draw tree Layout chart in vega

I want to have a tree chart of my data using vega in kibana 7.9.0, but I don't know how to write the query for that.
the below code is an example of tree chart from github website. I want such layout for my own index which I have it in kibanan.
Help me how to do that.
tree chart example
Sample tree chart code:
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"description": "An example of Cartesian layouts for a node-link diagram of hierarchical data.",
"width": 1000,
"height": 1600,
"padding": 5,
"signals": [
{
"name": "labels", "value": true,
"bind": {"input": "checkbox"}
},
{
"name": "layout", "value": "tidy",
"bind": {"input": "radio", "options": ["tidy", "cluster"]}
},
{
"name": "links", "value": "diagonal",
"bind": {
"input": "select",
"options": ["line", "curve", "diagonal", "orthogonal"]
}
},
{
"name": "separation", "value": false,
"bind": {"input": "checkbox"}
}
],
"data": [
{
"name": "tree",
"url": "data/flare.json",
"transform": [
{
"type": "stratify",
"key": "id",
"parentKey": "parent"
},
{
"type": "tree",
"method": {"signal": "layout"},
"size": [{"signal": "height"}, {"signal": "width - 100"}],
"separation": {"signal": "separation"},
"as": ["y", "x", "depth", "children"]
}
]
},
{
"name": "links",
"source": "tree",
"transform": [
{ "type": "treelinks" },
{
"type": "linkpath",
"orient": "horizontal",
"shape": {"signal": "links"}
}
]
}
],
"scales": [
{
"name": "color",
"type": "linear",
"range": {"scheme": "magma"},
"domain": {"data": "tree", "field": "depth"},
"zero": true
}
],
"marks": [
{
"type": "path",
"from": {"data": "links"},
"encode": {
"update": {
"path": {"field": "path"},
"stroke": {"value": "#ccc"}
}
}
},
{
"type": "symbol",
"from": {"data": "tree"},
"encode": {
"enter": {
"size": {"value": 100},
"stroke": {"value": "#fff"}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"fill": {"scale": "color", "field": "depth"}
}
}
},
{
"type": "text",
"from": {"data": "tree"},
"encode": {
"enter": {
"text": {"field": "name"},
"fontSize": {"value": 9},
"baseline": {"value": "middle"}
},
"update": {
"x": {"field": "x"},
"y": {"field": "y"},
"dx": {"signal": "datum.children ? -7 : 7"},
"align": {"signal": "datum.children ? 'right' : 'left'"},
"opacity": {"signal": "labels ? 1 : 0"}
}
}
}
]
}
I recommend going and learning the Kibana Vega interaction, Vega spec and elasticsearch search api. This is too broad of a question, you are basically asking someone to do the work for you which is easily deducible from the the documentation.
https://www.elastic.co/guide/en/kibana/current/vega.html
https://vega.github.io/vega/docs/
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html

How do I make the charts c two type: normal and stacked?

I want to have two typecharts in one visual.
Desired example
Current example, Editor
You can use 2 layers having bar chart instead of column. First layer will be a stacked layer in which filter is applied for used and free fields. Second layer can have the total field. Using xOffset you can position both the bar charts. Refer the code below or editor:
{
"$schema": "https://vega.github.io/schema/vega-lite/v5.json",
"config": {"view": {"stroke": "transparent"}, "axis": {"domainWidth": 1}},
"width": 300,
"data": {
"values": [
{"branch": "V", "free": 300, "used": 800, "total": 1000},
{"branch": "K", "free": 100, "used": 400, "total": 500},
{"branch": "D", "free": 100, "used": 200, "total": 300}
]
},
"transform": [
{"calculate": "[ datum.used, datum.free, datum.total]", "as": "table"},
{"calculate": "['used', 'free', 'total']", "as": "headline"},
{"flatten": ["table", "headline"]},
{
"calculate": "datum.headline +':'+ datum.table + ' ('+round(datum.table *100/ datum.total)+'%)'",
"as": "tooltip"
}
],
"encoding": {
"y": {"aggregate": "sum", "field": "table", "axis": {"grid": false}},
"x": {
"field": "branch",
"axis": {"grid": false, "domain": false, "labelAngle": 0, "ticks": false},
"sort": {"op": "sum", "field": "table", "order": "descending"}
},
"tooltip": {"field": "tooltip", "type": "nominal"},
"color": {
"field": "headline",
"type": "nominal",
"scale": {"range": ["#409b66", "#878787", "#1b5c9e"]}
}
},
"layer": [
{
"transform": [
{"filter": {"field": "headline", "oneOf": ["used", "free"]}}
],
"mark": {"type": "bar", "width": 15, "xOffset": -10}
},
{
"mark": {"type": "bar", "width": 15, "xOffset": 8},
"encoding": {
"y": {"field": "total", "axis": {"grid": false}, "stack": false}
}
}
]
}

Vega: Use filter as input and dynamically change data URL

I am trying to create a Vega visualisation using an external URL and I want to construct the URL dynamically using some filters as input to the visualisation. My current Vega code is this:
{
"$schema": "https://vega.github.io/schema/vega/v5.json",
"width": 400,
"height": 200,
"padding": 5,
"data": {
"name": "table",
"url":"https://raw.githubusercontent.com/aryak93/vega/master/data.json"
},
"signals": [
{
"name": "tooltip",
"value": {},
"on": [
{"events": "rect:mouseover", "update": "datum"},
{"events": "rect:mouseout", "update": "{}"}
]
}
],
"scales": [
{
"name": "xscale",
"type": "band",
"domain": {"data": "table", "field": "category"},
"range": "width",
"padding": 0.05,
"round": true
},
{
"name": "yscale",
"domain": {"data": "table", "field": "amount"},
"nice": true,
"range": "height"
}
],
"axes": [
{ "orient": "bottom", "scale": "xscale" },
{ "orient": "left", "scale": "yscale" }
],
"marks": [
{
"type": "rect",
"from": {"data":"table"},
"encode": {
"enter": {
"x": {"scale": "xscale", "field": "category"},
"width": {"scale": "xscale", "band": 1},
"y": {"scale": "yscale", "field": "amount"},
"y2": {"scale": "yscale", "value": 0}
},
"update": {
"fill": {"value": "steelblue"}
},
"hover": {
"fill": {"value": "red"}
}
}
},
{
"type": "text",
"encode": {
"enter": {
"align": {"value": "center"},
"baseline": {"value": "bottom"},
"fill": {"value": "#333"}
},
"update": {
"x": {"scale": "xscale", "signal": "tooltip.category", "band": 0.5},
"y": {"scale": "yscale", "signal": "tooltip.amount", "offset": -2},
"text": {"signal": "tooltip.amount"},
"fillOpacity": [
{"test": "isNaN(tooltip.amount)", "value": 0},
{"value": 1}
]
}
}
}
]
}
What I'm trying is to change the URL from "url":"https://raw.githubusercontent.com/aryak93/vega/master/data.json" to something like "url":"https://raw.githubusercontent.com/aryak93/vega/master/data.json?projectName={projectName}" and I am expecting the project name to be populated when a user sets the filter.
So far, I have struggled to find something relevant from the Vega and Vega lite documentations. Please let me know whether this'll be possible.

VEGA Kibana - multi view chart FLATTEN Transformation

I am having a hard time figuring out how to build a multiview chart in Vega using aggregated/nested.
I believe it is related to the way I am trying to transform my data to use in Vega multi view chart. Can anyone give me a hand to understand this?
I know flatten transformation is working as debug shows
But what I get is this
This is what I would like to achieve
This is my schema to build the chart
{
"$schema": "https://vega.github.io/schema/vega/v3.json",
"width": 400,
"height": 200,
"padding": 5,
"data": [
{
"name": "source",
"values": {
"aggregations": {
"order_labels": {
"buckets": [
{
"key": "USD/CAD",
"doc_count": 1,
"orders": {
"doc_count": 40,
"orders_id": {
"buckets": [
{
"key": 5241,
"doc_count": 1,
"orders_price": {"value": 0.01991}
},
{
"key": 5242,
"doc_count": 1,
"orders_price": {"value": 0.02021}
}
]
}
}
},
{
"key": "CAD/COD",
"doc_count": 1,
"orders": {
"doc_count": 40,
"orders_id": {
"buckets": [
{
"key": 5041,
"doc_count": 1,
"orders_price": {"value": 0.00002953}
},
{
"key": 5042,
"doc_count": 1,
"orders_price": {"value": 0.00002971}
}
]
}
}
}
]
}
}
},
"format": {"property": "aggregations.order_labels.buckets"},
"transform": [
{
"type": "flatten",
"fields": ["orders.orders_id.buckets"],
"as": ["orders"]
}
]
}
],
"mark": "bar",
"encoding": {
"row": {
"field": "orders.key",
"type": "ordinal"
},
"x": {
"aggregate": "sum",
"field": "orders.orders_price.value",
"type": "quantitative",
"scale": { "zero": false }
},
"y": {
"field": "key",
"type": "ordinal",
"scale": { "rangeStep": 12 }
}
}
}
I did many things but can`t understand what is wrong with it
Kibana 6.4 supports flatten transform for sure. I think it was also supported in 6.3.

Resources