Goal: I want to create a dashboard which shows user requests made to my website. For this, I created a filter in my java web-app and started capturing user requests and storing them in an ES index. The document is in the form of:
{
'user': 'user1',
'url': 'domain.com/page1',
'hitcount': 12
}
So, now I have an index which contains the information as to how many times a user requested which URLs.
Now, I want to create visualizations to show usage trends per user.
Question:
Which visualizations should be used for this use-case?
If I need to show the change in user-trends over time, how should I save the data? For e.g. is there a visualization where I could show, that a user has stopped/reduced requesting a page and now accesses a different page more frequently.
Any direction will be helpful.
Note: I understand, this could be done with grafana + prometheus, but I wish to do this with elastic stack.
I’d recommend logging user requests to a log file and have filebeat read and index them into ES. It is better to send non aggregated data into ES and then let ES aggregate it to create required visualizations
Related
I am trying to Use my Elastic search server installed in my local machine to use Postman .i.e., With the help of Postman I want to Post Data and retrieve it with a get operation but unable to do it as I am getting error unknown key [High] for create index
So please help me with the same.
If you want to add a document to your index,
your url should look something like this ( for document ID 1 ) :
PUT http://localhost:9200/test/_doc/1
A good place to start :
https://www.elastic.co/guide/en/elasticsearch/reference/current/getting-started-index.html
For indexing document in the index
PUT http://localhost:9200/my_index/_doc/1
Retrieving indexed document
GET http://localhost:9200/my_index/_doc/1
Introduction:
Elasticsearch is a distributed, RESTful search and analytics engine capable of addressing a growing number of use cases. As the heart of the Elastic Stack, it centrally stores your data for lightning fast search, fine‑tuned relevancy, and powerful analytics that scale with ease.
Kibana is a free and open user interface that lets you visualize your Elasticsearch data and navigate the Elastic Stack. Do anything from tracking query load to understanding the way requests flow through your apps.
Logstash is a free and open server-side data processing pipeline that ingests data from a multitude of sources, transforms it, and then sends it to your favorite “stash.” .
Elasticsearch exposes itself through rest API so in this case you don't have to use logstash as we are directly adding data to elastic search
How to add it directly
you can create an index and type using :
{{url}}/index/type
where index is like a table and type is like just a unique data type that we will be storing to the index. Eg {{url}/movielist/movie
https://praveendavidmathew.medium.com/visualization-using-kibana-and-elastic-search-d04b388a3032
I'm using ElasticSearch 7.1.1 as a full-text search engine. At the beginning all the documents are accessible to every user. I want to give users the possibility to edit documents. The modified version of the document will be accessible only to the editor and everyone else will only be able to see the default document.
To do this I will add two array to every document:
An array of users excluded from seeing the doc
An array with the only user that can see the this doc
Every time someone edit a document I will:
Add to the excluded users list the user that made the edit
Create document containing the edit available only to that user.
This way in the index I'll have three types of documents:
Documents accessible to everyone
Documents accessible to everyone except some users
Documents accessible only to a specific users
I use ElasticSearch not only to fetch documents but also to calculate live aggregations (e.g. sums of some field) so query-time I will be able to fetch user specific documents.
I don't expect a lot of edits, less than 1% of the total documents.
Is there a smarter, and less query intensive, way to obtain the same results?
You could implement a document level security.
With that you can define roles that restrict the read-access to certain documents that match a query (e.g. you could use the id of the document).
So instead of updating the documents each time via your proposed array-solution, you would instead update the role respectively granting the roles to the particular users. This would of course require that every user has an elasticsearch user.
This feature is the only workaround to fulfill your requirements that Elasticsearch brings on the table "out of the box" as far as I know.
I hope I could help you.
Scenario: I have an application where my java application pushes user data from database to Elastic search which is accessed using Kibana dashboards. I also have a Content application which allow users to create/edit data which is saved in database using my java application.
Use case: When user slices data in Kibana dashboards and reaches a point where he realize an error in data, he would want to make change to the data point. E.g. certain company is shown in a particular city in the dashboard which seems to be an incorrect data. User would want to change the city to the correct one.
Problem case: I am not able find a way to either allow the data to be edited within Kibana or have some kind of deeplink in Kibana which takes user from Kibana to my Content application so that the data point can be edited by user.
Currently the user can go to the Content application, search for the company, search for the addresses and make a change there, however thats very cumbersome to do with millions of companies and millions of data points in database.
Haven't found editing possibilities up to now ... but linking is possible:
when you head to "Kibana/Mgmt/Index Patterns" you can define fields to render as a clickable URL (e.g. to be used in the "Data Table" vis).
If you have a field containing e.g. some ID myid you can have kibana output a clickable link instead pointing to e.g. https://mysite/?id=myid
See https://www.elastic.co/guide/en/kibana/current/field-formatters-string.html for details.
If you need more complex linking options (e.g. your effective link needs to incorporate multiple fields of a document) you can create a so called scripted field, there you have access to multiple fields of an elasticsearch document and can construct your link more or less freely).
We use that a lot to link from overview kibana dashboards to other systems with detailed data on the respective item, see for example this Data Table:
I am new for developing the ELK. So, I have an idea to do with the Elasticsearch data with some dashboard design using Kibana. I have tried we are able to create dashboards like pie, graph etc. However, they are all based on the count and average related. So, I want data like table with whole data. But, a way of the form the query based on that it will generate table. I would like to know whether it is possible in Kibana?
I want to use Kibana to visualize data on a dashboard and make a lot of users on the internet can access the dashboard.
The problem is, Kibana will do a query every time, but the data will update about every 30 minutes, so it's a waste of cpu to do query evertime.
So, I want to cache the elasticsearch result in some place like redis and let the kinana to fetch data from the cache.
So:
is there any software that act as a proxy, which can accept kibana request and fetch data from cache and then send response to the kibana? In another word, I only want to use kibana as a UI framework and customize the data source
Is there any other UI framework that can easily visually elasticsearch query result?
There is no need - Elasticsearch will cache the results.