i am using spring boot application to load messages into elastic search. i have a use case where i need to query the elastic search data , get some id value and populate it in elastic search json document before inserting it into elastic search
Querying the elastic search before insertion . Will this be expensive ? If yes is there some other way to approach this issue.
You can use update_by_query to do it in one step.
But otherwise it shouldn't be slow if you do it in two steps (get + update). It depends on many things - how often you do it, how much data is transferred, etc.
Related
Can you help me with how to Archive data in Elastic search. That I don't know what is curator and data shrink. I am fresher to elastic search. from where I have to start studying and what all things I have to do.
Elastic search is an fulltext index. You can use this technology to index some data to get fast an powerful access to you data.
But, it's an index.
I don't think, elastic search is the right place to archive data.
Especially not if the archive is to fulfill certain archival standards.
You can archive your data somewhere else and use elastic search to search over your archived data.
If I were you, I would use a specialized tool for storing and archiving data and index this data with Elasticsearch for powerful search.
you need to look at using ILM for this, it's the replacement for curator and will handle it much cleaner
We are using ElasticSearch to save and manage information on complex transactions. We might need to add more information for every transaction, on the near future.
How about including JSON doc version?
Is it possible for elastic search, to include different versions of JSON docs, to save and to search?
How does this affects performance on ElasticSearch?
It's completely possible, By default elastic uses the dynamic mappings for every new documents such as your JSON documents to index them. For each field in your documents elastic creates a table called inverted_index and the search queries executed against them so regardless of your field variation as long as you know which field you want to execute query the data throughput and performance will not be affected.
How to serve search or query operation uninterrupted from a elastic server while we index/re-index the data into that particular elastic search server at the same time.
You need to look at using aliases:
https://www.elastic.co/guide/en/elasticsearch/guide/current/index-aliases.html
Essentially you create new index with a unique name, and swap the alias out with the old index, in one atomic step
Just going to implement elastic search log related task.
Have some questions about elastic:
What is elastic? does it mean flexible stuff?
What is elastic search? (https://www.elastic.co/products/elasticsearch)
what is elastic cache?
what is the relationship between elastic search and elastic cache?
Thanks
I'm not sure what relates ES with EC, but simply Elastic Search is where you index all the data you need, let it be log files or the data from a database. You could store them as docs within an index and then query in order retrieve data from the index.
This is what I got from my neighborhood friend Google:
Elasticsearch is a search engine based on Lucene. It provides a
distributed, multitenant-capable full-text search engine with an HTTP
web interface and schema-free JSON documents.
I'm using CouchDB river plugin with Elastic Search. In my web application, I am using CouchDB's bulk insert to insert documents into CouchDB. This triggers the changes feed and ES reads this to index my documents. The problem now is that my web ui isn't showing anything because ES is still indexing the documents.
I'm using PyES to "talk" to ES by the way. Is there any function I can call to know whether Elastic Search is busy indexing?
Thanks a million.
Even if ES is indexing, ES should answer to queries.
Could you check with a
curl localhost:9200/_search?q=*
That your index has docs in it while indexing from couchDb?
[UPDATE]
You have to know that Elasticsearch is a Near Real Time search engine. So, you have to wait some seconds to be able to search for your docs.
You can retrieve your docs immediatly but you need to wait for the refresh process.
You can trigger manually the refresh API. But it could slow down dramatically your insertions.
Does it help?