As of now my elastic search set up exists only on one machine.Now I want to set up elastic search cluster using two nodes.If I make my existing machine as master and data node new machine,then will my existing indexex and data get lost from my master/existing machine?
As per my finding/experiment it got lost,please update if I am wrong.
Please visit this link for the same.
Related
I'm working on a Janusgraph application. To improve gremlin query performance we are creating two mixed indexes, one for vertices and one for edges.
Now Janusgraph can query indexes for property keys that are created and indexed at the time of index creation i.e in the same transaction. If I'm creating and indexing a new property key in a new transaction then Janusgraph is not able to query them using indexing, instead, it does a complete graph scan.
Using Janusgeaph management API I checked that all property keys are indexed and enabled, even then Janusgraph is scanning a complete graph for querying on an indexed property key.
Is there anything I'm missing? Any help would be greatly appreciated.
Backend index engine -> ElasticSearch
Backend Storage -> Cassandra
Have faced this problem once. Try to reindex the created index once (Index created in some other transaction). It worked for me. Hope it works for you too.
Please find the steps below:-
For Reindex:
mgmt = graph.openManagement()
i = mgmt.getGraphIndex('IndexName')
mgmt.updateIndex(i, SchemaAction.REINDEX)
mgmt.commit()
For Enable the index:
ManagementSystem.awaitGraphIndexStatus(graph, 'IndexName').status(SchemaStatus.ENABLED).call()
NOTE: if you get "false" in enabling the index, Try enabling it 2 3 times using the same command (ManagementSystem.awaitGraphIndexStatus(graph, 'IndexName').status(SchemaStatus.ENABLED).call()). It would work eventually.
I'm new to elastic search however had to work with it. I have successfully set it up using logstash to connect it to my oracle database(one particular table). Now if new records are added to one of the tables in my oracle database(which I built the index on), what should be done?
I have thought of two solutions,
Re-build the indices by running the logstash conf file.
On insert into the table, also POST to elastic search.
The first solution is not working like it should. I mean that if 'users' is the table that I have updated with new records, then on re-building indices(for the 'users' table) in elastic search, the new records also should be reflected in the logstash get query.
The first should would help as a POC.
So, Any help is appreciated.
Thank you Val for pointing me in the right direction.
However, for the first brute-force solution it was about changing the document type in the logstash conf file.
{"document_type":"same_type"}
This must be consistent with the previously mentioned type. I had run it with different type, first time(Same_type). After adding new records, I used same_type. So, the elastic search as thrown an exception for multiple mapping rejection.
For further clarification, it looked up here.
Thank you guys.
I have ELK setup. with elastic search version 2.3.x. I wanted to know how many nodes by default it should create. I have noticed a weird situation. When I restarted elastic search it started with 3 nodes. There are multiple folder in data path ( /var/lib/elasticsearch/0 , 1 , 2 ,3). But when I restart it again it took only one node. I wanted to know how it is defining a number of nodes.
# curl -s -XGET "http://localhost:9200/_cat/nodes?v"
Because of this many shared are showing un-assigned due to lack of nodes
Your single elasticsearch installation on one machine is one node, if you want to have more u need to have multiple elastic installations running. See here for further details.
Currently I have existing
1. Elastic search
2. Logstash
3. Kibana
I have existing data on them.
Now i have setup ELK cluster with 3 Master nodes , 5 data nodes 3 client nodes.
But i am not sure how can i get existing data into them.
Is it possible that if i make the existing ES node as data node and then attach it to the cluster . Then will that data gets replicated to other data nodes as well? and then take that node offline
Option 1
How about just try with fewer nodes? It is not hard to test if it is supported if you setup one node, feed some data, and add one more and configure them as a cluster to see if data get synchronized.
Option 2
Another option is to use an elasticsearch migration tool like https://github.com/taskrabbit/elasticsearch-dump, basically, you could setup a clean cluster and migrate all your data in old node to this cluster.
How can I get an elasticsearch index to a file and then insert that data to another cluster?
I want to move data from one cluster to another but I can't connect them directly.
If you no need to keep _id the same and only important bit is _source you may use logstash with config:
input { //from one cluster } output { //to another cluster }
here is more info: http://www.logstash.net/docs/1.4.2/
Yes it's method is weird, but I tried it for instant data transfer between clusters index by index and it is working as a charm (of course if you no need to keep _id generated by elasticsearch)
There is script which will help you to backup and restore indices from one cluster to another. i didn't tested this but may be it will fix your needs.
check this Backup and restore an Elastic search index
And you can also use perl script to copy index from one cluster to another (or the same cluster).
check this link clintongormley/ElasticSearch.pm
I recently tried my hands around this and there are a couple of approaches that can help you.
Use Elasticsearch's Snapshot and Restore APIs.
You can take a snapshot at the source cluster and use that snapshot to restore data to your destination cluster.
If your setup allows installing external packages, you can use Elasticdump as well.
HTH!