Is there any way to copy all the data of an index from elasticsearch from my computer to cloud elastic ?
i'm working on localhost and now i want to migrate it to cloud.elastic.co
Cheers!
you can do this with a snapshot of your local cluster into s3, then a restore of that on Elastic Cloud.
Which cloud provider are you using? If it is AWS OpenSearch, it does not allow remote index operation from local ElasticSearch. Allows if it's an ElasticSearch in the AWS cloud and https is broadcasting.
If the data is critical, you can pull the data and send bulk requests to ElasticSearch in the cloud. I had to do so.
You can write your own application or you can send requests multi-threaded with a tool like Jmeter.
Related
I deployed a Laravel application on AWS Elasticbeanstalk.
I want to incorporate caching with Redis as my cache driver as well as Elasticsearch.
I managed to run these 2 features locally (redis on port 6379 and elasticsearch on 9200),
but now I want them to run on remote servers and I simply specify their endpoints in my .env file.
Can anyone let me know how I can obtain remote URLs for Redis and Elasticsearch?
Update:
I found out that Heruko offers the ability to create a Redis instance and thereby one can obtain a URL for Redis. I presume a similar thing is for Elasticsearch.
If this is not the right way to do so, please let me know how it works
Is there any easy way to configure it from Kibana? I have full control of the LogStash server and running Kibana on Elastic cloud
The only possible route I can think of is to create a custom button on Kibana that call a web service, setup a web service on LogStash server that manually start logstash on call
Is there any other way around this?
I'm trying to import some data from AWS RDS to Elasticsearch of Hosted Elastic Cloud
- It's not AWS Elasticsearch Service
What I want to do is below.
What: Data import
From: AWS RDS MySQL
To: Elasticsearch in Elastic Cloud
How: Using Logstash of Elastic Cloud
However, my AWS RDS MySQL is in AWS VPC and Elastic Cloud doesn't provide static ip address (please see Elasticsearch F&Q)
So Logstash can't access to AWS RDS MySQL preserving security rule of AWS VPC.
In previous data transfer case, I used to add trasferer's ip address to whitelist of VPC. For this case, it can't be done.
I totally don't know whether this trial is wrong or not.
How can I handle this case?
After some research, I concluded that there is no way to do that for now. However, there is compromise plan. By setting Logstash EC2 instance in Amazon VPC, Logstash can access to AWS RDS MySQL. With Elastic cloud's credential, it can also send data to Elasticsearch in Elastic Cloud.
I'm new to ELK. We have a Spring Boot backend on a dedicated AWS instance. We have ELK stack on another instance (To the outside world only Kibana is available). Information gathering to ELK is carried out via Amazon SQS.
These information include logs and some business history about user (registration, any other action, etc).
In this case, I have a question. Is it possible to get back information by action, by user and use it in the backend responses?
I am guessing you want to use data present in Elasticsearch to be available to spring boot application. It is definitely possible. You will need to open up elasticsearch port on elasticsearch machine to specifically to EC2 instance on which spring boot is running. How to open port will depend on if they are on same vpc, different vpc, different aws account etc. Once port is open, you can either use Spring Data Elasticsearch or just rest calls to access elasticsearch api.
I am trying to understand how Kibana communicates with ElasticSearch. Does this communication happen b/w the browser and the ElasticSearch server directly (Bypassing the Kibana server) or all Kibana requests go through Kibana server which then pass them on to ElasticSearch.
I am trying to figure out how to implement security around my ElasticSearch & Kibana based servers.
In Kibana3, the browser would talk to elasticsearch directly. Any protection of elasticsearch had to be done via a front-end web server, typically nginx with a bunch of proxy rules to prevent certain actions, etc.
In Kibana4, they realized that having a server would be beneficial, so your browser talks to the kibana server, which talks to elasticsearch.
The "official" way to protect elasticsearch is with "shield", which handles authentication and authorization for actions against the server.
in kibana 5 a request is sent from a web browser to kibana backend(developped using hapi js) then this request will be sent to elasticsearch using elasticsearch plugin wich is one of the kibana core plugins.