connect localhost elastic search cluster on mirage - elasticsearch

I'm trying to use Mirage to build my elastic search queries. I have started elastic search on localhost and when i do curl localhost:9210 on terminal, i get below details :
"name" : "RN48HFb",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "some number",
"version" : {
"number" : "5.6.0",
"build_hash" : "something",
"build_date" : "some date",
"build_snapshot" : false,
"lucene_version" : "6.6.0"
},
"tagline" : "You Know, for Search"
}
So, I am trying to connect this elastic search cluster on mirage to build certain elastic search queries.
What appname and elastic search cluster i use to connect it on mirage?
I'm trying to put http://localhost:9210/ in URL section, but it doesn't work? what should be the appname : is it "RN48HFb"?
Please help I am new to this.
I have installed chrome extension for Mirage and the URL to mirage is :
https://github.com/appbaseio/mirage
chrome-extension://dcnlpfmnpoggchflmdnkgiepijgljoka/site/index.html

sorry for the late response
You just need to update your ES config
Steps:
Stop elasticsearch
Update ES config: $DIR_ELASTIC/elasticsearch/elasticsearch.yml
Add this code:
http.cors.enabled: true
http.cors.allow-origin: "*"
http.cors.allow-credentials: true
http.cors.allow-headers: "Authorization, X-Requested-With, Content-Type, Content-Length"
http.cors.allow-methods: "OPTIONS, HEAD, GET, POST, PUT, DELETE"
Start Elasticsearch
You can Open console and try to make a request to be sure that mirage should work.
JS Code:
var xhr = new XMLHttpRequest();
xhr.open('POST', 'http://localhost:9200/_search', true);
xhr.withCredentials = true;
xhr.setRequestHeader('Content-Type', 'application/json');
xhr.setRequestHeader('Authorization', 'Basic ' + btoa('test:test'));
xhr.send('{"query":{"match_all":{}}}');
If it's response status is 200 then try with mirage.
Appname : ElasticSearch Index
URL : ElasticSearch url cluster (http://localhost:9200)

Related

Elasticsearch "certificate has expired" from Kibana Dev Tools

I have an Elasticsearch and Kibana Helm charts deployed on my Kubernetes cluster for a couple of years now, and I've been working with Kibana's Dev Tools to query my Elasticsearch.
Since a few days ago I started to get the following error when running any query from Kibana's Dev Tools:
{"statusCode":502,"error":"Bad Gateway","message":"certificate has expired"}
But when I try using curl command or simply opening the browser and entering my Elasticsearch's url and some uri it works and I get anything I need.
Moreover, when I try to fetch the /_ssl/certificates field it says that the certificate's expiry is in about a year so I do get to see that the used certificate is valid, but still for some reason I get 'certificate expired' from the Dev Tools.
Anyone knows if there are other certificates used I should check?
Edit: Adding output of field /_ssl/certificates:
$ curl -k -u elastic:*** "https://localhost:9200/_ssl/certificates?pretty"
[
{
"path" : "/usr/share/elasticsearch/config/certs/tls.crt",
"format" : "PEM",
"alias" : null,
"subject_dn" : "CN=***, O=***, L=***, ST=***, C=***",
"serial_number" : "***",
"has_private_key" : true,
"expiry" : "2024-01-19T23:59:59.000Z"
},
{
"path" : "/usr/share/elasticsearch/config/certs/tls.crt",
"format" : "PEM",
"alias" : null,
"subject_dn" : "CN=***, O=***, L=***, ST=***, C=***",
"serial_number" : "***",
"has_private_key" : false,
"expiry" : "2024-01-19T23:59:59.000Z"
},
{
"path" : "/usr/share/elasticsearch/config/certs/tls.crt",
"format" : "PEM",
"alias" : null,
"subject_dn" : "CN=DigiCert TLS RSA SHA256 2020 CA1, O=DigiCert Inc, C=US",
"serial_number" : "***",
"has_private_key" : false,
"expiry" : "2031-04-13T23:59:59.000Z"
},
{
"path" : "/usr/share/elasticsearch/config/certs/tls.crt",
"format" : "PEM",
"alias" : null,
"subject_dn" : "CN=DigiCert Global Root CA, OU=www.digicert.com, O=DigiCert Inc, C=US",
"serial_number" : "***",
"has_private_key" : false,
"expiry" : "2031-11-10T00:00:00.000Z"
}
]
Note: Replaced sensitive information with '***'.
It looks like SSL certificates on one or more nodes have expired. To find that node go to kibana.yml and check elasticsearch.hosts. Which node Kibana is querying, that node's certificate has expired.
You can renew the certificate with the help of this article.

Is there anyway to check if elasticsearch cluster exists or not?

I am working on an elasticsearch (es) cluster monitoring dashboard where I want to onboard all my es clusters. I am developing the dashboard from scratch. So, I wanted to add a button on the dashboard by clicking on that user will be able to enter the name of the es cluster address/IP(first time onboarding the cluster) then hit the submit button. If that es cluster exists then user should be able to monitor the cluster, if not then, it should show some error message to the user(on the dashboard) saying that "Sorry you have entered a wrong cluster address/IP". So, how can I determine if an es cluster exists or not?
A simple curl call to the ES cluster address and port should be enough to verify if an ES cluster exists or not.
For e.g. if we want to verify whether an ES cluster exists at http://localhost:9200, we would fire a curl call as follows:-
curl -XGET "http://localhost:9200/"
If the ES cluster exists/ has permissions to access, it would return a JSON as follows:
{
"name" : "es01",
"cluster_name" : "elasticsearch7",
"cluster_uuid" : "xu49eNE6SuC1Z857kG2Q5g",
"version" : {
"number" : "7.16.3",
"build_flavor" : "default",
"build_type" : "docker",
"build_hash" : "4e6e4eab2297e949ec994e688dad46290d018022",
"build_date" : "2022-01-06T23:43:02.825887787Z",
"build_snapshot" : false,
"lucene_version" : "8.10.1",
"minimum_wire_compatibility_version" : "6.8.0",
"minimum_index_compatibility_version" : "6.0.0-beta1"
},
"tagline" : "You Know, for Search"
}
Else, it would return an error as follows:-
curl: (7) Failed to connect to localhost port 9200: Connection refused
Please note, that you would need to use appropriate curl syntax according to the programming language. For the example, I have considered a bash script.

Elk stack, why I can't create index of rabbitmq messages?

I recently developed a C# web app that produce and consume messages on a RabbitMQ exchange of topic type, everything is working very good. Than I decided to use the ELK stack to analyze the RabbitMQ logs and it also working very good as expected, than my troubles starts when I decided to try to log all the messages that are produced and consumed.
I followed this guide to deploy the ELK stack.
How to Install ELK Stack on Debian 9
Than my trouble started..
this is an extract of the curl -XGET 'localhost:9200'
{
"name" : "dvv7m8h",
"cluster_name" : "elasticsearch",
"cluster_uuid" : "545-XOzEQ7W2C982ISVnng",
"version" : {
"number" : "6.8.4",
"build_flavor" : "default",
"build_type" : "deb",
"build_hash" : "bca0c8d",
"build_date" : "2019-10-16T06:19:49.319352Z",
"build_snapshot" : false,
"lucene_version" : "7.7.2",
"minimum_wire_compatibility_version" : "5.6.0",
"minimum_index_compatibility_version" : "5.0.0"
},
"tagline" : "You Know, for Search"
}
and as the official documentation states (
Rabbitmq input plugin) I need to enable the plugin by running this command bin/logstash-plugin install logstash-input-rabbitmq, but there is no bin/logstash-plugin command available for me! I tried to look nearly everywhere on the world wide web but after three days still no results. As reference I'll post my logstash config file as well.
input {
rabbitmq {
host => 'xxx.yyy.zz.nn:5672'
exchange => "my_exchange"
exchange_type => "topic"
id => "rabb"
}
}
}
output {
elasticsearch {
hosts => ["http://localhost:9200"]
index => "rabtest-%{+YYYY.MM.dd}"
}
}
Can anyone tell me what I'm missing? Is the plugin already shipped as a bundle inside logstash v.6.8.x? Why I don't have there forementioned command to install the plugin? Thanks.

Cant connect to my proxied elasticsearch node

I'm having issues with connecting from my Go client to my es node.
I have elasticsearch behind an nginx proxy that sets basic auth.
All settings are default in ES besides memory.
Via browser it works wonderfully, but not via this client:
https://github.com/olivere/elastic
I read the docs and it says it uses the /_nodes/http api to connect. Now this is probably where I did something wrong because the response from that api looks like this:
{
"_nodes" : {
"total" : 1,
"successful" : 1,
"failed" : 0
},
"cluster_name" : "elasticsearch",
"nodes" : {
"u6TqFjAvRBa3_4FndfKh4w" : {
"name" : "u6TqFjA",
"transport_address" : "127.0.0.1:9300",
"host" : "127.0.0.1",
"ip" : "127.0.0.1",
"version" : "5.6.2",
"build_hash" : "57e20f3",
"roles" : [
"master",
"data",
"ingest"
],
"http" : {
"bound_address" : [
"[::1]:9200",
"127.0.0.1:9200"
],
"publish_address" : "127.0.0.1:9200",
"max_content_length_in_bytes" : 104857600
}
}
}
}
I'm guessing I have to set the IPs to my actual IP/domain (my domain is like es01.somedomain.com)
So how do i correctly configure elastisearch so that my go client can connect?
My config files for nginx look similar to this: https://www.elastic.co/blog/playing-http-tricks-nginx
Edit: I found a temporary solution by setting elastic.SetSniff(false) in the Options for the client, but I think that means I can't scale ES horizontally. So still looking for an alternative.
You are looking for the HTTP options, specifically http.publish_host and http.publish_port, which should be set to the publicly reachable address and port of the Nginx server proxying the ES node.
Note that with Elasticsearch listening on 127.0.0.1:9300 for the transport, you won't be able to form a cluster with nodes on other hosts. The transport can be configured similarly with the transport options.

Elasticsearch basics : transportclient or not?

I set up a graylog stack (graylog / ES/ Mongo) everything went smooth (well almost), yesterday I tried to get some info using the following command :
curl 'http://127.0.0.1:9200/_nodes/process?pretty'
{
"cluster_name" : "log_server_graylog",
"nodes" : {
"Znz_72SZSyikw6DEC4Wgzg" : {
"name" : "graylog-27274b66-3bbd-4975-99ee-1ee3d692c522",
"transport_address" : "127.0.0.1:9350",
"host" : "127.0.0.1",
"ip" : "127.0.0.1",
"version" : "2.4.4",
"build" : "fcbb46d",
"attributes" : {
"client" : "true",
"data" : "false",
"master" : "false"
},
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 788,
"mlockall" : false
}
},
"XO77zz8MRu-OOSymZbefLw" : {
"name" : "test",
"transport_address" : "127.0.0.1:9300",
"host" : "127.0.0.1",
"ip" : "127.0.0.1",
"version" : "2.4.4",
"build" : "fcbb46d",
"http_address" : "127.0.0.1:9200",
"process" : {
"refresh_interval_in_millis" : 1000,
"id" : 946,
"mlockall" : false
}
}
}
}
I does look like (to me at least that there is 2 nodes running, someone on the ES IRC told me that there might be a transport client running (which show up as a second node)...
I really don't understand why where this transport client comes from, also, the guy from IRC told me it used to be a common setup (using transport client) but this is discouraged now, how can I reverse the config to follow ES best practices ? (which I couldn't find on the docs)
FYI, my config file :
cat /etc/elasticsearch/elasticsearch.yml
cluster.name: log_server_graylog
node.name: test
path.data: /tt/elasticsearch/data
path.logs: /tt/elasticsearch/log
network.host: 127.0.0.1
action.destructive_requires_name: true
# Folowing are useless as we are defining swappiness to 1, this shloud prevent ES memeory space from being sawpped, unless emergency
#bootstrap.mlockall: true
#bootstrap.memory_lock: true
Thanks
I found the answer using the graylog IRC, the second client is the graylog client created by.... Graylog server :)
So everything is normal and as expected.

Resources