I need to get information about all indicies created before ES5.0.
i try to get info with:
curl -XGET 'localhost:9200/_cat/indices?v&pretty'
but in the response I don't have ES version of index. I planing migration to ES6.7 and I want to reindex old indexes.
for getting the version on the index you can use the following curl
curl -XGET {host}:{port}/{index}/_settings?pretty
your required information will be in "version" key, for my index which was created with ES 6.5 -
"version" : {
"created" : "6050499"
}
for getting information on all the indices you can drop the {index} part and your curl would look like -
curl -XGET {host}:{port}/_settings?pretty
Related
I've updated ElasticSearch to 2.1 version.
Before of that, I deleted every document in a type using:
curl -XDELETE '<server_node>:<port>/<index>/<type>' -i
curl -XDELETE 'http://ESNode01:9201/living/inputs' -i
From now then, this is the response:
HTTP/1.1 400 Bad Request Content-Type: text/plain; charset=UTF-8
Content-Length: 61
No handler found for uri [/living/inputs] and method [DELETE]
What am I doing wrong?
Deleting a mapping type is not supported anymore since 2.0.
If you just need to delete the documents, then you may use the delete by query plugin, otherwise you should create a new index without the mapping you want to delete and reindex your data.
I wanted to try Elasticsearch with Polish language support, but I have some problems with it.
I installed Stempel Analysis Plugin, I'm trying to create an index that uses Polish analyzer:
curl -XPUT localhost:9200/polisz -d '{
"mappings" : {
"_default_" : {
"properties" : {
"text_entry" : { "type": "string", "analyzer": "polish" }
}
}
}
}
'
But I get an error about not recognized analyzer:
{
"status" : 400,
"error" : "MapperParsingException[mapping [_default_]]; nested: MapperParsingException[Analyzer [polish] not found for field [text_entry]]; "
}
Should I do anything after installing the plugin and rebooting ES?
I can't find any specific instructions about using the plugin so maybe I'm just doing something obviously wrong?
Some more details on how I set up my environment:
I installed and run docker image with ES and kibana by commands:
docker pull minimum2scp/es-kibana
docker run -d -p 8080:80 -p 9200:9200 --name es minimum2scp/es-kibana
I installed the Stempel plugin by command:
host$ docker exec -it es bash
root#docker-es:/# /usr/share/elasticsearch/bin/plugin install elasticsearch/elasticsearch-analysis-stempel/2.4.2
Then I rebooted elasticsearch, by:
root#docker-es:/# service elasticsearch restart
I'll be grateful for any help!
Krzysztof
OK, I got it. It seems that my plugin didn't install correctly. Even that plugin install command doesn't return any errors, neither elasticsearch restart command, there was a Lucene version mismatch in Elasticsearch( I don't remember, but below 4.10.2) and the plugin (4.10.3).
It was enough to look into elasticsearch.log file to find it out...My bad.
BUT there is more to it: I switched to the most popular (by stars) elasticsearch docker image, which is: dockerfile/elasticsearch. It has the ES version 1.4.2 that is based on Lucene 4.10.2, still mismatching the plugin Lucene 4.10.3. That causes an error even though authors of plugin states it plugin in 2.4.2 (current stable) support 1.4 ES version(s).
Citing an error for future web searching the problem:
[2015-02-13 10:57:11,850][INFO ][node ] [Necromantra] version[1.4.2], pid[1], build[927caff/2014-12-16T14:11:12Z]
[2015-02-13 10:57:11,851][INFO ][node ] [Necromantra] initializing ...
[2015-02-13 10:57:11,884][ERROR][plugins ] [Necromantra] cannot start plugin due to incorrect Lucene version: plugin [4.10.3], node [4.10.2].
[2015-02-13 10:57:11,884][WARN ][plugins ] [Necromantra] failed to load plugin from [jar:file:/data/plugins/analysis-stempel/elasticsearch-analysi
s-stempel-2.4.2.jar!/es-plugin.properties]
Now I chose a path to downgrade the plugin to 2.4.1, which agreed with my ES 1.4.2. Although in the long term I would look for docker image that has 1.4.3 ES which, hopefully,upgraded Lucene version as well.
Dadoonet, thank you for having a closer look on my problem.
I want to see all queries executed against an elasticsearch instance. Is it possible to run elasticsearch in a debug mode, or to tell it to store all queries executed against it?
The purpose is to see which queries are launched from a software using elasticsearch for analysis.
In versions of ElasticSearch prior to 5, you can accomplish this by changing the ElasticSearch.yml configuration file. At the very bottom of this file, you can adjust the logging time to record all:
index.search.slowlog.threshold.query.warn: 10s
index.search.slowlog.threshold.query.info: 5s
index.search.slowlog.threshold.query.debug: 2s
index.search.slowlog.threshold.query.trace: 500ms
index.search.slowlog.threshold.fetch.warn: 1s
index.search.slowlog.threshold.fetch.info: 800ms
index.search.slowlog.threshold.fetch.debug: 500ms
index.search.slowlog.threshold.fetch.trace: 200ms
index.indexing.slowlog.threshold.index.warn: 10s
index.indexing.slowlog.threshold.index.info: 5s
index.indexing.slowlog.threshold.index.debug: 2s
index.indexing.slowlog.threshold.index.trace: 500ms
Adjust the settings and restart your node, then consulting the logs to view the queries executed against your node. Note if in production log files will rapidly increase in size.
In version 5.x, you have to set slow log logging per index.
Command line:
curl -XPUT 'http://localhost:9200/myindexname/_settings' -d '{
"index.indexing.slowlog.threshold.index.debug" : "0s",
"index.search.slowlog.threshold.fetch.debug" : "0s",
"index.search.slowlog.threshold.query.debug" : "0s"
}'
Or, if you are using Kibana, go to the Dev Tools bar and enter:
PUT /myindexname/_settings
{"index.indexing.slowlog.threshold.index.debug": "0s",
"index.search.slowlog.threshold.fetch.debug" : "0s",
"index.search.slowlog.threshold.query.debug": "0s"}
#1: Apply to ALL indices
You can apply the setting to ALL indices with the following command:
PUT /_all/_settings
{"index.indexing.slowlog.threshold.index.debug": "0s",
"index.search.slowlog.threshold.fetch.debug" : "0s",
"index.search.slowlog.threshold.query.debug": "0s"}
#2: Preserve existing settings
If you don't want to overwrite existing settings, but just add new ones, add '''preserve_existing=true''' after _settings, like this:
PUT /_all/_settings?preserve_existing=true
{"index.indexing.slowlog.threshold.index.debug": "0s",
"index.search.slowlog.threshold.fetch.debug" : "0s",
"index.search.slowlog.threshold.query.debug": "0s"}
The above request will ONLY add the settings if they don't exist. It will not change them if they are already there.
#3: All available log settings
All available slow log settings are here and below for your reference:
PUT /test_index/_settings
{
"index.search.slowlog.threshold.query.warn": "60s",
"index.search.slowlog.threshold.query.info": "5s",
"index.search.slowlog.threshold.query.debug": "1s",
"index.search.slowlog.threshold.query.trace": "0.1s",
"index.search.slowlog.threshold.fetch.warn": "30s",
"index.search.slowlog.threshold.fetch.info": "5s",
"index.search.slowlog.threshold.fetch.debug": "1s",
"index.search.slowlog.threshold.fetch.trace": "0.1s",
"index.indexing.slowlog.threshold.index.warn": "6s",
"index.indexing.slowlog.threshold.index.info": "5s",
"index.indexing.slowlog.threshold.index.debug": "1s",
"index.indexing.slowlog.threshold.index.trace": "0.1s",
"index.indexing.slowlog.level": "info",
"index.indexing.slowlog.source": "1000"
}
Starting with Version 5 ElasticSearch charges money for this functionality. It's called "Audit log" and is now part of X-Pack. There is a basic license available that is free, but this license only gives you a simplistic monitoring functionality. Authentication, query logging and all these rather basic things cost money now.
Yes, it's possible to tell Elasticsearch to log all queries executed against it and you can configure logging levels, such as DEBUG. You can change it in ES 7.13.x using curl:
curl -X PUT "localhost:9200/_cluster/settings?pretty" -H 'Content-Type: application/json' -d'
{
"transient": {
"logger.org.elasticsearch.discovery": "DEBUG"
}
}
'
On macOS log files are stored on $ES_HOME by default. Please check the docs about Elasticsearch Logging
When I'm trying to connect to ElasticSearch (elasticsearch-0.90.3) installed on EC2 from a none local machine using play2-elastic plugin it throws the following exception (the plugin works fine when connecting locally)
error] application - ElasticSearch : No ElasticSearch node is available. Please check that your configuration is correct, that you ES server is up and reachable from the network. Index has not been created and prepared.
org.elasticsearch.client.transport.NoNodeAvailableException: No node available
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:205) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:85) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.client.support.AbstractIndicesAdminClient.exists(AbstractIndicesAdminClient.java:147) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:43) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59) ~[elasticsearch-0.90.3.jar:na]
I have used different methods to test the elasticsearch server is up and running, examples:
curl -XGET '184.72.55.204:9300/_analyze?analyzer=standard' -d 'this is a test'
curl: (52) Empty reply from server
telnet 184.72.55.204 9300
Trying 184.72.55.204...
Connected to ec2-184-72-55-204.us-west-1.compute.amazonaws.com.
Escape character is '^]'.
In some google groups I also saw other people having similar problem, they seem to be able to fix the problem with turning sniffing to off, so I have this in my application.conf
elasticsearch.client="184.72.55.204:9300"
elasticsearch.sniff=false # I ADDED THIS BUT DID NOT HELP
elasticsearch.index.name="phonotags"
elasticsearch.index.settings="{ analysis: { analyzer: { my_analyzer: { type: \"custom\", tokenizer: \"standard\" } } } }"
elasticsearch.index.clazzs="indexing.*"
elasticsearch.index.show_request=true
my build.scala file contains these:
"com.clever-age" % "play2-elasticsearch" % "0.7-SNAPSHOT"
resolvers += Resolver.url("play-plugin-releases", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/"))(Resolver.ivyStylePatterns),
resolvers += Resolver.url("play-plugin-snapshots", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/"))(Resolver.ivyStylePatterns)
I appreciate your help.
thanks
It seems your node is not available
curl -XPUT '184.72.55.204:9200/twitter/tweet/1' -d '{ "user": "kimchy", "post_date" : "2011-08-18T16:20:00", "message" : "trying out Elastic Search" }'
Can you check this ?
I am following this tutorial to learn Elasticsearch, running it on EC2:
http://exploringelasticsearch.com/book/an-overview/up-and-running.html
So I am able to run the server and, using the query tool provided (http://elastichammer.exploringelasticsearch.com/) I can get the status response. However, when I try creating an index with a PUT, I get back the error:
{ "error": "IndexCreationException[[planet] failed to create index];
nested: NoClassDefFoundError[Could not initialize class
org.elasticsearch.index.codec.postingsformat.PostingFormats]; ",
"status": 500 }
The same thing happens when I use cURL...I can get the sample response with
curl -XGET http://ec2-54-218-36-27.us-west-2.compute.amazonawscom:9200/
as well as the status with:
curl -XGET http://ec2-54-218-36-27.us-west-2.compute.amazonawscom:9200/_status
Has anybody experienced this?
Thank you!