How to use custom analyzer in ElasticSearch? - elasticsearch

I wanted to try Elasticsearch with Polish language support, but I have some problems with it.
I installed Stempel Analysis Plugin, I'm trying to create an index that uses Polish analyzer:
curl -XPUT localhost:9200/polisz -d '{
"mappings" : {
"_default_" : {
"properties" : {
"text_entry" : { "type": "string", "analyzer": "polish" }
}
}
}
}
'
But I get an error about not recognized analyzer:
{
"status" : 400,
"error" : "MapperParsingException[mapping [_default_]]; nested: MapperParsingException[Analyzer [polish] not found for field [text_entry]]; "
}
Should I do anything after installing the plugin and rebooting ES?
I can't find any specific instructions about using the plugin so maybe I'm just doing something obviously wrong?
Some more details on how I set up my environment:
I installed and run docker image with ES and kibana by commands:
docker pull minimum2scp/es-kibana
docker run -d -p 8080:80 -p 9200:9200 --name es minimum2scp/es-kibana
I installed the Stempel plugin by command:
host$ docker exec -it es bash
root#docker-es:/# /usr/share/elasticsearch/bin/plugin install elasticsearch/elasticsearch-analysis-stempel/2.4.2
Then I rebooted elasticsearch, by:
root#docker-es:/# service elasticsearch restart
I'll be grateful for any help!
Krzysztof

OK, I got it. It seems that my plugin didn't install correctly. Even that plugin install command doesn't return any errors, neither elasticsearch restart command, there was a Lucene version mismatch in Elasticsearch( I don't remember, but below 4.10.2) and the plugin (4.10.3).
It was enough to look into elasticsearch.log file to find it out...My bad.
BUT there is more to it: I switched to the most popular (by stars) elasticsearch docker image, which is: dockerfile/elasticsearch. It has the ES version 1.4.2 that is based on Lucene 4.10.2, still mismatching the plugin Lucene 4.10.3. That causes an error even though authors of plugin states it plugin in 2.4.2 (current stable) support 1.4 ES version(s).
Citing an error for future web searching the problem:
[2015-02-13 10:57:11,850][INFO ][node ] [Necromantra] version[1.4.2], pid[1], build[927caff/2014-12-16T14:11:12Z]
[2015-02-13 10:57:11,851][INFO ][node ] [Necromantra] initializing ...
[2015-02-13 10:57:11,884][ERROR][plugins ] [Necromantra] cannot start plugin due to incorrect Lucene version: plugin [4.10.3], node [4.10.2].
[2015-02-13 10:57:11,884][WARN ][plugins ] [Necromantra] failed to load plugin from [jar:file:/data/plugins/analysis-stempel/elasticsearch-analysi
s-stempel-2.4.2.jar!/es-plugin.properties]
Now I chose a path to downgrade the plugin to 2.4.1, which agreed with my ES 1.4.2. Although in the long term I would look for docker image that has 1.4.3 ES which, hopefully,upgraded Lucene version as well.
Dadoonet, thank you for having a closer look on my problem.

Related

elasticsearch 7.9.0 jarhell lucene50.Lucene50StoredFieldsFormat$1 debian 10 cannot start

Any help on finding the issue with this.
I manually installed elasticsearch 7.9.0 (https://www.elastic.co/guide/en/elasticsearch/reference/current/deb.html)
I had to add lucene-backward-codecs-8.7.0.jar in /usr/share/elasticsearch/lib or I get the error below.
[2022-01-22T21:25:39,352][ERROR][o.e.b.Bootstrap ] [guest] Exception
java.lang.IllegalStateException: jar hell!
class: org.apache.lucene.codecs.lucene50.Lucene50StoredFieldsFormat$1
jar1: /usr/share/elasticsearch/lib/lucene-backward-codecs-8.7.0.jar
jar2: /usr/share/elasticsearch/lib/lucene-core-8.6.0.jar
before and if I remove the lucene-backward-codecs-8.7.0.jar
[2022-01-22T21:32:05,863][ERROR][o.e.b.Bootstrap ] [guest] Exception
java.lang.IllegalArgumentException: Could not load codec 'Lucene87'. Did you forget to add lucene-backward-codecs.jar?
Just encountered this exact issue while running elasticsearch in a docker container. I resolved it by removing all containers/images related to elasticsearch and repulling them.
I didn't see you mention Docker anywhere in your post, so I'm not sure how relevant this will be, but I figured I'd toss it out there just incase.

How to know elastic search installed version from kibana?

Currently I am getting these alerts:
Upgrade Required Your version of Elasticsearch is too old. Kibana requires Elasticsearch 0.90.9 or above.
Can someone tell me if there is a way I can find the exact installed version of ELS?
from the Chrome Rest client make a GET request or
curl -XGET 'http://localhost:9200' in console
rest client: http://localhost:9200
{
"name": "node",
"cluster_name": "elasticsearch-cluster",
"version": {
"number": "2.3.4",
"build_hash": "dcxbgvzdfbbhfxbhx",
"build_timestamp": "2016-06-30T11:24:31Z",
"build_snapshot": false,
"lucene_version": "5.5.0"
},
"tagline": "You Know, for Search"
}
where number field denotes the elasticsearch version. Here elasticsearch version is 2.3.4
I would like to add which isn't mentioned in above answers.
From your kibana's dev console, hit following command:
GET /
This is similar to accessing localhost:9200 from browser.
Hope this will help someone.
You can check version of ElasticSearch by the following command. It returns some other information also:
curl -XGET 'localhost:9200'
{
"name" : "Forgotten One",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.3.4",
"build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
"build_timestamp" : "2016-06-30T11:24:31Z",
"build_snapshot" : false,
"lucene_version" : "5.5.0"
},
"tagline" : "You Know, for Search"
}
Here you can see the version number: 2.3.4
Typically Kibana is installed in /opt/logstash/bin/kibana . So you can get the kibana version as follows
/opt/kibana/bin/kibana --version
navigate to the folder where you have installed your kibana
if you have used yum to install kibana it will be placed in following location by default
/usr/share/kibana
then use the following command
bin/kibana --version
To check Version of Your Running Kibana,Try this:
Step1. Start your Kibana Service.
Step2. Open Browser and Type below line,
localhost:5601
Step3. Go to settings->About
You can See Version of Your Running kibana.
Another way to do it on Ubuntu 18.0.4
sudo /usr/share/kibana/bin/kibana --version
You can use the Dev Tools console in Kibana to obtain version information about Elasticsearch.
You click "Dev Tools" to navigate into console.
In the Dev Tools Console, you do a below query
GET /
You will see version and number like below with other details also.
{
"version" : {
"number" : "6.5.1",
...
}
}
You can Try this,
After starting Service of elasticsearch Type below line in your browser.
localhost:9200
It will give Output Something like that,
{
"status" : 200,
"name" : "Hypnotia",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "1.7.1",
"build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
"build_timestamp" : "2015-07-29T09:54:16Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
If you have installed x-pack to secure elasticseach, the request should contains the valid credential details.
curl -XGET -u "elastic:passwordForElasticUser" 'localhost:9200'
Infact, if the security enabled all the subsequent requests should follow the same pattern (inline credentials should be provided).
If you are logged into your Kibana, you can click on the Management tab and that will show your Kibana version. Alternatively, you can click on the small tube-like icon and that will show the version number.
From Kibana host, a request to http://localhost:9200/ will not be answered, unless ElasticSearch is also running on the same node. Kibana listens on port 5601 not 9200.
In most cases, except for DEV, ElasticSearch will not be on the same node as Kibana, for a number of reasons.
Therefore, to get information about your ElasticSearch from Kibana, you should select the "Dev Tools" tab on the left and in the console issue the command: GET /
If you looking for version in kibana ui

How to enable dynamic scripting in ElasticSearch with its gem?

I'm getting the error nested: ScriptException[dynamic scripting for [groovy] disabled]; because of this aggregation I'm making:
agg :category_aggregation do
{
terms: {
script: "doc['categories.id'].value + '|' + doc['categories.name'].value",
size: 30
}
}
end
I'm using the official elasticsearch gem and also tried with chewy but couldn't find how to enable the dynamic_search anywhere.
ElasticSearch version in my OS X: 1.5.2 installed with homebrew.
Dynamic scripting can only be enabled from the elasticsearch.yml configuration file in your ES cluster.
Add this to the file on every node and restart your cluster:
script.disable_dynamic: false
UPDATE
Since you've installed ES via homebrew, you can find the elasticsearch.yml file in /usr/local/Cellar/elasticsearch/1.5.2/config

ElasticSearch plugin for New Relic not showing metrics

I'm trying to make the new relic elastic search plugin work, the following is the website of the author:
New Relic Plug in Github
I've followed all the instructions and even see that the plugin has been installed succesfuly when I run in my server the command
'curl -XGET http://localhost:9200/_newrelic?pretty'
, I even get the response that should confirm everything has installed just fine:
' {
"configuration" : {
"agents" : {
"http" : true,
"pool" : true,
"transport" : true,
"fs" : true,
"indices" : true,
"network" : true
},
"refreshInterval" : 10
}
}'
However, when I log to my newrelic account not a single statistic is shown, either for JVM or whatsoever, nor I see the indices being monitored.
Has anyone encountered this problem before? PS: ElasticSearch is not running as a service and I DO have the new relic java agent properly installed. Also I have configured bin/elasticsearch.in.sh as it should be.
Thanks in advance!
JM.

play2-elastic does not work when ElasticSearch is installed on EC2 server

When I'm trying to connect to ElasticSearch (elasticsearch-0.90.3) installed on EC2 from a none local machine using play2-elastic plugin it throws the following exception (the plugin works fine when connecting locally)
error] application - ElasticSearch : No ElasticSearch node is available. Please check that your configuration is correct, that you ES server is up and reachable from the network. Index has not been created and prepared.
org.elasticsearch.client.transport.NoNodeAvailableException: No node available
at org.elasticsearch.client.transport.TransportClientNodesService.execute(TransportClientNodesService.java:205) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.client.transport.support.InternalTransportIndicesAdminClient.execute(InternalTransportIndicesAdminClient.java:85) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.client.support.AbstractIndicesAdminClient.exists(AbstractIndicesAdminClient.java:147) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.admin.indices.exists.indices.IndicesExistsRequestBuilder.doExecute(IndicesExistsRequestBuilder.java:43) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:85) ~[elasticsearch-0.90.3.jar:na]
at org.elasticsearch.action.ActionRequestBuilder.execute(ActionRequestBuilder.java:59) ~[elasticsearch-0.90.3.jar:na]
I have used different methods to test the elasticsearch server is up and running, examples:
curl -XGET '184.72.55.204:9300/_analyze?analyzer=standard' -d 'this is a test'
curl: (52) Empty reply from server
telnet 184.72.55.204 9300
Trying 184.72.55.204...
Connected to ec2-184-72-55-204.us-west-1.compute.amazonaws.com.
Escape character is '^]'.
In some google groups I also saw other people having similar problem, they seem to be able to fix the problem with turning sniffing to off, so I have this in my application.conf
elasticsearch.client="184.72.55.204:9300"
elasticsearch.sniff=false # I ADDED THIS BUT DID NOT HELP
elasticsearch.index.name="phonotags"
elasticsearch.index.settings="{ analysis: { analyzer: { my_analyzer: { type: \"custom\", tokenizer: \"standard\" } } } }"
elasticsearch.index.clazzs="indexing.*"
elasticsearch.index.show_request=true
my build.scala file contains these:
"com.clever-age" % "play2-elasticsearch" % "0.7-SNAPSHOT"
resolvers += Resolver.url("play-plugin-releases", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-releases/"))(Resolver.ivyStylePatterns),
resolvers += Resolver.url("play-plugin-snapshots", new URL("http://repo.scala-sbt.org/scalasbt/sbt-plugin-snapshots/"))(Resolver.ivyStylePatterns)
I appreciate your help.
thanks
It seems your node is not available
curl -XPUT '184.72.55.204:9200/twitter/tweet/1' -d '{ "user": "kimchy", "post_date" : "2011-08-18T16:20:00", "message" : "trying out Elastic Search" }'
Can you check this ?

Resources