I am using hadoop2.7.2 on ubuntu 16.04 LTS and developed a single node setup. I am running Rumen to create job traces from log history.
I am using following command:
/usr/local/hadoop/bin/hadoopjar/usr/local/hadoop/share/hadoop/tools/lib/hadoop-rumen-2.7.2.jar org.apache.hadoop.tools.rumen.TraceBuilder
file:///usr/local/hadoop/share/hadoop/tools/sls/sample-data/job-trace.json
file:///usr/local/hadoop/share/hadoop/tools/sls/sample-data/job-topology.json
hdfs:///usr/local/hadoop/logs/userlogs/application_1486821591386_0001
But it throws an error "WARN rumen.TraceBuilder: no job found in traces" and same time it successfully create folders 'job-trace.json' and 'job-topology.json'. But job-trace.json is empty and job-topology.json has some values look like:
{
"name" : "root"
"children" : [ ]
}
I don't know why is this happening. I have already searched everywhere but not getting anything.
I solved the problem... I was giving wrong input path to Rumen. The actual path should be like this:
hdfs:///tmp/hadoop-yarn/staging/history/done/2017/03/11/000000
Currently I am getting these alerts:
Upgrade Required Your version of Elasticsearch is too old. Kibana requires Elasticsearch 0.90.9 or above.
Can someone tell me if there is a way I can find the exact installed version of ELS?
from the Chrome Rest client make a GET request or
curl -XGET 'http://localhost:9200' in console
rest client: http://localhost:9200
{
"name": "node",
"cluster_name": "elasticsearch-cluster",
"version": {
"number": "2.3.4",
"build_hash": "dcxbgvzdfbbhfxbhx",
"build_timestamp": "2016-06-30T11:24:31Z",
"build_snapshot": false,
"lucene_version": "5.5.0"
},
"tagline": "You Know, for Search"
}
where number field denotes the elasticsearch version. Here elasticsearch version is 2.3.4
I would like to add which isn't mentioned in above answers.
From your kibana's dev console, hit following command:
GET /
This is similar to accessing localhost:9200 from browser.
Hope this will help someone.
You can check version of ElasticSearch by the following command. It returns some other information also:
curl -XGET 'localhost:9200'
{
"name" : "Forgotten One",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "2.3.4",
"build_hash" : "e455fd0c13dceca8dbbdbb1665d068ae55dabe3f",
"build_timestamp" : "2016-06-30T11:24:31Z",
"build_snapshot" : false,
"lucene_version" : "5.5.0"
},
"tagline" : "You Know, for Search"
}
Here you can see the version number: 2.3.4
Typically Kibana is installed in /opt/logstash/bin/kibana . So you can get the kibana version as follows
/opt/kibana/bin/kibana --version
navigate to the folder where you have installed your kibana
if you have used yum to install kibana it will be placed in following location by default
/usr/share/kibana
then use the following command
bin/kibana --version
To check Version of Your Running Kibana,Try this:
Step1. Start your Kibana Service.
Step2. Open Browser and Type below line,
localhost:5601
Step3. Go to settings->About
You can See Version of Your Running kibana.
Another way to do it on Ubuntu 18.0.4
sudo /usr/share/kibana/bin/kibana --version
You can use the Dev Tools console in Kibana to obtain version information about Elasticsearch.
You click "Dev Tools" to navigate into console.
In the Dev Tools Console, you do a below query
GET /
You will see version and number like below with other details also.
{
"version" : {
"number" : "6.5.1",
...
}
}
You can Try this,
After starting Service of elasticsearch Type below line in your browser.
localhost:9200
It will give Output Something like that,
{
"status" : 200,
"name" : "Hypnotia",
"cluster_name" : "elasticsearch",
"version" : {
"number" : "1.7.1",
"build_hash" : "b88f43fc40b0bcd7f173a1f9ee2e97816de80b19",
"build_timestamp" : "2015-07-29T09:54:16Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
If you have installed x-pack to secure elasticseach, the request should contains the valid credential details.
curl -XGET -u "elastic:passwordForElasticUser" 'localhost:9200'
Infact, if the security enabled all the subsequent requests should follow the same pattern (inline credentials should be provided).
If you are logged into your Kibana, you can click on the Management tab and that will show your Kibana version. Alternatively, you can click on the small tube-like icon and that will show the version number.
From Kibana host, a request to http://localhost:9200/ will not be answered, unless ElasticSearch is also running on the same node. Kibana listens on port 5601 not 9200.
In most cases, except for DEV, ElasticSearch will not be on the same node as Kibana, for a number of reasons.
Therefore, to get information about your ElasticSearch from Kibana, you should select the "Dev Tools" tab on the left and in the console issue the command: GET /
If you looking for version in kibana ui
I have successfully installed both the license plugin and the shield plugin on my client nodes. The logs show it starting correctly, and i am able to authenticate using the credentials is supplied. However when i connect i am getting a 503 error. I went back through the docs to see if i missed something, but i don't see anything about configuring the data nodes after enabling shield. What am i missing?
{
"status" : 503,
"name" : "Vertigo",
"cluster_name" : "cluster01",
"version" : {
"number" : "1.7.2",
"build_hash" : "e43676b1385b8125d647f593f7202acbd816e8ec",
"build_timestamp" : "2015-09-14T09:49:53Z",
"build_snapshot" : false,
"lucene_version" : "4.10.4"
},
"tagline" : "You Know, for Search"
}
From the client logs
2015-10-28 03:14:52,235][INFO ][io.fabric8.elasticsearch.discovery.k8s.K8sDiscovery] [Vertigo] failed to send join request to master [[Abominatrix][T6zFRQO7RG-thZmOWVk2Xw][es-master-e6mj9][inet[/10.244.85.2:9300]]{data=false, master=true}], reason [RemoteTransportException[[Abominatrix][inet[/10.244.85.2:9300]][internal:discovery/zen/join]]; nested: RemoteTransportException[Failed to deserialize exception response from stream]; nested: TransportSerializationException[Failed to deserialize exception response from stream]; nested: InvalidClassException[failed to read class descriptor]; nested: ClassNotFoundException[org.elasticsearch.shield.authc.AuthenticationException]; ]
Andrei,
I figured it out. Since i am running containers that separate the master, data, and client nodes, i had only installed the plugin on the client nodes. Once installed the plugin on the master and data nodes, uploaded the image to docker hub and rebuilt the cluster, it all started working.
Thanks
-winn
I am trying to import a mysql table from mysql server to elasticsearch on my MAC OSX Mavericks.
I have installed elasticsearch 1.3.1 with homebrew
Installed jdbc-river 1.3.0.4 with elasticsearch plugin --install
Installed jdk 1.7.0_67
Downloaded mysql-connector-java-5.1.28-bin.jar into
$ES_HOME/plugins/jdbc (I had to create the folders 'plugins' and
'jdbc' myself) and gave chmod 777 permission for the .jar file.
Then I ran ./bin/elasticsearch and called this command in postman in order to create a river:
PUT request.
URL: localhost:9200/_river/my_jdbc_river/_meta
Raw data:
{
"type" : "jdbc",
"jdbc" : {
"url" : "jdbc:mysql://localhost:3306/<databaseName>",
"user" : "<MysqlUserName>",
"password" : "<MysqlUserPass",
"sql" : "select * from <TableName>"
}
}
And I received the following error in the elasticsearch log in the terminal:
[2014-08-26 15:38:39,300][ERROR][org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource] while opening read connection: jdbc:mysql://localhost:3306/xcollector No suitable driver found for jdbc:mysql://localhost:3306/xcollector
java.sql.SQLException: No suitable driver found for jdbc:mysql://localhost:3306/xcollector
at java.sql.DriverManager.getConnection(DriverManager.java:596)
at java.sql.DriverManager.getConnection(DriverManager.java:215)
at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.getConnectionForReading(SimpleRiverSource.java:196)
at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.execute(SimpleRiverSource.java:315)
at org.xbib.elasticsearch.river.jdbc.strategy.simple.SimpleRiverSource.fetch(SimpleRiverSource.java:287)
at org.xbib.elasticsearch.plugin.feeder.jdbc.JDBCFeeder.fetch(JDBCFeeder.java:335)
at org.xbib.elasticsearch.plugin.feeder.jdbc.JDBCFeeder.executeTask(JDBCFeeder.java:179)
at org.xbib.elasticsearch.plugin.feeder.AbstractFeeder.newRequest(AbstractFeeder.java:362)
at org.xbib.elasticsearch.plugin.feeder.AbstractFeeder.newRequest(AbstractFeeder.java:53)
at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:87)
at org.xbib.pipeline.AbstractPipeline.call(AbstractPipeline.java:14)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
I have followed many posts in google to try and find the cause of the problem, does anyone have any idea what I am missing?
And if anyone knows if this procedure can be more automatic to deploy like having some kind of package manager (like npm for node.js).
Thanks in advance,
So the clue to your problem is that you had to create the folders "plugins" and "jdbc" in step 4. Both of those folders are created when you install the mysql river plugin. I can see that the mysql river plugin installed correctly in your error message - it's running but it is unable to find the jdbc driver.
Look on your drive for the correct folder - $ES_HOME should have the following folders in it:
bin
config
data
lib
logs
plugins
If it does not then $ES_HOME is set incorrectly. Copy your jdbc driver as directed into the correct folder and you should be able to resolve this problem.
I'm trying to make the new relic elastic search plugin work, the following is the website of the author:
New Relic Plug in Github
I've followed all the instructions and even see that the plugin has been installed succesfuly when I run in my server the command
'curl -XGET http://localhost:9200/_newrelic?pretty'
, I even get the response that should confirm everything has installed just fine:
' {
"configuration" : {
"agents" : {
"http" : true,
"pool" : true,
"transport" : true,
"fs" : true,
"indices" : true,
"network" : true
},
"refreshInterval" : 10
}
}'
However, when I log to my newrelic account not a single statistic is shown, either for JVM or whatsoever, nor I see the indices being monitored.
Has anyone encountered this problem before? PS: ElasticSearch is not running as a service and I DO have the new relic java agent properly installed. Also I have configured bin/elasticsearch.in.sh as it should be.
Thanks in advance!
JM.