I have a lot of log files from JBoss Fuse that I want to visualize in Kibana.
I've installed Elasticsearch and Kibana.
I have also installed the plugin ingest-geoip (bin/elasticsearch-plugin install ingest-geoip).
Now I am trying to install Filebeat.
I've done this OK:
curl -L -O https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-6.2.2-darwin-x86_64.tar.gz
tar xzvf filebeat-6.2.2-darwin-x86_64.tar.gz
cd filebeat-6.2.2-darwin-x86_64/
But when I want to run it I get the following:
sindre#selite:/usr/lib/filebeat$ ./filebeat modules enable system
bash: ./filebeat: cannot execute binary file: Exec format error
NB! This is my first time using Kibana. Please point me in the right direction if I am using it wrong. As I wrote earlier I want to use it for Jboss Fuse Log files.
filebeat-6.2.2-darwin-x86_64
There's your clue. darwin is the name given to the core OS-X unix distribution
https://en.wikipedia.org/wiki/Darwin_(operating_system)
It is extremely unlikely that a compiled darwin binary would be compatible with a linux system.
You really want to be looking at the Linx X86 64 package
If you have running instance of Kibana on your system, you can easily configure it for any underlying operating system(Linux/macOS) with a few provided commands:
visit: Home>>Add data>>System logs
current_url_for_demo: http://localhost:5601/app/kibana#/home/tutorial/systemLogs?_g=()
Visual Explanation:
Related
I have followed the instructions in the documentation: https://docs.datastax.com/en/dsbulk/doc/dsbulk/install/dsbulkInstall.html
However, after doing the following:
curl -OL https://downloads.datastax.com/dsbulk/dsbulk-1.6.0.tar.gz
and
tar -xzvf dsbulk-1.6.0.tar.gz
inside an application directory, followed by the command
dsbulk --version
I get the output
Unable to find java 8 (or later) executable. Check JAVA_HOME and PATH environment variables.
What am I doing wrong here?
Im using an AWS ec2 t2.medium instance - do I have to install java on this in order for dsbulk to work?
Yes, DSBulk doesn’t include Java into it, so you need to install Java yourself - via apt, or whatever you use
We have a 3-node cluster with ElasticSearch 6.4.3 on Ubuntu 16.04. There is nothing existing outside of the fresh install of ES - no indexes, no Kibana, no Beats, no Logstash, etc.
I have been asked to downgrade to version 6.4.2. I have limited Linux experience, but enough to be able to run command line commands and understand the output. Google has lead me to bits and pieces around accomplishing this, but I'd feel a lot less anxiety around it if someone with ES experience may be able to point me to something that's a bit more step-by-step.
I do have this link to download 6.4.2, but one of the things I need to know is which file to download: https://www.elastic.co/downloads/past-releases/elasticsearch-6-4-2
Sure here you go with step by step guide, As I did this for you, using your version.
Using this link https://www.elastic.co/downloads/past-releases/elasticsearch-6-4-2, which you mentioned, download the tar file to your local system.
Use SCP to transfer the .tar file to your ubuntu instance, I used my AWS ubuntu instance.
scp -i ~/your-identity-file ~/Desktop/elasticsearch-6.4.2.tar.gz
ubuntu#aws-ec2-instance-ip:/home/ubuntu
Untar file using tar -xvf elasticsearch-6.4.2.tar.gz command.
Go to config folder like cd elasticsearch-6.4.2/config/ and set the proper values in elasticsearch.config.
Start the elasticsearch from bin folder ./elastic command.
Update:- Based on the chat with OP, Adding official ES link https://www.elastic.co/guide/en/elasticsearch/reference/current/targz.html and https://www.elastic.co/guide/en/elasticsearch/reference/current/install-elasticsearch.html for detailed instruction.
I am running PostgreSQL 9.6 on a Windows 7 laptop that is tightly managed by corporate IT. I do not have admin privileges on this laptop, but can do "many" things, including software installation, through elevated rights granted through BeyondTrust.
I use this database to import a number of medium-sized datasets from various sources and perform some in-depth analysis and reporting. These datasets have a crazy variation of encoding and formatting which creates challenges while importing them. For one dataset, I first import it into MySQL, strip offending characters from the data, export it back out to CSV, and finally import this cleaned-up data into PostgreSQL.
As an experiment, I installed PostgreSQL 9.6 and pg_loader on my home iMac running macOS 10.12, and it is able to import all of the datasets with no problem.
Given all this as a backdrop, I have a need to use pg_loader to import various datasets. However, the website does not offer either a Windows installer or instructions for compiling it on Windows.
Can anyone point me in the direction of one of the following, with the list going from most desirable to least desirable?
Windows installer of pg_loader
Windows installer of a comparable alternative to pg_loader
Detailed instructions for compiling pg_loader on Windows
Suggestions for using Python to clean up the data prior to import (caveat: I have very little experience with Python)
You can install the Windows Subsystem for Linux, install a Linux distro, then install pgloader with apt or other package manager -
$ sudo apt install pgloader
$ pgloader --version
pgloader version "3.4.1"
compiled with SBCL 1.3.3.debian
$ cd /mnt/c/Users/bburns/Desktop
$ pgloader ....
This solution may come a little late to the original asker, but for those still searching for a windows solution. I can confirm that I was able to get pgloader working using Docker for Windows.
My environment is a local MySQL install on a Windows 10 PC. I installed Docker for Windows. I then installed PostgreSQL via docker:
docker run --name some-postgres -p 5432:5432 -e POSTGRES_PASSWORD=mysecretpassword -d postgres
Finally, I ran pgloader via docker to pull the schema & data from mysql to postgres using the following:
docker run --rm --name pgloader dimitri/pgloader:latest pgloader
mysql://root:root#host.docker.internal/my_db_name
pgsql://postgres:mysecretpassword#host.docker.internal/postgres
This created a schema called "my_db_name" on my postgres instance. The "root:root" and "postgres:mysecretpassword" are my username:password details. This is obviously a production ready environment! The other useful bit "host.docker.internal" is docker's reference to the parent host environment. Also, I ignored the SBCL warning.
I'm still processing the results. Most everything looks like it migrated successfully. I'm missing triggers (unsupported) and some foreign keys (bug?). Anyways, I hope this might be helpful to others.
I am trying to learn about the ELK stack (ElasticSearch, Logstash and Kibana). In an effort to get started, I've been running through the "Getting Started" guide found at http://logstash.net/docs/1.4.0/tutorials/getting-started-with-logstash. I've been following these steps on a Mac OS X Mavericks installation. I did do one thing different, though.
After downloading the Logstash and Elasticsearch tarballs, I copied their contents into the following locations:
Logstash: /Applications/dev/logstash/1.4.0/
Elasticsearch: /Applications/dev/Elasticsearch/1.1.0
I then start Elasticsearch using the following command from a terminal:
sudo /Applications/dev/Elasticsearch/1.1.0/bin/elasticsearch
I then start a new terminal window. In that new terminal window, I start Logstash using the following command:
sudo /Applications/dev/logstash/1.4.0/bin/logstash -e 'input { stdin { } } output { elasticsearch { host => localhost } } '
Logstash appears to start just fine. If I enter 'you know, for logs' as mentioned in the getting started guide, I can see the results using POSTMAN. However, I'm stuck on Kibana.
I've downloaded Kibana using the following command:
git clone https://github.com/elasticsearch/kibana.git kibana
I've placed the downloaded files into /Applications/dev/kibana/3.0.0. I'm now stuck on how to actually bring Kibana up in a web browser. I know that somehow I need to serve these files. I see server stuff in this sample. However, I'm not sure if
There is a recommended web server for Kibana
How to setup a a web site on a Mac OS X machine. If I were on Windows, I'd just use IIS and point it at my directory. I'm sure there's something similar on Mac OS X. However, I'm not sure what that is.
Thank you!
While downloading the packages is perfectly fine, your install can be much easier on OS X if you use homebrew. (http://brew.sh)
Simply run:
brew install elasticsearch
brew install logstash
And if you are missing the proper Java JDK or JRE files, homebrew will tell you where to get them from.
If you are running the OS X Server App (see App Store) and have websites service enabled you can also simply drop the Kibana files into the default website directory:
/Library/Server/Web/Data/Sites/Default/
or a subdirectory like:
/Library/Server/Web/Data/Sites/Default/kibana
Hope this helps!
Kibana 3
Kibana is a html/Js website. So, you only need to prepare a web server for it! For example, an apache server. Mavericks has already install apache server. After you start apache server, put the kibana folder to the apache document root. Then you can visit Kibana by
http://localhost/kibana
Remember to config the the elasticsearch server in config.js.
Update:
Kibana 4 has buildin web server. After you start your Kibana, you can directly visit your page at : http://localhost:5601
I updated to the latest Java Version by downloading JDK Here: http://www.oracle.com/technetwork/java/javase/downloads/index.html
Once Java is installed, I installed via brew:
brew install elasticsearch
brew install logstash
Download the Kibana binary here for mac: http://www.elastic.co/downloads/kibana
Extract and run the bin/kibana executable and point your browser to localhost:5601
No need to download any third party servers.
Cheers!
UPDATE: Instead of step 3) & 4) You can also install kibana via brew: brew install kibana
I am running mac osx. I installed elasticsearch with "pip install elasticsearch". I cannot start the elasticsearch server. None of the start commands I tried work. For example, "bin/elasticsearch -f" resulted in "No such file or directory". Please help.
What is the version of elasticsearch you are using and from where you have downloaded it?
On my Mac OS I have downloaded the zip file from elasticsearch download website.
Unzipped it on my machine.
and Started the ES using "./bin/elasticsearch -f" command.
Check that you have permission on this files in bin directory.
I solved the problem. Basically using "pip install elasticsearch" only allows you to use elasticsearch directly in python (my theory). So I installed elasticsearch a different way and it worked!
Procedure:
a) brew install elasticsearch
b) elasticsearch -f -D es.config=/usr/local/opt/elasticsearch/config/elasticsearch.yml
For mac if installed using homebrew
Navigates to /usr/local/Cellar/elasticsearch/6.7.0(your version)/libexec/config/elasticsearch.yml
and from below mentioned block
Use a descriptive name for your cluster:
change cluster name as show below
cluster.name: elasticsearch_your_system_name >>> cluster.name:elaticsearch