I'd like to get an Elkstack running on my virtual machine. I read up on the topic already a while ago but just recently figured out how to use docker-compose. I found a complete compose file on github (https://github.com/deviantony/docker-elk) which is capable of composing the whole stack, however I ran into two questions: Kibana is awaiting input from Elasticsearch on the url "http://elasticsearch:9200" - The url is obviously not resolving to anything - Hows that supposed to work? I made changes to it so it tries to connect to localhost:9200 but Elasticsearch is refusing the connection - I checked the running containers and saw that elasticsearch is indeed running but using "lsof -i :9200" (or 9300) did not show anything which means it is not listening on the port right? Or would it now show up due to the fact that it is running in docker?
Thanks in advance.
Related
I need to add Solr to a DDEV project but am encountering errors when attempting to gather information about available services.
I'm following guidance here:
https://ddev.readthedocs.io/en/stable/users/extend/additional-services/
When I attempt to list all available services: ddev get --list, I receive this response after approx 30 seconds:
Failed to list available add-ons: Unable to get list of available services: Get "https://api.github.com/search/repositories?q=topic:ddev-get+fork:true+org:drud": dial tcp: lookup api.github.com: i/o timeout
I'm not sure what the problem is. If I curl the URL from the error message, ie curl https://api.github.com/search/repositories?q=topic:ddev-get+fork:true+org:drud, I receive a JSON response from Github with information about the repository.
This has happened for over two days now. I may be overlooking something but am not sure what, exactly. I'm able to run DDEV projects using the standard installation (mariadb, nginx, nodejs, mailhog) but continue to run into errors re listing add-ons.
I have ddev v.1.21.4 installed.
I'm using an M1 Mac on macOS 13.1.
Thank you.
Your system is unable to do a DNS lookup of the hostname api.github.com, and this is happening on your macOS host. Are you able to ping api.github.com? Have you tried rebooting?
You may want to temporarily disable firewall, VPN, virus checker to see if that changes things. But you'll want to be able to get to where you can ping api.github.com.
There is an obscure golang problem on macOS affecting situations where people have more than one DNS server, so that could be it if you're in that category. You also might want to consider changing the DNS server for your system to 1.1.1.1, as this can sometimes be a problem with your local DNS server (but of course the fact that you can curl the URL argues against that).
I have a self-hosted elasticsearch in my server. I have integrated with my code after certain days my existing data is deleted. Only If I create a new entry this will store in the index
I have used Amazon aws free tier server. I have used rabbitmq in this server also. I know the more ram needed for the elastic search but I need to know the reason for the data loss
THis is my server log link
Elasticsearch never deletes an index. It means that you probably have something, someone running a DELETE index query from somewhere.
If you look at your logs you should see something like this.
Data is stored on disk in the data dir.
You have been attacked by a bot. If that happens again, try to list indices by GET /_cat/indices. If you see something like 'meow' or 'warn' there, then it is known bot attackers.
I faced same issue and did some investigations. Port 9200 was open on my server! Despite of iptables rules which restrict everything except 443, 8443 and 22. And the cause was docker! It added one more rule after all mine.
sudo iptables -S | grep 9200
-A DOCKER -d 172.18.0.2/32 ! -i br-cab97908df43 -o br-cab97908df43 -p tcp -m tcp --dport 9200 -j ACCEPT
How is that possible? Why docker does this crazy thing? The reason is default docker-compose.yml which I took from elasticsearch website.
Change
ports:
- 9200:9200
to
ports:
- 127.0.0.1:9200:9200
Happened the same to me once, I was working on some queries when all my indices got suddenly deleted.
I had in mind securing the whole thing but I kept on postponing it. It just took an hour to install x-pack and figure out all the errors that may arrive because of that, but no suddenly deleted indices ever since :D.
I installed Docker-ce(ver 17.03.1-ce-mac12 17661) on macOS Sierra(ver 10.12.5)
I created a container and run a simple socket echo server.
And then attempted to connect to the container's echo server from the host.
Initially, when the number of open sockets reached 370, a connection failure occurred,
I found the following via Google search.
https://github.com/docker/for-mac/issues/1009
To summarize, the docker for mac has its own maximum number of connections.
I modified the maximum number of connections moderately according to this content.
And I connected to the docker host in the following way.
http://pnasrat.github.io/2016/04/27/inside-docker-for-os-x-ii
I changed the ulimit configuration of the docker host as well, and changed the osx and container settings accordingly.
And again, I tried again, but this time the number of sockets exceeded the 370 limit mentioned above, but it is also blocked at about 930 ~ 940.
I try to change the settings like this, but it does not get better.
Note that a docker running on top of an Ubuntu server does not need to change any settings, and works well without any socket restrictions.
An echo server running inside the container of a docker running on Ubuntu can maintain at least 4000 sockets.
The problem only occurs with the docker for mac.
If you are aware of this situation, can anyone suggest a solution?
Thank you.
I have an elasticsearch server with host h1 running at port 9200. And i have a separate kibana server with host h2 running on port 5601. I have configured both of them properly and then tried to see the GUI of kibana.
It says Status:Red and in the error i see "Unable to connect to Elasticsearch at h1:9200"
I ran the basic troubleshoot mentioned in the elk documentation i.e. to check whether kibana.yml's elasticsearch.url field is properly referring to h1:9200 or not and vice versa for elasticsearch.yml's fields. Everything seems to be fine. What could be any other reason for this error? How to fix the same?
Could be for a number of reasons, connectivity/network configuration usually the prime suspect - check out this troubleshooting article for possible solutions
On my server I have installed elasticsearch-2.2.1 and couchbase server version 4.1.0. The aim is to transfer data from bucket x on couchbase to elastic search.
Ive installed the transport-couchbase plugin on elastic-search which will basically allow for xdcr from the server to elastic search.
As I understand it, transport-couchbase listens by default on port 9091 so in essence I'm supposed to create a cluster reference that points to that port (both couchbase and elastic search are installed on the same machine).
When I try create the reference I get an internal server error. The logs don't give me much information regarding the issue and I can ping the port. However when I try to telnet the machine on the port it refuses connection.
the server is sitting behind a proxy and i am starting to think that the issue lies within either couchbase server or elasticsearch ( transport-couchbase plugin)
Im going out on a limb here but I think maybe im supposed to configure the plugin so that it accepts requests going through tthe proxy. If this is the issue, is there a way to embed proxy settings into the plugin so that it can accept connections for xdcr?
PS: When I did this whole process on a separate machine that isnt sitting behind a proxy, everything worked fine. So I have a strong suspicion that it is proxy issues
If you can't telnet or browse to port 9091, this most likely indicates a network config issue. The plugin binds to the interface that elasticsearch binds to. The first thing to check is that the bind_host and publish_host in elasticsearch.yml is configured to bind to an interface that allows connections from wherever the proxy is located and that the proxy is really connecting on that interface.
There is a thread in github for the bug in transport plugin where it might not bind to all interfaces :
https://github.com/couchbaselabs/elasticsearch-transport-couchbase/issues/134
The above solutions didn't work for me, however I added this line:
-Djava.net.preferIPv4Stack=true
to /etc/elasticsearch/jvm.options and it seemed fixed the issue in my case