Recently, I began to learn jhipster. After I have biuld a jhipster project and run it ,but it throws a error :
ERROR 16725 --- [ restartedMain] .d.e.r.s.AbstractElasticsearchRepository : failed to load elasticsearch nodes : org.elasticsearch.client.transport.NoNodeAvailableException: None of the configured nodes are available: [{#transport#-1}{yaU7NNKDRzK_pZir0MCKNw}{localhost}{127.0.0.1:9300}]
and here is my elasticsearch.yml:
version: '2'
services:
cmpmanager-elasticsearch:
image: elasticsearch:5.6.5
# volumes:
# - ~/volumes/jhipster/CMPManager/elasticsearch/:/usr/share/elasticsearch/data/
ports:
- 9200:9200
- 9300:9300
command: -Enetwork.host=0.0.0.0 -Ediscovery.type=single-node
If you are access the ES nodes from another host check the node status using below command,
curl '192.168.x.y:9200/_cat/indices?v'
If it failed to show status, you have to open ES to access via public.
host=0.0.0.0
Not works for me also.
Related
I hope someone can help me with this issues, as I'm no expert with docker.
I have a Java Spring Boot application (let's call it my-app) that uses ScyllaDB. So far, I have been running the application with Spring Boot embedded Apache Tomcat build, and the database is running in Docker with no issues.
Here is the docker-compose file for the 3 Scylla nodes:
version: "3"
services:
scylla-node1:
container_name: scylla-node1
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9042:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
scylla-node2:
container_name: scylla-node2
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9043:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
scylla-node3:
container_name: scylla-node3
image: scylladb/scylla:4.5.0
restart: always
command: --seeds=scylla-node1,scylla-node2 --smp 1 --memory 750M --overprovisioned 1 --api-address 0.0.0.0
ports:
- 9044:9042
volumes:
- "./scylla/scylla.yaml:/etc/scylla/scylla.yaml"
- "./scylla/cassandra-rackdc.properties.dc1:/etc/scylla/cassandra-rackdc.properties"
networks:
- scylla-network
Using the node tool, I can see the DB is fine:
Datacenter: DC1
-- Address Load Tokens Owns Host ID Rack
UN 172.27.0.3 202.92 KB 256 ? 4e2690ec-393b-426d-8956-fb775ab5b3f9 Rack1
UN 172.27.0.2 99.5 KB 256 ? ae6a0b9f-d0e7-4740-8ebe-0ce1d2e9ea7e Rack1
UN 172.27.0.4 202.68 KB 256 ? 7a4b39bf-f38a-41ab-be33-c11a4e4e352c Rack1
In the application, the Java driver I'm using is the DataStax Java driver 3.11.2.0 for Apache Cassandra. The way I connect with the DB is the following:
#Bean
public Cluster cluster() {
Cluster cluster = Cluster.builder().addContactPointsWithPorts(
new InetSocketAddress("127.0.0.1", 9042),
new InetSocketAddress("127.0.0.1", 9043),
new InetSocketAddress("127.0.0.1", 9044))
.build();
return cluster;
}
#Bean
public Session session(Cluster cluster, #Value("${scylla.keyspace}") String keyspace) throws IOException {
final Session session = cluster.connect();
setupKeyspace(session, keyspace);
return session;
}
When running the application with the tomcat server, I receive a lot of connection errors at the start:
2022-07-19 22:42:38.424 WARN 28228 --- [r1-nio-worker-3] com.datastax.driver.core.Connection : Error creating netty channel to /172.27.0.4:9042
However, after a small spam of log errors, the app eventually connects and its totally usable. I do have to wait for the node tool to execute and confirm that all nodes are up, though.
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] c.d.d.c.p.DCAwareRoundRobinPolicy : Using data-center name 'DC1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /172.27.0.3:9042 added
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /172.27.0.2:9042 added
2022-07-19 23:25:12.324 INFO 25652 --- [ restartedMain] com.datastax.driver.core.Cluster : New Cassandra host /127.0.0.1:9044 added
Then, I recently added "my-app" to my docker-compose file, but the app can't start and instantly shuts down even if I wait for the node status tool to confirm that all nodes are up.
Caused by: java.net.ConnectException: Connection refused
Caused by: com.datastax.driver.core.exceptions.NoHostAvailableException: All host(s) tried for query failed (tried: /127.0.0.1:9042
Is there something wrong with the way I'm connecting with the DB? I wonder why the embedded tomcat build works and the docker one instantly shuts down. I was hoping someone here could help me find a way for the docker-compose build to wait for all the scylla nodes to be up before starting my-app (I assume I can do it with a script in the dockerfile? Maybe?), but I can't even seem to start the app in docker the same way I did with the tomcat. Maybe I'm missing smething regarding the port and host when using docker.
Any ideas in what I could try to solve this? Thanks in advance!
Docker compose file edited with the app:
my-app:
container_name: my-app
build:
context: .
dockerfile: Dockerfile
image: my-app
ports:
- 8082:8082
depends_on:
- scylla-node1
- scylla-node2
- scylla-node3
networks:
- scylla-network
You need to use IP addresses in the contact points which are accessible from outside the containers, not localhost.
Typically, it will be the IP address you've configured for CASSANDRA_RPC_ADDRESS (environment variable) or rpc_address (in your yaml).
If you didn't set the RPC addresses for the containers, you need to tell Cassandra what IP address to advertise to other nodes and clients by specifying a broadcast address with CASSANDRA_BROADCAST_ADDRESS or broadcast_rpc_address.
The important thing is that you need to use IP addresses which are reachable from your Spring Boot app. Cheers!
I have some issues to dockerize my Spring boot Cassandra application.
When I use Idea to run my project it's work fine correctly but when I turn my application into a docker container I have this issues :
"RetryingCassandraClusterFactoryBean : All host(s) tried for query failed (tried: localhost/127.0.0.1:9042 (com.datastax.driver.core.exceptions.TransportException: [localhost/127.0.0.1:9042] Cannot connect))"
my application.properties:
spring.data.cassandra.keyspace-name=keyspace
spring.data.cassandra.contact-points=127.0.0.1
spring.data.cassandra.port=9042
spring.data.cassandra.schema-action=create_if_not_exists
server.port=8081
my cassandra :
cassandradb:
image: "cassandra:latest"
container_name: "cassandradb"
ports:
- "9042:9042"
- "7191:7191"
- "7001:7001"
- "9160:9160"
- "7000:7000"
environment:
CASSANDRA_CLUSTER: "myCluster"
CASSANDRA_ENDPOINT_SNITCH: "GossipingPropertyFileSnitch"
CASSANDRA_DC: "data"
CASSANDRA_LISTEN_ADDRESS: "auto"
photos:
image: "photos:latest"
container_name: "photo"
ports:
- "8081:8081"
links:
- "cassandradb"
Error happens, becouse your spring cant reach cassandradb. You should connect or via host ip, or usecassandradb` host, anc configure docker-compose.yml links part
I develop a simple example about the CRUD spring boot application by Cassandra.
You can check it in github. here are a few differences in the comparison of mine.
We have a Ruby project where we are using Wercker as Continuous Integration.
We need to start an Elastic Search service in order to run some integration tests.
Locally, we added the Elastic configuration to the docker file and everything runs smoothly:
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.5.1
container_name: elasticsearch
environment:
- discovery.type=single-node
ports:
- "9200:9200"
- "9300:9300"
In The Wercker.yml file, we tried several things, but we cannot reach the elastic service.
Our wercker.yml contains:
services:
- id: elasticsearch:6.5.1
env:
ports:
- "9200:9200"
- "9300:9300"
We have this king of error when trying to use Elastic in our tests:
Errno::EADDRNOTAVAIL: Failed to open TCP connection to localhost:9200 (Cannot assign requested address - connect(2) for "localhost" port 9200)
Do you have any idea of what we are missing?
So, we found a solution:
In wercker.yml
services:
- id: elasticsearch:6.5.1
cmd: "/elasticsearch/bin/elasticsearch -Ediscovery.type=single-node"
And we added a step to check the connection:
build:
steps:
- script:
name: Test elasticsearch connection
code: curl http://elasticsearch:9200
Using Elasticsearch:latest (v5.1) from the Docker public repo, I created my own image containing Cerebro. I am now attempting to get Elasticsearch networking properly configured so that I can connect to Elasticsearch from Cerebro. Cerebro running inside of the container I created, renders properly on my host at: http://localhost:9000.
After committing my image, I created my Docker container with the following:
sudo docker run -d -it --privileged --name es5.1 --restart=always \
-p 9200:9200 \
-p 9300:9300 \
-p 9000:9000 \
-v ~/elasticsearch/5.1/config:/usr/share/elasticsearch/config \
-v ~/elasticsearch/5.1/data:/usr/share/elasticsearch/data \
-v ~/elasticsearch/5.1/cerebro/conf:/root/cerebro-0.4.2/conf \
elasticsearch_cerebro:5.1 \
/root/cerebro-0.4.2/bin/cerebro
my elasticsearch.yml in ~/elasticsearch/5.1/config currently has the following network and discovery entries specified:
network.publish_host: 192.168.1.26
discovery.zen.ping.unicast.hosts: ["192.168.1.26:9300"]
I have also tried 0.0.0.0 and not specifying the values to default to the loopback for these settings. In addition, I've tried specifying network.host with a combination of values. No matter how I set this, elasticsearch logs on startup:
[info] play.api.Play - Application started (Prod)
[info] p.c.s.NettyServer - Listening for HTTP on /0:0:0:0:0:0:0:0:9000
[error] p.c.s.n.PlayDefaultUpstreamHandler - Cannot invoke the action
java.net.ConnectException: Connection refused: localhost/127.0.0.1:9200
… cascading errors because of this connection refusal...
No matter how I set the elasticsearch.yml networking, the error message on Elasticsearch startup does not change. I verified that the elasticsearch.yml is being picked-up inside of the Docker container. Please let me know were I'm going wrong with this configuration.
Well, it looks like I"m answering my own question after a days-worth of battle with this! The issue was that elasticsearch wasn't started inside of the container. To determine this, I got a terminal into the container:
docker exec -it es5.1 bash
Once in the container, I checked service status:
service elasticsearch status
To this, the OS responded with:
[FAIL] elasticsearch is not running ... failed!
I started it with:
service elasticsearch start
I add a single script that I'll call from docker run to start elasticsearch and cerebro and that should do the trick. However, I would still like to hear if there is a better way to configure this.
I made a github docker-compose repo that will spin up a elasticsearch, kibana, logstash, cerebro cluster
https://github.com/Shuliyey/elkc
========================================================================
On the other hand, in regard to the actual problem (elasticsearch_cerebro not working).
To get the elasticsearch and cerebro working in one docker container. Need to use supervisor
https://docs.docker.com/engine/admin/using_supervisord/
will update with more details
No need to use supervisor at all. A very simple way to solve this is to use docker-compose and bundle Elasticsearch and Cerebro together, like this:
docker-compose.yml:
version: '2'
services:
elasticsearch:
build: elasticsearch
volumes:
- ./elasticsearch/config/elasticsearch.yml:/usr/share/elasticsearch/config/elasticsearch.yml
- ./elasticsearch/data:/usr/share/elasticsearch/data
ports:
- "9200:9200"
- "9300:9300"
environment:
ES_JAVA_OPTS: "-Xmx1500m -Xms1500m"
networks:
- elk
cerebro:
build: cerebro
volumes:
- ./cerebro/config/application.conf:/opt/cerebro/conf/application.conf
ports:
- "9000:9000"
networks:
- elk
depends_on:
- elasticsearch
networks:
elk:
driver: bridge
elasticsearch/Dockerfile:
FROM docker.elastic.co/elasticsearch/elasticsearch:5.5.1
cerebro/Dockerfile:
FROM yannart/cerebro
Then you run docker-compose build and docker-compose up. When everything is started, you can access ES at http://localhost:9200 and Cerebro at http://localhost:9000
I am trying to migrate by Dockerfile from using links to networking but I am experiencing some problems. I will first sketch my original setup:
Original - Links
Dockerfile:
FROM java:8
VOLUME /tmp
ADD docker_micro_maven-0.0.1-SNAPSHOT.jar app.jar
RUN bash -c 'touch /app.jar'
EXPOSE 8080
ENTRYPOINT ["java","-Dspring.data.mongodb.uri=mongodb://mongodb/micros", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
docker-compose.yml:
mongodb:
image: mongo
employee:
image: jdruwe/docker_micro_maven
links:
- mongodb
ports:
- "8080"
New - Networking
Dockerfile:
FROM java:8
VOLUME /tmp
ADD docker_micro_maven-0.0.1-SNAPSHOT.jar app.jar
RUN bash -c 'touch /app.jar'
EXPOSE 8080
ENTRYPOINT ["java","-Dspring.data.mongodb.uri=mongodb://${MONGO_URI_DOCKER}/micros", "-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]
docker-compose.yml:
mongodb:
container_name: "db"
image: mongo
employee:
image: jdruwe/docker_micro_maven
environment:
- MONGO_URI_DOCKER=db:27017
ports:
- "8080"
When I called my rest endpoint in the spring boot app (original situation) I always got a response (some json data). When I call it now I get the following error response:
{
"timestamp": 1453405937191,
"status": 500,
"error": "Internal Server Error",
"exception": "org.springframework.dao.DataAccessResourceFailureException",
"message": "Timed out after 10000 ms while waiting to connect. Client view of cluster state is {type=Unknown, servers=[{address=db:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.UnknownHostException: db: unknown error}}]; nested exception is com.mongodb.MongoTimeoutException: Timed out after 10000 ms while waiting to connect. Client view of cluster state is {type=Unknown, servers=[{address=db:27017, type=Unknown, state=Connecting, exception={com.mongodb.MongoException$Network: Exception opening the socket}, caused by {java.net.UnknownHostException: db: unknown error}}]",
"path": "/employee"
}
The video from Devoxx I used as reference: https://youtu.be/aSATsLG59Zs?t=53m33s
It seems that I keep doing something wrong but I am not really sure what it is, any help is welcome. Thanks in advance!
In docker-compose 1.5.x, the new networking is still an experimental feature that needs to be enabled using the --x-networking option.
The new networking features will be moved out of experimental in docker-compose 1.6 and are automatically used if the new (2.0) file format is used for your docker-compose.yml.
Read the release notes of docker-compose 1.6-rc1 for more information;
https://github.com/docker/compose/releases/tag/1.6.0-rc1