I have a 3 node kubernetes cluster, a master and two nodes on AWS that I created with kubeadm (https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/)
I have created some deployments from the master node and I can see that pods are created on the 2 nodes for each of the deployments.
But the issue is I can't access the pod ip from the master or from the other node. So the pod ip is only accessible on the node where is pod is running.
I have a service of nodeport type, so when the service(pod1:port) hits the other pod(pod2), it hangs and times out
Thanks.
It works either by disabling the firewall or by running below command.
I found this bug in my search. Looks like this is related to docker >=1.13 and flannel
refer: https://github.com/coreos/flannel/issues/799
Related
I have a cluster of consul servers in two datacenters. each datacenter consists of 3 servers each. When I execute consul members -wan command I can see all 6 servers.
I want to separate these two into individual clusters and no connection between them.
I tried to use the command force-leave and leave as per the consul documentation:
https://www.consul.io/commands/force-leave: When I used this command
the result was a 500 - no node is found. I tried using the node name as server.datacenter, full FQDN of the server, IP of the server, none of them worked for me.
https://www.consul.io/commands/leave: When I used this command from
the node which I want to remove from the cluster, the response was
success but when I execute consul members -wan I still can see this
node.
I tried another approach where in I stopped the consul on the node I want to remove from cluster. Then executed the command: consul force-leave node-name. Then the command: consul members -wan showed this node as left. When I started the consul on this node, the node is back in cluster.
What steps am I missing here?
I think I solved the problem I had. I followed the instructions here:
https://support.hashicorp.com/hc/en-us/articles/1500011413401-Remove-WAN-federation-between-Consul-clusters
I have an EKS cluster running with both Linux and Windows nodes. On the Windows nodes i am scheduling pods. They run for about 30 minutes and then get removed. The first thing any pod does is download some data from S3 using the AWS cli installed on it.
I am facing some intermittent connectivity issues. Pods get spun up on and sometimes give a fatal error:
Could not connect to the endpoint URL: "https://sts.eu-west-1.amazonaws.com
As far as i can see this only happens when I schedule more then one pod on a node. I do use a smaller instance type (M5.large) but i am not close to reaching the pod limit of this instance type. When there is 1 pod per node they can all connect and download data from S3.
Reading the documentation I can see it is possible to schedule more then 1 pod per EC2 instance. But I am unsure what the requirements are to the EC2 instance to give all those pods access to download data from S3. I did try to add more ENIs to the EC2 instances but this prevented the EC2 instances to be registered as nodes in the EKS cluster.
I tried to find the answer in previous post, but i did not find it !
My question seems dumb, i'm just trying to figure it out :)
I'm new to docker and kubernetes, i'm trying to understand the architecture of kubernetes cluster, nodes, and pods.
I'm using two machines with docker installed, each machine have two containers running, i want to install MicroK8s to start playing with kubernetes, my questions are :
As below image > Can I install it on separate machine and connect it to my docker host machines so it will manage my containers their with support of some sort of (agent/ maybe services) ?, Or kubernetes/MicroK8s must be installed on the machine that will host the containers ?
Can i add my running docker containers directly to a pod ? or i must re-create them ?
Many thanks
You can play with any VM software(cpu virtualization required).
You can set up 3 VMs(master, node1,node2). You have to install kubernetes in each VM. When you connect them thru calico they communicate each other. When you make pods with app or db , you can loadbalance to node1 and node2 or more from master. Then you can create a service to export route to the pods. Or If you want to run everything in one big server, you can. Horizontal scaling or vertical scaling is your choice.
you cant mount a running docker container to the pod but you can load a docker image from any registry.
I have 2 node K8 cluster (1 linux + 1 win node).
For windows I'm using 1803 version. The issue I'm facing with is that Pods in Windows node can't resolve domain of services created on K8 cluster and which points to Pods on linux machine.
I can run container inside Windows node, tried with iis and containers are started successfully but they cannot resolve dns -> if you try nslookup or even ping clusterIP od service.
On linux node everything is working fine and pods see each other. Any suggestions or doubts where is the problem?
After restarting my 3 masters in my DC/OS cluster, the DC/OS dashboard is showing 0 connected nodes. However from the DC/OS cli I see all 6 of my agent nodes:
$ dcos node
HOSTNAME IP ID
172.16.1.20 172.16.1.20 a7af5134-baa2-45f3-892e-5e578cc00b4d-S7
172.16.1.21 172.16.1.21 a7af5134-baa2-45f3-892e-5e578cc00b4d-S12
172.16.1.22 172.16.1.22 a7af5134-baa2-45f3-892e-5e578cc00b4d-S8
172.16.1.23 172.16.1.23 a7af5134-baa2-45f3-892e-5e578cc00b4d-S6
172.16.1.24 172.16.1.24 a7af5134-baa2-45f3-892e-5e578cc00b4d-S11
172.16.1.25 172.16.1.25 a7af5134-baa2-45f3-892e-5e578cc00b4d-S10`
I am still able to schedule tasks in Marathon both from the dcos cli and from the Marathon gui, they then are properly scheduled and executed on the agents. Also, from the mesos interface on :5050 I can see all of the agents in the slaves page.
I have restarted agent nodes and master nodes. I have also rerun the DC/OS GUI installer and run preflight check, which of course fails with an "already installed" error.
Is there a way to re-register the node with DC/OS GUI short of uninstalling/reinstalling a node?
For anyone who is running into this, my problem was related to our corporate proxy. In order to get the Universe working in my cluster I had to add proxy settings to /opt/mesosphere/environment. I then restarted the dcos-cosmos.service and life was good. However, upon server restart, dcos-history-service.service was now running with the new environment and was unable to resolve my local names with our proxy server. To solve, I added a NO_PROXY to the /opt/mesosphere/environment and DCOS dashboard is again happy.