This is my first attempt to install and use Kubernetes. I am trying to install an environment on Mac for developing my own apps and deploying them for test locally with Kubernetes. I am familiar with using Vagrant, VirtualBox and Docker for the same purpose. When I saw this page https://github.com/GoogleCloudPlatform/kubernetes/blob/master/docs/getting-started-guides/vagrant.md I assumed it would be trivial. I executed these lines:
export KUBERNETES_PROVIDER=vagrant
curl -sS https://get.k8s.io | bash
This created a master VM and a Minion, but Kubernetes seems to have failed to start on the master. On the master /var/log/salt/master is full of python Traceback errors, like this:
2015-07-17 22:14:42,629 [cherrypy.error ][INFO ][3252] [17/Jul/2015:22:14:42] ENGINE Started monitor thread '_TimeoutMonitor'.
2015-07-17 22:14:42,736 [cherrypy.error ][ERROR ][3252] [17/Jul/2015:22:14:42] ENGINE Error in HTTP server: shutting down
Traceback (most recent call last):
File "/usr/lib/python2.7/site-packages/cherrypy/process/servers.py", line 187, in _start_http_thread
self.httpserver.start()
File "/usr/lib/python2.7/site-packages/cherrypy/wsgiserver/wsgiserver2.py", line 1824, in start
raise socket.error(msg)
error: No socket could be created
Vagrant is version 1.7.3. VirtualBox is version 4.3.30
Have I made an obvious stupid mistake?
I don't yet know the fix but I know what is going wrong since it happens to me as well:
OS X 10.10.3
Vagrant 1.7.4
VirtualBox 4.3.30
Kubernetes 1.0.1
When I run the default configuration of this (which creates one "master" and one "minion" VM) I see that the static IP address is not being assigned to the "eth1" interface, and I also see that the Salt API server is sitting in what appears to be an infinite retry loop because it is trying to listen on that IP address.
Also, the following message happened during boot:
[vagrant#kubernetes-master ~]$ dmesg | grep eth1
[ 9.321496] IPv6: ADDRCONF(NETDEV_UP): eth1: link is not ready
So basically, the static IP address didn't get assigned because eth1 wasn't ready when the system first booted, and Salt is waiting for it to get assigned.
I could fix this after boot by sshing to the box using "vagrant ssh" and running the command:
sudo /etc/init.d/network restart
on each host.
This "fixes" eth1 by assigning the static IP address, and after that Salt begins to do its thing, installs Docker, boots various containers, and so on.
What I don't know is how to make this work every time without manual intervention. It appears to be some sort of a race condition between Vagrant and VirtualBox.
If you just want to kick the tires with Kubernetes, I'd recommend installing boot2docker and then following the Running kubernetes locally via Docker getting started guide. Once you are comfortable interacting with the Kubernetes API and want a more complex local setup, you can then work on installing Vagrant.
If the Vagrant instructions aren't working, you should also feel free to file a bug in the github repository.
The tutorial pointed by Robert is realy easy to run. Just change the version to 0.21.2 (maybe 0.21.3 works too).
Else, if you prefer a vagrant solution, try with pires cluster on vagrant. It runs with almost nothing to change.
Running Kubernetes inside VirtualBox requires 4 networks and some adjustments to the configuration:
The VirtualBox HOST ONLY network will be the network used to access the Kubernetes master and nodes from the Mac or PC.
The NAT Network to download packages from the Internet.
The internal connections between Kubernetes PODs uses a tunnel network TUN
The Kubernetes Cluster IP Network is a private IP range used inside the cluster to give each Kubernetes service a dedicated IP
Vagrantfile needs to pass the node public IPs to the Ansible roles that configure Kubernetes to set KUBELET_EXTRA_ARGS environment variable with the public IP of each node (required for reading logs using kubectl).
NodePort needs to be used to publish applications running inside the Kubernetes cluster as Load Balancers are not available in VirtualBox.
See the full example and download the code at Building a Kubernetes Cluster with Vagrant and Ansible (without Minikube), it has been tested in Ubuntu but should work on a MAC as well.
Related
I'm very new to kubernetes and trying to conceptualize it as well as set it up locally in order to try developing something on it.
There's a confound though that I am running on a windows machine.
Their "getting started" documentation in github says you have to run Linux to use kubernetes.
As docker runs on windows, I was wondering if it was possible to create a kubernetes instance as a container in windows docker and use it to manage the rest of the cluster in the same windows docker instance.
From reading the setup instructions, it seems like docker, kubernetes, and something called etcd all have to run "in parallel" on a single host operating system... But part of me thinks it might be possible to
Start docker, boot 'default' machine.
Create kubernetes container - configure to communicate with the existing docker 'default' machine
Use kubernetes to manage existing docker.
Pipe dream? Wrongheaded foolishness? I see there are some options around running it in a vagrant instance. Does that mean docker, etcd, & kubernetes together in a single VM (which in turn creates a cluster of virtual machines inside it?)
I feel like I need to draw a picture of what this all looks like in terms of physical hardware and "memory boxes" to really wrap my head around this.
With Windows, you need docker-machine and boot2docker VMs to run anything docker related.
There is no (not yet) "docker for Windows".
Note that issue 7428 mentioned "Can't run kubernetes within boot2docker".
So even when you follow instructions (from a default VM created with docker-machine), you might still get errors:
➜ workspace docker run --net=host -d -v /var/run/docker.sock:/var/run/docker.sock gcr.io/google_containers/hyperkube:v0.14.2 /hyperkube kubelet --api_servers=http://localhost:8080 --v=2 --address=0.0.0.0 --enable_server --hostname_override=127.0.0.1 --config=/etc/kubernetes/manifests
ee0b490f74f6bc9b70c1336115487b38d124bdcebf09b248cec91832e0e9af1d
➜ workspace docker logs -f ee0b490f74f6bc9b70c1336115487b38d124bdcebf09b248cec91832e0e9af1d
W0428 09:09:41.479862 1 server.go:249] Could not load kubernetes auth path: stat : no such file or directory. Continuing with defaults.
I0428 09:09:41.479989 1 server.go:168] Using root directory: /var/lib/kubelet
The alternative would be to try on a full-fledge Linux VM (like the latest Ubuntu), instead of a boot2docker-like VM (based on a TinyCore distro).
All k8s components can be raised up with hyperkube, which helps you bring up a containerized one.
If you're able to run docker on windows, it would probably work. I haven't tried it on windows personally.
When I attempt to initiate 'vagrant up' the script executes as normal until it gets to the last line, where NFS shared drives are mounted.
I have tried deleting the exports file in /etc/ followed by a nfsd restart and vagrant destroy / vagrant up but to no avail.
After some considerable amount of time the console outputs the following [certain details redacted]:
*==> default: Mounting NFS shared folders...*
*The following SSH command responded with a non-zero exit status. Vagrant assumes that this means the command failed!*
*mount -o 'nolock,vers=3,udp,noatime' XXX.XXX.XX.X:'/Users/dhatton/Google Drive/moodle-doodle/site' /var/www/site*
*Stdout from the command:*
*Stderr from the command:*
*mount.nfs: Connection timed out*
UPDATE
The above problem was encountered when using a VPN into the office network. Upon logging in on-site without the VPN, everything works again.
For macOS Monterey 12.1 with virtualBox 6.1.30 and vagrant Vagrant 2.2.19/18:
create vbox folder in /etc
create a file inside /etc/vbox named networks.conf
add the following inside networks.conf
* 0.0.0.0/0 ::/0
Note: if you get the ip address range error, add your IP here too.
I had similar issue. I searched a lot, and tried following solutions:
Check /etc/exports and /etc/hosts files, if there are invalid entries in file, remove them.
Check your firewall is not blocking access
Restart NFS system
install vagrant plugin install vagrant-vbguest plugin
do vagrant reload --provision
Reboot your pc
Reinstall vagrant
For me reinstalling vagrant worked.
I've ran across this before and the problem turned out to be related to my companies VPN. If I tried running vagrant up connected to the VPN it would hang on mounting NFS, but if I disconnected from VPN and tried again it worked. Once running I could connect to VP Probably goes back to it needing a stable internet connection.
Assuming you are trying to mount from guest to host (host being OSX?) trying mounting to a different path. You might be encountering issues with the space in Google Drive?
Vagrant downloads binaries from its cloud while configuring a VM, so a stable internet connection is needed. In fact, an internet connection is necessary for using most of the Hashicorp products.
I installed boot2docker as explained on the docker website. Here are some command runs to show that I have things installed correctly:
$$:~ kv$ boot2docker start
Waiting for VM and Docker daemon to start...
...................ooo
Started.
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/ca.pem
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/cert.pem
Writing /Users/kvantum/.boot2docker/certs/boot2docker-vm/key.pem
Your environment variables are already set correctly.
$$:~ kv$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
ubuntu 14.04 b39b81afc8ca 11 days ago 188.3 MB
hello-world latest e45a5af57b00 3 weeks ago 910 B
After this, I ran the following command:
docker run -t -i ubuntu:14.04 /bin/bash
Inside the container, I installed zeromq, and started a zeromq server on port 5555 using tcp.
My questions are following:
If I exit out of the container, will it save all the work I do inside it?
I have no idea how to connect to the server running on port 5555. I read something about exposing a port, but I am not sure how to go about doing that. I did an ifconfig inside the container, and tried to connect to the server from the host like this:
$$:~ kv$ ./zmq_client tcp://container_ip:5555
This did not work. Can someone please lists the steps I need to take in order to connect to the server running within the container.
For completion sake, I am providing the list of my environment variables:
TERM_PROGRAM=Apple_Terminal
TERM=xterm-256color
SHELL=/bin/bash
TMPDIR=/var/folders/km/5kbpdx4s7cg4rmyc6d5q9l9r0000gq/T/
DOCKER_HOST=tcp://192.168.109.103:2376
Apple_PubSub_Socket_Render=/tmp/launch-1tWMHJ/Render
TERM_PROGRAM_VERSION=326
OLDPWD=/Users
TERM_SESSION_ID=262CBC8B-0A74-4B70-9F28-D9FA51FF713C
USER=kv
SSH_AUTH_SOCK=/tmp/launch-ZTWNGL/Listeners
__CF_USER_TEXT_ENCODING=0x1F7:0:0
DOCKER_TLS_VERIFY=1
__CHECKFIX1436934=1
PATH=/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin
PWD=/Users/kv
DOCKER_CERT_PATH=/Users/kv/.boot2docker/certs/boot2docker-vm
HOME=/Users/kv
SHLVL=1
LOGNAME=kv
LC_CTYPE=UTF-8
DISPLAY=/tmp/launch-rco9zt/org.macosforge.xquartz:0
_=/usr/bin/env
One last question I have is about code performance. So within my Mac OS X, I have a docker container running (which runs Ubuntu). If I run the application, like a zeromq based server inside the container, will it not be slower as compared to running it on Mac OS X directly. Please explain the benefits of using docker in such a scenario..
You should really do some more reading and research before turning to SO, then ask about anything you can't figure out. But:
No. If the container is "exited" you can restart it and your files will still be there, but once it is removed your files are gone. You can use docker commit to save them to an image, but the best bet is to use a Dockerfile.
docker run -p 5000:8000 image will expose port 8000 in the container as port 5000 on the host.
Yes, it will be slower due to the boot2docker VM. It would not be slower if you were running on a Linux host. The advantage is that zeromq is now running in an isolated container with all its dependencies.
I have been following this tutorial to try and setup a single node mesosphere cluster from their
official tutorial:
http://mesosphere.com/docs/getting-started/developer/single-node-install/
I followed all the commands without any issues, and I also added the ports 5050 and 8080 to my security group. When I try to access the console for mesos/marathon, I get a "Internet Explorer cannot display the webpage" message.
They also recommend checking it the following way:
MASTER=$(mesos-resolve `cat /etc/mesos/zk`)
mesos-execute --master=$MASTER --name="cluster-test" --command="sleep 5"
But that comes up with an error:
WARNING: Logging before InitGoogleLogging() is written to STDERR
F0106 17:03:08.126703 20993 process.cpp:1561] Failed to initialize, gethostbyname2: Unknown host
*** Check failure stack trace: ***
I am not really sure how to troubleshoot this either, and there are not many tutorials I could find on how to install mesos on ubuntu.
I checked the contents of the zk file, seems to be the default value.
$ cat /etc/mesos/zk
zk://localhost:2181/mesos
I would really appreciate any clues on how to go about this one.
Edit: The process is definitely running too - just an fyi:
root 31545 8.5 5.9 187464 35604 ? Ssl 17:28 0:00 /usr/local/sbin/mesos-slave --master=zk://localhost:2181/mesos --log_dir=/var/log/mesos
root 31563 28.5 2.1 116304 12856 ? Rs 17:28 0:00 /usr/local/sbin/mesos-master --zk=zk://localhost:2181/mesos --port=5050 --log_dir=/var/log/mesos --quorum=1 --wo
Mesos uses gethostbyname2 to resolve hostnames to IPs. The first thing I would recommend, is to try "ping localhost" and "ping hostname", and verify that there are no strange settings in /etc/hosts. If you're doing a multi-node cluster, I'd recommend that hostname map to the public IP address (not 127.0.x.1).
If that doesn't help, you can try setting the --ip and --hostname flags when starting mesos-master and mesos-slave, to bypass the gethostbyname2 resolution. These can also be set by writing to the file-based parameters, e.g. /etc/mesos/mesos-master/ip
For additional troubleshooting, try running wget http://localhost:5050 (or curl -L) from the mesos master, to verify that it is locally visible. Also try wget http://<public_ip>:5050 to verify that the web server is up and serving to the public IP. Depending on how your (EC2?) node is setup, you may need to expose/forward the port, or connect to a VPN.
Thanks Adam. I ran the wget and curl commands, and nothing was actually listening on port 8080 or 5050. I did open those ports in the ec2. A simple reboot did the trick however, once I ssh'ed into the ec2 instance after the reboot, both mesos and marathon were running and both ports are now showing after I ran
netstat -ntln.
I'm using Vagrant to run an Ubuntu VM with Microsoft Azure provider, but I need to create an endpoint on this VM.
I've done it installing azure command-line tools with
npm install azure-cli -g
and after that
azure vm endpoint create {VM-Name} 80 8080
VM is running fine without problems. But I don't want to configure the endpoint manually anymore. I would like to create it when I run vagrant up. I don't want to use vagrant provision, because this shell instruction should not be executed in my VM, but in my physical machine.
There is no options such as verbose in vagrant, which would have permitted me to see what it really does with the Vagrantfile, and what I should do to implement a shell command like this one in it.
Ansver to initial question, do the following:
in vagrantfile add following string in azure provider section:
azure.tcp_endpoints = '80:8080'
hope this help.
answer to your question about verbose output:
you may run vagrant up --provider=azure --debug
for additional debug information.
if you need to run local scripts you may try to use vagrant-triggers plugin