Vanilla image for sahara cluster - hadoop

I'm trying to install an image for sahara cluster. But the documentation is a little restrictive. I didn't find the qcow2 image adapted to sahara. the mirrors that exist now are no longer up to date.Therefore, I tried to follow this doc Building Images for Vanilla Plugin
In summary, I followed these steps:
Clone repository
“https://git.openstack.org/cgit/openstack/sahara-image-elements”
locally.
Use tox to build images.
tox -e venv -- sahara-image-create -u
The following error is throwing up
INFO diskimage_builder.block_device.blockdevice [-] State already cleaned - no way to do anything here
ERROR: InvocationError: '/opt/stack/sahara-image-elements/.tox/venv/bin/sahara-image-create'
_______________________________________________________________________________________________________ summary ________________________________________________________________________________________________________
ERROR: venv: commands failed
I am using centos7 and openstack queens.
I would be grateful for any suggestion.

Try this:
tox -e venv -- sahara-image-create -p vanilla -v 2.7.1 -i centos7

Related

Run docker container with open62541 server on Raspberry Pi

Description
I am connected to my raspberry pi 1 ( via putty and I have already followed the steps as described here at "Building Debian Packages inside Docker Container with CMake on Ubuntu or Debian"
I have cloned the docker-deb-builder repo and build both the dockerfile-ubuntu-18.04 and
dockerfile-ubuntu-17.04.
Then, I have cloned the open62541 repo and checked out pack/1.0.
Afterwards, I have created the output folder in the docker-deb-builder folder and tried to run the
following command in the development folder:
./build -i docker-deb-builder:18.04 -o output ~/ma/development/open62541
But it fails with following error message:
pi#raspberrypi:~/ma/dockerVersion/docker-deb-builder $ sudo ./build -i docker-deb-builder:18.04 -o output ~/ma/dockerVersion/open62541
Running docker:
docker run -it -it -v /home/pi/ma/development/open62541:/source-ro:ro -v /home/pi/ma/development/docker-deb-builder/output:/output -v /home/pi/ma/development/docker-deb-builder/build-helper.sh:/build-helper.sh:ro -e USER=0 -e GROUP=0 --rm docker-deb-builder:18.04 /build-helper.sh
Unable to find image 'docker-deb-builder:18.04' locally
docker: Error response from daemon: pull access denied for docker-deb-builder, repository does not exist or may require 'docker login': denied: requested access to the resource is denied.
See 'docker run --help'.
What I am trying to accomplish is to run a docker container (with an open62541 server running in it) on my raspberry pi.
Questions:
- How can I solve the problem described above?
- How or where can I integrate my customized open62541 server code?
Background Information / Reproduction Steps
Running the command cat /etc/os-release gives me the following information:
pi#raspberrypi:~/ma/dockerVersion/docker-deb-builder $ cat /etc/os-release PRETTY_NAME="Raspbian GNU/Linux 9 (stretch)"
NAME="Raspbian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=raspbian
ID_LIKE=debian
HOME_URL="http://www.raspbian.org/"
SUPPORT_URL="http://www.raspbian.org/RaspbianForums"
BUG_REPORT_URL="http://www.raspbian.org/RaspbianBugs"
Issue 1 - running a container by tag (but not finding it by tag)
Firstly, the error docker: Error response from daemon: pull access denied for docker-deb-builder occurs when you docker run using a image name that cannot be found locally, so it goes off and tries to find docker-deb-builder on hub.docker.com (publicly) and if you go look there are none with that exact name.
The error is basically telling you;
I couldn't find docker-deb-builder locally
I couldn't find docker-deb-builder on hub.docker.com (public)
I want to try hub.docker.com (private) but you are not logged in
So you might first try to build the container and docker tag it with docker-deb-builder so you can then find it locally with the command you provided in this issue.
Issue 2 - building on armv7 (raspberry pi)
The page you linked to has instructions for;
git clone https://github.com/tsaarni/docker-deb-builder.git
Looking at tsaarni/docker-deb-builder on github for the 18.04 image you're targetting i found the Docekrfile and it is using ubuntu:18.04 and you will need to modify this to use ubuntu:18.04#sha256:60a99a670b980963e4a9d882f631cba5d26ba5d14ccba2aa82a4e1f4d084fb1f which is the signature for the armv7.
Might just add, to make sure in case you're not aware, to run on armv7 you must also docker build it on armv7

How to run a library in docker - confusion

I am trying to use microsoft's cntk library on a mac; for this purpose I am using Docker. I am not an expert in all this, though, so I am having a hard time figuring out how to make it work.
From my understanding, Docker provides a way to run an app in a virtualized environment, without having to virtualize the entire operating system. So you download (or create) images, and you run them in "containers".
Alright, so I have followed the required steps to make the cntk library work on Docker; and if I list the images, I find
$: docker images
REPOSITORY TAG IMAGE ID CREATED SIZE
microsoft/cntk latest c2c192036e19 7 days ago 5.92 GB
ubuntu 14.04 7c09e61e9035 5 weeks ago 188 MB
hello-world latest 48b5124b2768 2 months ago 1.84 kB
At this point I want to run of the tutorials that are in the cntk repository. I have downloaded the master branch of the cntk repository on my desktop and try to run one of the examples in the "Tutorial" folder, but I get the following error:
terminal~ username$ docker run -w /Users/username/Desktop/CNTK-master/Tutorials microsoft/cntk configFile=lr_bs.cntk
container_linux.go:247: starting container process caused "exec: \"configFile=lr_bs.cntk\": executable file not found in $PATH"
docker: Error response from daemon: oci runtime error: container_linux.go:247: starting container process caused "exec: \"configFile=lr_bs.cntk\": executable file not found in $PATH".
ERRO[0001] error getting events from daemon: net/http: request canceled
terminal~ username$
Essentially I call docker run with the -w flag to inform him of where the files are, but it does not work. I tried searching online but it's not clear to me how to solve the issue. Should I create a new image? Should I call the docker run command with different parameters?
The -w flag sets the working directory, which is just the default directory inside the container. Your directory is on the host, so it won't work here. Instead you need to use volumes to mount your directory on the host into the container. The final paragraph in the document you link has an example:
$ docker run --name cntk_container1 -ti -v /project1/data:/data -v /project1/config:/config cntk bash

Creating image in Docker fails

I'm trying to create my own image on Docker container. I wrote my Docker file as given below:
FROM ubuntu:latest
RUN apt-get update && apt-get install astyle ruby
But on running
docker build -t username/newname .
It gives an error Error response from daemon:
mkdir /var/lib/docker/tmp/docker-builder705720973: no space left on device
I'm new to Docker. Any help would be appreciated. Thanks in advance
You must have pulled a lot of images i guess. Check all your images using following command:
sudo docker images
If you see many images, try deleting some that you don't need anymore using the following command
sudo docker rmi <iamge_id>
Hope that helps

TensorFlow docker dev workflow on mac

There is an official guide on how to install it that doesn't say much about actually developing in it.
From what I understand, there is a quite big challenge in developing with Docker in general. Not to mention there could be deeper technical complications about going with it for TensorFlow, maybe mostly thanks to GPUs. So there is a lot of stuff to after pulling the docker image...
Does anyone have a step by step guide on how to get development going here?
You could mount a local directory to the docker container so that you can still use your preferred editor in osx. Here's a command to start the container with a mounted directory and run a command:
docker run --name tensorflow --rm -v /Users/me/Code/web/tensorflow_dev:/tensorflow_dev b.gcr.io/tensorflow/tensorflow /bin/sh -c 'cd /tensorflow_dev && python mnist.py'
-v will mount the local directory and the -c will run the specified command. So your flow might look like:
Edit python script in your favorite editor
Run the above command to excute your script
However, I actually use pycharm so that I can place breakpoints and run the python script interactively within the editor.
Hope this helps.

Vagrant, puppet and virtual box

Getting this error when I run the command - vagrant up
Error: Execution of '/usr/bin/yum -d 0 -e 0 -y list emacs' returned 1:
Error: Cannot retrieve repository metadata (repomd.xml) for
repository: nodesource. Please verify its path and try again
Although some details around OS etc. would have helped but the error seems to be coming when trying to do an "yum update". Similar error has been answered here. So technically it is not even a Puppet error. If you need a basic sandbox for trying puppet using Vagrant then you can use the repo here

Resources