I'm currently working on a server-side product which is a bit complex to deploy on a new server, which makes it an ideal candidate for testing out in a VM. We are already using Hudson as our CI system, and I would really like to be able to deploy a virtual machine image with the latest and greatest software as a build artifact.
So, how does one go about doing this exactly? What VM software is recommended for this purpose? How much scripting needs to be done to accomplish this? Are there any issues in particular when using Windows 2003 Server as the OS here?
Sorry to deny anyone an accepted answer here, but based on further research (thanks to your answers!), I've found a better solution and wanted to summarize what I've found.
First, both VirtualBox and VMWare Server are great products, and since both are free, each is worth evaluating. We've decided to go with VMWare Server, since it is a more established product and we can get support for it should we need. This is especially important since we are also considering distributing our software to clients as a VM instead of a special server installation, assuming that the overhead from the VMWare Player is not too high. Also, there is a VMWare scripting interface called VIX which one can use to directly install files to the VM without needing to install SSH or SFTP, which is a big advantage.
So our solution is basically as follows... first we create a "vanilla" VM image with OS, nothing else, and check it into the repository. Then, we write a script which acts as our installer, putting the artifacts created by Hudson on the VM. This script should have interfaces to copy files directly, over SFTP, and through VIX. This will allow us to continue distributing software directly on the target machine, or through a VM of our choice. This resulting image is then compressed and distributed as an artifact of the CI server.
Regardless of the VM software (I can recommend VirtualBox, too) I think you are looking at the following scenario:
Build is done
CI launches virtual machine (or it is always running)
CI uses scp/sftp to upload build into VM over the network
CI uses the ssh (if available on target OS running in VM) or other remote command execution facility to trigger installation in the VM environment
VMWare Server is free and a very stable product. It also gives you the ability to create snapshots of the VM slice and rollback to previous version of your virtual machine when needed. It will run fine on Win 2003.
In terms of provisioning new VM slices for your builds, you can simply copy and past the folder that contains the VMWare files, change the SID and IP of the new VM and you have a new machine. Takes 15 minutes depending on the size of your VM slice. No scripting required.
If you use VirtualBox, you'll want to look into running it headless, since it'll be on your server. Normally, VirtualBox runs as a desktop app, but it's possible to start VMs from the commandline and access the virtual machine over RDP.
VBoxManage startvm "Windows 2003 Server" -type vrdp
We are using Jenkins + Vagrant + Chef for this scenario.
So you can do the following process:
Version control your VM environment using vagrant provisioning scripts (Chef or Puppet)
Build your system using Jenkins/Hudson
Run your Vagrant script to fetch the last stable release from CI output
Save the VM state to reuse in future.
Reference:
vagrantup.com
I'd recommend VirtualBox. It is free and has a well-defined programming interface, although I haven't personally used it in automated build situations.
Choosing VMWare is currently NOT a bad choice.
However,
Just like VMWare gives support for VMWare server, SUN gives support for VirtualBOX.
You can also accomplish this task using VMWare Studio, which is also free.
The basic workflow is this:
1. Create an XML file that describes your virtual machine
2. Use studio to create the shell.
3. Use VMWare server to provision the virtual machine.
Related
I am looking for a way to be able to do the following:
Create an instance of Windows with installed prerequisites and configuration
An isolated environment would be recommended (As in it will not modify the existing configuration on local machine only in that VM-like environment)
Ability to use the internet within that environment
Using it sort of like a "check-point" (Start working on it, doing something wrong and being able to start once again from the instance that we created)
Ability to share the environment
Possibility of creating multiple different environments
Low disk usage if possible
Fast deployment of environment on local machine
I have looked into Docker which seems pretty good for what I need, but I want to investigate other options as well because it requires Windows 10 x64 Enterprise
.
Something that works on Windows 7/Server/8/8.1 would be nice
I would also love to get arguments on why X option is better than Y option.
Thanks in advance!
If you want a completely separate environment, creating a Virtual Machine will be worth considering.
There are products from VMware and Oracle to create your virtual machine. I have been using Oracle Virtualbox (Oracle's virtual machine software) for some time now and find it pretty useful.
With a virtual machine it addresses all your concerns:
Create an instance of Windows with installed prerequisites and
configuration - A virtual machine will run on top of your installed OS without making
any modifications in current installation
An isolated environment would be recommended (As in it will not
modify the existing configuration on local machine only in that
VM-like environment) - It runs completely isolated like a separate
machine.
Ability to use the internet within that environment - You can use
internet inside of a virtual machine
Using it sort of like a "check-point" (Start working on it, doing
something wrong and being able to start once again from the instance
that we created) - You can take a snapshot and save the state. Next time when you start the VM it will be started from this state only.
Ability to share the environment - Export a created VM and it can be
reused.
Possibility of creating multiple different environments - You can run
multiple VMs on your machine. Configure the disk usage and RAM
accordingly.
Low disk usage if possible - Configurable while creating a virtual
machine.
Fast deployment of environment on local machine - Yes, you'll need
the .iso image of your Operating System
I'm trying to introduce continuous integration in an old project, and we've got quite specific situation - it's possible to put the CI server only on our test server that runs on CentOS. The server has quite a lot of unused RAM and CPU capability.
However, we need to run Ant builds on Windows (this also used to be how the project did packaging before), however it turned out that not the same output (after binary compare) is produced by just using Unix versions of Java and Ant.
I drew up a diagram of how in my mind it could work, but I'm really wondering whether that is even possible (with already given tools).
The black part is implemented, I'm curious whether the red part could be possible. Could the Jenkins slave communicate with master on different OS?
It should be possible. I have a feeling you will need to play with your network settings. But if before you start changing anything see if you can start a headless slave by following these directions: https://wiki.jenkins-ci.org/display/JENKINS/Step+by+step+guide+to+set+up+master+and+slave+machine
Using VirtualBox for CentOS, it will possible to run a Windows VM on your CentOS host.
I'm not sure you need Docker to launch your Jenkins slave.
It maybe better to use a standard JNLP Windows service to connect your Windows slave to Dockerised Jenkins master.
If the master is not able to view the Windows node using this method, you may have to tweak your network configuration on the Windows VM.
But I'm not sure it's necessary.
First with the background...
We have a Linux server that supports multiple projects.
The Clearcase server and repository are installed on this Linux server.
Different projects require different cross-compilers and libraries, and all of them are installed on the server.
User can choose different tool sets by running different scripts, which exports different environment variable values such as include paths and compilers.
User needs to run cleartool to mount the repository.
Developers develop in Eclipse and have two options:
SSH into the server and run Eclipse through with X11 tunneling.
Install Eclipse locally on their Windows machine and invoke builds from the SSH terminal.
Now:
Problem with #1 is that Eclipse operations (typing, content assist, etc) are extremely laggy.
Problem with #2 is that the developers need to go through extra hoops to build their code.
This is what I have tried:
Set up Remote System Explorer, which allows remote editing of files and remote running of the compiler:
How to build a c++ project on a remote computer in Eclipse?
This approach works perfectly for files that do not need special environment variable values and mounting of Clearcase repository, but I could not figure out how to get all of these things to integrate.
It would be great if someone can let me know how I can direct RSE to run a script (may be different per project) to set the environment variables and to run the cleartool commands to mount the repository so that it can locate the files.
The cleartool command arguments would be different per user for setting up a particular view.
Some extra info that may help:
I have root access to the development server
The Clearcase filesystem is mapped to a drive on the Windows machine
Thanks in advance for saving me hours of frustration dealing with a slow network!
==================
Additional detail per comments:
- The VOB storage is located locally on the Linux server. We would SSH to the server and start Eclipse there, therefore the delay should not be due to dynamic vs snapshot view and GUI performance seems to be the real problem.
- We also mount the same view on Windows by using Region Synchronizer. When running the local copy of Eclipse installed on Windows, there is no performance problems.
So this question can probably be solved by answering either question:
1. How to improve X11 performance such that development on Linux will suffice?
2. How to set up Windows Eclipse to perform all the steps mentioned above when building projects?
I came here a similar question to your part two, but alas, no one has answered it. However, I have an answer to your part one: https://www.nomachine.com/. It speeds up X11 forwarding considerably.
I am trying to create an application that runs on windows. I want this application to download a "disk image" from the network (from pre-assigned server) and create a virtual machine based on it. This VM would run for a specified number of hours and then shutdown.
I want to use VirtualBox by scripting it. I found VBoxManage command and it seems to be what I am looking for. However, it seems that VirtualBox tools store their configuration as XML files in User home directory. I did learn that i can change the value of VBOX_USER_HOME environment variable to control where they are stored. However, I am not sure whether this is enough.
My problem is that the user may already have installed VirtualBox on his/her computer. I do not want my application code (and it's packaged VirtualBox binaries and conf) to mess with the existing installation.
How do I cleanly isolate my application specific VirtualBox binaries and configuration from the potentially pre-existing installed VirtualBox setup? (Even if both instances of VirtualBox binaries are being used at the same time)
I chose VirtualBox because of it's open source license and applicability of commercial use (if I compile my own binaries from the source) and because it works on Windows too (heard QEMU support for windows is still not stable). Will VirtualBox suffice for my use-case or should I look elsewhere?
Thanks for reading so far :)
If the VBoxManage command really does rely on the environment variable VBOX_USER_HOME then you could write your scripts to change the environment variable to reflect your deployment for the execution of that script and its children, staying away from user data.
Check out the VirtualBox SDK http://download.virtualbox.org/virtualbox/SDKRef.pdf
Is there a way to run an Amazon EC2 AMI image in Windows? I'd like to be able to do some testing and configuration locally. I'm looking for something like Virtual PC.
If you build your images from scratch you can do it with VMware (or insert your favorite VM software here).
Build and install your linux box as you'd like it, then run the AMI packaging/uploading tools in the guest. Then, just keep backup copies of your VM image in sync with the different AMI's you upload.
Some caveats: you'll need to make sure you're using compatible kernels, or at least have compatible kernel modules in the VM, or your instance won't boot on the EC2 network. You'll also have to make sure your system can autoconfigure itself, too (network, mounts, etc).
If you want to use an existing AMI, it's a little trickier. You need to download and unpack the AMI into a VM image, add a kernel and boot it. As far as I know, there's no 'one click' method to make it work. Also, the AMI's might be encrypted (I know they are at least signed).
You may be able to do this by having a 'bootstrap' VM set up to specifically extract the AMI's into a virtual disk using the AMI tools, then boot that virtual disk separately.
I know it's pretty vague, but those are the steps you'd have to go through. You could probably do some scripting to automate the process of converting AMI's to vdks.
The Amazon forum is also helpful. For example, see this article.
Oh, this article also talks about some of these processes in detail.
Amazon EC2 with Windows Server - announced this morning, very exciting
http://aws.amazon.com/windows/
It's a bit of a square peg in a round hole ... kind of like running MS-Office on Linux.
Depending on how you value your time, it's cheaper to just get another PC and install Linux and Xen.