I'm lost as far as how do I work with existing deployment of VMSS which I perfomed using template in powershell. For example I want all VMs to have additional extension installed now and this was not part of original template. How do I add this extensions to all machines?
You can simply deploy the template again. It will only deploy the difference (so make sure you use the same username/password etc!)
minor edit: if you have upgradePolicy.mode set to "Manual", you will also have to do a "Update-AzureRmVmssInstance" call on each VM you want updated; if it's "Automatic", it will go out to all VMs automatically in parallel; if it's "Rolling" (preview here: https://github.com/Azure/vm-scale-sets/tree/master/preview/upgrade), it rolls out in batches.
You can use the Add-AzureRmVmssExtension PowerShell cmdlet to add an extension. Install the latest version of Azure PowerShell if you have not already.
Or 'az vmss extension set' if using CLI, for example in the Azure Cloud Shell.
Related
My version of Visual Studio 2013 Release Management Client/Server is Release 4.0. I am using the ReleaseTfvcTemplate.12.xaml template. Is there any way to access the system variables from a release management component (deployer configuration) when using an AGENT based release template? I get NULL when I try to pass $applicationPath or $(applicationPath) or $(ApplicationPath).
Here is my status on the deployment of a component in RM that has a custom powershell deployment script:
I can successfully queue a release build
The build output is properly copied to \servername\BuildDrop\
I have an AGENT (not VNEXT) based RM template that gets triggered when the build is completed
The build properly triggers the RM template to release the build
This RM template has one component that gets deployed via a CUSTOM powershell script
The powershell script needs the full absolute path to the builddrop location in order to perform its work (what the powershell script does is not relevant to the discussion)
Example: In my deployment configuration for the component in RM, I need to pass the TFS build drop location without having to hardcode it
Based on the MSDN documentation there is a system variable $applicationPath which is hydrated by RM but ONLY when using VNEXT based templates, not Agent based templates
According to an accepted StackOverflow answer here related to accessing system variables from Agent Based RM templates, it would appear the MSDN documentation does not tell the whole story and I should be able to set a line in my component deployment configuration (See below)
As Graham mentioned in the comments, the build output is copied locally and it is your working directory when executing your Powershell.
You also can use this to access the build drop location: $(PackageLocation)
http://incyclesoftware.zendesk.com/entries/24422737-Deployment-Metadata
Just found a most recent and up to date post from Donovan Brown: http://donovanbrown.com/post/What-are-the-System-variables-for-PS-DSC-deployments-in-Release-Management.aspx
I need to periodically distribute an updated set of files (DLL, javascript) as a patch release to multiple windows PCs of various versions (vista/XP/7 etc.). Currently I have a link on my website for each user to download the setup.exe file and manually install the patch (replaces the existing DLLs). In order to eliminate the manual process, I'm researching how to run a program automatically on windows (I assume that means a service) which will check my server periodically for an update, download it and automatically trigger the setup.exe (or some similar process to get the patch files replaced).
I'm not a windows developer per se, but have downloaded and tried to check various options (in order to not reinvent the wheel) such as Sparkle, DDay, CSAutoUpdater, Npackd and just started looking into Google Omaha - but most assume that check happens when the windows application is started using their libraries/components. I need to run an autoupdater independently of the app I'm trying to update - so I need to first get all users to run a setup.exe which should install and start my windows service so that it can run with windows daily to run a program to check, download and install the update.
I found some sample code for writing windows service on stackoverflow, codeproject also, but couldn't find how to autoinstall it (ideally with no UAC hassles for newer windows versions). All required using instalutil or some other manual process to install and start the service.
Any guidance on this would be great! Thanks - and apologies to the long winded question. Will update with additional results as I try out other products.
As you mentioned Npackd I would like to describe how this can be done using it.
I agree with selbie that the simplest solution would be to just create a task with Windows scheduler.
For Npackd you would need your own repository - an XML file accessible via HTTP, for example as http://www.yourserver.com/rep.xml . Here is a simple example for a repository: https://gist.github.com/raw/4132983/dabecde48c796d4fdfa2f645bb744ac58640572c/TestRepository.xml . A user would download Npackd (http://code.google.com/p/windows-package-manager/downloads/list) and add the URL to the list of repositories.
I would define 2 packages: one for the program itself and one for the auto-updater. This way if somebody wants to update the program manually, he can do this too.
You could create or delete a Windows task using the command line tool schtasks.exe available on every Windows system.
The update command itself would be also very simple:
npackdcl update --package=<package>
In order to eliminate the manual process, I'm researching how to run a program automatically on windows (I assume that means a service) which will check my server periodically for an update, download it and automatically trigger the setup.exe (or some similar process to get the patch files replaced).
You likely want the Windows Task Scheduler service for the "periodic" part of your statement. You use that to launch whatever code you would need to actually do the "check for update", download, copy, etc... I don't think you need a Windows Service.
You can likely script a Task Scheduler event. Or do it more programmatically with the API.
You can do as follow:
Maintain the version of the of the dll,exe etc in DB.
Check the version of the Installed version with the version in DB, before the application start up.
If the versions are diff, ask the user to update to the latest version from server.
If he clicks on YES. Update the files from server.
Prasad.
AWS released a new AMI recently which has CloudFormation tools installed by default on their Windows Server 2008 R2. The AMI itself can be found here :
[https://aws.amazon.com/amis/microsoft-windows-server-2008-r2-base-cloudformation]
When using this AMI directly within a CloudFormation template and launching the stack, I am able to launch my stack easily and the instance downloads my files located in S3 without any problem during boot up, all the folders created by cfn-init command can also be seen as expected.
However, if I modify the AMI to customize (just enabling IIS) it and recreate a new AMI and use this AMI within the template, the files don't get downloaded neither are the other folders suppose to be created by cfn-init command can be seen.
Any suggestions ?! Am I missing something ?!
Most probable cause of this is that the custom AMI was created without using EC2Config Service's Bundle tab.
CloudFormaion support on Windows depends on EC2Config service's functionality of running commands specified in User Data on first boot. This functionality automatically gets disabled after first boot so that the subsequent boots do not cause re-runs of the same commands.
If the custom AMI is created using EC2Config's Bundle tab , it ensures that the resulting AMI has the User Data command execution functionality enabled. Hence it is necessary (and always recommended) to create the custom AMI using EC2Config's Bundle tab.
Hope this helps.
Regards,
Shon
It seems that the stock bootstrapping process is a bit lacking on Windows.
Linux has cloud-init which will install packages, store files, and run a bash script from user data.
Windows has ec2config but there is currently no support to run a cmd or powershell script when the system is "ready"--meaning that all the initial reboots are completed.
There seem to be third party options. For example RightScale has the RightLink agent which performs this function.
Are there open source options available?
Are there any plans to add this feature to Ec2Config?
Do I have to build this my self?
Am I missing something?
It appears that EC2Config on the Amazon-provided AMIs now supports "User Data Scripts" as of the 11-April-2012 updates.
The documentation has not yet been updated, so it's hard to tell if it supports PowerShell or just cmd.exe scripts. I've posted a question on the AWS forums to try and get some more detail, and will update here when I learn more.
UPDATE: It looks like cmd.exe batch syntax is supported, which can in turn invoke PowerShell. There's a new version of the EC2Config documentation included on the AMI. Quoting from it:
[EC2Config] will read in the user data specified for the instance and then check if it contain the tags <script> and </script>. If it finds both then it will take the information between those two tags and save it to a batch file located in the Settings folder of this application. It will then execute the batch file during the start of an instance.
The batch file will only be created and executed on the first launch of an instance after a sysprep. If you want to have the batch file created and executed again set the Ec2HandleUserdata plugin state to Enabled.
UPDATE 2: My interpretation is confirmed by Shon from the AWS Team
UPDATE 3: And as of the May-2012 AMIs, PowerShell is supported using the <powershell/> tag.
Cloudbase.it have opensourced a python windows service they call cloudbase-init which follows the configdrive and HTTP datasources.
http://www.cloudbase.it/cloud-init-for-windows-instances/
github here
https://github.com/stackforge/cloudbase-init/
I had to build one myself however it was very easy. Just made a service that reads the user-data when starts up and executes the file as a powershell script.
To get around the issue of not knowing when to start the service I just made the service start type as "delayed-auto" and that seemed to fix the problem. Depending on what you need to do to the system that may or may not work for you however in my case that was all I had to do.
I added a new codeplex project that already has this tool built for windows. Looking forward to some feedback.
http://cloudinitnet.codeplex.com/
We had to build it ourselves; we did it with a custom service and built our own AMIs. There's no provision currently within EC2Config to do it.
Even better, there is no easy way to determine when the instance is "ready". We had to do it by tailing the logfile of EC2Config.
I've recently found nssm (at nssm.cc) which easily wraps a simple batch file (or pretty much anything else) as a service. You can then us sc config servic1 depend= service0 to force the batch file to be run at a particular point in the service initialization sequence. I am using it in between ex2config and sql express to create a folder on d, for instance. You'll have to use the services tool to make it run as network services and change the AppExit property to Ignore using regedit, but it works once you get it all in place.
We are currently using VSS for version control. Quite few of our developers are interested in a distributed model (And want to get rid of VSS). Our network is full of Windows machines and while our IT department has experience maintaining Linux machines they would prefer not to.
What DVCS systems can host their central repository on Windows while providing..
Push access to the repository.
Basic authentication. Mostly just a way to allow or deny access to the whole repository. No need for fine grained access.
Server process so users don't need write right to the repository reducing the risk of accidentally messing with it.
On the client side a GUI such as Tortoise would be more or less a requirement (Sorry, Windows shell sucks. :|). Ease of installation would be a huge plus as our IT department is already quite low on resources. And using windows credentials for authentication would be an advantage but not a requirement as long as the client is able to store the credentials.
I have had a (really) quick look at Git, Mercurial and Bazaar.
Git seemed to use ssh or simple WebDAV for repository access, requiring write permission for the users.
Mercurial had a built in http server, but this seemed to be only for pull purposes. Update: Mercurial supports push as well.
Bazaar Seemed to use sftp for repository access, again requiring a write permission for the users.
Are there windows server processes for any DVCS systems and has anyone managed to set one up in a Windows land?
And apologies if this is a duplicate question. I couldn't find one.
Update
Got Mercurial working for push purposes! Detailed list what was required can be found as an answer below.
Mercurial's almost certainly your easiest option on Windows.
If you didn't care about authentication, you actually can trivially allow hg serve to permit push. To do so, you merely need to add the following to the .hg/hgrc file in the repository you wish to serve:
[web]
allow_push = *
push_ssl = false
The first line says that anyone may push to this repository. The second tells Mercurial to allow pushing without SSL, since hg serve does not currently natively support HTTPS. At this point, users can push to your repository without having an account anywhere. If you're simply a small shop, that's probably fine--especially since you can use Mercurial's ability to sign changesets to guarantee a much higher level of verifiability than HTTP Basic will provide, anyway.
For a larger, shop, though, you'd be totally right in wanting at least a simple barrier for committing. To do that, you need to make two changes. First, you'll need to put Mercurial behind a web server with either reverse proxy support or CGI support. Thankfully, recent versions of IIS support both. You can consult the CGI directions in the Mercurial Redbook for Mercurial-specific steps, and Microsoft's guide to setting up CGI applications in IIS 6 for help on the IIS side.
Next, you'll need to set up some basic authentication. IIS provides HTTP Basic out-of-the-box, which, as a bonus, can authenticate directly against your domain, keeping administrative overhead to a minimum.
Finally, you'll want to change the allow_push line to support only specific users by specifying a comma-delimited list of user names. For example:
allow_push = benjamin, ted, the_cow
That's it. Mercurial will now allow push from users who can authenticate via HTTP Basic authentication, and allow pull from everyone else.
After Benjamin pointed out the HTTP serving CGI scripts I decided to try those out and managed to get a repository hosted over HTTP. The Redbook which Benjamin linked was of much help as were two Mercurial wiki articles. One which describes Mercurial publishing in general and another containing step by step instructions for setting up the HgWebDir CGI script.
These instructions weren't completely foolproof though so I had to poke around a bit. Most likely as I'm running 64bit Vista. The instructions below document what I did. Now that I've done it once I'd probably do things in another order so don't consider these step by step instructions.
Mercurial
First I acquired the Mercurial binary from http://mercurial.berkwood.com/ which got installed into d:\dev\Mercurial. I created a repository for testing under d:\dev\testRepo repository using hg init. The d:\dev\Mercurial\library.zip contains Mercurial library files required by the CGI script so they were extracted to d:\dev\Mercurial\library. Something which confused me at first is that when I opened the zip file I received an error message and saw no contents. Just extracting the file to a directory worked though.
For the web script, I downloaded Mercurial source which contained the hgwebdir.cgi which got moved and renamed to d:\dev\Mercurial\webroot\hgwebdir.py. The step by step article contains good instructions for modifying the hgwebdir script for Windows. They also contain instructions for hgweb.config which in my case ended up looking like this:
[paths]
/hg/hgwebdir.py/test = D:\dev\Mercurial\testRepo
Also the repository wanted the following config so I could push there without SSL. Note I am using Basic Authentication to authenticate users currently. I had to create the config in D:\dev\Mercurial\testRepo\.hg\hgrc and add the following lines to it:
[web]
allow_push = *
push_ssl = false
Python
The CGI script is a Python script so it requires Python. It's seems pretty picky on which Python version executes it. One of the articles mentioned that running it requires same version that was used to build the Mercurial. In the end I got it working on Python 2.5 x86 after trying Python 2.6 x64, Python 2.4, Python 2.5 x64.
IIS
Two things I missed and had to install were CGI support and Basic Authentication. Both of these were installed through Control Panel, Programs and Features. Once done with installation I created a virtual directory (Which I later changed to an Application) in IIS pointing to D:\dev\Mercurial\webroot. The virtual directory required an CGI handler for *.py files which could be added from Handler Mappings. The executable was D:\dev\SDKs\Python25_x86\Python.exe %s. Once IIS had permissions to the webroot directory I could navigate to http://localhost/hg/hgwebdir.py/test and see the repository.
So now the read access was working. When I tried pushing to the repository I received weird error messages telling me it wasn't a real repository.
After an hour of debugging I ended up copying the whole D:\dev\Mercurial\library\mercurial tree under webroot so that Python could find D:\dev\Mercurial\webroot\mercurial\hgweb\hgwebdir_mod.pyc. After this Wireshark was reportting Access Denied errors in the stack trace. No idea what the real reason to this was but changing the virtual directory into an Application in IIS and moving it on top of an application pool which ran using Local System account the access denied errors went away.
Also at some point I gave HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\WinSock2\Parameters registry key more permissions so IIS could access it. Doubt that it requires these after using Local System account.
Once these were done pushing stuff to the repository using hg push http://localhost/hg/hgwebdir.cgi/test was working!
Problems and solutions
Where to find the library files.
They were in the library.dll under Mercurial installation folder. I just had to extract them even if my unzip program refused to view me its contents.
How to get the Python script to run
Download the correct Python version for x86 architecture as the script uses some x86 libraries. The correct Python version depends on the Mercurial version. For 1.2.1 it was Python 2.5 x86.
Alternatively you could try building Mercurial from sources with whatever Python version you want but in my case this failed when building extensions.
How to set CGI up in IIS
First make sure CGI is installed in IIS. This wasn't assumed to be true in the IIS instructions Benjamin posted.
Create a new Module Mapping for *.py in IIS Handler Mappings. The correct Module is CgiModule and the executable is your Python executable + %s
How to allow the CGI script to write to the repository
Make sure the script has everything it requires. I had to move the library\mercurial\hgweb\hgwebdir_mod.pyc to another place.
Make sure the script has permissions to everywhere it wants. I solved this by Creating a new Application Pool for the CGI script that used Local System account, converting the Virtual Directory to an Application in IIS and selecting the new Application Pool.
After reading Mikko's Answer which almost worked for me, I came up with my own notes for installation. My setup was designed to be a "non protected and open" repository that members of my team could use installed on a Windows 2008 Server.
1. Install Python.
The version of Python I used was Python 2.6.2 and I used the Windows x86 MSI Installer.
Install for all Users.
Install to C:\Mercurial\Python
Use Default Feature Options.
2. Install MinGW.
The version of Minimalist GNU for Windows I used was MinGW 5.1.4
Install the MinGW-5.1.4.exe.
Choose the Download and Install Option.
Choose the Current Package Option to Install.
For the Components to Install Select the "Minimal" option.
Install to C:\Mercurial\MinGW
3. Modify your path.
You need to add in locations to your environmental path at this point.
Add 'C:\Mercurial\Python26;C:\Mercurial\MinGW\bin' to the path (Order Matters.)
4. Install Mercurial.
The version of mercurial that I used was the latest release in the stable branch and I did not use the binaries, but used the source code. I wanted to compile mercurial myself so that it would work with whatever version of Python I had installed so I didn't have to worry about any compatability issues which I found to be the biggest challenge with other install methods. The easist way to get the source is by downloading the "zip" file.
Mercurial Stable Release
Extract Zip File to C:\Mercurial\Source.
Build the Source at command prompt.
python setup.py build --force -c mingw32
python setup.py install --force --skip-build
5. Modify your path.
You need to insert into your environmental path another location for the 'hg' command.
Add 'C:\Mercurial\Python26\Scripts;C:\Mercurial\Python26;C:\Mercurial\MinGW\bin' to the path (Order Matters.)
6. Create your Config file.
You need to have a default user name set if your going to do any commits locally on this server.
Create file '"C:\Documents and Settings{username}.hgrc"'
[ui]
editor = Notepad
username = your_name
6. Test your Install.
Open up a new command window and test with 'hg debuginstall' to validate. You should see something like the following.
Checking encoding (cp1252)...
Checking extensions...
Checking templates...
Checking patch...
Checking commit editor...
Checking username...
No problems detected
7. Setup Web Directory.
Create Directory 'C:\Mercurial\Web'
Copy the hgwebdir.cgi file from the 'C:\Mercurial\Source' to 'C:\Mercurial\Web'
8. Configure IIS7 for Centralized Repository.
I used the DefaultAppPool which is using .Net 2.0, Pipeline=Integrated, Identity = ApplicationPoolIdentity.
Ensure CGI features are available in IIS7.
Control Panel/Programs/Windows Features/IIS/App Development Features/CGI
Add App into IIS on the Website you wish.
Alias=Mercurial -- Physical Path=C:\Mercurial\Web
On the App select HTTP Modules and add a new Module Mapping.
Request Path=*.cgi, Module=CgiModule, Executable=C:\Mercurial\Python26\python.exe %s, Name=Mercurial.
When Prompted to add entry to ISAPI and CGI restrictions list say yes.
9. Test your Web Setup.
You should now be able to browse http://localhost/Mercurial/hgwebdir.cgi and see and empty repository list.
10. Configure IIS7 for Friendly URL
I did not like having the unfriendly URL and this step allows us to remap the URL to something more friendly. Install the URL Rewrite Moduel 1.1 Extension for IIS.
On the Mercurial IIS Application in IIS Manager featurs View select URL Rewrite Component and install a new Rule.
Choose Add Rules, then the Template 'Rule with rewrite map.' Rule Action=Rewrite, Specify Rewrite Map=Mercurial
Add a mapping Entry. OriginalValue='/Mercurial/Repo', New Value='/Mercurial/hgwebdir.cgi'
11. Create Mercurial Repository
You can now create a test repository.
Create a Directory C:\Mercurial\Repository and ensure IUSR account has the permissions to write to the directory. (If on Domain account is more like IUSR_{ComputerName}.
Create file C:\Mercurial\Web\hgweb.config to list the repositories.
[paths]
/ = C:\Mercurial\Repository\**
Add a directory C:\Mercurial\Repository\Test and initialize the repository with 'hg init'
** If you want now to be able to push without ssl create in the .hg directory of the repository a hgrc file the following lines.
[web]
allow_push = *
push_ssl = false
References:
Mercurial Wiki Windows Install
HG Book
Step by Step
Publishing Mercurial Repositories
For a team taking the first step away from VSS I would have suggested using SubVersion for source control and either TortoiseSVN or VisualSVN for the client.
But if the team has made the decision to switch to a DVCS then I'd suggest Mercurial because of it's better support for HTTP and windows on the client via TortoiseHg.
If you're looking for:
Distributed development support
Run Windows servers seamlessly
And a great GUI
You're exactly describing Plastic SCM
Excuse my necroposting and shameless self-promotion, but I've just released an alpha version of HgLab, which is a Mercurial Server for Windows with full pull-push support and Active Directory integration.
SCM-agnostic (to some degree) Windows-solution with Repository-frontent and management today may be SCM-Manager (Git, Mercurial, SVN repo out of a box with a single requirement of JVM)