I have seen this answer Deploying SSAS cube to environments
Which describes how deployment of a SSAS cube can be automated, however what I can't see is how to configure the project to be able to deploy to multiple environments i.e. Development, UAT and Production, where each is on a different server and has a different data source.
I can see in the Visual Studio SSDT Analysis Services project that the configuration manager can be used to set multiple deployment configs in the USER SPECIFIC!! .dwproj.user file.
This allows me to set multiple SSAS deployment locations and when building for DEV or UAT etc.. the .deploymenttargets file gets set correctly.
What this doesn't do is allow me to set a different data source automatically during a build to auto change the .configsettings file.
Does anyone know how to do this?
I believe the dwproj file holds the different SQL connection strings per environment as described here:
http://www.artisconsulting.com/blogs/greggalloway/2008/3/19/analysis-services-project-configurations
I have seen a bug when you first set this up that the dwproj file is never flagged as dirty so those connection string changes are never saved to disk. If you have an issue with this I usually add a new Role object then immediately delete it. That marks the dwproj file as needing to be saved.
Related
I developed and have been following a development changes (aka Change Request / CR) methodology for all my web development for many years which includes the following procedural constraints:
Version control Trunk equals Production at all times except for small windows during production releases.
Development code is stored in its own independent directory structure and contain all files necessary for deployment of the changes to dev, test or production. This includes all DB schema and web resources in that change set.
Each CR directory will NOT contain any files that are not being modified in the change set.
In a team based environment, these change set directories would be branches in version control so all team members could see all open changes and also enable automated validation / CI.
Each Change is named with a unique identifier like a date/timestamp or a formal change identifier in an issue tracker like Jira.
When Deploying changes to dev, test or prod, the web directory can just be pushed over to the web server being tested by SCP/SFTP, sometimes using the cool WinSCP directory syncing tool.
When production release is complete and validated, files can be copied / merged to Version Control Trunk and the CR folder moved to an archive for future reference.
Now, I am now trying to get my head around how to maintain large web projects using Visual Studio whilst following my change set methodology and have hit a wall on how to make this work. It would be great to keep my changes separate but also be able to use Visual Studio to step through code to debug things when log based debugging is insufficient. The Web projects I am maintaining are massive.
Has anybody come across the ability in Visual Studio to have a Web Project based on two separate directory structures like WEB_BASE (containing the entire website) and CR1234 (containing the actively development code) and have Visual Studio use files by the following order of precedence: Use any files from CR1234 first and then WEB_BASE second, ignoring the duplicates in WEB_BASE?
Are there any other ways to keep track of the discrete changes required for change sets whilst using Visual Studio and minimizing the UI noise of scrolling through resources that you are not changing to find the files you are working on?
Here's the scenario: I have multiple developers on an asp.net mvc 4 project. Each developer has a local database. The source control system is TFS at http://tfs.visualstudio.com. We're using Azure websites to host the site. We have also configured Azure websites for continuous deployment.
The source control system could be git, mercurial, TFS, etc. Honestly, I don't think it matters.
My question is how to accomplish these three things:
Each developer has his/her own connection string(s) locally (without them being in source control)
Azure has its own connection string(s) (without it being in source control)
Source Control doesn't show any connection information
The ability for each developer to F5 and run/debug/test the app locally.
We accomplished #1 by adding our individual connection strings to our machine.config so that there's no conflict between developer workstation setups.
I originally removed the connectionstrings section from web.config. In the Azure website (using the management portal, under Configure), I configured the connection strings, and after watching a Scott Hanselman video was under the impression that these would be dynamagically merged into my web.config upon deployment, but that doesn't appear to happen. Whenever I go to any page that hits the db, I get an error saying can't find the connection string (or some other db error related to the connection)
If I put the Azure connection string directly in web.config, Things work on Azure, but then the connection details are in source control visible to everybody.
After reading some more posts from Scott and David Ebbo it seems that I could put a blank connection string in web.config (with the correct name) and then Azure will overwrite the values correctly. I would then have to have the developers put their connection strings in their web.debug.config and then install the Slow Cheetah plugin so that they could F5 and test locally. They would also have to not check in the web.debug.config into source control. (Not that easy with TFS) This seems like a seriously burdensome kludge, that's bound to fail somewhere along the line.
I have to believe that this isn't that rare of a problem. How do other teams accomplish this?
After looking around, it appears that what I was asking isn't actually supported without a bunch of command line hacks to the pre/post build process. What we ended up doing is forcing developers to all create their own local databases, use trusted authentication, and establish a SQL alias that was used by all developers in the web.config. That way, it works locally for everybody, it doesn't expose any user names/passwords within source control, and Azure can still overwrite it when automatically pulled from source control.
Slow Cheetah is actually a nice solution. It's an extension to web.config transformations. Those transformations let you keep one web.config file and then for each deployment scenario you specify which changes you want to it. For example, your Release configuration will probably remove the debug attribute.
This can also be used to change connection strings. The transformations are applied during the deployment of your project to Azure.
What I've done in the past to make this also work with local development machines is use a web.config with an externalized connections.config file. Each developer created a connection.machinename.config file that was copied to connection.config on build in the post-build step. Those files don't have to be checked in and they can never cause conflicts because each machine name is unique.
The release/staging/.. configurations used a web.config transformation to replace the connection string element with a specific connection string for that deployment (and that way remove the dependency on the external config file).
Slow Cheetah offers some nice helpers for checking the result of these transformations at design time.
Scenario:
In my organisation, we separately develop multiple applications. In the end however, many of the applications (and their databases) are deployed to the same SQL instance, so they share the same master database.
We use Visual Studio 2010 database and server projects to source control said databases.
To try and standardise some things, I want to do the following:
Create a 'Core' Server database project which has all the server settings, core logins etc. Things like SET TRUSTWORTHY ON and server-level ANSI settings etc.
Have each Application's own Server.dbproj specify the logins and roles etc specific to that application.
Have each Application's own ApplicationDatabase.dbproj reference the ApplicationX.Server.dbproj
In theory, each Application in source control would only contain the items specific to it, rather than keeping server related settings or configuration synchronised across many projects.
Problem
However, in practice I can get this far:
Done. Produces a .schema file which I reference in future steps
Done. Server.dbproj happily references Core.dbschema and 'extends' it with it's own logins and roles etc. Is happy to deploy this anywhere I point it.
Nadda. I add a reference from ApplicationDatabase.dbproj to Server.dbproj (assuming Server would pull in the items from Core) and it complains about any logins that are actually in Core.
So I then added both Server and Core as references to ApplicationDatabase as it settled down. Compiles fine.
However, when you deploy, you get the same problem described here: http://social.msdn.microsoft.com/Forums/uk/vstsdb/thread/23cb9132-00d4-42ed-b34c-ab49027cddf7
Error TSD01234: The source model contains 2 server option elements.
Only one element can be contained in a model that can be deployed
The problem I think is that ApplicationDatabase essentially has two Server projects that it knows about, and therefore duplicate settings.
Microsofts documentation makes no mention of using partial projects in Server projects, but neither is it listed as a limitation.
So the question is...
Has anybody used partial projects successfully for Server projects, or is there a way you can see to achieve the same thing?
I'll be honest and say I won't just 'remove the Server projects' to make the problem disappear - we had it working very well up until I tried to improve things!
I have an SQL 2008 database project in Visual Studio 2010 that is sync'ed on a regular basis from a schema comparison during the development phase. This same project is also under TFS source control. I have two environments, Debug and Production. Each environment is a single machine that runs both IIS and SQL Server. The production environment however has different data and log paths for the database D:\Data\ and E:\Logs\ versus my development server at the standard c:\program files\sql....\data.
What I'm trying to do is setup the way I transact my deployments from the debug to production environments. I've gotten WebDeploy 2.1 setup and I build my deployment packages in Visual Studio via the right-click context menu on the website project. I want to manually copy deployment packages to the production server via RDP, so there's no over the wire concerns here. The deployment package settings are setup to include all databases configured in Package/Publish SQL tab. In the Package/Publish SQL tab I don't pull data from data/schema from an existing database because I want to deploy from the SQL database project instead. So I just point to the pre-generated .sql script file located in my database project's /sql/release folder. To top it off, I generate the .sql script in the post-build events in the SQL project via VSDBCMD.exe /dd:- /a:Deploy /manifest:... so that a simple solution rebuild all, then website project deploy ensures I always have the latest .sql script in the deployment package.
This is great and all, but I have a major problem here I can't seem to overcome. It has to do with the database data and log files paths being different from debug to production environments. I actually receive an exception during the WebDeploy in IIS on the production server that says it can't find c:\programs files...\MyDatabase.mdf file. And what's scarey is after this exception, the entire database is deleted. The empty databases I create right before doing the deployment. Happen both times I tried messing around with it. I'm not sure how I feel about that, but I'm hoping I could find a reliable solution to this.
I have been feverishly looking for a way to change the paths during a deployment and have found many places that mention changing the paths in the *.sqlfiles.sql files under Schema Objects\Database level objects\Storage\Files because the path it tries to deploy to is the path specified in those because of the Schema Comparisons and Writes from the Debug SQL server database. Changing the paths here will work temporarily, until I do my next schema comparison and write, then the sqlfiles.sql files will get overwritten with the info from the Debug database again. And I don't want to have to remember to never update these files during a schema comparison because any mistake has the potential to delete the production database.
I think my salvation lies in my Release.sqlcmdvars file. It's a tease actually, I can see a place I "could" type the default database path, but it appears to a read-only field as it mentions "Location where database files are created by default (set when you deploy)." It would be grand if I could specify the paths here. Is there any way at all to specify the path in a variable here that would override the paths from the *.sqlfiles.sql files?
In the solution where I work at, there are two custom variables in the sqlcmdvars called Path1 and Path2 that I thought were reserved names that do such that. However, this doesn't work in my solution and the difference between the two solutions are the other solution gets deployed via TFS build controller. Doing the TFS build controller route isn't an option really because I opted out to save money while using a third party source control service.
Any help with this would be great. I have even gone so far as to create separate *.sqlfiles.sql files for debug and release and configured the dbproj file to use one or the other depending on the Configuration, but this doesn't seem to be working either. Also, using the custom PATH1 variable in the sqlfile.sql file like FILENAME = '$(PATH1)\Cameleon_log.ldf', doesn't work either. I seriously think it shouldn't be this difficult. Am I missing something simple here??
Thanks!
Okay, this was an exercise in futility. Apparently with out syncing with the target database during the script generation the script would be exactly what is needed to build the database from scratch. Even if I could override the file paths, the deployment would complain about database objects already existing. I needed to specify the connection string of the target database in the deploy settings so a comparison is done during the script generation and only the relevant differences are added to the script. I really wanted to avoid exposing my production SQL server to the outside world, but it is what it is. No need to override the paths anymore because it looks the database file paths are conveniently ignored during this comparison!!
Do I have to manually edit the Azure connection strings myself to switch between production and development, or is there something comparable to the Transformation Visual Studio applies to Web.Config?
To add to what Brent has said. I use a special small configuration-only (Config) project that contains a folder for every deployment type - inside each folder there is a collection of .config and .cscfg files that are tailored toward a specific deployment (a few partial .config files too). During every compile via Pre-Build event step, Visual Studio copies the files from the correct folder into the root folder of that Config project.
This is the command I use in the Pre-Build Event Command Line:
xcopy /Y "$(ProjectDir)$(ConfigurationName)\*.config" "$(ProjectDir)"
xcopy /Y "$(ProjectDir)$(ConfigurationName)\*.cscfg" "$(ProjectDir)"
Every other project in the solution links to the configuration files from the root folder of the Config project.
I also use config transformations as well, for Production vs. Non-production environments. Everything non-Production (local development environment, Azure-QA development environment) has a lot of debug and tracing built in - errors are returned completely to the clients/etc. Production environment has that locked down.
Edit: wrote a blog about this finally: http://www.paraleap.com/blog/post/Managing-environments-in-a-distributed-Azure-or-other-cloud-based-NET-solution.aspx
As Brent pointed out, it is not a good idea to have Staging area to be a full-blown Testing site. It is more geared toward a quick smoke test as well as a great way to deploy a new package into Azure without taking your main site down. (IP swap between Production and Deployment usually does not cause any issues to users)
Hope this helps
First a question, are you referring to configuration settings in the traditional configs? Or in the cloud service config (cscfg)?
If the later, then ATM I'm not aware of any support for configuration transformation. The approach I've seen most folks taking is to maintain number configuration files and use the appropriate file when doing your azure deployment.
This subject also touches on usage of the "Staging" environment. I've seen some folks using it as a parallel testing environment. In practice, its more affective to use it as a staging area to smoke test a new deployment before rolling it into production. If you have a need for a longer term test environment, I've found it better to deploy those services to their own unique namespaces that are then sandboxed away from the production services.
This again touches on ALM best practices for Azure (versioning, deployments, etc..). Something I believe the PNP team is working on and will hopefully have recommendations for soon.
Like knightpfhor said, you can use Visual Studio config transformations if you edit the .ccproj file manually. My answer to a similar question lists the minimum number of steps required to get transformations working.
You can use CloudConfigurationManager in Azure SDK 1.7 http://msdn.microsoft.com/en-us/LIBRARY/microsoft.windowsazure.cloudconfigurationmanager
This starts by looking in the ServiceConfiguration.cscfg e.g. ServiceConfiguration.Cloud.cscfg for config setting. If it isn't there it falls back to web.config and app.config
For example
CloudConfigurationManager.GetSetting("StorageConnectionString")
Will look in the appropriate cscfgfile for StorageConnectionString setting, then it will search the web.config and then app.config.
The simple answer to your question is yes, but you have to mess around with the .ccproj file manually to do it. A full description on how to to do this can be found here