In Windows 7(i.e. MSI 5.0), there is a property called MSIFASTINSTALL which will improve the performance of your installer. Or else, you can turn off the rollback option by setting property DISABLEROLLBACK. This property is available in earlier version of MSI 5.0 too.
Please share your knowledge to improve the install experience. Also, I cannot find the right way to improve the performance of Uninstall. We use huge set of files/folders (more than 70,000) and components like 35000. It hangs in the file costing process and do not know how to avoid this delay. Sometime it hangs for more than 2 or 3 hours to uninstall in XP or Vista machines
Edit:
I did some hack in my install by zipping the folders which has huge file sets and reduced the components size like Christopher said. It improved the performance drastically. yes ofcourse, I lost the MSI installer pattern by doing this concept and it is not recommended approach. However, it is trade off when we want to achieve this and our user really do not want to have file version details when we uninstall/upgrade the patches.
I had a similar situation, though the number of files was a bit less, about 25k. Most of those files were icons, which were never changed from one release to another. Only a major release (once per 2 years) might bring some changes to this area. A "quick & dirty" solution was zipping those icons and include this single file into the installation (not a component, just a file side by side with the MSI). During the installation this ZIP was extracted in the background thread, and RemoveFile table was used to delete icons on uninstall. It was faster than installing those 20k icons as separate components, even as components with many files. A good and correct solution was to convince the main application developers to put all those 20k icons into the 20 zip archives. Now these 20 zip files are installed as regular MSI components, and the application knows how to extract an icon on demand and cache it.
I would not recommend you to disable rollback. Though you'll save quite some installation time, you lose a standard guaranteed rollback option.
Uninstall takes more time than install because of rollback feature again. The way I understand it, when you uninstall, the MSI firstly creates a copy of every single file, then uninstalls every single file, and in case of success, drops every single backed up file. Hence, the uninstall time is about three times as much as the install time. I experienced the same problem when I took a default option to have 1 file per component. Though it is recommended, you should make a trade-off if you deal with an outstanding case.
Hope this clears up the things for you a bit.
The best option for improving the performance of your app is to reduce the number of files and components. While there may be a couple tweaks you can do to your MSI to improve the performance, the excessive number of files/components is the core issue and will be the gating factor on any performance improvements you make. Why do you need to install 70k files?
Related
For my Wix project I am harvesting 4 directories, via the pre-build-event of visual studio, which will result in about 160mb of data, and about 220 files, but the build process tooks very long.
How can i speed that process up? I have one embedded media.cab file which will hold all the files. Is it the size or the amount of files that will slow the process down? Or is it the harvesting with the heat tool in the pre-build-event? Would it be faster with the HeatDirectory element?
Anyone made some experience with speeding this up?
For us, the vast majority of the time was spent invoking light (for the linking phase).
light is very slow at compressing cabinets. Changing the DefaultCompressionLevel in the .wixproj from high to mszip (or low or none) helps a lot. However, our build was still too slow.
It turns out that light handles cabinets independently, and automatically links them on multiple threads by default. To take advantage of this parallelism, you need to generate multiple cabinets. In our .wxs Product element, we had the following that placed everything in a single cabinet:
<MediaTemplate EmbedCab="yes" />
It turns out you can use the MaximumUncompressedMediaSize attribute to declare the threshold (in MB) at which you want files to be automatically sharded into different .cab files:
<MediaTemplate EmbedCab="yes" MaximumUncompressedMediaSize="2" />
Now light was much faster, even with high compression, but still not fast enough for incremental builds (where only a few files change).
Back in the .wixproj, we can use the following to set up a cabinet cache, which is ideal for incremental builds where few files change and most cabinets don't need to be regenerated:
<CabinetCachePath>$(OutputPath)cabcache\</CabinetCachePath>
<ReuseCabinetCache>True</ReuseCabinetCache>
Suppressing validation also gives a nice speedup (light.exe spends about a third of its time validating the .msi by default). We activate this for debug builds:
<SuppressValidation>True</SuppressValidation>
With these changes, our final (incremental) build went from over a minute to a few seconds for a 32 MB .msi output, and a full rebuild stayed well under a minute, even with the high compression level.
WiX Help File: How To: Optimize build speed. In other words: 1) Cabinet reuse and 2) multi-threaded cab creation are built-in mechanisms in WiX to speed up builds.
Hardware: The inevitable "throw hardware at it". New SSD and NVMe disks are so much faster than older IDE drives that you might want to try them as another way to improve build speed and installation speed. Obvious yes, but very important. It can really improve the speed of development. See this answer.
Challenges with NVMe drives?: 1) They run hot, 2) they usually have limited capacity (size), 3) they might be more vulnerable than older 2.5" drives when used in laptops (I am not sure - keep in mind that some NVMe drives are soldered solid to the motherboard on laptops), 4) data rescue can be a bit challenging if you don't have good quality external enclosures (form factor etc...), 5) NVMe drives are said to burn out over time, 6) They are still somewhat pricey - especially the larger capacity ones, and there are further challenges for sure - but overall: these drives are awesome.
Compression: You can try to compile your setup with a different compression level (for example none for debug builds). No compression makes builds faster. Here are illustrations for doing the opposite, setting higher compression (just use none instead of high for your purpose):
CompressionLevel: Msi two times larger than msm
MediaTemplate: How can I reduce the size of a 1GB MSI file using Orca?
A related answer on compression: What is the compression method used by MSI files?
Separate Setup: If you still go compressed, you could put prerequisites and merge modules in a separate setup to avoid compressing them for every build (or use release flags if you are in Installshield, or check the Preprocessor features in Wix).
External Source Files: I suppose you could use external source files if that's acceptable - then you don't have a lengthy compression operation taking place during the build, just a file copy (which keeps getting faster - especially with flash drives).
Shim: Another technique is to shim all the files you install to be 1 KB if what you are testing is the setup itself and its GUI and custom actions. It is then just a "shell" of a setup - which is a great way to test new custom actions to your setup. Many have written tools for this, but I don't have a link for you. There is always github.com to search.
Release Flags: Another way to save time is to use special release flags (Installshield only) to compile smaller versions of the setup you are working on at the moment (leaving out many features). WiX has similar possibilities via its preprocessor. More on WiX preprocessor practical use.
Debug Build: I usually use combinations of these techniques to make a debug build.
I normally use external source files when I experiment and add new features and keep rebuilding and installing the setup all the time.
Release flags to compile only part of the setup, cabinet reuse and release flags combined can save a lot of time depending on the size of your setup, the number of files and your hardware configuration.
Perhaps the most effective is a separate setup in my opinion (provided it is stable and not changing that often). Beware though: Wix to Install multiple Applications (the problems involved when it comes to splitting setups).
My take on it: go for a prerequisites-only separate setup. This is good also for Large Scale Deployment scenarios where corporate users want to use their own, standardized prerequisites and are annoyed with lots of embedded "junk" in a huge setup. A lot of package preparation time in large companies is spent taking out outdated runtimes and prerequisites. You can also deliver updates to these prerequisites without rebuilding your entire setup. Good de-coupling.
Links:
How can I speed up MSI package install and uninstall?
Simply put, don't harvest files. Please see my blog article: Dealing with very large number of files
The third downside is that your build will take A LOT longer to
perform since it's not only creating your package but that it's also
authoring and validating your component definitions.
I maintain an open source project on CodePlex called IsWiX. It contains project templates (scaffolding) and graphical designers to assist you in setting up and maintaining your WiX source. That said, it was designed around merge modules which slows the build down a bit as the .MSM has to be built and then merged into the .MSI. Pure fragments would be faster if you are really concerned about pure speed. That said I have many installers around 160mb and it doesn't take long at all.
And of course don't forget about having a fast build machine. CPU, RAM and SSD disk I/O all contribute to fast generation of MSIs. For my consulting, I use Microsoft Visual Studio Online (VSO). I have a Core i7-2600k Hyper-V server with 32GB of ram and a Samsung 850evo SSD. My build server (VM) runs a TFS proxy server for local SCC caching.
For fun, on the above machine, I took a 220 files from my system32 folder totaling 160MB. It took 30 seconds to build the MSM and 30 seconds to build the MSI for a total of 60 seconds. This is 'fast enough' for me. I would expect an MSI authored using only fragments to take 30 seconds.
I have a WIX 3.0 installer that is building 88 slightly different builds (cross product of 32 and 64-bit, 11 locales, four editions (Beta, Retail, Evaluation, Different Evaluation).
Each build has slightly different contents in addition to localized UI, so I can't just build one configuration with multiple locales.
The resulting MSI is about 120MB. I'm already using the CabCache.
The installer takes about 3-5 minutes per release to build, resulting in a pretty lengthy overall build time.
The install appears to be heavily disk bound during linking (light.exe).
Clearly making the disks faster could help. Does anybody have advice on how to set up a machine that could crank through these installers faster? (or advice on reconfiguring my WIX project to build more efficiently?)
Get an SSD. Like one of those with internal RAID architecture from e.g. OCZ. SSD is every developer's upgrade of the decade. Plus more RAM if swapping is an issue.
If you have common parts (that are not localized) you can create a merge module with the common parts and then just add the differencing stuff to each build.
I am not sure if you have any say or communication with the developers of the application that you are installing, but if you have to create that many MSI's mainly because of languages, have you considered just offering one Language MSI that delivers all the language specific files to a resources directory and then the user can choose which language they would like to use (but only install this if they need something other than the default language). Also it might be worth looking into having the product made in such a way that the user can pick from within which language is best, then having all the languages installed from the start.
As for your question about speeding up the build, that is a tricky one. Using Merge Modules I would rule out right away, as I don't see any actual gain coming out of that. Of course updating the hardware (as you said) will give some results, but again, I am not sure how much of a jump you would be making so it is hard to tell what kind of gain that would give. I think it might be best to go over your WXS with a fine tooth comb and see what is really going on in there. You can sometimes find things that are left over from the developement of the package, or from a previous tool that are really slowing you down. One example would be that my company recently switched to WiX from a more automated setup creation utility (leaving the name out on purpose cause I am listing the problems with it :P ) and it automatically created every folder under Windows that might possibly be needed in the running of a windows application, as well as the common files folder, the current user profile, and many many more. I think I ended up erasing in all over 100 empty directories that this old technology was nice enough to add for me. That is just one example of optimization that was done. It is amazing what can be found when you take the time to REALLY review what is going on under the hood.
In your wixproj setup file add this just before the end of file in <PropertyGroup> tag
<IncrementalGet>true</IncrementalGet>
This will tell WIX to compile only those files which are changed after the previous build.
I need to find out whether my apps are being flagged as viruses by the most popular anti-virus packages (not best, but biggest by user base). I therefore would like to know how others go about this. Some background:
I have an application written in Delphi. Ever since the Delphi virus was found, I've had problems with false positives on my applications, particularly my demonstration versions for some reason (they all share the same code). AVG has been good, and I can now whitelist my files easily, but then I got the latest DevExpress installer and it was false-positived too. Given this is getting more widespread, it struck me that I need to find out if my apps are being flagged by the most popular anti-virus packages. I therefore would like to know how others go about this. I don't want people to be downloading our demonstration versions, getting an AV warning, and deciding not to try it.
The only options I have so far are buying a load of AV packages and putting them in a VM, or using a service like VirusTotal. The latter seemed an ideal option but for the fact that they limit the test to files under 20Mb, and my files are bigger than this. There is no paid for option either to expand the capability. (I thought this an odd limit, but Kaperskis free checker is limited to 1Mb!)
How do you check your applications?
VirusScan.jotti.org and VirusTotal.com may help
http://online.us.drweb.com/
i couldn't see any file size limit on it
My thoughts on this are as follows:
I set up a computer (nothing special) with a lot of disk space. I'll call this the ScanPC. Every time I do a build, the script will copy the new files to the ScanPC into a build specific directory. This will ensure that I have an archive of all builds that can be examined. Any one may have been released to customers.
Now, I then install VMWare server, and set up a number of virtual PCs. In each, I set up the anti-virus software to scan the network share, but in a read-only mode so that no scanner can accidentally modify or remove the false positive. Each VM can then be automatically updated from the vendor, and hopefully they will have an email option to tell me when they spot a virus, which I will then know is a false positive and can report to the vendor.
The benefit of this is that I have a complete build archive (something I need anyway), and it means that old versions out with customers that trigger the AV are identified as well as the most recent. It means I can add or remove AV products as appropriate. It means that I only need a single computer (performance is not important).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I recently had to struggle with one installation project (which uses most popular product for creating installations: InstallShield) to make it work for product upgrades (migrating from one version to another). In the end it turned out that I needed to use one long package code but was using some other. It wasted my 8 hours (testing and debugging installers is a pain).
Now if I think about it, once you are done all the hard part of coding, all you want to is that correct applications, libraries are copied to target computer and user just runs it. Period. This apparently simple task normally turns out to be a tricky one and "being closed to finish date" makes in even harder.
Don't you think deploying a product is made damn difficult on windows which should have been simpler? (or installer really deserves that much attention and I am just being crazy about it?)
Have you ever used simpler deployment schemes such as "copy the folder to wherever you like and run the exe. When you want to remove it, just delete the folder!"? Was it effective and made things simpler?
Painful as it is you need to wrestle with the windows installer for the benefit of your customers. Otherwise you will need to do a lot more work to
Handle situations where for some reason an error occurs during the installation. What do you do next?
Handle issues like security. What if the installing user does not have rights to particular folders/registry keys?
Correctly cleanup after installation
Patching and patch management
Performing additional tasks -- registering COM objects, creating databases, creating shortcuts, creating an un-installation shotcut and so on
Installing prerequisites
Letting users choose which features to install
Your own custom scripts to solve all these problems eventually become a bigger problem than the installation itself!
I recommend that you check out Wix. It's not exactly child's play but it gets the job done. If you install Votive as a visual studio add in you get intellisense to help you strucutre the tags correctly. With the help file you can create pretty functional flexible installations
I don't think you'll see too many disagreements here, especially regarding MSI. I think one thing to keep in mind is to watch the way many programs are using MSI files these days. Displaying UI dialogs and making complex configuration choices with an MSI is very weak simply due to the way Windows Installer was designed, so I've noticed a lot of programs being split into a bunch of baby MSIs that are installed with the minimal UI by a parent setup program. The SQL Server 2008 setup wizard does this. UPS WorldShip does this. And Paint.NET does this, too--the wizard you see is a Windows Forms app, and it launches msiexec itself (you can see the minimal UI of the Windows Installer pop up on top of the white wizard window), passing any configuration parameters as property arguments to msiexec.
A common scenario where this comes up is where someone is tasked with building an installer for an application that has both server and client counterparts. If the user chooses the server option, then they may or may not want a new database to be installed, which means installing SQL Server. But you can't just install SQL Server while you're in the middle of your own installation because Windows Installer won't let you do that. So a frequent solution is to write an app that displays a wizard that allows the user to configure all of the setup options, and then your app launches the MSI files as needed for SQL Server, your server application, and your client application in the minimal UI mode; basically, eschewing the "features" aspect of Windows Installer entirely and moving it up to the MSI level. 4.5's multiple-package installations seems to be a step further in this direction. This format is also especially useful if you also need to loop in non-MSI installers from third parties as part of your installation process, like installing a printer driver for some bizarre point of sale printer.
I'll also agree that Windows Installer lacks built-in support for common deployment scenarios. It's meant for when setup isn't XCOPY, but they seem to miss the fact that setup usually isn't just "files + shortcuts + registry keys," either. There are no built-in actions for setting up IIS Web sites, registering certificates, creating and updating databases, adding assemblies to the GAC, and so on. I guess they take the opinion that some of this should happen on first run rather than being a transactional part of the install. The freely available tooling and documentation has been awful--flat out awful--for the better part of a decade. Both of these issues are largely addressed by the WiX project and DTF (which lets you finally use managed code custom actions), which is why we're all so grateful to Rob Mensching and others' work on that project.
I've had the same experience. Installation can quickly suck up your time as you go down the rabbit hole of "Oh God, I guess I have to become an expert in this too." I second the idea that's it's best to address it early on in your project and keep it maintained as part of your build process. This way, you can help avoid that scenario of having developed a practically uninstallable product. (Trac was an example of this for a while, requiring to track down specific versions of weird Python libraries.)
(I could go on about how Windows Installer sometimes decides to use my slow, external USB hard drive as a place to decompress its files, how it seems to sit there doing nothing for minutes on end on computers that have had lots of MSI installs on them, and how that progress bar resetting itself a bazillion times during a single install is the most idiotic thing I have ever seen, but I'll save those rants for another day. =)
My two cents; please note that I really just know enough about Windows Installer to do damage, but this is my assessment coming from a small business developer just trying to use it. Good luck!
Well, its a lot easier if you build your installer first, make it part of your build system, and let it grow with your project.
I agree, the windows installer drives me insane. But there are a lot of situations that xcopy just doesn't solve. Sometimes you want to install for multiple users, not just the current user. Sometimes you have to register COM objects. Sometimes you have to make a whole bunch of changes to the system, such as registering services to run at startup, connecting to network servers, etc. Sometimes you have users that can't use a command prompt. And you always want to be able to role the whole thing back when something fails halfway through.
Was the whole MSI database approach the best way of doing it? I'm not sure. Would I rather pound nails into my head than write another line of WiX code? Probably. But you have to admit, it does a good job of doing everything you could ever possibly want. And when it doesn't there is always the CustomAction option.
Really, what I would like to see, is better documentation (really, what is a type 50 action? How about giving it a name?) and a lot more easy-to-usurp templates.
And the WiX users group alias does a good job of answering questions.
You should read RobMen's blog. He does a good job explaining why things are the way they are. He has done a lot of thinking (more than any human should) about the problems of setup.
Have you looked at NSIS: http://en.wikipedia.org/wiki/Nullsoft_Scriptable_Install_System ?
And 1: Yes, 2: No
Personally, I mostly agree with #Conrad and #John Saunders. I wrote about this topic a long time ago on my old blog. I think #jeffamaphone has a point about the Windows Installer complexity (and my over attention to setup, in general ) but I believe the Windows Installer is still the best all round option for installation on Windows.
"Once you have done all the hard part of coding", you haven't done a thing if all your hard work doesn't install. Installers need to be built and tested on every nightly build, every night, almost from day one. You need to test that the installer can be built and run, and you need to verify the installation.
Otherwise, who cares how much hard work you've done coding - nobody will ever see your work if it doesn't install!
Note that this also applies to XCOPY.
Another thing: what is your QA testing if they're not testing what your installer installs? You have to test what the customer will get!
For exactly the reasons you state, we've done internal releases, handled by the dev team by copying the required files, and then done the rest of the setup using scripts and our own utilities.
However, for end users you have to have some kind of hand holding wizard, I've used the MS installer from within VS and found it confusing and clunky. After that experience I've avoided the pain by getting others to do the installation step. Can anyone recommend a good .Net installer?
I use Installshield and if you are not trying to do anything too fancy (I why would you) then it's pretty straighforward - set initial setting, select files, set up shortcuts and create setup.exe.
All future updates I handle inside my code - much more convinient to the user
Pretty much we've all done an installer here and there - and all of us did an installation of some behemoth of a program. Why do some installations take so much time? Case in point: Adobe CS suite (with newer versions you can take a vacation) or Visual Studio.
I know there are files to copy - most of the time unpack even. There are some registry keys to set (if under Windows), maybe a service or couple to start. Some installations probably even check hardware/software combination. All of this does not justify sllloooow installation time in some of the programs.
How can I speed it up?
It obviously depends what you're installing As Colin Pickard pointed out, you'll be shifting huge quantities of data onto the disk (+optional virus check etc.).
For installations I've built recently, we have to request the shut down of some Windows services, wait for that, and check that they really have shut down before continuing. That takes time.
I confess that in the above, that's not parallelised, whereas it could be. I suspect that installations are not necessarily optimised. They may well be the last thing that the team put together prior to release, and they may well figure that you're only going to do it once (and forget the pain upon completion). Obviously not an ideal state of affairs!
Visual Studio on my machine is 3.03GB - 16,842 files in 1,979 folders. Passing 3GB through virus scan and auditing software and onto the filesystem is too much for my (dualcore,2GB,sata2) system - it's CPU or IO bound the whole way through the process. That's why it takes so long.
Most installers not only pack, but also compress their contents, so at installation time all of these files must be decompressed. All of the data that is decompressed must be written to disk after it is decompressed as well.
Look at the time a zip operation takes on several files. It's also slow.
Many installers maintain a log that is flushed to the disk after each primitive operation so that even if installation encounters a fatal failure the log is preserved and can be sent to the software vendor. Such flushing sums up and significantly contributes to overall time.