I have a WIX 3.0 installer that is building 88 slightly different builds (cross product of 32 and 64-bit, 11 locales, four editions (Beta, Retail, Evaluation, Different Evaluation).
Each build has slightly different contents in addition to localized UI, so I can't just build one configuration with multiple locales.
The resulting MSI is about 120MB. I'm already using the CabCache.
The installer takes about 3-5 minutes per release to build, resulting in a pretty lengthy overall build time.
The install appears to be heavily disk bound during linking (light.exe).
Clearly making the disks faster could help. Does anybody have advice on how to set up a machine that could crank through these installers faster? (or advice on reconfiguring my WIX project to build more efficiently?)
Get an SSD. Like one of those with internal RAID architecture from e.g. OCZ. SSD is every developer's upgrade of the decade. Plus more RAM if swapping is an issue.
If you have common parts (that are not localized) you can create a merge module with the common parts and then just add the differencing stuff to each build.
I am not sure if you have any say or communication with the developers of the application that you are installing, but if you have to create that many MSI's mainly because of languages, have you considered just offering one Language MSI that delivers all the language specific files to a resources directory and then the user can choose which language they would like to use (but only install this if they need something other than the default language). Also it might be worth looking into having the product made in such a way that the user can pick from within which language is best, then having all the languages installed from the start.
As for your question about speeding up the build, that is a tricky one. Using Merge Modules I would rule out right away, as I don't see any actual gain coming out of that. Of course updating the hardware (as you said) will give some results, but again, I am not sure how much of a jump you would be making so it is hard to tell what kind of gain that would give. I think it might be best to go over your WXS with a fine tooth comb and see what is really going on in there. You can sometimes find things that are left over from the developement of the package, or from a previous tool that are really slowing you down. One example would be that my company recently switched to WiX from a more automated setup creation utility (leaving the name out on purpose cause I am listing the problems with it :P ) and it automatically created every folder under Windows that might possibly be needed in the running of a windows application, as well as the common files folder, the current user profile, and many many more. I think I ended up erasing in all over 100 empty directories that this old technology was nice enough to add for me. That is just one example of optimization that was done. It is amazing what can be found when you take the time to REALLY review what is going on under the hood.
In your wixproj setup file add this just before the end of file in <PropertyGroup> tag
<IncrementalGet>true</IncrementalGet>
This will tell WIX to compile only those files which are changed after the previous build.
Related
I am supporting a vb6 application. I am trying to transfer the executable and DLL to a new server and I am prompted with component not registered errors. I have got round this by manually registering the components on the new server.
I have found two files with file extensions of 000 and 001 that have registry commands in them (registering components) Can anyone explain how these files are generated? I have experience creating installation files in vb.net to a certain extent.
Repackaging and redeployment is not a developer issue and really doesn't belong here. Such issues are more appropriate for someplace like ServerFault.
It is one thing to have lost all of the source code of an application, but even worse in some ways to have lost the deployment package. Sadly some shops fail to archive either of these.
However it was also common enough for shops to see RAD tools like VB6, Delphi, PowerBuilder, etc. as things to shove off on the worst of the worst of their developers. These poor slobs seldom got official Microsoft training that should have emphasized the importance of creating proper installers. For that matter even those courses tended to marginalize the topic. It doesn't help that the Web is full of "Mort teaching Mort" half-baked development even today, or that the pioneers who wrote many of the early serious VB programming books tended to be loose cannons and contrarians who didn't really believe deployment was a serious concern.
The end result is that lots of shops have machines with VB6 programs shoehorned onto them in a half-baked way. Often when deadlines loomed they let Old Mort install VB6 right onto the production server and let him hack away right there! So it's no wonder people get into trouble once a server needs to be replaced or its OS updated.
Those REG files with .000, .001, etc. extensions aren't anything normal that I'm aware of. For all I know they've fallen out of REGMON runs or some 3rd party packaging tool. Manual registry exports created using REGEDIT would normally have .REG extensions.
If you are actually "supporting" this application it implies that you have the source code, VB6 compiler, developer install packages for any 3rd party controls, and a writeup describing any special packaging and installation requirements (target machine DCOM/COM+ configuration, system requirements such as IIS or MSMQ or 3rd party DBMS Providers and Drivers, special folder requirements, software firewall rules, etc.).
From those it ought to be possible to compile a clean new copy of the EXE, DLLs, etc. and create a clean deployment package - even if some configuration still needs to be done manually before and after running the installer.
Without those you are a computer janitor and your question belongs over at ServerFault. It is no fun, I know. I've had to take part in such janitorial services myself all too often.
I saw and done myself a lot of small products where a same piece of software is separated into one executable and several DLLs, and those DLLs are not just shared libraries done by somebody else, but libraries which are done exclusively for this software, by the same developer team. (I'm not talking here about big scale products which just require hundreds of DLLs and share them extensively with other products.)
I understand that separating code into several parts, each one compiling into a separate DLL, is good from the point of view of a developer. It means that:
If a developer changes one project, he has to recompile only this one, and dependent ones, which can be much faster.
A project can be done by a single developer in a team, while other developers will just use provided interfaces, without stepping into the code.
Auto updates of the software may sometimes be faster, with lower server impact.
But what about the end user? Isn't it just bad to deliver a piece of software composed of one EXE & several DLLs, when everything could be grouped together? After all:
The user may not even understand what are those files and why they fill memory on his hard disk,
The user may want to move a program, for example save it on an USB flash drive. Having one big executable makes things easier,
Most anti-virus software will check each DLL. Checking one executable will be much faster than the smaller executable and dozens of libraries.
Using DLLs makes some things slower (for example, in .NET Framework, a "good" library must be found and checked if it is signed),
What happens if a DLL is removed or replaced by a bad version? Does every program handle this? Or does it crash without even explaining what's wrong with it?
Having one big executable has some other advantages.
So isn't it better from end users point of view, for small/medium size programs, to deliver one big executable? If so, why there are no tools allowing to do it easily (for example a magic tool integrated in common IDEs which compiles the whole solution into one executable, not each time, of course, but on-demand or during deployment).
This is someway similar to putting all CSS or all JavaScript files into one big file for the user. Having several files is much smarter for the developer and easier to maintain, but linking each page of a website to two files instead of dozens optimizes performance. In the same manner, CSS sprites are awful for the designers, because they require much more work, but are better from users point of view.
It's a tradeoff
(You figured that out yourself already ;))
For most projects, the customer doesn't care about how many files get installed, but he cares how many features are completed in time. Making life easier for developers benefits the user, too.
Some more reasons for DLL's
Some libraries don't play well together in the same build, but can be made to behave in a DLL (e.g. one DLL may use WTL3, the other requires WTL8).
Some of the DLL's may contain components to be loaded into other executables (global hooks, shell extensions, browser addons).
Some of the DLL's might be 3rd party, only available as DLL.
There may be reuse within the company - even if you see only one "public" product, it might be used in a dozen of internal projects using that DLL.
Some of the DLL's might have been built with a different environment thats not available for all developers in the company.
Standalone EXE vs. Installed product
Many products won't work as standalone executable anyway. They require installation, and the user not touching things he's not supposed to touch. Having one or more binaries doesn't matter.
Build Time Impact
Maybe you underestimate the impact of build times, and maintaining a stable build for large projects. If a build takes even 5 minutes, you could ephemistically call that "make developers think ahead, instead of tinker until it seems to work ok". But it's a serious time eater, and creates a serious distraction.
Build time of a single project is hard to improve. Working on VC9, build parallelization within one project is shaky, as is the incremental linker. Link times are especially hard to "optimize away" by faster machines.
Developer Independence
Another thing you might underestimate.
To use a DLL, you need a .dll and a .h.
To compile and link source code, you usually need to set up include directories, output directories, install 3rd party libraries, etc. It's a pain, really.
Yes, it is better IMHO - and I always use static linking for exactly the reasons you give, wherever possible. Lots of the reasons that dynamic linkage was invented for (saving memory, for example) no longer really apply. OTOH, there are architectural reasons, for example plugin architectures, why dynamic linking may be preferable to static.
I think your general point about considering carefully the final packaging of deliverables is well made. In the case of JavaScript such packaging is indeed possible, and with compression makes a significant difference.
Done lots of projects, never met an end-user which has any problem with some dll files residing on his box.
As a developer I would say yes it could matter. As an end-user who cares...
Yes, it may often be better from the end user's point of view. However, the benefits to the developer (and the development process) that you mention often mean that a business will prefer the cost-effective option.
It's a feature that too few users will appreciate, and that will cost a non-trivial amount to deliver.
Remember that we on StackOverflow are "above average" users. How many (non-geek) family members and friends do you have that would really value the ability to install their software to a USB stick?
The big advantages for dll are linked to the introduction of borders and independance .
For example in C/C++ only symbols exported are visible. Imagine a module A with a global variable "scale" and a module B with another global variable "scale" if you put all together you go to desaster ; in this case a dll may help you.
You can distribute those dll as component for customers without exactly the same compiler / linker options ; and this is often a good way to do cross language interop.
Last year, in 2009 GSoC, I participated with an organization called Winlibre. The basic idea is having a project similar to Aptitude (or Apt-get) and a GUI like Synaptic but for Windows and just to hold (initially), only open source software. The project was just ok, we finished what we considered was a good starting point but unfortunately, due to different occupations of the developers, the project has been idle almost since GSoC finished. Now, I have some energy, time and interest to try to continue this development. The project was divided in 3 parts: A repository server (which i worked on, and which was going to store and serve packages and files), a package creator for developers, and the main app, which is apt-get and its GUI.
I have been thinking about the project, and the first question that came to my mind is.. actually is this project useful for developers and Windows users? Keep in mind that the idea is to solve dependencies problems, and install packages "cleanly". I'm not a Windows developer and just a casual user, so i really don't have a lot of experience on how things are handled there, but as far as I have seen, all installers handle those dependencies. Will windows developers be willing to switch from installers to a packages way of handling installations of Open source Software? Or it's just ok to create packages for already existing installers?
The packages concept is basically the same as .deb or .rpm files.
I still have some other questions, but basically i would like to make sure that it's useful in someway to users and Windows developers, and if developers would find this project interesting. If you have any questions, feedback, suggestions or criticisms, please don't hesitate posting them.
Thanks!!
be sure to research previous efforts on this. Google turns up several similar/relevant efforts.
http://en.wikipedia.org/wiki/Package_management_system#Microsoft_Windows
http://windows-get.sourceforge.net
http://pina.plasmite.com
IIRC there was an rpm for windows at some point
Also I think there was some guy (who used to work at MS) in the news recently that basically is starting up a very similar project. I can't find a link to this now.
But anyway, yeah, it would be awesome if there was such a standard tool and repository.
I can only speak for myself, but obviously I could definitely make use of such a tool as I found your post through googling! ;)
My two use cases for this tool would the following ones:
1. I generally avoid to re-install my system as long as possible (in fact I manage to do so only for switching to a reasonable (not each an every) new version of Windows every few years or to setup new computers). But still I'd like my software to be up-to-date. Neither do I want to have to go to all the web pages and check manually if there are compatibility issues with the new version of Doxygen, Graphviz and the latest version of MikTeX for example, nor do I want to have to navigate to the download pages and run the setups all by myself. I just want to schedule ONE SINGLE (!) tool, which checks whether there are new updates or not and updates those applications which are not in conflict with any other application version.
If it unavoidably happens to me that I have to re-install my system, I don't want to get the new setups neither (and check compatibility). I even don't want to wait for one setup to finish in order to start the next one, I just want to check the tools I need, or even better, I want to simply load my "WinApt XML" batch installation file, which gets the installers and handles the setups sequentially all by itself.
I don't know enough about the architecture of .deb or .rpm but IMHO the most reasonable would be to maintain a DB with only the names, versions, dependencies and the location of the different versions' download locations. I mean, most of the tools available for Windows provide .msi packages anyways, which (I guess) is the application itself and some custom installation properties (really not sure how scripting is handled, but I know that creating a MSI in Visual Studio has very limited abilities to create custom installation steps and I can only imagine this is due to limitations of MSI protocol).
I guess a GUI will be mandatory for Windows users ;) but I personally would prefer the additional ability to handle the setups with the console.
Well, I like the idea and would love to hear from that (or such a) tool in the future.
Cheers
Check out NSIS. It's an open source MSI creator. You might be able to use it as part of your package creation software.
http://nsis.sourceforge.net/Main_Page
For the ALT-.Net tool/lib stack there have been some affords in this direction: Horn Get
However, the usability in a real world project has been subject in this SO question.
I need to find out whether my apps are being flagged as viruses by the most popular anti-virus packages (not best, but biggest by user base). I therefore would like to know how others go about this. Some background:
I have an application written in Delphi. Ever since the Delphi virus was found, I've had problems with false positives on my applications, particularly my demonstration versions for some reason (they all share the same code). AVG has been good, and I can now whitelist my files easily, but then I got the latest DevExpress installer and it was false-positived too. Given this is getting more widespread, it struck me that I need to find out if my apps are being flagged by the most popular anti-virus packages. I therefore would like to know how others go about this. I don't want people to be downloading our demonstration versions, getting an AV warning, and deciding not to try it.
The only options I have so far are buying a load of AV packages and putting them in a VM, or using a service like VirusTotal. The latter seemed an ideal option but for the fact that they limit the test to files under 20Mb, and my files are bigger than this. There is no paid for option either to expand the capability. (I thought this an odd limit, but Kaperskis free checker is limited to 1Mb!)
How do you check your applications?
VirusScan.jotti.org and VirusTotal.com may help
http://online.us.drweb.com/
i couldn't see any file size limit on it
My thoughts on this are as follows:
I set up a computer (nothing special) with a lot of disk space. I'll call this the ScanPC. Every time I do a build, the script will copy the new files to the ScanPC into a build specific directory. This will ensure that I have an archive of all builds that can be examined. Any one may have been released to customers.
Now, I then install VMWare server, and set up a number of virtual PCs. In each, I set up the anti-virus software to scan the network share, but in a read-only mode so that no scanner can accidentally modify or remove the false positive. Each VM can then be automatically updated from the vendor, and hopefully they will have an email option to tell me when they spot a virus, which I will then know is a false positive and can report to the vendor.
The benefit of this is that I have a complete build archive (something I need anyway), and it means that old versions out with customers that trigger the AV are identified as well as the most recent. It means I can add or remove AV products as appropriate. It means that I only need a single computer (performance is not important).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
I recently had to struggle with one installation project (which uses most popular product for creating installations: InstallShield) to make it work for product upgrades (migrating from one version to another). In the end it turned out that I needed to use one long package code but was using some other. It wasted my 8 hours (testing and debugging installers is a pain).
Now if I think about it, once you are done all the hard part of coding, all you want to is that correct applications, libraries are copied to target computer and user just runs it. Period. This apparently simple task normally turns out to be a tricky one and "being closed to finish date" makes in even harder.
Don't you think deploying a product is made damn difficult on windows which should have been simpler? (or installer really deserves that much attention and I am just being crazy about it?)
Have you ever used simpler deployment schemes such as "copy the folder to wherever you like and run the exe. When you want to remove it, just delete the folder!"? Was it effective and made things simpler?
Painful as it is you need to wrestle with the windows installer for the benefit of your customers. Otherwise you will need to do a lot more work to
Handle situations where for some reason an error occurs during the installation. What do you do next?
Handle issues like security. What if the installing user does not have rights to particular folders/registry keys?
Correctly cleanup after installation
Patching and patch management
Performing additional tasks -- registering COM objects, creating databases, creating shortcuts, creating an un-installation shotcut and so on
Installing prerequisites
Letting users choose which features to install
Your own custom scripts to solve all these problems eventually become a bigger problem than the installation itself!
I recommend that you check out Wix. It's not exactly child's play but it gets the job done. If you install Votive as a visual studio add in you get intellisense to help you strucutre the tags correctly. With the help file you can create pretty functional flexible installations
I don't think you'll see too many disagreements here, especially regarding MSI. I think one thing to keep in mind is to watch the way many programs are using MSI files these days. Displaying UI dialogs and making complex configuration choices with an MSI is very weak simply due to the way Windows Installer was designed, so I've noticed a lot of programs being split into a bunch of baby MSIs that are installed with the minimal UI by a parent setup program. The SQL Server 2008 setup wizard does this. UPS WorldShip does this. And Paint.NET does this, too--the wizard you see is a Windows Forms app, and it launches msiexec itself (you can see the minimal UI of the Windows Installer pop up on top of the white wizard window), passing any configuration parameters as property arguments to msiexec.
A common scenario where this comes up is where someone is tasked with building an installer for an application that has both server and client counterparts. If the user chooses the server option, then they may or may not want a new database to be installed, which means installing SQL Server. But you can't just install SQL Server while you're in the middle of your own installation because Windows Installer won't let you do that. So a frequent solution is to write an app that displays a wizard that allows the user to configure all of the setup options, and then your app launches the MSI files as needed for SQL Server, your server application, and your client application in the minimal UI mode; basically, eschewing the "features" aspect of Windows Installer entirely and moving it up to the MSI level. 4.5's multiple-package installations seems to be a step further in this direction. This format is also especially useful if you also need to loop in non-MSI installers from third parties as part of your installation process, like installing a printer driver for some bizarre point of sale printer.
I'll also agree that Windows Installer lacks built-in support for common deployment scenarios. It's meant for when setup isn't XCOPY, but they seem to miss the fact that setup usually isn't just "files + shortcuts + registry keys," either. There are no built-in actions for setting up IIS Web sites, registering certificates, creating and updating databases, adding assemblies to the GAC, and so on. I guess they take the opinion that some of this should happen on first run rather than being a transactional part of the install. The freely available tooling and documentation has been awful--flat out awful--for the better part of a decade. Both of these issues are largely addressed by the WiX project and DTF (which lets you finally use managed code custom actions), which is why we're all so grateful to Rob Mensching and others' work on that project.
I've had the same experience. Installation can quickly suck up your time as you go down the rabbit hole of "Oh God, I guess I have to become an expert in this too." I second the idea that's it's best to address it early on in your project and keep it maintained as part of your build process. This way, you can help avoid that scenario of having developed a practically uninstallable product. (Trac was an example of this for a while, requiring to track down specific versions of weird Python libraries.)
(I could go on about how Windows Installer sometimes decides to use my slow, external USB hard drive as a place to decompress its files, how it seems to sit there doing nothing for minutes on end on computers that have had lots of MSI installs on them, and how that progress bar resetting itself a bazillion times during a single install is the most idiotic thing I have ever seen, but I'll save those rants for another day. =)
My two cents; please note that I really just know enough about Windows Installer to do damage, but this is my assessment coming from a small business developer just trying to use it. Good luck!
Well, its a lot easier if you build your installer first, make it part of your build system, and let it grow with your project.
I agree, the windows installer drives me insane. But there are a lot of situations that xcopy just doesn't solve. Sometimes you want to install for multiple users, not just the current user. Sometimes you have to register COM objects. Sometimes you have to make a whole bunch of changes to the system, such as registering services to run at startup, connecting to network servers, etc. Sometimes you have users that can't use a command prompt. And you always want to be able to role the whole thing back when something fails halfway through.
Was the whole MSI database approach the best way of doing it? I'm not sure. Would I rather pound nails into my head than write another line of WiX code? Probably. But you have to admit, it does a good job of doing everything you could ever possibly want. And when it doesn't there is always the CustomAction option.
Really, what I would like to see, is better documentation (really, what is a type 50 action? How about giving it a name?) and a lot more easy-to-usurp templates.
And the WiX users group alias does a good job of answering questions.
You should read RobMen's blog. He does a good job explaining why things are the way they are. He has done a lot of thinking (more than any human should) about the problems of setup.
Have you looked at NSIS: http://en.wikipedia.org/wiki/Nullsoft_Scriptable_Install_System ?
And 1: Yes, 2: No
Personally, I mostly agree with #Conrad and #John Saunders. I wrote about this topic a long time ago on my old blog. I think #jeffamaphone has a point about the Windows Installer complexity (and my over attention to setup, in general ) but I believe the Windows Installer is still the best all round option for installation on Windows.
"Once you have done all the hard part of coding", you haven't done a thing if all your hard work doesn't install. Installers need to be built and tested on every nightly build, every night, almost from day one. You need to test that the installer can be built and run, and you need to verify the installation.
Otherwise, who cares how much hard work you've done coding - nobody will ever see your work if it doesn't install!
Note that this also applies to XCOPY.
Another thing: what is your QA testing if they're not testing what your installer installs? You have to test what the customer will get!
For exactly the reasons you state, we've done internal releases, handled by the dev team by copying the required files, and then done the rest of the setup using scripts and our own utilities.
However, for end users you have to have some kind of hand holding wizard, I've used the MS installer from within VS and found it confusing and clunky. After that experience I've avoided the pain by getting others to do the installation step. Can anyone recommend a good .Net installer?
I use Installshield and if you are not trying to do anything too fancy (I why would you) then it's pretty straighforward - set initial setting, select files, set up shortcuts and create setup.exe.
All future updates I handle inside my code - much more convinient to the user