Every source I found online says a full installation of Cygwin takes over 1 GB, but mine is only 100 MB. I was pretty sure I downloaded everything from the mirror servers, but the install took less than 5 minutes to complete instead of hours, as I'd expect if it were installing gigabytes of software.
Did Cygwin get a huge clean-up during 2012~2013, or did I do something wrong in the installation?
A full Cygwin installation can range from 23 to 112 GiB, depending on how you define "full."
Your 100 MB number tells me that you just clicked through the defaults presented by Cygwin's setup-*.exe program, selecting no optional packages, because that installs only the Base package set, which currently amounts to 0.1 GiB. Cygwin follows the modern net-connected software distribution model: it assumes you can just run setup-*.exe again and select new packages as you need them.
The Cygwin maintainers try to keep the Base category's package set as small as practical.¹ A Cygwin Base install gives you something much like an old-style Unix installation, covering little more than what POSIX specifies.
So How Do You Get a Full Installation?
The Cygwin installer does not have an obvious way to get a "full" installation, on purpose, because no one needs literally every package in the Cygwin repository.²
There is, however, a sneaky way to install everything. At the Select Packages screen...
...switch to Category view, then click the "Default" text to the right of the "All" group header. It will change to "Install," as will the corresponding text in all of the groups underneath it. This marks everything for installation.
I include this tip for completeness only. You do not want to do this! It will install gigs and gigs of stuff you will never use. Currently, there are 11242 packages in Cygwin,³ and installing every last one of them took 93 GiB of disk space for the installation tree plus 19 GiB for the download tree⁴ the last time I tried it. That gives the 112 GiB upper limit above.
All that unused software carries several costs. Even if disk space, download time, installation time, rebasing time, and local bandwidth use are of no consequence to you, consider the generous free mirror's wasted bandwidth.
I've done the experiment, so now you don't have to.
An Intelligent "Complete" Installation
I've come up with a simple set of package exclusion rules that results in a much smaller installation:
Skip all of the -debuginfo packages. Few people need these, and they take up a lot of space. Savings: about 53 GiB in the installation tree alone; more in the download tree.
It's easy to apply this rule. After selecting all packages for installation with the sneaky trick above but before you move on to the next screen, click the "Install" text next to the "Debug" category header until it switches back to "Default."
If you've already installed the debug packages, click that text until it says "Uninstall" instead.
Do not explicitly install any of the lib* packages. Let Cygwin's setup-*.exe automatically install libraries to satisfy package dependencies. Savings: about 5 GiB ⁵
To apply this rule, switch the "Libs" category to "Default" or "Uninstall" as you did with the "Debug" category. The installer will figure out which libraries you actually need in a later step.
Skip the cross-compilers and associated packages. Again, few people need these.⁶ Savings: About 4 GiB
There are two major sets of cross-development tools in Cygwin: the set for creating Cygwin executables of the other word size (i.e. 64-bit tools and libraries for 32-bit Cygwin, or vice versa) and the set for building MinGW executables of the same word size as your Cygwin installation.
To apply this rule for a 64-bit Cygwin installation, while still on the "Select Packages" screen, type cygwin32- in the package name search box at the top of that screen, then click the Default text next to each top-level category until it cycles to Default or Uninstall, as above.
Repeat that for mingw64-.
The idea is the same for 32-bit Cygwin, except that you search for and exclude packages with cygwin64- and mingw32- in their names instead.
By following this rule set, I was able to install nearly everything, taking only about 23 GiB.
Paring That Down
We can get the installation to be even smaller by excluding several other notorious disk hogs:
X11, the desktop environments, and the GUI apps together require about 11 GiB.⁷
A Cygwin Base + Devel installation comes to about 10 GiB.
A Cygwin Base + TeX category installation takes about 5 GiB. If you install only your native language's support package, it comes to about 3.7 GiB instead.
All of the -doc packages combined chew up about 5 GiB of disk space.
Someone who isn't a software developer, who doesn't use Cygwin for GUI stuff, who uses the Web for docs, and who doesn't use TeX for document creation could thus have a "full" Cygwin installation in only about 1 GiB.⁸
If you use Cygwin the way it is intended to be used, installing the base and only the extra packages you need at the moment, you probably won't even get your installation size to even those levels. I use Cygwin this way, and my installations are typically well under 1 GiB, yet they are "complete" by my lights, since they meet my current needs.
For Extra Disk-Filling Fun...
All of this testing was done with the 64-bit version of Cygwin. You can roughly double the above space requirements by installing a parallel 32-bit Cygwin installation.⁹
Doing so is not pointless. It is a viable alternative to using cross-compilers, for one thing. For another, the fundamental incompatibility of the two Cygwins means you may have need of both.
Footnotes
There are occasional threads on the Cygwin mailing list where someone argues that some very common package should be included in the Base category, such as Perl, and the result is usually that the maintainers decide not to add it to Base.
You also occasionally see the opposite: some package accidentally slips into Base, usually via an incorrect dependency. Shortly after the problem is brought to the attention of the Cygwin maintainers, the status quo ante is restored. (example)
Perhaps I am wrong.
There could be a vision-impaired Czech immigrant musician who completes US government software development contracts on the side while not busy brushing up on his technical Hindi vocabulary by translating electrical engineering reports into his adopted Mandarin.
I want to meet him. He sounds like an interesting guy.
Plus, I think I can help him with his plan to create a Tcl/Tk GUI front end for Orpie. Naturally, I will try to talk him into porting it from Ocaml to C++/Qt.
I mean, Tcl/Tk, seriously? In 2018? Let's not be ridiculous.
curl -s https://cygwin.com/packages/package_list.html | grep -c x86_64/
Cygwin's setup-*.exe doesn't delete the downloaded package files after installing them. This is useful at sites where you have multiple Cygwin installations, since you can put the download directory on a shared network drive. Each package then only has to be downloaded once at that site.
My 105 GiB upper limit assumes you will download and install to the same drive, and that you will keep the download tree in case you need to reinstall it later.
Not only does setup-*.exe not delete downloaded package files after installing them, it doesn't auto-purge old versions, so your download tree grows and grows over the years you use Cygwin. (There are scripts floating about the net to solve this problem, such as this one.)
All data storage values given in this answer are apparent disk usage numbers — du -bhs — rather than actual disk usage numbers, which would account for the file system overhead, since that varies between systems. This affects the installation tree to a much greater degree than it does the download tree since the proportion of small files is much greater in the installation tree. Expect something like +1% in the download tree and +5% in the installation tree.
You may wonder why there are libraries in the Cygwin package repository that you don't need even though you've installed "all" packages. There are several reasons:
some libraries are obsolete, but are still present on the mirrors
some libraries come in multiple alternative forms, so that people who know they need something other than the default can choose it
some libraries are there only for people writing their own programs, not to support any existing Cygwin package
Pretty much the only people who need the Cygwin cross-compilers are the people maintaining Cygwin packages, since maintainers are expected to build for both 32-bit and 64-bit Cygwin unless there is a good reason not to.
There are probably more people with a good justification for MinGW cross-development tools, but there's also the option of using MinGW and MSYS instead of Cygwin. Also, I am guessing that the number of people who do dual-stack Cygwin + MinGW development is smaller than the set of people who use one or the other exclusively, or nearly so.
It is not easy to do this exclusion, because GUI packages are scattered throughout the Cygwin packaging system, and their top-level category often contains non-GUI software, so you can't simply exclude the whole category. (e.g. Math.)
The first pass is to exclude the X11, GNOME, KDE, LXDE, MATE, Xfce, and Games categories using the same Categories view technique as above.
Then, using the search box as we did for the cross-compiler exclusion above, remove packages matching gtk, gnome, qt, and kde, optionally excluding those in the Devel, Debug or Libs packages, if you need those.
Finally, you'll have to switch to the Pending view and manually exclude a bunch of packages that weren't caught by either of those two broad exclusions: Abiword, Calligra, Celestia, Dia, Evince, Geany, gEdit, Geomview, Gimp, gLabels, GnuCash, Gnumeric, Kalzium, KMyMoney, KolourPaint, Konqueror, Krita, KStars, LyX, Marble, Pidgin, QupZilla, Scribus, Skrooge, Spectacle, Stellarium, Tellico, and Vinagre. If you don't exclude these, they'll drag back in much of what you excluded above as dependencies!
You may notice that the size of all the excluded package sets adds up to more than the 23 GiB of the "intelligent complete" installation. This is because of shared dependencies. That is to say, these sizes overlap to some extent.
If you put both setup-*.exe programs in the same directory, they will share the download tree so that all of the noarch packages are downloaded only once for both Cygwins. Between that and the fact that 32-bit and 64-bit Intel compilers generate code that differs in size, installing both Cygwins doesn't quite double the disk space requirement.
When installing packages for the first time, setup*.exe does not install every package. Only the minimal base packages from the Cygwin distribution are installed by default.
Clicking on categories and packages in the setup*.exe package installation screen will provide you with the ability to control what is installed or updated.
Clicking on the "Default" field next to the "All" category will provide you with the opportunity to install every Cygwin package.
Installing and Updating Cygwin Packages
I just did a full installation of Cygwin x64 (2018-01-03) on Windows 7 SP1 x64 Ultimate:
Cygwin folder (which by default is C:\cygwin64) has the following properties:
Size: 89.1 GiB (95,671,245,179 bytes)
Size on disk: 92.3 GB (99,148,398,592 bytes)
Contains: 1,433,494 Files, 94,363 Folders
Cygwin temporary folder where downloaded packages are stored prior to being installed (can be removed after Cygwin is installed):
Size: 18.0 GiB (19,363,639,932 bytes)
Size on disk: 18.0 GiB (19,383,767,040 bytes)
Contains: 9,711 Files, 10,354 Folders
I would like to add to this thread. This approach gives you a leaner, meaner, bare-bones, minimal Cygwin install, with just the tools/items you need. No dependency bloat, no unwanted packages, files etc.
I have been experimenting with Cygwin attempting to get a "bare-bones", minimal install. I do find that installing utilities like grep, gawk, sed and similar tools has dependencies on cygwin, base-Cygwin and sometimes unwanted tools like bash, coreutils etc.
I wanted to get only the tools and their required dlls installed and started examining the Cygwin package. I discovered that not using the setup.exe supplied by Cygwin is an alternative way to accomplish minimal Cygwin installs.
And this is how I got it done:
Download only the packages you want from any of the Cygwin mirror
sites using ftp or http. Alternatively you can use the setup.exe
supplied by Cygwin to download all the packages - download only and
no install.
Once the download is successfully completed, individual
packages like zlib, gawk, grep, libiconv are found under the
x86/release or x86_64/release directory Each package is 'tar'red and
compressed using tool 'xz' or bzip and stored in respective
directories.
To install a specific tool like sed or gawk, all that
needs to be done, is to extract the tool executable and its
dependencies (.dll)
Before you attempt the following, please ensure you have a tool like 7z.exe, xz.exe, bzip2 or other that is capable of uncompressing an .xz or bzip archive
Installing gawk example below :
Extract gawk.exe from gawk-4.1.3-1.tar.xz archive using the command 7z.exe e -so gawk-4.1.3-1.tar.xz | tar xvf -
Once that is done, you should find gawk.exe in a subfolder usually, usr/bin under the release/gawk folder
Find the dependencies for gawk, you can do this in a couple of ways.
Examine the Cygwin setup.ini file found in x86 or x86_64 folder.
Look for the string '# gawk' and in the lines after this line you should find a "requires:" line that lists the dependencies.
Mine reads like this - "requires: bash cygwin libgmp10 libintl8 libmpfr4 libreadline7"
For gawk to run, bash is not a must since we have the windows command shell. (bash is included to get a few other dlls required by gawk. However, that causes a lot more unecessary files to be installed.) The other dependencies contain files that gawk needs to run.
Extract each of the above packages using tools like 7z or xz into individual files.
After all the dependencies are extracted, copy your needed tool(s) (grep/sed/gawk) to a folder and all the dependent .dlls
You should now be able to run your tool with the minimum set of .dlls required in a bare-bones cygwin installation.
Caution : It may not be sufficient to just extract the dependencies listed in setup.ini for each tool. Sometimes, you may need to execute/run the tool to discover that there are more dlls required.
There are other means of finding out the dlls required by an .exe - you can use dumpbin from MS or dependency walker, ndepends or similar tools to find the list of dependent dlls
Consult - How do I detect the DLLs required by an application?
How do I find out which dlls an executable will load?
I also brute forced this dll info by just running the tool and installing the missing dlls listed one by one by extracting from the required packages.
When you run a tool and it errors out with a missing .dll message, search for the package that contains the dll here - https://cygwin.com/cgi-bin2/package-grep.cgi . Enter the full/partial name of the missing dll to find the name of the package containing the dll.
Eventually, I have ended up with a bare-bones cygwin install with only the tools and dlls that I need.
Example : gawk - gawk.exe and the following dlls - cygwin1.dll, cyggmp-10.dll, cygiconv-2.dll, cygintl-8.dll, cygmpfr-4.dll, cyggcc_s-seh-1.dll, cygncursesw-10.dll, cygreadline7.dll
sed - sed.exe and dlls - cygwin1.dll, cygintl-8.dll
Hope this is found useful. The Cywin installer also does dll re-basing, which I will not venture into here.
Under the devel you may need only the following:
gcc-core:GNU Compiler collection(C Open MP)
gcc-fortran:GNU Compiler collection(Fortran)
gcc-g++:GNU Compiler collection(C++)
gcc-objc:GNU Compiler collection(Objective-C)
gdb: The GNU Debugger
make: The GNU version of the "make" utility
Photo
I've created a patch with Advanced Installer by using an old (target image) msi and the new one (upgrade image). Inspecting MSP file I've discovered that it contains both modified and completely new files. The problem is that during the installation it installs only the "added" files. Existing files are ignored. I've already tried MSIEXEC switches like:
REINSTALL=ALL
REINSTALLMODE=sumo / aums / omus etc...
UPGRADE="Yes"
IS_MINOR_UPGRADE = "1"
..in different order and combinations (i.e. "REINSTALLMODE=aums REINSTALL=ALL"), so don't reply or comment just by telling me to try REINSTALLMODE=omus or something similar.
When creating a patch there is a set of rules that need to be followed, have you checked those? Breaking one of those can lead to unexpected behavior, such as the one you are now encountering.
To check for the rules you can start with a diff between the project files, as they are standard XML ones, and check for their product code, component GUIDs, etc... For example folder synchronization is a common problem encountered when a patch is created, as this changes the component GUIDs.
For some odd reason all the core components (exe, dlls) I wanted to update were set to "Do not register this component with Windows Installer".
I suppose this is some kind of project bug, because it's very old and was migrated through different Advanced Installer versions (7, 8 and 9).
Anyway, I was not able to update my application correctly even with a fixed patch. Windows Installer kept on asking me to browse to target image msi file (cached installer of the previous version).
However not all of my customers keep those files (usually this cached files are stored in %APPDATA% folder). So I found a workaround:
I've applied "Hash files" option in order to create a MsiFileHash table
I've packed my msp patch in a bootstrapper (exe file) that starts it with the TWICE with following command-line parameters:
first time:
"myPatch.msp" /n {150F6CE2-8C12-414B-9377-F087A62E6B67} REINSTALLMODE=c /qb
second time:
"myPatch.msp" /n {150F6CE2-8C12-414B-9377-F087A62E6B67} REINSTALLMODE=dep /qb
REINSTALLMODE=c switch forces file compare algorithm based on hashes, so it doesn't require the original setup sources anymore
REINSTALLMODE=dep restores all the other missing files, files with unknown or different (from target) version
I hope this workaround will be useful to people that use MSI/MSP authoring tools other than Advanced Installer
We are doing some upload testing to the server, specifically taking in .msi package installer files. Currently I can create a 2GB total file size .msi with the limited knowledge and Visusl Studio tools that I have, but am wondering if I can create a larger one.
None of my googling has shown me a definitive answer, but I need to test the max file size that can be uploaded, which seems to be restricted by the max .msi file size, which I can't find or create. I assume it's larger than 2GB, anyone know or can point me to the right resource?
What is the max .msi file size? If greater than 2GB, how do I create one?
The limit of 2 GB is for the CAB files included in the MSI package, not for the MSI database itself. To create a larger setup package you need to configure it to place the resources in multiple CAB files, next to the MSI package.
This article should answer all your questions about MSI size as well as warn about various other limitations related to the MSI packages with lots of files / components.
Additional links from MSDN:
reduce the size of the MSI package
authoring a large package
Hope this will bring you to the right solution.
How do patches or service packs work? I don't know how to explain my question but I will give a try
Take Windows for example. It has files which altogether consume 100s of MB. Now a single service pack (may be 300 MB file) updates the whole windows OS.
Similarly I have seen updates happening for softwares like adobe reader etc. In all these cases the main exe is much larger compared to the update. How does the process work? If the main file refers any dependancy files and if the update change the version or size. Will it not affect the exe?
Patches and service packs usually only need to update core shared libraries of the system. These libraries are replaced or patched from a compressed archive, hence their size. Once the libraries are updated the rest of the software of the OS can continue using the new versions.
Applications nowadays are designed to be modular and to use external libraries which can be updated easily. Sometimes the main application or any media used does not need to be replaced, only the library that's changed.
To complement earlier answers, back in the day, when file size really mattered, some patches were delivered as binary diffs, meaning, the patch itself was an executable that knew what files needed to be changed, and how, and it actually changed only a certain part of the files' zeroes and ones, locally, instead of replacing the files entirely.
Following URL may be of interest to you in knowing architecture.
http://msdn.microsoft.com/en-us/library/aa387291(VS.85).aspx
Patches (also called deltas) are only the differences between two files. If only few bytes of 1GB file change, patch will have only few bytes of size. For text files diff is used, for binary files xdelta or similar. Service packs are collections of patches.