To make things easier when switching between machines (my workstation at the office and my personal laptop) I have thought about trying an external hard drive to store my working directory on. Specifically I am looking at Firewire 800 drives (most are 5400 rpm 8mb cache). What I am wondering is if anyone has experience with doing this with Visual Studio projects and what sort of performance hit they see.
It depends on the size of the project. The throughput is low and the latency is high, so you're going to get hit every which way, but due to the latency you'll be hit harder if you have a lot of little files rather than a few large ones.
Have you considered simply carrying around a GIT or other distributed repository and updating the machine repositories as you move around? Then you can compile locally and treat the drive and a roving server. Since only changes will be moved across, it should be faster, and your code will be 'backed up' in more places.
If you forget the drive, it breaks, or is lost/stolen, then you can still sit down at a PC and program with no code missing if you're at the last PC you used, or very little code missing (which will be updated later with a resync anyway).
And it's just a hop skip and a jump away from simply using the network to move the changes between the systems if you don't want to carry the drive around later.
I use vmware and the virtual machines are on an external usb drive. Performance is fine. You might have some issues with the drive name changing - not an issue if you use virtual machines.
Granted I work in an industry were Personal Information and Intellectual Property are king, but I don't like that idea at all. That hard drive disappears and you have a big problem.
Why not Remote Desktop into the work machine?
EDIT Stipud Spelingg
Related
I'm doing very time consuming ffmpeg video editing. That's why I put my commands into a .bat batch file and run them over night. Usually that works fine, but from time to time when I look the next moring I see an error message of this kind:
From that state on, I didn't find any good way to close the console. When I press the [x] button in the top right corner, it freezes. When I try to kill the application using the task manager nothing happens. Even explorer.exe cannot be closed using the task manager. A shutdown won't do anything. During the last month I had this problem about three times and the only way I could close it was to long press the power button of the computer until it was turned off "the bad way".
Any ideas what to in such situations?
Or even better: How to prevent those situations?
What can the reason(s) be for the error?
Do you understand the message?
When the computer is started again the next morining and I run the same .bat file again everything works fine. So the same error does not repeat and the video is edited nicely!
Edit: Now, about one week after posting this question the problem occurred many more times! It is very annoying. I guess it has to do with the external hard drive connected by USB. Sometimes it randomly interrupts the connection! That might be the reason for the behavior. Whatever its causing the error, I want to learn a solution how to deal with this in future. I don't want to always push the reset button of my computer. I want a proper way to be able to shut it down.
To narrow down the cause, what is causing this error, and what is not, here is a list of seven seven seemingly isolated solutions that each alone or all together should fix your problem:
The .bat Batch File
Apparently there is nothing wrong with your coded .bat batch files.
If that was the case, then none of your past videos would have rendered.
But just to be sure, try to run your .bat in a different laptop or computer on the heaviest and most demanding video editing project files just to make sure that the .bat files in fact and without a doubt flawless.
The Computer CPU
Make sure that your CPU runs flawlessly not just for 30 minutes but for the hours long burn tests that are the video
projects at night you mention. Poor contact between a concave or convex heatsink and cpu or lack of or too much of thermal paste can make cpu too hot and unstable during prolonged cpu intensive burn tests. A software like OCCT or Intel
Burn Test should be able to run for hours in your case without a
single fault.
The Computer RAM
To test your memory you can use MemTest86 or my favourite the open source MemTest86+ which should run for hours without a single memory error.
The OS Integrity
Run CMD as admin, and type chkdsk c: /f or chkdsk c: /f /r /x and press Y to check and repair (after a reboot) the local hard drive c: or any other partitions that are the source or destination of your rendering projects. When your computer encounters a sudden shutdown or detects a corrupted file system, sometimes this is the cause of a corrupted OS file. This checks for the integrity of the most important system files. Also sfc /scannow is another way to check System Files which scans and repairs system files.
The Harddrive
Connect your external drive locally, and run both a short and deep long test to make sure the harddrive has zero cluster faults. A SMART test from Crystical Disk Info famout for their Crystal Disk Test, can be a good way to see all the past errors on a Harddrive. Also, try to run the nightly batch files on the HDD connected internally. That way you can rule out the next item:
The Cable Quality
Cat rated UTP networking and USB cables are notoriously known for their poor manufacturing quality and low reliability. Not just over time, but new out of the box they can be the cause of disconnects, bad connections and low throughput. There is not something like they work 100% or they work 0%. Sometimes they sit right in between and "work, but to a degree" enough to be sold, with the absolute bare minimum and sometimes under minimum quality strands that are anything but cupper. So check your cables, replace the cables with other cables that you have laying around. CCA (Copper Cladded Aluminium) is the garbage to stay away from. Get proper Cupper only cables.
USB to SATA (HDD) or M.2 NVMe (SSD) Adapter Chip
Some USB-to-SATA adapters are notorious for their low stamina, stopping working when the adapter chips become exhausted in professional usage over prolonged continous workloads, resulting in disconnects even if they would be connected via a cupper USB 3.2 cable to the computer! The internet is full of forums with people having problems with older generation cheaper JMicron chips causing interruptions causing failures in copying files from or to the PC. Realtek chips are somewhat better, but often the solutions on the last page shows all problems went away when they bought an expensive adapter that uses an ASMedia chip.
Here is my conundrum.
On the project I work on I need to switch branches a few times a day, when I do my machine grinds to a halt, it makes me really unproductive for 10 minutes while Visual Studio reloads the solution.
Compilation time is really slow as well this is 2-5 minutes of coffee making time because my machine is totally unusable during this.
Now my work machine is no beast but it's no desk clerk POS either. High spec i5 with 8GB of ram. HDD is possibly a cheap junker.
Our solution have roughly 11K of files and it's going to keep growing.
What can I do to speed things up?
I was thinking SSD possibly 4GB more RAM, setup a RAM drive?
Any suggestions welcome, if I do go the SSD route, any suggestions what goes on the SSD and what not.
Turn off anti-virus
more ram
SSD
In that order
An SSD will make a huge difference, as will more RAM, but in the meantime if you have a second HDD around, try installing that and put the source on it. In the days before SSDs we found a huge improvement in build times and machine usability when the source disk was separate to the OS. When it was all on the same disk the whole machine ground to a halt on big builds, but on separate disks the machine became usable again even while building.
An SSD is the way to go, disk read/write peformance for small files in random access pattern is most important to make Visual Studio build faster. So go shopping for an SSD but be careful to check some benchmarks to find a drive that has good performance for small files and random access read/writes.
I would recommend something like the Samsung 840 Pro or similar, Intel also has some drives with good performance.
Another thing about your solution is that the number of Projects has an impact on build performance. You should keep the number of output assemblies small and also make sure all the Projects are using one and the same output folder and then change all references between Projects to assembly references and make sure the copy local setting is set to false. This will improve build times dramatically as it eliminated lots of assembly copy operations.
I know this is not so much a programming question but it is relevant.
I work on a fairly large cross platform project. On Windows I use VC++ 2008. On Linux I use gcc. There are around 40k files in the project. Windows is 10x to 40x slower than Linux at compiling and linking the same project. How can I fix that?
A single change incremental build 20 seconds on Linux and > 3 mins on Windows. Why? I can even install the 'gold' linker in Linux and get that time down to 7 seconds.
Similarly git is 10x to 40x faster on Linux than Windows.
In the git case it's possible git is not using Windows in the optimal way but VC++? You'd think Microsoft would want to make their own developers as productive as possible and faster compilation would go a long way toward that. Maybe they are trying to encourage developers into C#?
As simple test, find a folder with lots of subfolders and do a simple
dir /s > c:\list.txt
on Windows. Do it twice and time the second run so it runs from the cache. Copy the files to Linux and do the equivalent 2 runs and time the second run.
ls -R > /tmp/list.txt
I have 2 workstations with the exact same specs. HP Z600s with 12gig of ram, 8 cores at 3.0ghz. On a folder with ~400k files Windows takes 40seconds, Linux takes < 1 second.
Is there a registry setting I can set to speed up Windows? What gives?
A few slightly relevant links, relevant to compile times, not necessarily i/o.
Apparently there's an issue in Windows 10 (not in Windows 7) that closing a process holds a global lock. When compiling with multiple cores and therefore multiple processes this issue hits.
The /analyse option can adversely affect perf because it loads a web browser. (Not relevant here but good to know)
Unless a hardcore Windows systems hacker comes along, you're not going to get more than partisan comments (which I won't do) and speculation (which is what I'm going to try).
File system - You should try the same operations (including the dir) on the same filesystem. I came across this which benchmarks a few filesystems for various parameters.
Caching. I once tried to run a compilation on Linux on a RAM disk and found that it was slower than running it on disk thanks to the way the kernel takes care of caching. This is a solid selling point for Linux and might be the reason why the performance is so different.
Bad dependency specifications on Windows. Maybe the chromium dependency specifications for Windows are not as correct as for Linux. This might result in unnecessary compilations when you make a small change. You might be able to validate this using the same compiler toolchain on Windows.
A few ideas:
Disable 8.3 names. This can be a big factor on drives with a large number of files and a relatively small number of folders: fsutil behavior set disable8dot3 1
Use more folders. In my experience, NTFS starts to slow down with more than about 1000 files per folder.
Enable parallel builds with MSBuild; just add the "/m" switch, and it will automatically start one copy of MSBuild per CPU core.
Put your files on an SSD -- helps hugely for random I/O.
If your average file size is much greater than 4KB, consider rebuilding the filesystem with a larger cluster size that corresponds roughly to your average file size.
Make sure the files have been defragmented. Fragmented files cause lots of disk seeks, which can cost you a factor of 40+ in throughput. Use the "contig" utility from sysinternals, or the built-in Windows defragmenter.
If your average file size is small, and the partition you're on is relatively full, it's possible that you are running with a fragmented MFT, which is bad for performance. Also, files smaller than 1K are stored directly in the MFT. The "contig" utility mentioned above can help, or you may need to increase the MFT size. The following command will double it, to 25% of the volume: fsutil behavior set mftzone 2 Change the last number to 3 or 4 to increase the size by additional 12.5% increments. After running the command, reboot and then create the filesystem.
Disable last access time: fsutil behavior set disablelastaccess 1
Disable the indexing service
Disable your anti-virus and anti-spyware software, or at least set the relevant folders to be ignored.
Put your files on a different physical drive from the OS and the paging file. Using a separate physical drive allows Windows to use parallel I/Os to both drives.
Have a look at your compiler flags. The Windows C++ compiler has a ton of options; make sure you're only using the ones you really need.
Try increasing the amount of memory the OS uses for paged-pool buffers (make sure you have enough RAM first): fsutil behavior set memoryusage 2
Check the Windows error log to make sure you aren't experiencing occasional disk errors.
Have a look at Physical Disk related performance counters to see how busy your disks are. High queue lengths or long times per transfer are bad signs.
The first 30% of disk partitions is much faster than the rest of the disk in terms of raw transfer time. Narrower partitions also help minimize seek times.
Are you using RAID? If so, you may need to optimize your choice of RAID type (RAID-5 is bad for write-heavy operations like compiling)
Disable any services that you don't need
Defragment folders: copy all files to another drive (just the files), delete the original files, copy all folders to another drive (just the empty folders), then delete the original folders, defragment the original drive, copy the folder structure back first, then copy the files. When Windows builds large folders one file at a time, the folders end up being fragmented and slow. ("contig" should help here, too)
If you are I/O bound and have CPU cycles to spare, try turning disk compression ON. It can provide some significant speedups for highly compressible files (like source code), with some cost in CPU.
NTFS saves file access time everytime. You can try disabling it:
"fsutil behavior set disablelastaccess 1"
(restart)
The issue with visual c++ is, as far I can tell, that it is not a priority for the compiler team to optimize this scenario.
Their solution is that you use their precompiled header feature. This is what windows specific projects have done. It is not portable, but it works.
Furthermore, on windows you typically have virus scanners, as well as system restore and search tools that can ruin your build times completely if they monitor your buid folder for you. windows 7 resouce monitor can help you spot it.
I have a reply here with some further tips for optimizing vc++ build times if you're really interested.
The difficulty in doing that is due to the fact that C++ tends to spread itself and the compilation process over many small, individual, files. That's something Linux is good at and Windows is not. If you want to make a really fast C++ compiler for Windows, try to keep everything in RAM and touch the filesystem as little as possible.
That's also how you'll make a faster Linux C++ compile chain, but it is less important in Linux because the file system is already doing a lot of that tuning for you.
The reason for this is due to Unix culture:
Historically file system performance has been a much higher priority in the Unix world than in Windows. Not to say that it hasn't been a priority in Windows, just that in Unix it has been a higher priority.
Access to source code.
You can't change what you can't control. Lack of access to Windows NTFS source code means that most efforts to improve performance have been though hardware improvements. That is, if performance is slow, you work around the problem by improving the hardware: the bus, the storage medium, and so on. You can only do so much if you have to work around the problem, not fix it.
Access to Unix source code (even before open source) was more widespread. Therefore, if you wanted to improve performance you would address it in software first (cheaper and easier) and hardware second.
As a result, there are many people in the world that got their PhDs by studying the Unix file system and finding novel ways to improve performance.
Unix tends towards many small files; Windows tends towards a few (or a single) big file.
Unix applications tend to deal with many small files. Think of a software development environment: many small source files, each with their own purpose. The final stage (linking) does create one big file but that is an small percentage.
As a result, Unix has highly optimized system calls for opening and closing files, scanning directories, and so on. The history of Unix research papers spans decades of file system optimizations that put a lot of thought into improving directory access (lookups and full-directory scans), initial file opening, and so on.
Windows applications tend to open one big file, hold it open for a long time, close it when done. Think of MS-Word. msword.exe (or whatever) opens the file once and appends for hours, updates internal blocks, and so on. The value of optimizing the opening of the file would be wasted time.
The history of Windows benchmarking and optimization has been on how fast one can read or write long files. That's what gets optimized.
Sadly software development has trended towards the first situation. Heck, the best word processing system for Unix (TeX/LaTeX) encourages you to put each chapter in a different file and #include them all together.
Unix is focused on high performance; Windows is focused on user experience
Unix started in the server room: no user interface. The only thing users see is speed. Therefore, speed is a priority.
Windows started on the desktop: Users only care about what they see, and they see the UI. Therefore, more energy is spent on improving the UI than performance.
The Windows ecosystem depends on planned obsolescence. Why optimize software when new hardware is just a year or two away?
I don't believe in conspiracy theories but if I did, I would point out that in the Windows culture there are fewer incentives to improve performance. Windows business models depends on people buying new machines like clockwork. (That's why the stock price of thousands of companies is affected if MS ships an operating system late or if Intel misses a chip release date.). This means that there is an incentive to solve performance problems by telling people to buy new hardware; not by improving the real problem: slow operating systems. Unix comes from academia where the budget is tight and you can get your PhD by inventing a new way to make file systems faster; rarely does someone in academia get points for solving a problem by issuing a purchase order. In Windows there is no conspiracy to keep software slow but the entire ecosystem depends on planned obsolescence.
Also, as Unix is open source (even when it wasn't, everyone had access to the source) any bored PhD student can read the code and become famous by making it better. That doesn't happen in Windows (MS does have a program that gives academics access to Windows source code, it is rarely taken advantage of). Look at this selection of Unix-related performance papers: http://www.eecs.harvard.edu/margo/papers/ or look up the history of papers by Osterhaus, Henry Spencer, or others. Heck, one of the biggest (and most enjoyable to watch) debates in Unix history was the back and forth between Osterhaus and Selzer http://www.eecs.harvard.edu/margo/papers/usenix95-lfs/supplement/rebuttal.html
You don't see that kind of thing happening in the Windows world. You might see vendors one-uping each other, but that seems to be much more rare lately since the innovation seems to all be at the standards body level.
That's how I see it.
Update: If you look at the new compiler chains that are coming out of Microsoft, you'll be very optimistic because much of what they are doing makes it easier to keep the entire toolchain in RAM and repeating less work. Very impressive stuff.
I personally found running a windows virtual machine on linux managed to remove a great deal of the IO slowness in windows, likely because the linux vm was doing lots of caching that Windows itself was not.
Doing that I was able to speed up compile times of a large (250Kloc) C++ project I was working on from something like 15 minutes to about 6 minutes.
Incremental linking
If the VC 2008 solution is set up as multiple projects with .lib outputs, you need to set "Use Library Dependency Inputs"; this makes the linker link directly against the .obj files rather than the .lib. (And actually makes it incrementally link.)
Directory traversal performance
It's a bit unfair to compare directory crawling on the original machine with crawling a newly created directory with the same files on another machine. If you want an equivalent test, you should probably make another copy of the directory on the source machine. (It may still be slow, but that could be due to any number of things: disk fragmentation, short file names, background services, etc.) Although I think the perf issues for dir /s have more to do with writing the output than measuring actual file traversal performance. Even dir /s /b > nul is slow on my machine with a huge directory.
I'm pretty sure it's related to the filesystem. I work on a cross-platform project for Linux and Windows where all the code is common except for where platform-dependent code is absolutely necessary. We use Mercurial, not git, so the "Linuxness" of git doesn't apply. Pulling in changes from the central repository takes forever on Windows compared to Linux, but I do have to say that our Windows 7 machines do a lot better than the Windows XP ones. Compiling the code after that is even worse on VS 2008. It's not just hg; CMake runs a lot slower on Windows as well, and both of these tools use the file system more than anything else.
The problem is so bad that most of our developers that work in a Windows environment don't even bother doing incremental builds anymore - they find that doing a unity build instead is faster.
Incidentally, if you want to dramatically decrease compilation speed in Windows, I'd suggest the aforementioned unity build. It's a pain to implement correctly in the build system (I did it for our team in CMake), but once done automagically speeds things up for our continuous integration servers. Depending on how many binaries your build system is spitting out, you can get 1 to 2 orders of magnitude improvement. Your mileage may vary. In our case I think it sped up the Linux builds threefold and the Windows one by about a factor of 10, but we have a lot of shared libraries and executables (which decreases the advantages of a unity build).
How do you build your large cross platform project?
If you are using common makefiles for Linux and Windows you could easily degrade windows performance by a factor of 10 if the makefiles are not designed to be fast on Windows.
I just fixed some makefiles of a cross platform project using common (GNU) makefiles for Linux and Windows. Make is starting a sh.exe process for each line of a recipe causing the performance difference between Windows and Linux!
According to the GNU make documentation
.ONESHELL:
should solve the issue, but this feature is (currently) not supported for Windows make. So rewriting the recipes to be on single logical lines (e.g. by adding ;\ or \ at the end of the current editor lines) worked very well!
IMHO this is all about disk I/O performance. The order of magnitude suggests a lot of the operations go to disk under Windows whereas they're handled in memory under Linux, i.e. Linux is caching better. Your best option under windows will be to move your files onto a fast disk, server or filesystem. Consider buying an Solid State Drive or moving your files to a ramdisk or fast NFS server.
I ran the directory traversal tests and the results are very close to the compilation times reported, suggesting this has nothing to do with CPU processing times or compiler/linker algorithms at all.
Measured times as suggested above traversing the chromium directory tree:
Windows Home Premium 7 (8GB Ram) on NTFS: 32 seconds
Ubuntu 11.04 Linux (2GB Ram) on NTFS: 10 seconds
Ubuntu 11.04 Linux (2GB Ram) on ext4: 0.6 seconds
For the tests I pulled the chromium sources (both under win/linux)
git clone http://github.com/chromium/chromium.git
cd chromium
git checkout remotes/origin/trunk
To measure the time I ran
ls -lR > ../list.txt ; time ls -lR > ../list.txt # bash
dir -Recurse > ../list.txt ; (measure-command { dir -Recurse > ../list.txt }).TotalSeconds #Powershell
I did turn off access timestamps, my virus scanner and increased the cache manager settings under windows (>2Gb RAM) - all without any noticeable improvements. Fact of the matter is, out of the box Linux performed 50x better than Windows with a quarter of the RAM.
For anybody who wants to contend that the numbers wrong - for whatever reason - please give it a try and post your findings.
Try using jom instead of nmake
Get it here:
https://github.com/qt-labs/jom
The fact is that nmake is using only one of your cores, jom is a clone of nmake that make uses of multicore processors.
GNU make do that out-of-the-box thanks to the -j option, that might be a reason of its speed vs the Microsoft nmake.
jom works by executing in parallel different make commands on different processors/cores.
Try yourself an feel the difference!
I want to add just one observation using Gnu make and other tools from MinGW tools on Windows: They seem to resolve hostnames even when the tools can not even communicate via IP. I would guess this is caused by some initialisation routine of the MinGW runtime. Running a local DNS proxy helped me to improve the compilation speed with these tools.
Before I got a big headache because the build speed dropped by a factor of 10 or so when I opened a VPN connection in parallel. In this case all these DNS lookups went through the VPN.
This observation might also apply to other build tools, not only MinGW based and it could have changed on the latest MinGW version meanwhile.
I recently could archive an other way to speed up compilation by about 10% on Windows using Gnu make by replacing the mingw bash.exe with the version from win-bash
(The win-bash is not very comfortable regarding interactive editing.)
I have the following setup for my daily/main/only development environment
Hardware/Tin = 4gb ram, 2.6ghz dual core CPU, 2x250gb HD's, usual array of periperhals
One the tin above, I currently have Windows XP installed, in Windows XP I have VMWare Workstation installed and I run a Windows Server 2003 deelopment environment. This includes,Visual Studio 2003/2005/2008, Sql Sever 2005/2008, Full MS Office suite, some producitivity tools (e.g. Redgate Sql/Data Compare, DevXpress Coderush, TestDriven.net etc).
I have problems with this, it runs slow (15 minutes to boot), the Watch/Autos windows in VS freeze up when debugging, I can't have more than 2-3 copies of VS open, the Errors window freezes up, WinGrep and COm+ constantly runs out of Virtual Desktop Memory and so forth (In fact, I would attribue most of the issues to Virtual Desktop Memory)
Now, I've tried every tweak in the book, I have second HD for VMWare, my paging file is on a differnt drive, I've adjusted my Ram split between guest and host, I've hacked the reg key for Virtual Desktop Memory and all of this to no avail.
Now, I could increase my Ram or CPU, but I'm not able to.
My question is, has anybody experienced the above, and if so, how did you solve it? Did you try ESXi? or shift your environment to raw tin?
IMHO, you've tried just about every tweak in the book. I'd suggest that you should just move to native for your main setup, and restrict VM use for testing.
I use a VM as my main dev env, but I don't run as much stuff as you, so I don't hit a big performance wall.
I guess the trick you didn't try was to run less things on your VM. 2-3 copies of VS are a recipe for slowness. Running Sql Server, same thing. Bump up memory would be good, but at least run services (iis, sql server) on another vm or better yet, another box. You are taxing your VM waay too much, it is not VM's fault.
The problem you run into most of the time on VPS is IO wait.
Do you run your virtual machine off of a disk image, if so try defragmenting your drive.
Or did you dedicate a partition to it?
Edit:
I would suggest to:
either try defragmenting the drive that has the disk image
either try dedicating a partition to the virtual machine, instead of using a disk image all together. (ideally the first partition on the drive, since this will have the lowest random access time)
Running off a disk image works, but since you're working on top of a filesystem, the disk image might be fragmented throughout the disk.
Good luck, hope it helps...
How do I improve Subversion client update performance? It appears to be disk bound on the client.
Details:
CollabNet Windows client version 1.6.2 (r37639)
Windows XP SP2
3 GB RAM with PF Usage around 1 GB and System Cache of 1.1 GB.
Disk has write caching enabled
Update takes 7-15 minutes (when very little to update).
Checkout has 36,083 directories/files (from svn list)
Repository has 58,750 revisions.
Checkout takes about 2.7 GB
Perf monitor shows % Disk Write time stays near 90% during update.
Max Disk Read Bytes/sec got up to 12.8M and write got up to 5.2M
CPU, paging file usage, and network usage are all low.
Watching the server performance seems to show that it isn't a bottleneck.
I'm especially interested in answers besides getting a faster disk (especially configuration changes).
Updates from some of the suggestions:
I need the whole thing so sparse directories won't work.
Another client (TortoiseSVN) takes 7 minutes also
TortoiseSVN icon overlays have be configured so they don't cause the problem.
Anti-virus is configured to to skip that directory is it isn't causing the problem.
I experience exatly the same thing. Recently replaced Perforce with svn, but if we cannot overcome the performance problems on Windows me must consider another tool.
Using svn 1.6.6, Win XP and Vista clients. RedHat server.
My observations matches yours:
Huge disk-write activity.
Antivirus not a bottleneck.
No matter witch svn-clients are used.
No server or network bottleneck.
Complementary info
More than 3 times faster operations on:
Linux (Ubuntu).
Linux (Ubuntu) running on VirtualBox at Win Vista host.
Win XP running on VMWare at RedHat host.
Do you need every bit of the repository on your working copy? If you truly only care about particular portions of the tree, look into Subversion's Sparse Directories (a.k.a. "Sparse Checkouts") feature. It allows you to manipulate your working copy so it only contains those directories of interest.
Just as an example, you might use this to prune documentation, installer-related files, etc. Depending on what you truly need on your local machine, embracing this approach could make a serious dent in your wait times.
Try svn client version 1.5.. It helped me on my Vista laptop. Versions 1.6. are extremely slow.
This is more likely to be your network and the amount of data moved as well as your client. Are you using Tortoise? I find it to be a bit slow myself when moving that much data!
Are you using TortoiseSVN? If so, the Icon Overlays do slow down operations. If you go to TortoiseSVN Settings/Icon Overlays there are several settings you can tweak to control the level to which you want to use the Overlays, including turning them off completely. See if that affects your performance.
Do you run a virus checker that uses on-access scanning? That can really make it crawl. If so, turn it off and see if that helps. Most scanners will have a way to exclude specific directories if that helps.
Nobody seems to be pointing out the one reason that I often consider a design flaw. Subversion creates a second "pristine" copy of the checkout for offline operations. If you're checking out 4G of files, it's actually writing 8G to disk.
Compare a checkout to an export. That will show you the massive difference when writing those second copies.
There's nothing you can do about that.
Upgrade to svn 1.7
From Discussion of Slow Performance of SVN Update:
The update process in svn 1.6 goes something like this:
search the entire working copy, to see what's there at the moment, and locking it so no one changes the answer during the next steps
tell that to the server
receive from the server whatever new stuff you need, applying the changes to the files as you go
recurse over the entire working copy again, unlocking it
If there are many directories and files, steps 1 and 4 can take up a
lot of time. This would be consistent with your observation of long
delays with no network traffic.
Working copy format was changed in svn 1.7. Now all meta information is stored in SQLite database in root folder of working copy and there is no need to perform steps 1 and 4 any more which consumed most of the time durring svn update.