As compilation is mostly reading small files, I wonder if buying a fast usb key to work on can speed up compilation time compared to a standard SATA drive, and with a lower price than an SSD drive (16Gb keys are < 30$).
Am I right ?
PS: I working with .Net, VS and the whole MS' tools, but I think this is true for all languages...
[Edit] According Toms's hardware, a recent hdd drive read access time is near 14ms average where the slowest USB key is 1.5ms
That's why I'm wondering if my hypothesis is right
One year later, SSD drives cost has melt.
This is now the simplest solution, as you can host the operating system and all the dev tools on a fast SSD drive, which costs less than $ 120 (100 €)
The specifications on Wikipedias SATA page suggest otherwise. The SATA revision 3.0 is 50% faster than USB 3.0 according to the spec.
Related
I tried the example provided by atmel's ASF on USB mass storage host to send/read a file to a USB flash storage device. When reading a file, i'm getting 1.7 MB/s speed, I tried a lot of solutions, which include :
Made sure its running on High speed mode, and the board is running
on 300 mhz
Tried increasing the buffer size for the F_read function, and I
managed to increase it to 2.2 MB/s
I tested the file system it self, which is FAT32 on a virtual memory
example, and got 30MB/s on read operations ( not sure if thats
helpful for speed debugging purposes)
I tried using the same program, except reading from an SD card, which
gave me an output of 1 MB/s
I also tested it on full speed mode and it gave me an output of 0.66
MB/s.
one extreme idea i tested was running two boards, one in host mode,
and the other in device mode then I tested the transfer speed of the
USB, it gave me an output of 1.66 MB/s on Bulk Mode. (running on HS)
tried the Keil examples which gave me worst results than Atmel's.
can someone please suggest solutions? I've read all documentation regarding USB communication provided by Atmel and Keil.
Atmel's mass storage USB stack lacks multi-sector read and write, though the SCSI layer indeed implements the proper command to get many sectors in a row (see uhi_msc_scsi_read_10). The abstraction layer reading data above the SCSI commands (uhi_msc_mem_read_10_ram and uhi_msc_mem_write_10_ram for instance) only read sector by sector, yielding in very poor performance.
In order to achieve USB High Speed performance (~35 MB/s) you will have to hack these functions (and all the layers above) to use multi-sectors read/write.
What is the minimum amount of RAM required to run Linux kernel on an Embedded device? In Linux-0.11 for 80x86, the minimum RAM required was 2MB to load the kernel data structures and interrupt vectors.
How much is the minimum needed RAM for present Linux-3.18 kernel? Does different architectures like x86 and ARM have different requirements for minimum RAM required for booting? How does one calculates the same?
It's possible to shrink it down to ~600 KiB. Check the work done by Tom Zanussi from Intel.
Presentation from Tom and Wiki page about the topic.
UPDATE. Tom published interesting statistics about memory use by different subsystems in the kernel. He did research during that time when he was working on the project.
Yet another interesting project is Gray486linux.
This site suggests:
A minimal uClinux configuration could be run from 4MB RAM, although
the recommendation we are giving to our customers is that they should design
in at least 16 MB's worth of RAM.
If you are using SDRAM, the problem would be getting a part any smaller than 16Mb at reasonable volume cost and availability, so maybe it is a non-problem? For SRAM however, that is a large and relatively expensive part.
eLinux.org has a lot of information on embedded kernel size, how to determine it, and how to minimise it.
It depends how you define Linux. If you ask for current operating systems then we are talking about way above 100MByte, better 1000MByte of memory.
If we are talking about "Linux from Scratch" then we are also talking about how much pain you are willing to suffer. In the mid-1990 I build a Linux system by compiling every binary myself and made it run on a 386sx16, 1,5MByte of memory. While it had a 40MByte harddrive it was mostly empty. I compiled my own Kernel 1.0.9, my own libc5, my own base tools, SVGAlib. That system was somewhat useable for using textmode and SVGAlib applications. Increasing the memory to 2MByte did help a lot. And believe me, the system was extremely bare. Today all components need at least twice the memory but then there is also ulibc instead of libc and busybox.
At 8MByte of memory I can create a very basic system today from scratch. At 512MByte of memory you might have a somewhat modern looking but slow desktop system.
When using the desktop PC's in my university (Which have 4Gb of ram), calculations in Matlab are fairly speedy, but on my laptop (Which also has 4Gb of ram), the exact same calculations take ages. My laptop is much more modern so I assume it also has a similar clock speed to the desktops.
For example, I have written a program that calculates the solid angle subtended by 50 disks at 500 points. On the desktop PC's this calculation takes about 15 seconds, on my laptop it takes about 5 minutes.
Is there a way to reduce the time taken to perform these calculations? e.g, can I allocate more ram to MATLAB, or can I boot up my PC in a way that optimises it for using MATLAB? I'm thinking that if the processor on my laptop is also doing calculations to run other programs this will slow down the MATLAB calculations. I've closed all other applications, but I know theres probably a lot of stuff going on I can't see. Can I boot my laptop up in a way that will have less of these things going on in the background?
I can't modify the code to make it more efficient.
Thanks!
You might run some of my benchmarks which, along with example results, can be found via:
http://www.roylongbottom.org.uk/
The CPU core used at a particular point in time, is the same on Pentiums, Celerons, Core 2s, Xeons and others. Only differences are L2/L3 cache sizes and external memory bus speeds. So you can compare most results with similar vintage 2 GHz CPUs. Things to try, besides simple number crunching tests.
1 - Try memory test, such as my BusSpeed, to show that caches are being used and RAM not dead slow.
2 - Assuming Windows, check that the offending program is the one using most CPU time in Task Manager, also that with the program not running, that CPU utilisation is around zero.
3 - Check that CPU temperature is not too high, like with SpeedFan (free D/L).
4 - If disk light is flashing, too much RAM might be being used, with some being swapped in and out. Task Manager Performance would show this. Increasing RAM demands can be checked my some of my reliability tests.
There are many things that go into computing power besides RAM. You mention processor speed, but there is also number of cores, GPU capability and more. Programs like MATLAB are designed to take advantage of features like parallelism.
Summary: You can't compare only RAM between two machines and expect to know how they will perform with respect to one another.
Side note: 4 GB is not very much RAM for a modern laptop.
Firstly you should perform a CPU performance benchmark on both computers.
Modern operating systems usually apply the most aggressive power management schemes when it is run on laptop. This usually means turning off one or more cores, or setting them to a very low frequency. For example, a Quad-core CPU that normally runs at 2.0 GHz could be throttled down to 700 MHz on one CPU while the other three are basically put to sleep, while it is on battery. (Remark. Numbers are not taken from a real example.)
The OS manages the CPU frequency in a dynamic way, tweaking it on the order of seconds. You will need a software monitoring tool that actually asks for the CPU frequency every second (without doing busy work itself) in order to know if this is the case.
Plugging in the laptop will make the OS use a less aggressive power management scheme.
(If this is found to be unrelated to MATLAB, please "flag" this post and ask moderator to move this question to the SuperUser site.)
I've now saved a bit of money for the hardware upgrade. What I'd like to know, which is the easiest way to measure which part of hardware is the bottleneck for compiling and should be upgraded?
Are there any clever techniques I could use? I've looked into perfmon, but it has too many counters and isn't very helpful without exact knowledge what should be looked at.
Conditions: Home development, Windows XP Pro, Visual Studio 2008
Thanks!
The question is really "what is maxed out during compilation?"
If you don't want to use perfmon, you can use something like the task monitor.
Run a compile.
See what's maxed out.
Did you go to 100% CPU for the whole time? Get more CPU -- faster or more cores or something.
Did you go to 100% memory for the whole time? Which number matters on the display? The only memory you can buy is "physical" memory. The only factor that matters is physical memory. The other things you see on the meter are not things you buy, they're adjustments to make to the way Windows works.
Did you go to "huge" amounts of I/O? You can't easily tell what's "huge", but you can conclude this. If you're not using memory and not using CPU, then you're using the only resource that's left -- you're I/O bound and you need a faster bus -- which usually means a whole new machine.
A faster HDD is of little or no value -- the bus clock speed is one limiting factor. The bus width is the other limiting factor. No one designs an ass-kicking I/O bus and then saddles it with junk HDD's. Usually, they design the bus that fits a specific cost target based on available HDD's.
A faster HDD is of little or no value -- the bus clock speed is one limiting factor. The bus width is the other limiting factor. No one designs an ass-kicking I/O bus and then saddles it with junk HDD's. Usually, they design the bus that fits a specific cost target based on available HDD's.
Garbage. Modern HDDs are slow compared to the I/O buses they are connected to. Name a single HDD that can max out a SATA 2 interface (and that is even a generation old now) for random IOPS... A hard drive is lucky to hit 10MB/s when the bus is capable of around 280MB/s.
E.g. http://www.anandtech.com/show/2948/3. Even there the SSDs are only hitting 50MB/s. It's clear the IOPs are NOT the bottleneck otherwise the HDD would do just as much as the SSDs.
I've never seen a computer IOPs bound rather than HDD bound. It doesn't happen.
Using the task monitor has already been suggested but the Sys Internals task monitor gives you more information than the built-in Windows task monitor:
Sys Internals task monitor
You might also want to see what other things are running on your PC which are using up memory and / or CPU processing power. It may be possible to remove or only run on demand things which are affecting performance.
Windows XP will only support 3GB of memory using a switch that you have to turn on and
I seem to remember that applications need to be written to actually take this into consideration.
In designing any desktop applications, are there any general rules on how much memory should the application uses?
For heavy-weight applications, those can be easily understood or at least profiled such as Firefox or Google Chrome. But for smaller utilities or line-of-business application, how much is an acceptable amount of memory usage?
I asked because I've recently come across a trade-off between memory usage and performance and wonder if there is any general consensus regarding it?
EDIT: Platform is Windows XP for users with machine just capable of running rich internet applications.
My specific trade-off problem is about caching a lot of images in memory. If possible, I'd love to have my app cache as much as the user's memory will allow. I have done it so that the application will cache upto a certain maximum limit considering memory pressure at the moment..
But what would be a good number? How do you come up with one? That's the point I'm asking.
There is no absolute answer for this. It depends on too many variables.
Here are some trade-offs for consideration:
What device/platform are you developing for?
Do you expect your user to use this software as the main purpose for their computer (example maybe you are developing some kind of server software)
Who is your target audience, home users? power users?
Are you making realistic expectations for the amount of RAM a user will have?
Are you taking into consideration that the user will be using a lot of other software on that computer as well?
Sometimes it's possible to have your cake and eat it too. For example if you were reading a file and writing it back out, you could read it chunk by chunk instead of reading the whole file into memory and then writing it out. In this case you have better memory usage, and no speed decrease.
I would generally recommend to use more RAM to get better speed if you must. But only if the RAM requirements are realistic for your target audience. For example if you expect a home user that has 1GB of RAM to use your program, then don't use 600MB of RAM yourself.
Consider using more RAM in this instance to get better speed, and to optimize some other part of your code to use less RAM.
Edit:
About your specific situation of caching images. I think it would be best for you to allow the user to set the amount of caching they would like to perform as an option. That way people with a lot of RAM can put it higher for better performance, and the people with little RAM can set it low.
This depends entirely on your target platform, which is more or less a business decision. The more memory you will need, the less customers will be able to use your software. Some questions to ask: How much memory do your customers (or potential customers) have installed in their computers? What other applications will they run simultaneously with your application? Is your application something assumed to be running exclusively (like a full screen computer game), or a utility which is supposed to run mostly in background, or to be switched into it from other applications often?
Hear is one example of a survey showing a distribution of installed RAM in systems of people playing games via Steam (source: Valve - Survey Summary Data):
Less than 96 Mb 0.01 %
96 Mb to 127 Mb 0.01 %
128 Mb to 255 Mb 0.21 %
256 Mb to 511 Mb 5.33 %
512 Mb to 999 Mb 19.81 %
1 Gb to 1.49 Gb 30.16 %
1.5 Gb to 1.99 Gb 6.10 %
2.0 Gb 38.37 %
A conclusion I would draw from a survey like this in my domain (computer games) is I can reasonably expect almost all our users having 512 MB or more, and vast majority having 1 GB or more. For a computer game which is supposed to run exclusive this means working set around 400 MB is rather safe and will limit almost no one out, and if it provides a significant added value for the product, it may have a sense to have a working set around 800 MB.
This depends on your target PC hardware. If your application uses too much memory then it will be slow while windows pages. TEST! Try both options in your compromise, and some in between if it makes sense. Run the tests on a typical machine that your users would use and with a sensible number of other applications open. So for most people that is Outlook and probably an instance or 2 of Internet Explorer (or the mail client/browser of your choice). I work in an organistion where uses of my application are also likely to be running some other custom applications so we test with those running as well. We have found that our application used too much memory, and makes switching application painfully slow so we have slowed our application slightly to reduce its memory usage.
If you are interested our target hardware was originally 512Mb machines becuase that was what our common standard spec workstation was. Several PC's had to be upgraded to 1Gb though becuase of this application. We have now trimmed its RAM usage a bit but it is written in VB .NET and most of the memory used seems to be the framework. PerfMon says the process is using aroung 200Mb (peak) but that the managed heap is only around 2Mb!