How to increase heap space in Google Go - go

I'm running the Go code from Rober Hundt's benchmark to compare performance of Go with the other languages. When I run the Go code from that benchmark (see http://code.google.com/p/multi-language-bench/source/browse/trunk/src/#src%2Fhavlak%2Fgo) I get an out of memory exception:
................runtime: out of memory: cannot allocate 1048576-byte block (1270808576 in use)
throw: out of memory
My question is how I can increase Go's memory space. Is there some start-up parameter or some flag for the compiler that can be set accordingly? The Makefile file of the Go source doesn't reveal anything...
Thanks, Oliver

If you share your OS version, if it is 32 vs 64 bits and your go version result, we can probably help you more.
The 32-bit version of Go has known heap memory issues and is generally not used in production. Make sure you are using the 64-bit version of Go on a 64-bit platform.
Go tip, the branch of go that will become Go 1.1 has upgraded the maximum heap space from 8 GB to 128 GB. The core Go developers I have spoken with recommend using Go tip for memory intensive production uses.
Go on Windows is less used and thus less tested, this could be a bug. Go is much more tried and true on Linux ans to a lesser extent OS X.
As peterSO mentioned, Robert Hundt's benchmark is flawed- read more here: Profiling Go Programs

One can only wildly guess - for the lack of more details.
If you're running a 32 bit Go version, try the 64 bit one instead and use the tip revision.
If possible, use a *nix platform - it has IMO better memory management.

Robert Hundt's benchmark is flawed. It's not a valid benchmark for Go. Read Profiling Go Programs for details.

For those that might be interested I sent a mail to mr. hundt himself asking whether there is some (undocumented) compiler switch with which the heap size could be adapted or parameterized somehow. This was because I first mistakenly assumed that such a switch must exist, but is not documented. And nobody hat replied to my post here seemed to know, either. I guess I was "too much inspired" here by how the jvm works ...
Anyway, he recommended to try different compilers (gcc based compiler, or the plan-9 based compiler) as one might work better than the other. Then try on a 64-bit machine, of course.
He also pointed my to some good article about Go's memory management: http://lwn.net/Articles/428100/
But no, there is no compiler switch to play around with memory settings.
Regards, Oliver

Related

Most simple architecture available as GCC target

I'm looking for CPU architecture, which is supported by GCC (and is still maintained) for which is easiest to implement software simulator.
It should be something simple, with flat memory model, 16bit+ address space, 16-32 bit ALU and good code dencity is prefered as for it will be running programs with program memory limitations.
Just few words about origin of those requirements. I need virtual CPU for running 'sandboxed' programs. That will be running on microcontrollers with ~5 KBytes RAM, ARM CPU ~20 MHz clock speed.
Performance is non an issue at all, what I really need is writing C/C++ programs and then running them in sandbox without stdlib. For writing programs GCC can help, just need implement vcpu for one of target architectures.
I've got acquainted with ARMv7-m, avr32 references and found them pretty accaptable but some more powerfull then I need. The less/simpler code I need to write for vcpu implementation, the sooner I will have what I need and less bugs will be there.
UPDATE:
Seems like I found what I need. Is was already answered here: What is the smallest, simplest CPU that gcc can compile for?
Thank you all.

CUDAGRIND - MEMORY TRANSACTION CHECKING FOR CUDA in windows [duplicate]

Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I was looking into Valgrind to help improve my C coding/debugging when I discovered it is only for Linux - I have no other need or interest in moving my OS to Linux so I was wondering if there is a equally good program for Windows.
As jakobengblom2 pointed out, valgrind has a suit of tools. Depending which one you are talking about there are different windows counter parts. I will only mention OSS or free tools here.
1. MemCheck:
Dr. Memory. It is a relatively new tool, works very well on Windows 7. My favorite feature is that it groups the same leaks' allocation stacks in the report.
http://code.google.com/p/drmemory/
I have also used UMDH( http://support.microsoft.com/kb/268343 ) and found it quiet useful and easy to setup. It works from Win2000 to Win7.
AppVerifier is a must have swissknife for windows native code developers, its "memory" checker does similar job
http://msdn.microsoft.com/en-us/library/dd371695%28v=vs.85%29.aspx
2. Callgrind:
My favorite is verysleepy ( http://www.codersnotes.com/sleepy ) It is tiny but very useful and easy to use.
If you need more features, AMD CodeAnalyst™ Performance Analyzer is free:
http://developer.amd.com/documentation/videos/pages/introductiontoamdcodeanalystperformanceanalyzer.aspx
Windows Performance Analysis tools is free from Microsoft, not very easy to use but can get the job done if you are willing to spend the time. http://blogs.microsoft.co.il/blogs/sasha/archive/2008/03/15/xperf-windows-performance-toolkit.aspx
Download:
http://msdn.microsoft.com/en-us/performance/cc752957
3. Massif:
Similar(not quite exact match) free tools on windows are:
VMMap from sysinternals : http://technet.microsoft.com/en-us/sysinternals/dd535533
!heap command in windbg : http://hacksoflife.blogspot.com/2009/06/heap-debugging-memoryresource-leak-with.html
4. Cachegrind:
Above mentioned Windows Performance Tools has certain level of L2 cache miss profiling capability but not quite as good and easy to use as Cachegrind.
5. DRD:
Haven't found anything free and as powerful on Windows yet, the only free tool for windows I can find that is slightly close is the "lock" checker in
AppVerifier:
http://msdn.microsoft.com/en-us/library/dd371695%28v=vs.85%29.aspx
Why not use Valgrind + Wine to debug your Windows app? See
http://wiki.winehq.org/Wine_and_Valgrind
(Chromium uses this to check the Windows version for memory errors; see
build.chromium.org
and look at the experimental or memory waterfalls, and search for wine.)
There's also Dr. Memory, see
dynamorio.org/drmemory.html
Some more good commercial tools:
Purify
Insure++
For Visual C++, try Visual Leak Detector. When I used it, it detected a memory leak from a new call and returned the actual line in source code of the leak. The latest release can be found at http://vld.codeplex.com/.
Development environment for Windows you are using may contain its own tools. Visual Studio, for example, lets you detect and isolate memory leaks in your programs
i would like to list some tool , hope will be useful
read this article for more detail
Purify
Bounds Checker
Coverity (basically its a code analyzer but, it will catch memory leak in static )
Glow Code
dmalloc
ccmalloc
NJAMD
YAMD
Valgrind
mpatrol
Insure++
Try DUMA
There is Pageheap.exe part of the debugging tools for Windows. It's free and is basically a custom memory allocator/deallocator.
See http://support.microsoft.com/kb/286470
In combination with Visual Studio I generally use Visual Leak Detector or simply _CrtDumpMemoryLeaks() which is a win32 api call. Both are nothing fancy but they get the job done.
I had the chance to use Compuware DevPartner Studio in the past and that was really good, but it's quite expensive.
A cheaper solution could be GlowCode, i just worked with a 5.x version and, despite some problems in attaching to a process i needed to debug, it worked quite well.
I've been loving Memory Validator, from a company called Software Verification.
Viusual Studio can help detecting memory leaks itself. See Microsoft Visual C++ Tips and Tricks -> "Memory Leaks" section.
See also this post in SO
Although real tracing is only possible with the Team Edtion of Visual Studio.
See the "Source Test Tools" link on the Software QA Testing and Test Tool Resources page for a list of similar tools.
I've used BoundsChecker,DevPartner Studio and Intel V-Tune in the past for profiling. I liked V-Tune the best; you could emulate various Intel chipsets and it would give you hints on how to optimize for that platform.
Does Jochen Kalmbach's Memory Leak Detector qualify?
PS: The URL to the latest version is buried somewhere in the article's comment thread.
LeakDiag, UMDH, App Verifier, DebugDiag, are all useful tools to improve robustness of code and find memory leaks.
The Boost Test library can detect memory leaks.
How about the Purify?
Try Intel's Inspector XE product which can help you detect both memory and threading issues: http://software.intel.com/en-us/articles/intel-inspector-xe/
Perhaps CodeSnitch would be something you're after? http://www.entrek.com/codesnitch.html
If you are developing with Borland/CodeGear/Embarcadero C++ Builder, you could use CodeGuard.
More or less all Profilers include checking for memory leaks and show you the stack when the memory was allocated.
I can recommend Intels Parallel Inspector. Simple to use and no recompilation needed. The trial version runs for 30 days.
GlowCode and AtromatedQA also include such capabilites. They all offer free trials.
Compuware DevPartner (aka BoundsChecker) in Contrast needs a slowed down "instrumentation" recompile and the application also runs slower when checking for errors. And BoundsChecker can not work with 64 Bit evsrions at all. We gave up on that tool.
The best tool I ever used is DevPartner BoundsChecker - it's not free but it has an evaluation period.
Another memory tool for your list: Memory Validator.
Not free, but nowhere near as expensive as Purify or Boundschecker.
If you're not afraid of mingw, here are some links (some might work with MSVC)...
http://betterlogic.com/roger/?p=1140
We are just completing a Memory Safety checking tool for Windows, that handles GCC and Micrsoft Visual C (not C++ yet), and are looking for Beta testers.
EDIT June 12, 2011: Not Beta anymore, now production for GCC and Microsoft Visual Studio C.
I found this SF project today:
http://sourceforge.net/p/valgrind4win/wiki/Home/
They are porting valgrind to Windows. Probably in several years we will have a reliable valgrind on windows.
Check out this question: Is there a good Valgrind substitute for Windows? . Though general substitute for valgrind is asked, it mainly discusses memory leak detectors and not race conditions detections.
I used Insure++ which does excellent job in finding c++ memory leaks/corruptions and many other bugs like uninitialized variables, pointer errors, strings etc., It also does visual "Code coverage" and run time memory usage etc.. which give more confident on your code.. You can try it for trail version..
You might want to read what Mozilla is doing regarding memory leaks. One tool in their toolbox is the Hans Boehm garbage collector used as memory leak detector.
You can give a try to RuntimeChecker trial ot to IBM Purify trial..
A free solution would be to use the following code in Visual Studio:
#ifdef _DEBUG
#define new DEBUG_NEW
#endif
Just write this in the top of all your cpp files.
This will detect memory leaks of your application whenc stopping debug run and list them in the output window. Double clicking on a memory leaks line will higlight you the line where memory is allocated and never released. This may help you : http://www.flipcode.com/archives/How_To_Find_Memory_Leaks.shtml

high performance runtime

It’s the first time I submit a question in this forum.
I’m posting a general question. I don’t have to develop an application for a specific purpose.
After a lot of “googling” I still haven’t found a language/runtime/script engine/virtual machine that match these 5 requirements:
memory allocation of variables/values or objects cleaned at run time
(e.g. a la C++ that use keyword delete or free in C )
language (and consequently the program) is a script or
pseudo-compiled a la byte code that should be portable on main
operating system (windows, linux, *bsd, solaris) & platform(32/64bit)
native use of multicore (engine/runtime)
no limit on the heap usage
library for network
The programming language for building application and that run on this engine is agnostic oriented (paradigm is not important).
I hope that this post won’t stir up a Holy-War but I'd like to put focus on engine behavior during program execution.
Sorry for my bad english.
Luke
I think Erlang might fit your requirement:
most data is either allocated in local scopes and therefore immediately deleted after use or contained in a library-powered permanent storage like ETS, DETS or Mnesia. There is Garbage Collection, though, but the paradigm of the language makes the need for it not as important.
the Erlang compiler compiles the source code to the BEAM virtual machine byte code, which, unlike Java is register-based and thus much faster. The VM is available for:
Solaris (including 64 bit)
BSD
Linux
OSX
TRU64
Windows NT/2000/2003/XP/Vista/7
VxWorks
Erlang has been designed for distributed systems, concurrency and reliability from day one
Erlang's Heap grows with your demand for it, it's initially limited and expanded automatically (there are numerous tweaks you can use to configure this on a per-VM-basis)
Erlang comes from a networking background and provides tons of libraries from IP to higher-level protocols

Profiling a benchmark compiled for the SPARC v8 on an x86

I'm trying to make a (small) improvement to the leon3 processor (instruction set is SPARC v8) for an academic exercise. Before I decide what to improve, I want to profile a couple of benchmark programs that I want to tailor the improvements to.
I don't have access to a SPARC v8 machine.
Currently, I'm using an evaluation version of 'tsim' (a leon3 simulator) which does profiling at the functional level. Which is not really all that useful.
I have tried weird stuff like compiling with loop unrolling enabled and then counting the interesting instructions in the assembly code, but gcc refuses to unroll the loops, probably because some of them go too deep (e.g. 4 nested 'for' loops).
Ideally, what I'm looking for is a SPARC v8 simulator that runs the benchmark and profiles it at the instruction level (stuff like: 'smul' was executed x times) so that I can decide where to start trying with the improvement. Of course if there are other ways I can do this if not a profiler, I won't mind.
Any ideas?
Simulating the processor in Modelsim could be an option. With Modelsim you can do a functional simulation of the complete LEON3 processor. Although the simulation will be quite slow and probably complete overkill for your purposes but Aeroflex Gaisler provides excellent scripts to work with Modelsim.
A student edition of modelsim can be found here:
http://www.mentor.com/company/higher_ed/modelsim-student-edition
If you really want to dig that deep into the hardware, you'll find a simulator useful that helps you with that.
Simics comes into mind. They used to have free academic licenses, but since they were bought by Intel, you now need to apply for one, which from my experience takes a couple of weeks. If you are willing to invest this time, you'll certainly get a tool that suits your needs, although they support LEON2, not LEON3, as a model, but for profiling this should be fine.
Qemu also has LEON support but as they're heavily recompiling, it will probably be hard to to instruction-level profiling with it.

Which is the best tool to test for Memory leak in Win32/COM application?

I'm looking for a tool which can monitor a running application (Win32/COM) for a long duration (1-3 days) and detect memory leaks if any. Any suggestions?
It is a .NET Windows application calling lots of unmanaged code.
You can try Memory Validator
iJeeves, the combination of BoundsChecker and .NET memory profiling should help you with your memory analysis. DevPartner Studio 10.5 ships February 4, 2011 with 64-bit application support. Depending on your application raw memory footprint, you may run x86 build configurations with the error dectection memory tracking analysis as long as you keep below the 2gb overall process virtual address limit, 3gb if you link the exe with LARGE_ADDRESS_AWARE and run on an x64 OS with extra RAM. The x64 build configuration will let you go up as high as your system RAM allows, at least until you start paging and performance grinds to a standstill. You can run BC error dection for your native code under the .NET process, but object leaks or held references in managed code require a second pass using the .NET memory profiler. We do not yet have a single pass analysis that can handle the mixed C++ and .NET code with full mixed stack traces but we can handle managed code above the line, any PInvokes that cross the line, and all native activity below the line in two passes. Shameless plug: I work on the DevPartner team. The links above pointing to microfocus.com acurately resolve to DevPartner pages. Look for DPS 10.5 when it ships and pull down the eval to see if it meets your needs.
AQTime is nice, I used it several times and it helped me with some tricky bugs.
I used to use Bounds Checker but nowadays I either use the Micrsoft inbuilt CRT library or build my own.
If your looking for a pay$'s tool then DevPartner is well worth using. It has memory leak detection for managed, and unmanaged code.
Application Verifier is free and from Microsoft. It detects memory leaks, double frees, overwrites and many other things. I use it all the time and it has helped me track down some nasty issues.

Resources