I have MPI program which I want to debug.
I use mpich 2 under Windows, so does anybody know tools that can help me? Is it even possible?
I have some experience of the parallel debuggers DDT and Totalview on Linux. I see that DDTLite is available as a plug-in for MS Visual Studio; I don't think that there is a version of Totalview for any Windows platform.
So, yes, it is possible, but DDTLite costs money; however I couldn't work without either DDT or Totalview.
It's possible that Microsoft, who now produce an HPC edition of Windows, have parallelised their debugger but I have no knowledge of that. You can use gdb on parallel programs, so an installation under Cygwin (or similar) might help you. Personally I have never found gdb for parallel debugging anything other than incredibly difficult and I am fortunate to work at a site where getting a parallel debugger has not proven difficult.
Related
How are operating systems typically debugged? They cannot be stepped through with a debugger like simple console programs, and the build times are too large to repeatedly make small changes and recompile the whole thing.
They aren't debugged as a multi-gigabyte programs! :)
If you mean the individual user-mode components, they can mainly be debugged just like normal programs and libraries (because they are normal programs/libraries!).
For kernel-mode components, though, each OS has its own mechanism; here is some information regarding the way that we do kernel debugging in Windows. It can be done using the help of another machine connected to the machine you're debugging, via a serial port or something. I'm not familiar with the process itself, but that's the gist of how they work. (You need to set some boot loader options so that the system is ready for the debugger to be connected as early as possible.)
It depends on which part of the operating system you're talking about. When I worked at MSFT, I worked on the IE team. We debugged IE and the shell (Windows Explorer) in Visual Studio and stepped through them line by line all day long. Though, sometimes, it's easier to debug using a command line tool such as NTSD.
If, however, you want to debug anything in Kernel land such as the OS kernel or device drivers, which I suspect is really what you're asking, then you must use the Kernel debugger. For Windows that is a command line tool called kd, and generally you run the debugger on one machine and remotely debug the target.
There are a whole set of techniques throughout history from flashing lights on the console, to the use of hardware devices like an ICE, to more modern techniques utilizing fairly standard debuggers. One technique that is more common among OS developers then application developers is the analysis of a core dump. Look at something like mdb on solaris for ideas about how Solaris kernel developers do some of their debugging. Also tracing technologies are used. Anywhere from fairly straightforward logging packages to more modern techniques like dtrace.
Also note that the techniques used depend on the layer of software. Initial boot tends to be a fairly hard place to get your fingers into. But after that the environment of modern operation systems looks more and more like the application setting you are use to. In the end, it is all code :)
I was wondering is there a free Windows version of GCC.
I know there is minigw and something else but I don't know how to use them.
Sorry if this should be on SU.
The main choices are either MinGW or CygWin.
CygWin is a more complete UNIX-like environment than MinGW as it offers quite a lot of tools over and above development stuff. Even to the point of a full X-Windows server so you can develop software that'll run on both UNIX-like systems and Windows.
The installer is good but I would suggest installing everything even if you think you don't need it. Disk space is cheap and I've had problems in the past trying to get stuff going on partial installs (whether 1.7, or even earlier, fixes this, I don't know - I always do full installs).
However, it relies on the CygWin UNIX emulation DLL which, if I remember rightly, has restrictions for non-free software.
MinGW is more concentrated on the development tools. It generates native Windows applications rather than running under a emulation DLL like CygWin.
It used to be difficult to install with having to do MinGW, MSYS and others separately but it's come a long since then and has an easy graphical installer.
I believe it can do graphical applications using native Windows calls rather than via X-Windows, since it just links to the normal Windows runtimes.
If you want to know how to use either of them, you really have to look through the docs found at those links I provided - check the Documentation link on the left for MinGW (particularly Getting started) or the CygWin FAQ.
As for GUIs for development, I've never used one for CygWin - I'm old enough that I remember mark sense cards so I'm not scared of the command line interface :-).
I used Eclipse with CDT running over MinGW and wasn't that impressed although admittedly that was an early version. Don't get me wrong Eclipse is a brilliant tool and we use it for both Java and Linux/C development, I just had a lot of troubles with Eclipse/CDT under Windows.
Code::Blocks, on the other hand, was absolutely brilliant but you should check them all out to see which one suits you best. As I said, the last time I looked was about five years ago, an eternity in the IT world.
I am a bit late, but since the question may still arise...
gcc for Windows (including Ada, C, C++ and Fortran compilers) can be obtained from MinGW-builds on SourceForge: http://sourceforge.net/projects/mingwbuilds/files/host-windows/releases/
As of august 2013, there are 32 and 64 bits versions of gcc-4.8.1.
What kind of debugging tools you've been using to debug working binaries?
Is there are debugging toolkits in addition to GDB?
The only reason is that I'm quite new for system debugging and I've been debugging my system service.
Sultan
I'd recommend Valgrind [1]. It's quite useful when dealing with memory leaks and segfaults.
The segfaults can be tracked by letting GDB run (without any breakpoint) and check the backtrace ('bt' command), after the crash.
P.S.: I don't remember if Valgrind is avaliable for other systems, but since you asked about alternatives to GDB, I'm assuming you're on a *nix box.
Have a nice debugging.
[1] http://valgrind.org/
Assuming you are on Linux systems, one of the most valuable tool is valgrind.
I don't really use anything else except for precondition/postcondition check in the code itself (i.e. assertion on methods' input values).
I'm writing a compiler that converts source code (written in a small imperative programming language) to Intel AT&T 32-bit assembler.
I tend to spend a lot of time debugging, because of nasty offset-mistakes etc. in the generated code, and I would like to know if anyone knows of a tool to "walk through" the generated assembler code step-by-step, visualizing what's on the stack etc.
I use Ubuntu Linux as my development platform, and I'm comfortable with the terminal -- a GUI-program would be nice though. Does it exist? Or is there a good reason it doesn't (maybe the problem isn't so straightforward..?)
If you have good ideas for approaching debugging tasks in assembly code, I'll be glad to hear from you!
I like EDB (Evan's Debugger) on Linux. It has a nice, easy-to-use, QT4-based GUI. Its developer's goal is to make it similar to OllyDbg. And it's being actively maintained:
EDB on FreshMeat
I'm pretty sure it's installable through Synaptic on Ubuntu as well. Enjoy!
Is the end result of the compile process something that you can actually execute, and therefore examine in a debugger? If so the Data Display Debugger (ddd) might be useful.
My experience with debuggers such as Olly and EDB is quite sparse, so I wasn't able to solve my problem with those. I ended up
scattering calls around to a Debug function in the source code, nailing down bad register values
letting the compiler output HTML-formatted code with useful metadata for different iterations in the liveness analysis etc.
I'm trying/thinking of making CppCMS - C++ Web Framework project little bit more cross platform.
Today I can easily support Linux, OpenSolaris, FreeBSD and even Cygwin. But when it comes to Native Windows it becomes really painful:
The overview of the situation:
I'm POSIX/Linux developer and I'm barely familiar with Native Windows development tools like Visual Studio and Win32 API. However I do some work for this platform so I understand the limitations and the fact that Windows is totally different world.
This is web project that uses APIs that popular in Unix world, like: CGI, FastCGI and SCGI that implemented in most UNIX web servers; but I understand that I would not be able to use it with IIS because it does not support FastCGI over TCP/IP (only Windows pipes).
So even when it would work it would probably run only with Windows port of Apache.
I relay heavily on POSIX API:
Pref-forking allows be keep high survivability in case of crashing (not supported under windows) so this feature would be missing.
I use some file-locking facilities (but I can probably give them up without forking)
I have intensive use of native pthreads, even I can replace them with Boost.Thread
I probably would never be able to support Visual Studio (maybe 2010 with C++0x support), because I relay on C++0x decltype/auto feature or typeof/__typeof__ extension that is supported by most compilers I worked with: gcc, intel, sun studio. (To be honest: I can work without them but it makes the life much easier to framework user.
I relay heavily on autotools and I can't replace them with CMake, bjam or friends, because when it comes to support of internationalization, cross copiling, package management, they just does not give me a solution.
There are many annoying points like missing gmtime_r, or localtime_r under windows and many others that just require from me to rewrite them or replace them with 3rd part libraries.
There are still many "UNIX like" libraries that ported to Win32 like: iconv, gcrypt and some others that are barely ported like libdbi that have many limitations on windows.
Bottom line:
There is lots of non-trivial work to do, and even when it would be complete, it would probably work only with MingW tools and not "native" tools that Windows programmers are
familiar with.
So, my questions are:
Does such MingW port worth an effort? Would this help to build bigger community?
Does anybody have experience on how painful porting big projects from POSIX environment to
Win32 API is?
Would it be useful for Windows developers at all?
Edit:
It is also important for me to understand, how many of windows developers prefer to use
Open source development tools, MingW over Microsoft development solutions like VS.
Edit #2: Clearification about "native" windows solusions and IIS.
In fact, running framework with IIS is really hard problem. I explain:
The project relates to standard web server API as FastCGI or SCGI that allows to accept many requests over sinlge socket. Thus, on application side, I accept new request proceed it and returns the answer. Sometimes several threads process several requests.
Thus, implementing one or two standard protocols I open communication with any existing server: Apache, lighttpd, nginx, cherokee... or any other servers; with small exception of IIS
IIS has implementation of FastCGI, but... It supports only 1 connection per local process only that controlled by web server...
So... there is absolutely no standard way to connect my application to IIS.
Please note, I implement standard Web server API, I do not implement Neither IIS proprietary ISAPI nor Apache proprietary API, even the second is more important as for targeting UNIX world.
So, just Windows IIS Web world is just not really ready for cooperation for such project, so if anybody would use it under Windows it would use it with more open web servers.
You should base your decision on user demand. Have users ever requested using the framework on Windows? If so, did they explain why they wanted to use Windows (e.g. what additional constraints they had, what webserver they wanted to use, etc.)?
Typically, Windows users do expect that things work the Windows way. That means Visual Studio support, IIS support, MSI installer, and so on. If something still feels like being Unix, I would rather use Unix proper, instead of fighting with a half-working port.
As a windows client app developer it sort of hurts me that the development environment division currently is essentially Win32 and everything else and that they are mostly incompatible. That's why I'm preparing to move to MinGW for my personal windows app projects and to try to make them cross-platform.
I would suggest gradually moving to more cross-platform libraries like, as you suggested, refactoring pthreads to boost::thread, or going from fork() to multi-process with IPC, probably also using boost's facilities. Date/time stuff can be dealt with Boost libs as well. As for database support, there are
Microsoft compiler support is not that important I think, as MinGW provides a decent build environment with all the IDEs that support it, Eclipse CDT and Dev-C++ being among the most popular. But if you are going to make your project msvc-compatible, make sure users will be able to use Express editions of Visual Studio 2010 (as soon as thay come out) - that way no one will have to fork out for a Visual Studio 2010 (upgrade) just to use your project and there will be no problem for you to require the latest in Microsoft technology.
Most likely you won't avoid some amount of ifdefs for a code base of your project's size, but surely the effort might be worth it, if not only for gaining valuable experience and expanding the community with a few new happy and grateful members.
Your saying that you can support Cygwin quite easily reminds me that I've seen commercial Windows software that simply bundled in cygwin1.dll to support some originally-Unix code. If adding cygwin1.dll to your installer is all it takes, try it out.
I think you only have to look at the questions asked on SO to work out that MinGW users on Windows (of which I am one) are in a minority in the development community - the vast majority of Windows developers are using MS tools. Anyway, the compiler is only half (or less) of the issue - if your architecture depends on forking lots of processes, using MinGW is not going to help you. My advice is, if you really want to do cross platform development:
look at how Apache do it
consider using the Apache libraries as your base
don't use very new or compiler-specific language features
use multi-threading rather than multi-process
Does such MingW port worth an effort? Would this help to build bigger community?
I am still working myself on this issue with my own large POSIX project and my conclusion is that if you need to later interface with Microsoft products, then its worth while, however then I would only use MingW if project is medium small, if it is very large, then I would go all the way with MSDN Microsoft development tools - Huge amount of help will be available there - however it will cost
Does anybody have experience on how painful porting big projects from POSIX environment to Win32 API is?
sofar my own conversion of my POSIX project have been constantly put on hold, because of the amount of time each issue takes to handle is enormous - not finished converting yet - If I ever will be
Would it be useful for Windows developers at all?
Sure working inside the Microsoft IDE using tools from MSDN will definately decrease development time, however it will increase your dependence on Microsoft libraries - something you need to decide from beginning if that is an issue
**
Actually you could just add the necessary cygwin dlls to your projects make and then you would beable to run it in windows
I managed to make my POSIX project run when I added following dlls
cygboost_filesystem.dll
cygboost_system-mt-1_53.dll
cygboost_thread.dll
cyggcc_s-1.dll
cygstdc++-6.dll
cygwin1.dll
Probably your project will have different dependencies, however if you think conversion is not worth it, then perhaps this is a solution for you
You could also add your libs as static, then you would end up with only having to provide the last cygwin1.dll