Create a Debugging IDE for proprietary language - debugging

I am using a rather obsure, proprietary langauge called WIL/Winbatch that had an awful IDE (winbatch studio).
I would like to develop an alternative environment; however, without the ability to set breakpoints, step, and examine variables, there is really no point. How does one begin even researching how to implementing a debugger for a proprietary language? Is it even legal?
I guess I'm kind of locked in a mindset that the debugger portion must be able to examine the statements that are provided to it in WIL as they are executed, right? So somehow i have to trap the output of the interpreter? Or is it just a matter of reading locations in memory using whatever language?
Thanks in advance.

Having been there and successfully completed the task, here are the things to keep in mind:
Build it as a plug-in/extension to an IDE your team is already familiar with and likes. They'll thank you for providing an interface consistent with what they really know how to use, plus you can focus entirely on the features that make your language different from others.
You'll have to learn the debugging protocol for your language. In our case, we had source access to the runtime for the interpreted language. In other cases, you may find documentation for GDB local or remote debugging interface, a library you can link to for the language's debugging protocols, or maybe even figure out what the call stacks look like and wrap the Windows Debugging API to analyze it behind the scenes.
Don't build in excess of what the language provides. Adding debugging features takes a lot of time, and they have a rather annoying habit of needing to be significantly altered or completely rewritten as versions of the target language are updated.

Why are you tied so closely to this language? If it's not well supported, there are many others you can use. Anyway, to actually answer your question, the difficulty depends on whether it is a compiled or interpreted language and whether or not you have access to any source code (which it seems of course, that you don't). That said, this would be a very challenging project as you would have to reverse engineer the compiled code for it to have any meaning. Your time would be better spent learning another (better) language.
Perhaps if you can give us an idea of why you want to use this language we could give you some help?

Related

Debugging Tcl/tk application for Windows

Does it possible debug Tcl/tk application for Windows, without the source code? The application have no support and there is no source code available. There is error shown during one of operation. Is it possible to find what cause an error and whether error can be fixed by patching code?
It's very difficult without the source code, as it is at the level of the source code where you'd need to make the patch. If the code is exposing itself via the send command mechanism (or the comm package) you can probably make a bit of progress (as there's quite good introspection capabilities in Tcl by default, so info commands, info vars and info body may help, along with many other info commands and some introspectors that are elsewhere too), but it's still really difficult, particularly if you don't understand the internal structure of the code. OTOH, I wouldn't expect a production application to expose itself this way; typically you disable that sort of thing when outside development.
A standard debugger like gdb won't help, especially with the Tcl 8.6 non-recursive runtime. (Tcl applications in that environment just tend to show up as effectively “doing stuff”; there's nothing to really indicate how it hangs together.)
But the first thing to check is whether you actually have the source code. It's possible that the code has just been packaged together as a Starpack and that you can use a tool like sdx to extract the things you need to make changes in. But you aren't guaranteed to succeed at that; if the code was compiled/obscured with a commercial tool like the TDK, you really don't have the source and can't really do much about it. (By the same general principles that mean that DRM doesn't work well, it's possible to decompile the .tbc files that the TDK produces, but that's really a last-ditch thing to do as it is technically illegal in some jurisdictions, morally rather dodgy, and inclined to produce really awful output.) Can you contact the original author(s) of the code instead? If that works, it'll be cheaper and simpler…
If you've actually got the human-readable code, even if packaged with sdx, then you can do a lot more.

BOINC: Is there an easy example how to code a programm for it and how to implement it into their client/server system?

I did a numeric method as my diploma thesis and coded it in java. It needs a lot of computational time when adequately executed. So I looked for an alternative and found BOINC. Unfortunately I didn't have time for doing my method in BOINC, because I'm an Aerospace student and not a programmer and I decided to keep my priority on my java program. Now it's finished an I still would like to port this to BOINC environment.
Unfortunately I'm learning in re-doing examples and I couldn't find any, neither on the official site http://boinc.berkeley.edu nor in the internet.
So do you know a good and easy example or do you have any experience in BOINC and would like to start a new platform for such a boinc project?
I'm realistic about my method, that it wouldn't run 24/7, because there aren't as many work units as for seti or folding projects. So I would like to have a platform for more than just my project so that another platform project can be worked on, when one part of the project does not have any work units at that moment.
But to start this, I would keep it simple and just want to know how to code it and use it in the client and server system. It doesn't matter what the example projects will work on, as long as it is simple enough, that I can understand it and extending it for my method.
Thank you in advance, Andreas! :)
PS: I know that BOINC supports JAVA as a programming language, and my method is coded in JAVA.
As far as I know, JavaApps is just an idea; I don't know if anyone actually tried it in a real BOINC project. And it's Windows-only. And it seems to be a bit of a pain to redistribute the entire JRE as part of the BOINC application (both technically and legally).
Also, I generally dislike using that kind of “wrapper” where the science app (using the BOINC API) starts another process that then does the real computation. It's usually unreliable. There are lots of things that could go wrong with the wrapper, especially related to controlling the child process (eg. if something kills the wrapper, the child process has to quit too).
However, I just found something pretty interesting that may let me do a better Java wrapper for BOINC... Stay tuned! (but don't hold your breath either; it's the holidays!)
Meanwhile, I suggest you start by reading BOINC wiki and setting up a server with a “hello world” application; and if you have any trouble, ask a specific question about your trouble either here or in the boinc_projects mailing list.
(Of course, payin’ me to install the server for you is also an option ;) but I can't guarantee anything; not even my mere availability at this time of the year)

Why is debugging better in an IDE? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 years ago.
Improve this question
I've been a software developer for over twenty years, programming in C, Perl, SQL, Java, PHP, JavaScript, and recently Python. I've never had a problem I could not debug using some careful thought, and well-placed debugging print statements.
I respect that many people say that my techniques are primitive, and using a real debugger in an IDE is much better. Yet from my observation, IDE users don't appear to debug faster or more successfully than I can, using my stone knives and bear skins. I'm sincerely open to learning the right tools, I've just never been shown a compelling advantage to using visual debuggers.
Moreover, I have never read a tutorial or book that showed how to debug effectively using an IDE, beyond the basics of how to set breakpoints and display the contents of variables.
What am I missing? What makes IDE debugging tools so much more effective than thoughtful use of diagnostic print statements?
Can you suggest resources (tutorials, books, screencasts) that show the finer techniques of IDE debugging?
Sweet answers! Thanks much to everyone for taking the time. Very illuminating. I voted up many, and voted none down.
Some notable points:
Debuggers can help me do ad hoc inspection or alteration of variables, code, or any other aspect of the runtime environment, whereas manual debugging requires me to stop, edit, and re-execute the application (possibly requiring recompilation).
Debuggers can attach to a running process or use a crash dump, whereas with manual debugging, "steps to reproduce" a defect are necessary.
Debuggers can display complex data structures, multi-threaded environments, or full runtime stacks easily and in a more readable manner.
Debuggers offer many ways to reduce the time and repetitive work to do almost any debugging tasks.
Visual debuggers and console debuggers are both useful, and have many features in common.
A visual debugger integrated into an IDE also gives you convenient access to smart editing and all the other features of the IDE, in a single integrated development environment (hence the name).
Some examples of some abilities that an IDE debugger will give you over trace messages in code:
View the call stack at any point in time, giving you a context for your current stack frame.
Step into libraries that you are not able to re-compile for the purposes of adding traces (assuming you have access to the debug symbols)
Change variable values while the program is running
Edit and continue - the ability to change code while it is running and immediately see the results of the change
Be able to watch variables, seeing when they change
Be able to skip or repeat sections of code, to see how the code will perform. This allows you to test out theoretical changes before making them.
Examine memory contents in real-time
Alert you when certain exceptions are thrown, even if they are handled by the application.
Conditional breakpointing; stopping the application only in exceptional circumstances to allow you to analyse the stack and variables.
View the thread context in multi-threaded applications, which can be difficult to achieve with tracing (as the traces from different threads will be interleaved in the output).
In summary, print statements are (generally) static and you'll need to re-compile to get additional information if your original statements weren't detailed enough. The IDE removes this static barrier, giving you a dynamic toolkit at your fingertips.
When I first started coding, I couldn't understand what the big deal with debuggers was and I thought I could achieve anything with tracing (granted, that was on unix and the debugger was GDB). But once you learn how to properly use a graphical debugger, you don't want to go back to print statements.
An IDE debugger lets you change the
values of variables at run-time.
An IDE
debugger lets you see the value of
variables you didn't know you wanted
to see when execution began.
An IDE
debugger lets you see the call stack
and examine the state of the
function passed weird values.
(think this function is called from
hundreds of places, you don't know
where these weird values are coming
from)
An IDE debugger lets you
conditionally break execution at any
point in code, based on a condition,
not a line number.
An IDE debugger will let you examine the state of the program in the case of an unhandled exception instead of just crapping out.
Here's one thing that you definitely cannot debug with "print" statement, which is when a customer brings you memory dump and says "your program crashed, can you tell me why?"
Print statements all through your code reduces readability.
Adding and removing them for debug purposes only is time consuming
Debuggers track the call stack making it easy to see where you are
Variables can be modified on the fly
Adhoc commands can be executed during a pause in execution to assist diagnosing
Can be used IN CONJUNCTION with print statements : Debug.Write("...")
I think debugging using print statements is a lost art, and very important for every developer to learn. Once you know how to do that, certain classes of bugs become much easier to debug that way than through an IDE. Programmers who know this technique also have a really good feel of what's useful information to put in a log message (not to mention you'll actually end up reading the log) for non-debugging purposes as well.
That said, you really should know how to use the step-through debugger, since for a different class of bugs it is WAY easier. I'll leave it up to the other excellent answers already posted to explain why :)
Off the top of my head:
Debugging complex objects - Debuggers allow you to step deep into an object's innards. If your object has, say, an array of array of complex objects, print statements will only get you so far.
The ability to step past code - Debuggers will also allow you to skip past code you don't want to execute. True, you could do this manually as well, but it's that much more code you have to inject.
As alternative to debug in IDE you can try great Google Chrome extension PHP Console with php library that allows to:
See errors & exception in Chrome JavaScript console & in notification popups.
Dump any type variable.
Execute PHP code remotely.
Protect access by password.
Group console logs by request.
Jump to error file:line in your text editor.
Copy error/debug data to clipboard (for testers).
I haven't been developing for nearly 20 years, but I find that using a IDE / debugger I can :
see all kinds of things I might not have thought to have included in a print statement
step through code to see if it matches the path I thought it would take
set variables to certain values to make code take certain branches
One reason to use the IDE might be that modern IDEs support more than simple breakpoints. For example, Visual Studio offers the following advanced debugging features:
define conditional breakpoints (break only if a condition is met, or only on the n-th time the statement at the breakpoint is executed)
break on an unhandled exception or whenever a (specific) ecxeption is to be thrown
change variable while debugging
repeating a piece of code by setting the next line to be executed
etc.
Also, when using the debugger, you won't have to remove all your print statements once you have finished debugging.
In my experience, simple printouts have one huge advantage that no one seems to mention.
The problem with an IDE debugger is that everything happens at real time. You halt the program at a certain time, then you step through the steps one at a time and it is impossible to go back if you suddenly want to see what happened before. This is completley at odds with how our brain works. The brain collects information, and gradually forms an oppinion. It might be necessary to iterate the events several times in doing so, but once you have stepped past a certain point, you cannot go back.
In contrast to this, a selected series of printouts/logging gives you a "spatial projection of the temporal events". It gives you a complete story of what happened, and you can go back and fourth several times very easily by just scrolling up and down. It makes it easy to answer questions like "did A occur before B happened". It can make you see patterns you wernt even looking for.
So in my experience. IDE and debuggers are fantastic tools to solve simple problems when something in one single call-stack went wrong, and explore the current state of the machine at a certain crash.
However, when we approach more difficoult problems where gradual changing of state is involved. Where for example one algorithm corrupted a data structure, that in turn caused anohter algorithm to fail. Or if we want to answer questions like "how often do this happen", "do things happen in the order and in the way as I imagine them to happen". etc. Then the "old fashined" logging/printout technique has a clear advantage.
The best things is to use either technique when it is most suitable, for example use logging/printouts to get to some bugs, and pause at a breakpoint where we need to explore the current state more in detail.
There are also hybrid approaches. For example, when you do console.log(object) you get a data-structure widget in the log that you can expand and explore more in detail.This is many times a clear advantage over a "dead" text log.
One thing that I'm surprised I haven't seen in another answer is that the 2 debugging methods are not mutually exclusive.
printf debugging can work quite nicely even if you're using a standard debugger (whether IDE based or not). In particular with a logging framework so you can leave all or most of in the released product to help with diagnosing customer problems.
As noted in pretty much all the other answers here, the key nice thing about a standard debugger is that it allows you to more easily examine (and potentially change) the details of the program state. You don't have to know up front what you might want to look at - it's all available at your fingertips (more or less).
Because debugging multi-threaded applications with print statements will drive you bananas. Yes you can still do it with print statements but you'd need a lot of them and unravelling the sequential print out of statements to emulate the multi-threaded executiong would take a long long time.
Human brains are only single-threaded unfortunately.
Since you asked for pointers to books... As far as Windows debugging goes, John Robbins has several editions of a good book on Windows debugging:
Debugging Applications for Microsoft .NET and Microsoft Windows
Note that the most recent edition (Debugging Microsoft .NET 2.0 Applications) is .NET only, so you might want an older one (like in the first link) if you want native code debugging (it covers both .NET and native).
I personally feel the answer is as simple as "A integrated debugger/IDE gives you a wealth of different information quickly without the need for punching in commands. The information tends to be there in front of you without you haven't tell it what to show you.
The ease in which the information can be retrieved is what makes them better than just command-line debugging, or "printf" debugging.
Advantages of a debugger over a printf (note not an IDE debugger but any debugger)
Can set watchpoints.
This is one of my favourite ways of finding memory corruptions
Can debug a binary that you can't recompile at the moment
Can debug a binary that takes a long time to recompile
Can change variables on the fly
Can call functions on the fly
Doesn't have the problem where debug statemenets are not flushed and hence timing issue can not be debugged acuratly
Debuggers help with core dumps, print statements dont'
This is what I use most on VS.NET debugging windows:
Call stack, which is also a great way to figure out someone else's code
Locals & Watches.
Immediate window, which is basically a C# console and also lets me change variable contents, initialize stuff etc.
The ability to skip a line, set the next statement to be executed somewhere else.
The ability to hover over variables and have a tool-tip showing me their values.
In summary, it gives me a 360 degree view of the state of my executing code, not just a small window.
Never found a book teaching this kind of stuff, but then again, it seems to be quite simple, it's pretty much WYSIWYG.
A debugger can attach to a running process
Often easier to debug threaded code from a debugger
With an IDE debugger you can see the values of ALL the variables in the current scope (all the way up the call stack) whenever you halt execution.
Print statements can be great but dumping so much information to the screen at any given place can produce a whole lot of print statements.
Also, many IDE debuggers let you type in and evaluate methods, and evaluate members while you are halted, which further increases the amount of print statements you'd have to do.
I do feel that debuggers are better for some languages than for others however...
My general opinion is that IDE debuggers are absolutely, amazingly wonderful for managed languages like Java or C#, are fairly useful for C++, and are not very useful for scripting languages like Python (but it could be that I just haven't tried a good debugger for any scripting languages yet).
I absolutely love the debugger in IntelliJ IDEA when I do Java development. I just use print statements when I use Python.
As someone said above: Debugger != IDE.
gdb and (back in the day) TurboDebugger (stand-alone) work just fine for the languages they support[ed], thank you. (or an even older technology: Clipper debugger linked into the xBase executable itself) -- none of these required an IDE
Also, though C/++ coding is more rare, printf statements sometimes mask off the very bug you are trying to find! (initialization problems in auto vars on the stack, for instance, or memory allocation/alignment)
Finally, as others stated, you can use both. Some real-time-ish problems almost require a print, or at least a judicious "*video_dbg = ( is_good ? '+' : '-');" somewhere into video memory. My age is showing, this was under DOS :-)
TMTOWTDI
In addition to much of what the other posters have said, I really like stepping through one line at a time along with the computer, as it forces me to think about one line at a time. Often I will catch the bug without even looking at variable values simply because I am forced to look at it as I click the 'next line' button. However, I don't think my answer will help you, Bill, because you probably have this skill already.
As far as learning resources go, I haven't used any -- I just explore all the menus and options.
Is this even real question from real programmer?
Anyone who spent even 5 mins debugging with print statements and debugging with IDE - it will OCCUR to him/her without even asking!
I've used both prints and IDEs for debugging and I would much rather debug using an IDE. The only time for me when that doesn't work is in time critical situations (like debugging online games) where you litter the code with print statements and then look at the log files after it has gone horribly wrong. Then if you still cannot figure it out, add more prints and repeat.
Just wanted to mention a useful feature of a console debugger vs printf and vs debugger in an IDE.
You can attach to a remote application (obvioustly, compiled in DEBUG mode) and inspect its state dumping the debugger output to a file using POSIX tee utility. Compared to printf, you can choose where to output the state in run-time.
It helped me a lot when I was debugging Adobe Flash applications deployed in an agressive environment. You just need to define some actions that print required state in each breakpoint, start the console debugger with fdb | tee output.log, and walk through some breakpoints. After that you can print the log and analyse the information by thorough comparison of the state in different breakpoints.
Unfortunatelly, this feature [logging to a file] is rarely available in GUI debuggers, making developers compare the state of objects in their head.
By the way, my opinion is that one should plan where and what to debug before staring a debugger.
Well another thing is that if you join a new old project and nobody really knows how the code is doing what it's doing, then you can't debug by echoing variables/objects/... b/c you have no idea what code is executed at all.
At my job I am facing exactly that kind of situation and visual XDebuging helps me getting an idea about what is going on and where, at all.
Best regards
Raffael
In addition to the many things that have been already mentioned, one of the most important advantages of a debugger over printf is that using printf statements assumes that you know in which function the bug resides. In many cases you don't, so you have to make a few guesses and add print statements to many other functions in order to localise it. The bug may be in framework code or somewhere far removed from where you think it is. In a debugger it is far easier to set breakpoints to examine the state in different areas of the code and at different points in time.
Also, a decent debugger will let you do printf-style debugging by attaching conditions and actions to breakpoints, so that you still retain the benefits of printf debugging, but without modifying the code.
Debugging in an IDE is invaluable in an environment where error logs and shell access are unavailable, such as a shared host. In that case, an IDE with a remote debugger is the only tool which allows you to do simple things such as view stderr or stdout.
A problem with using print statements is it makes a mess of your code. IE, you have a function with 10 parts to it and you know it crashes somewhere, but you're not sure where. So you add in 10 extra print statements to pinpoint where the bug is. Once you've found and solved your bug, you now have to clean up by removing all of those print statements. Maybe you'll do that. Maybe you'll forget and it'll end up in production and your user's console will be full of debug prints.
Wauw, do I like this question. I never dared to pose it...
It seems that people just have different ways of working.
For me what works best is:
Having a solid mind model of my code, including memory management
Using instrumentation (like print statements) to follow what's happening.
I've earned my living programming for over 40 years now, working at non-trivial technical and scientific applications in C++ and Python daily, and I have the personal experience that a debugger doesn't help me a bit.
I don't say that's good. I don't say that's bad. I just want to share it.
It's not just debugging. An IDE helps you build better software faster in a lot of ways:
refactoring tools
intellisense to make api's more discoverable, or remind of exact spelling/case of familiar items(not much use if you've used the same system for 15 years, but that's rare)
save on typing by autocompleting variable and class names
find certain kinds of errors before you even start to compile
Automatically jump to variable/method/class declarations/definitions, even if they're not in the same file or folder.
Break on unhandled and handled exceptions
I could go on.

VB6 (erk) - Inline functions?

I use VB6 for an application.
Is it possible to force the compiler to inline a function?
Or is there an add-in that achieves the same thing?
There's a secure part of my code that I want to make difficult to hack, by repeating the code at every point where it is used instead of being listed once as a function.
Hope someone can answer my question!
IMO if this is what you're doing as a security measure, you have bigger problems than getting VB to inline your function. And I don't think there is any provision in VB6 to do this. AND I tend to think that this technique would make it easier to hack your code, since you'd see the same really important function repeated over and over again... Sorry :-(
There is no support for inlining a function. However there are several things working in your favor.
First VB6 is notoriously difficult to decompile as witness by the lack of decompilers on the market over the history. The results has been less than useful for people trying to recover lost source code or hack VB6.
But...
If you are using ActiveX DLLs then it could very easy to hack your software by a person writing a compatible DLL. The best way my company found to deal with this is to make critical objects public non-creatable and exchange packed binary data.
The public non-creatable prevents somebody from referencing the DLL, creating instance of that object and then running tests to see what you are doing. The binary data is to obscure the data you are exchanging.
In you look in the literature about COM there are more secure ways of dealing with these issues but these are simple things you can do to make a ActiveX application more difficult to hack.
My company goal isn't to make it impossible for somebody to hack our software but make it difficult enough that it cost less for our competitor to deal with us rather than try to hack our system (A CAD/CAM system)
As Dave said there is no support for that in the VB compiler.
If you really want to this why not run a search & replace on a copy of your code and build that copy. Doing that on the command line shouldn't be too difficult.

Does it still make sense to learn low level WinAPI programming? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
Does it make sense, having all of the C#-managed-bliss, to go back to Petzold's Programming Windows and try to produce code w/ pure WinAPI?
What can be learn from it? Isn't it just too outdated to be useful?
This question is bordering on religious :) But I'll give my thoughts anyway.
I do see value in learing the Win32 API. Most, if not all, GUI libraries (managed or unmanaged) result in calls to the Win32 API. Even the most thorough libraries don't cover 100% of the API, and hence there are always gaps which need to be plugged by direct API calls or P/invoking. Some of the names of the wrappers around the API calls have similar names to the underlying API calls, but those names aren't exactly self-documenting. So understanding the underlying API, and the terminology used therein, will aid in understanding the wrapper APIs and what they actually do.
Plus, if you understand the nature of the underlying APIs that are used by frameworks, then you will make better choices with regards to which library functionality you should use in a given scenario.
Cheers!
I kept to standard C/C++ for years before learning Win32 API, and to be quite blunt, the "learning Win32 API" part is not the best technical experience of my life.
In one hand Win32 API is quite cool. It's like an extension of the C standard API (who needs fopen when you can have CreateFile. But I guess UNIX/Linux/WhateverOS have the same gizmo functions. Anyway, in Unix/Linux, they have the "Everything is a file". In Windows, they have the "Everything is a... Window" (no kidding! See CreateWindow!).
In the other hand, this is a legacy API. You will be dealing with raw C, and raw C madness.
Like telling one's structure its own size to pass through a void * pointer to some Win32 function.
Messaging can be quite confusing, too: Mixing C++ objects with Win32 windows lead to very interesting examples of Chicken or Egg problem (funny moments when you write a kind of delete this ; in a class method).
Having to subclass a WinProc when you're more familiar with object inheritance is head-splitting and less than optimal.
And of course, there is the joy of "Why in this fracking world they did this thing this way ??" moments when you strike your keyboard with your head once too many and get back home with keys engraved in your forehead, just because someone thought it more logical to write an API to enable the changing of the color of a "Window", not by changing one of its properties, but by asking it to its parent window.
etc.
In the last hand (three hands ???), consider that some people working with legacy APIs are themselves using legacy code styling. The moment you hear "const is for dummies" or "I don't use namespaces because they decrease the runtime speed", or the even better "Hey, who needs C++? I code in my own brand of object-oriented C!!!" (No kidding... In a professional environment, and the result was quite a sight...), you'll feel the kind of dread only condemned feel in front of the guillotine.
So... All in all, it's an interesting experience.
Edit
After re-reading this post, I see it could be seen as overly negative. It is not.
It is sometimes interesting (as well as frustrating) to know how the things work under the hood. You'll understand that, despite enormous (impossible?) constraints, the Win32 API team did wonderful work to be sure everything, from you "olde Win16 program" to your "last Win64 over-the-top application", can work together, in the past, now, and in the future.
The question is: Do you really want to?
Because spending weeks to do things that could be done (and done better) in other more high-level and/or object-oriented API can be quite de-motivational (real life experience: 3 weeks for Win API, against 4 hours in three other languages and/or libraries).
Anyway, you'll find Raymond Chen's Blog very interesting because of his insider's view on both Win API and its evolution through the years:
https://blogs.msdn.microsoft.com/oldnewthing/
Absolutely. When nobody knows the low level, who will update and write the high level languages? Also, when you understand the low level stuff, you can write more efficient code in a higher level language, and also debug more efficiently.
The native APIs are the "real" operating system APIs. The .NET library is (with few exceptions) nothing more than a fancy wrapper around them. So yes, I'd say that anybody who can understand .NET with all its complexity, can understand relatively mundane things like talking to the API without the benefit of a middle-man.
Just try to do DLL Injection from managed code. It can't be done. You will be forced to write native code for this, for windowing tweaks, for real subclassing, and a dozen other things.
So yes: you should (must) know both.
Edit: even if you plan to use P/Invoke.
On the assumption that you're building apps targeted at Windows:
it can sure be informative to understand lower levels of the system - how they work, how your code interacts with them (even if only indirectly), and where you have additional options that aren't available in the higher-level abstractions
there are times when your code might not be as efficient, high-performance or precise enough for your requirements
However, in more and more cases, folks like us (who never learned "unmanaged coding") will be able to pull off the programming we're trying to do without "learning" Win32.
Further, there's plenty of sites that provide working samples, code fragments and even fully-functional source code that you can "leverage" (borrow, plagiarize - but check that you're complying with any re-use license or copyright!) to fill in any gaps that aren't handled by the .NET framework class libraries (or the libraries that you can download or license).
If you can pull off the feats you need without messing around in Win32, and you're doing a good job of developing well-formed, readable managed code, then I'd say mastering .NET would be a better choice than spreading yourself thin over two very different environments.
If you frequently need to leverage those features of Windows that haven't received good Framework class library coverage, then by all means, learn the skills you need.
I've personally spent far too much time worrying about the "other areas" of coding that I'm supposed to understand to produce "good programs", but there's plenty of masochists out there that think everyone's needs and desires are like their own. Misery loves company. :)
On the assumption that you're building apps for the "Web 2.0" world, or that would be just as useful/beneficial to *NIX & MacOS users:
Stick with languages and compilers that target as many cross-platform environments as possible.
pure .NET in Visual Studio is better than Win32 obviously, but developing against the MONO libraries, perhaps using the Sharp Develop IDE, is probably an even better approach.
you could also spend your time learning Java, and those skills would transfer very well to C# programming (plus the Java code would theoretically run on any platform with the matching JRE). I've heard it said that Java is more like "write once, debug everywhere", but that's probably as true as (or even moreso than) C#.
Analogy: If you build cars for a living (programming), then its very pertinent to know how the engine works (Win32).
Simple answer, YES.
This is the answer to any question that is like.. "does it make sense to learn a low level language/api X even when a higher level language/api Y is there"
YES
You are able to boot up your Windows PC (or any other OS) and ask this question in SO because a couple of guys in Microsoft wrote 16-bit assembly code that loads your OS.
Your browser works because someone wrote an OS kernel in C that serves all your browser's requests.
It goes all the way up to scripting languages.
Big or small, there is always a market and opportunity to write something in any level of abstraction. You just have to like it and fit in the right job.
No api/language at any level of abstraction is irrelevent unless there is a better one competing at the same level.
Another way of looking at it: A good example from one of Michael Abrash's book: A C programmer was given the task of writing a function to clear the screen. Since C was a better (higher level) abstraction over assembly and all, the programmer only knew C and knew it well. He did his best - he moved the cursor to each location on the screen and cleared the character there. He optimized the loop and made sure it ran as fast as it could. But still it was slow... until some guy came in and said there was some BIOS/VGA instruction or something that could clear the screen instantly.
It always helps to know what you are walking on.
Yes, for a few reasons:
1) .net wraps Win32 code. .net is usually a superior system to code against, but having some knowledge of the underlying Win32 layer (oops, WinAPI now that there is 64-bit code too) bolsters your knowledge of what is really happening.
2) in this economy, it is better to have some advantages over the other guy when you are looking for a job. Some WinAPI experience may provide this for you.
3) some system aspects are not available through the .net framework yet, and if you want to access those features you will need to use p/invoke (see http://www.pinvoke.net for some help there). Having at least a smattering of WinAPI experience will make your p/invoke development effort a lot more efficient.
4) (added) Now that Win8 has been around for awhile, it is still built on top of the WinAPI. iOS, Android, OS/X, and Linux are all out there, but the WinAPI will still be out there for many many years.
Learning a new programming language or technology is for one of three reasons:
1. Need: you're starting a project for building a web application and you don't know anything about ASP.NET
2. Enthusiasm: you're very excited about ASP.NET MVC. why not try that?
3. Free time: but who has that anyway.
The best reason to learn something new is Need. If you need to do something that the .NET framework can't do (like performance for example) then WinAPI is your solution. Until then we keep ourself busy with learning about .NET
For most needs on the desktop you wont need to know the Win32, however there is a LOT of Win32 not in .NET, but it is in the outlaying stuff that may end up being less than 1% of your application.
USB support, HID support, Windows Media Foundation just off the top of my head. There are many cool Vista API's only available from Win32.
You will do yourself a large favor by learning how to do interop with a Win32 API, if you do desktop programing, because when you do need to call Win32, and you will, you won't spend weeks scratching your head.
Personally I don't really like the Win32 API but there's value in learning it as the API will allow more control and efficiency using the GUI than a language like Visual Basic, and I believe that if you're going to make a living writing software you should know the API even if you don't use it directly. This is for reasons similar to the reasons it's good to learn C, like how a strcpy takes more time than copying an integer, or why you should use pointers to arrays as function parameters instead of arrays by value.
Learning C or a lower level language can definitely be useful. However, I don't see any obvious advantage in using the unmanaged WinAPI.
I've seen low level Windows API code... it ain't pretty... I wish I could unlearn it. I think it benefits to learn low level as in C, as you gain a better understanding of the hardware architecture and how all that stuff works. Learning old Windows API... I think that stuff can be left to the people at Microsoft who may need to learn it to build higher level languages and API... they built it, let them suffer with it ;-)
However, if you happen to find a situation where you feel you just can't do what you need to do in a higher level language (few and far between), then perhaps start the dangerous dive into that world.
yes. take a look at uTorrent, an amazing piece of software efficiency. Half of it's small size is due to the fact that much of it's core components were re-written to not use gargatuian libraries.
Much of this couldn't be done without understanding how these libraries interface with the lower level API's
It's important to know what is available with the Windows API. I don't think you need to crank out code with it, but you should know how it works. The .NET Framework contains a lot of functionality, but it doesn't provide managed code equivalents for the entire Windows API. Sometimes you have to get a bit closer to the metal, and knowing what's down there and how it behaves will give you a better understanding of how to use it.
This is really the same as the question, should I learn a low level language like C (or even assembler).
Coding in it is certainly slower (though of course the result is much faster), but its true advantage is you gain an insight into what is happening at close to the system level, rather than than just understanding someone else's metaphor for what is going on.
It can also be better when things won't work well, or fast enough or with the sort of granularity that you need. (And do at least some subclassing and superclassing.)
I'll put it this way. I don't like programming to the Win32 API. It can be a pain compared to managed code. BUT, I'm glad I know it because I can write programs that otherwise I wouldn't be able to. I can write programs that other people can't. Plus it gives you more insight into what your managed code is doing behind the scenes.
The amount of value you get out of learning the Win32 API, (aside from the sorts of general insights you get from learning about how the nuts and bolts of the machine fit together) depends on what you're trying to achieve. A lot of the Win32 API has been wrapped nicely in .NET library classes, but not all of it. If for instance you're looking to do some serious audio programming, that portion of the Win32 API would be an excellent subject of study because only the most basic of operations are available from .NET classes. Last I checked even the managed DirectX DirectSound library was awful.
At the risk of shameless self-promotion....
I just came across a situation where the Win32 API was my only option. I want to have different tooltips on each item in a listbox. I wrote up how I did it on this question.
Even in very very high level languages you still make use of the API. Why? Well not every aspect of the API has been replicated by the various libraries, frameworks, etc. You need to learn the API for as long as you will need the API to accomplish what you are trying to do. (And no longer.)
Apart from some very special cases when you need direct access to APIs, I would say NO.
There is considerable time and effort required to learn to implement the native API calls correctly and the returning value is just not worth it. I would rather spend the time learning some new hot technology or framework that will make your life easier and programming less painful. Not decades-old obsolete COM libraries that nobody really uses anymore (sorry to COM users).
Please don't stone me for this view. I know a lot of engineers here have really curious souls and there is nothing wrong with learning how things work. Curiousity is good and really helps understanding. But from a managerial point of view, I would rather spend a week learning how to develop Android apps than how to calls OLEs or COMs.
If you planning to develop a cross platform application, If you use win32, then your application could easily run on linux through WINE. This results in a highly maintainable application. This is one of the advantages of learning win32.

Resources