Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I am developing an application using the wrong tools. I don't wish to get into the rights or wrongs of this - the platform has been blessed as Strategic and nothing is going to change that now. But I'm hoping somebody can give me some tips on how to make the best of the situation.
We use a server-side language, let's call it X, and client-side HTML/JS/CSS (on IE6). X is primitive from an application development point of view (but excellent for data processing, which is why we are using it); it doesn't even have the concept of user-defined functions, so trying to make the application modular in any way is a challenge. Think tens of thousands of lines of nested if/then/else statements.
My current assumption is that reducing the spaghetti-factor of the code will not be possible, and that really great documentation is my only weapon against this becoming a totally unsupportable nightmare that ends up on TheDailyWTF.
Anybody got any other ideas?
(I don't mention what language X is simply because I'm hoping for answers to the general problem of working with deficient tools, not any particular tactics for X.)
Edit:
Ok, for the morbidly curious, X is SAS. I didn't want the question to focus on whether function-style macros are functions (they are not, and cannot implement design patterns), or to blame it - given the constraints of this particular project, I actually agree with the decision to use it! I am also sure that the majority of software is developed in incredibly non-optimal environments (broken tools, bad management, overbearing legacy burden, etc.), and that there must be strategies for making things work even so.
Are you familiar with Church thesis?
If you can't solve "A" in Y but you can emulate Z in Y and Z can solve "A" then by definition Y can solve "A".
Maybe you can write some generalized routine that somehow makes X more effective for the problem at hand? A sort of extension to X, or, even better, a little-language implemented in X?
It seems that others tend to conflate "little language" with documentation. While you can try to go that way (in this case I suggest you have a look at Robodoc) I was thinking something closer to Wasabi, in approach - i.e. really using your tool X to create a sort of interpreter for X++ or even Y, without knowing what X is I can't be, of course, more specific than that.
Does X have comments?
Write your little language aka pseudo code in the comments.
In addition to documentation, choices of variable names and conventions for how they are used may help a bit. Also you may be able to set up some structural conventions in the code so that there is some regularity. Way back when, when folks wrote assembler good coders produced readable code.
hmmmm, sounds like another MUMPS/Intersystems Cache developer ;)
Seriously though, you might want to check if there are any tools for 'X' which could map the flow of the program, or as part of the documentation process break out something like Visio or another similar tool where you can walk through the code and map out what it does (more or less). Hardest part would probably be having to go back and stare at that wall of code and jump right back in so anything you can do to document it/graph it/chart it will help.
Is it possible to use a different technology, better suited to your problem between X and the client-side?
Alternatively, you could use more IF/Then/else statements to construct modular blocks of code, which might help with maintenance.
I find it hard to believe that you don't have any form of user defined functions available in X - even batch files have functions (kind of)
As soon as you have functions, you can make things at least fairly modular.
You could find a language you like, and implement the usual "slap some data into a template"-level web-app stuff in that, and implement wrappers to call out to 'X' for the things it is good at.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
Improve this question
Why is it more costly to discover a defect later in the process?
I've heard this a lot but I struggle to understand and put context/examples to this.
You're building a house. You're laying the sewer pipes into the foundations, but unknown to you one of the pipes is blocked by a dead hedgehog.
Would you rather find out:
Just before you pour the concrete
After the house is finished and the new owner tries to use the toilet?
(There's a "Stack Overflow" joke somewhere in this analogy . 8-)
The longer it takes to find a bug, then:
the more the behavior of the bug may have been accepted as correct, and the more other things may have become dependent on that behavior (Windows is notorious for this).
the more tightly integrated the system is likely to have become, and the harder the bug will be to extract.
the higher the likelihood that the bug's erroneous behavior will be duplicated elsewhere by virtue of copy-pasting or in clients that use the erroneous code.
the longer it's been since the code was originally written and the harder it may be to understand it.
the less likely it will be for people who understand that original part of the system to be around to fix it.
This can be illustrated in a simple (if not trivial) example.
Take a simple dialog with a message and just two buttons "OK" and "Cancel".
Assume that the error is a spelling mistake.
If this is found after the product is released then a new version of the product has to be released with all the costs associated with that. Manuals will need to be reprinted.
If this is found in final testing the manual will have to be reprinted. The code will need to be rewritten and tests re-run.
If this is found during development then there is just the cost of fixing the code.
If this is found during design then the code is written correctly first time - no cost.
The later you find out a bug, the worse. When you find a bug immediately after you code, you have all the behavior in mind, and know exactly what changes caused it. You will be able to focus on the problem, once you know where it resides.
When you take long, developers no longer remember exactly how it worked, and there are many more places to investigate to find the bug. Perhaps the developer who coded the bug is no longer working in the company also.
Also, as time goes by, more parts of the code will probably depend on the buggy code, and you may need to fix them as well.
Finally, there are issues involving users. If you find a bug after a release, more users will be frustrated by it, and your product image will be worse. Users may also be used to have a workaround for this bug, which may start to fail after you fix the bug.
Summary: When you take long to find a bug
Your scope to investigate is bigger
The developer who created the bug may not be there anymore, and the other developers will have to study the code more to find it, understand it, and fix it
You may also need to fix parts that depends on buggy code (and there will be more parts like that)
Users will already be frustrated by the bug, and the image of the product will be damaged
No-one ever understands the code as well as you do as you are writing it.
People may have come to depend on the bug being there.
You may have to fix up lots of bad data that the bug has saved away.
You may have to roll out a new version or patch of your software.
Your helpdesk may have to field a whole heap of calls.
You may have to fill in bunches of paperwork explaining why that bug exists and what problems it causes, and what you are going to do to make sure it never, ever happens again.
Because more people will have spend time with the defective software.
If you fix a bug at early on you and maybe a code reviewer will spend a little time on it.
If it gets released to customers and reported as an error, you will have coded it, someone may have reviewed it, someone may have tested it, somebody may even have documented it and so forth ...
There may be other dependencies (internal or external) which will affect the fixing of a defect.
For example - If I resolve this defect, I may have to fix something else
Imagine you're writing an essay on why it's more costly to discover a defect later in the process, and you suddenly realise one of the premises on which most of your essay content is based is false.
If you're still planning, you only have the half a page of plan to change. If your essay is nearly finished, you suddenly need to scrap the lot and start over. If you've already handed it in, the error is gonna cost you your grade.
Same reason.
For a shrink-wrapped software product:
If you find a bug after your product hits the stores, you will have to help users through support calls, suggest a workaround or even recall the product/issue a service pack.
For a website:
Site outages and delays cost you money.
Customer loss as a result of poor/malfunctioning site costs you more.
The debugging process is also costly itself.
It is probably an error by the question author, but the actual question is, "Why is it more costly to discover a defect later in the process" Within that question is the cost to discover the bug and we can hope it also means to fix it. Most of the answers do a good job at describing the cost to fix and why it is better to fix early versus fix later. And, I really don't disagree with any of them. But, that isn't the whole question.
I have a regular series of esoteric arguments with some about the discovery cost. How much testing would have been required to find a specific bug (without hindsight). Would it have take 3 man-months more of automated or manual testing before you would have been likely to find that test case and scenario ?
In practice, test as much as you can but finding that balance point isn't as easy as many would have you think. Most programs are too big to have 100% code coverage. And, 100% code coverage is usually just a fraction of all the possible scenarios the code must handle.
Another factor that comes into the cost of a bug is the business cost associated with the bug. Are there 5 million boxes out there holding the bug ? Would you have to do a product recall ? Will it generate X calls to your warranty help desk ? Will it trigger some clause in a contract holding you liable for damages. In very simple terms, this is why software written in the medical field costs more per LOC than those for website development.
Because of the development process and all the work involved in fixing the defect.
Imagine you find a problem in the function you coded yesterday, you just check out, fix, check in, period. It's still fresh in your mind, you know what it is about and that your fix won't have any side effect.
Now imagine finding the same bug in six month from now. Will you remember why the function was coded that way ? Will you still be working on this project/company ? You have to open a defect report, a new version of your software have to be issued, QA needs to validate the correction. If the software has been deployed, then all instances have to be upgraded, customers will call support ...
Now it's true that the curve showing the cost are made up to illustrate the point; it actually depends on the development process.
I would say that the most costly is to find a defect and let it be. The longer you allow the defect to live the more costly it becomes.
I was at a company at a time, where they had the policy, that once they had taken a decision, they stick with it. The system I worked on was loaded with bugs because of a stupid corporate framework that we were forced to use, and a deep misunderstanding of the proper usage of web services.
To this day, I believe that the cheapest way for that company to get a working, usable system, would be to ditch the entire system and rewrite it from scratch.
So my point is, that I don't think that finding a defect at a late stage is that problematic. But ignoring a defect until a late stage is extremely problematic.
What's your standard way of debugging a problem? This might seem like a pretty broad question with some of you replying 'It depends on the problem' but I think a lot of us debug by instinct and haven't actually tried wording our process. That's why we say 'it depends'.
I was sort of forced to word my process recently because a few developers and I were working an the same problem and we were debugging it in totally different ways. I wanted them to understand what I was trying to do and vice versa.
After some reflection I realized that my way of debugging is actually quite monotonous. I'll first try to be able to reliably replicate the problem (especially on my local machine). Then through a series of elimination (and this is where I think it's problem dependent) try to identify the problem.
The other guys were trying to do it in a totally different way.
So, just wondering what has been working for you guys out there? And what would you say your process is for debugging if you had to formalize it in words?
BTW, we still haven't found out our problem =)
My approach varies based on my familiarity with the system at hand. Typically I do something like:
Replicate the failure, if at all possible.
Examine the fail state to determine the immediate cause of the failure.
If I'm familiar with the system, I may have a good guess about to root cause. If not, I start to mechanically trace the data back through the software while challenging basic assumptions made by the software.
If the problem seems to have a consistent trigger, I may manually walk forward through the code with a debugger while challenging implicit assumptions that the code makes.
Tracing the root cause is, of course, where things can get hairy. This is where having a dump (or better, a live, broken process) can be truly invaluable.
I think that the key point in my debugging process is challenging pre-conceptions and assumptions. The number of times I've found a bug in that component that I or a colleague would swear is working fine is massive.
I've been told by my more intuitive friends and colleagues that I'm quite pedantic when they watch me debug or ask me to help them figure something out. :)
Consider getting hold of the book "Debugging" by David J Agans. The subtitle is "The 9 Indispensable Rules for Finding Even the Most Elusive Software and Hardware Problems". His list of debugging rules — available in a poster form at the web site (and there's a link for the book, too) is:
Understand the system
Make it fail
Quit thinking and look
Divide and conquer
Change one thing at a time
Keep an audit trail
Check the plug
Get a fresh view
If you didn't fix it, it ain't fixed
The last point is particularly relevant in the software industry.
I picked those on the web or some book which I can't recall (it may have been CodingHorror ...)
Debugging 101:
Reproduce
Progressively Narrow Scope
Avoid Debuggers
Change Only One Thing At a Time
Psychological Methods:
Rubber-duck debugging
Don't Speculate
Don't be too Quick to Blame the Tools
Understand Both Problem and Solution
Take a Break
Consider Multiple Causes
Bug Prevention Methods:
Monitor Your Own Fault Injection Habits
Introduce Debugging Aids Early
Loose Coupling and Information Hiding
Write a Regression Test to Prevent Re occurrence
Technical Methods:
Inert Trace Statements
Consult the Log Files of Third Party Products
Search the web for the Stack Trace
Introduce Design By Contract
Wipe the Slate Clean
Intermittent Bugs
Explot Localility
Introduce Dummy Implementations and Subclasses
Recompile / Relink
Probe Boundary Conditions and Special Cases
Check Version Dependencies (third party)
Check Code that Has Changed Recently
Don't Trust the Error Message
Graphics Bugs
When I'm up against a bug that I can't get seem to figure out, I like to make a model of the problem. Make a copy of the section of problem code, and start removing features from it, one at a time. Run a unit test against the code after every removal. Through this process your will either remove the feature with the bug (and hence, locate the bug), or you will have isolated the bug down to a core piece of code that contains the essence of the problem. And once you figure out the essence of the problem, its a lot easier to fix.
I normally start off by forming an hypothesis based on the information I have at hand. Once this is done, I work to prove it to be correct. If it proves to be wrong, I start off with a different hypothesis.
Most of the Multithreaded synchronization issues get solved very easily with this approach.
Also you need to have a good understanding of the debugger you are using and its features. I work on Windows applications and have found windbg to be extremely helpful in finding bugs.
Reducing the bug to its simplest form often leads to greater understanding of the issue as well adding the benefit of being able to involve others if necessary.
Setting up a quick reproduction scenario to allow for efficient use of your time to test any hypothosis you chose.
Creating tools to dump the environment quickly for comparisons.
Creating and reproducing the bug with logging turned onto the maximum level.
Examining the system logs for anything alarming.
Looking at file dates and timestamps to get a feeling if the problem could be a recent introduction.
Looking through the source repository for recent activity in the relevant modules.
Apply deductive reasoning and apply the Ockham's Razor principles.
Be willing to step back and take a break from the problem.
I'm also a big fan of using process of elimination. Ruling out variables tremendously simplifies the debugging task. It's often the very first thing that should to be done.
Another really effective technique is to roll back to your last working version if possible and try again. This can be extremely powerful because it gives you solid footing to proceed more carefully. A variation on this is to get the code to a point where it is working, with less functionality, than not working with more functionality.
Of course, it's very important to not just try things. This increases your despair because it never works. I'd rather make 50 runs to gather information about the bug rather take a wild swing and hope it works.
I find the best time to "debug" is while you're writing the code. In other words, be defensive. Check return values, liberally use assert, use some kind of reliable logging mechanism and log everything.
To more directly answer the question, the most efficient way for me to debug problems is to read code. Having a log helps you find the relevant code to read quickly. No logging? Spend the time putting it in. It may not seem like you're finding the bug, and you may not be. The logging might help you find another bug though, and eventually once you've gone through enough code, you'll find it....faster than setting up debuggers and trying to reproduce the problem, single stepping, etc.
While debugging I try to think of what the possible problems could be. I've come up with a fairly arbitrary classification system, but it works for me: all bugs fall into one of four categories. Keep in mind here that I'm talking about runtime problems, not compiler or linker errors. The four categories are:
dynamic memory allocation
stack overflow
uninitialized variable
logic bug
These categories have been most useful to me with C and C++, but I expect they apply pretty well elsewhere. The logic bug category is a big one (e.g. putting a < b when the correct thing was a <= b), and can include things like failing to synchronize access among threads.
Knowing what I'm looking for (one of these four things) helps a lot in finding it. Finding bugs always seems to be much harder than fixing them.
The actual mechanics for debugging are most often:
do I have an automated test that demonstrates the problem?
if not, add a test that fails
change the code so the test passes
make sure all the other tests still pass
check in the change
No automated testing in your environment? No time like the present to set it up. Too hard to organize things so you can test individual pieces of your program? Take the time to make it so. May make it take "too long" to fix this particular bug, but the sooner you start, the faster everything else'll go. Again, you might not fix the particular bug you're looking for but I bet you find and fix others along the way.
My method of debugging is different, probably because I am still beginner.
When I encounter logical bug I seem to end up adding more variables to see which values go where and then I go and debug line by line in the piece of code that causing a problem.
Replicating the problem and generating a repeatable test data set is definitely the first and most important step to debugging.
If I can identify a repeatable bug, I'll typically try and isolate the components involved until I locate the problem. Frequently I'll spend a little time ruling out cases so I can state definitively: The problem is not in component X (or process Y, etc.).
First I try to replicate the error, without being able to replicate the error it is basically impossible in a non-trivial program to guess the problem.
Then if possible, break out the code in a separate standalone project. There are several reasons for this: If the original project is big it quite difficult to debug second it eliminates or highlights any assumptions about the code.
I normally always have another copy of VS open which I use for the debugging parts in mini projects and to test routines which I later add to the main project.
Once having reproduced the error in the separate module the battle is almost won.
Sometimes it is not easy to break out a piece of code so in those cases I use different methods depending on how complex the issue is. In most cases assumptions about data seem to come and bite me so I try to add lots of asserts in the code in order make sure my assumptions are correct. I also disabling code by using #ifdef until the error disappears. Eliminating dependencies to other modules etc... sort of slowly circling in the bug like a vulture ..
I think I don't have really a conscious way of doing it, it varies quite a lot but the general principle is to eliminate the noise around the issue until it is quite obvious what it is. Hope I didn't sound too confusing :)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
When do you start to consider a code base to be getting too large and unwieldy?
-when a significant amount of your coding time is devoted to "where do I put this code?"
-when reasoning about side-effects starts to become really hard.
-when there's a significant amount of code that's just "in there", and nobody knows what it does or if it's still running but it's too scary to remove
-when lots of team members spend significant chunks of their time chasing down intermittent bugs caused by some empty string somewhere in the data where it wasn't expected, or something that you think would usually be caught in a well-written application, in some edge case
-when, in considering how to implement a new feature, "complete rewrite" starts to seem like a good answer
-when you dread looking at the mess of code you need to maintain and wish you could find work building something clean and logical instead of dumpster diving through the detritus of someone else's poorly organized thinking
When it's over 100 lines. Joke. This is probably the hardest question to answer, because it's very individual.
But if you structure the application well and use different layers for i.e. interfaces, data, services and front-end you will automaticly get a nice "base"-structure. Then you can dividie each layer into different classes and then inside the classes you point out the appropriet methods for the class.
However, there's not an "x amount of lines per method is bad" but think of it more like this, if there is possibility of replication, split it from the current peice and make it re-usable.
Re-using code is the basics of all good structure.
And splitting up into different layers will help the base to become more and more flexible and modular.
There exist some calculable metrics if that's what you're searching for. Static code analysis tools can help with that:
Here's one list: http://checkstyle.sourceforge.net/config_metrics.html
Other factors can be the time it takes to change/add something.
Other non-calculable factors can be
the risk associated to changes
the level intermingling of features.
if the documentation can keep up with the features / code
if the documentation represent the application.
the level of training needed.
the quantity of repeat instead of reuse.
Ah, the god-program anti-pattern.
When you can't remember at least the
outline of sections of it.
When you have to think about how
changes will affect itself or
dependencies.
When you can't remember all the
things it's dependant on or depend
on it.
When it takes more than a few
minutes(?) to download the source or
compile.
When you have to worry about how to
deploy new versions.
When you encounter classes which are
functionally identical to other
classes elsewhere in the app.
So many possible signs.
I think there are many thoughts to why some code base is too large.
It is hard to remain in a constant naming convention. If classes/methods/atributes can't be named consistently or can't be found consistently, then it's time to reorganize.
When your programmers are surfing the web and going to lunch in order to compile. Keeping compiling/linking time to a minimum is important for management. The last thing you want is a programmer to get distracted by twiddling their thumbs for too long.
When small changes start to affect many MANY other places of code. There is a benefit to consolidation of code, but there is also a cost. If a small change to fix one bug causes a dozen more, and this is commonly happens, then your code base needs to be spread out (versioned libraries) or possibly unconsolidated (yes, duplicate code).
If the learning curve of new programmers to the project is obviously longer than acceptable (usually 90 days), then your code base/training isn't set up right.
..There are many many more, I'm sure. If you think about it from these three perspectives:
Is it hard to support?
Is it hard to change?
Is it hard to learn?
...Then you will have an idea if your code fits the "large and unwieldy" category
For me, code becomes unwieldy when there's been a lot of changes made to the codebase that weren't planned for when the program was initially written or last refactored significantly. At this point, stuff starts to get fitted into the existing codebase in odd places for expediency and you start to get a lot of design artifacts that only make sense if you know the history of the implementation.
Short answer: it depends on the project.
Long answer:
A codebase doesn't have to be large to be unwieldy - spaghetti code can be written from line 1. So, there's not really a magic tripping point from good to bad - it's more of a spectrum of great <---> awful, and it takes daily effort to keep your codebase from heading in the wrong direction. What you generally need is a lead developer that has the ability to review others' code objectively, and keep an eye on the architecture and design of the code as a whole - no one line developer can do that.
When I can't remember what a class does or what other classes it uses off the top of my head. It's really more a function of my cognitive capacity coupled with the code complexity.
I was trying to think of a way of deciding based on how your collegues perceive it to be.
During my first week at a gig a few years ago, I said during a stand-up that I had been tracking a white rabbit around the ContainerManagerBean, the ContainerManagementBean and the ContextManagerBean (it makes me shudder just recalling these words!). At least two of the developers looked at their shoes and I could see them keeping in a snigger.
Right then and there, I knew that this was not a problem with my lack of familiarity with the codebase - all the developers perceived a problem with it.
If over years of development different people code change requests and bug fixes you will sooner or later get parts of code with duplicated functionality, very similar classes, some spaghetti etc.
This is mostly due to the fact that a fix is needed fast and the "new guy" doesn't know the code base. So he happily codes away something which is already there.
But if you have automatic checks in place checking the style, unit test code coverage and similar you can avoid some of it.
A lot of the things that people have identified as indicating problems don't really have to do with the raw size of the codebase, but rather its comprehensibility. How does size relate to comprehensibility? If at all...
I've seen very short programs that are just a mess -- easier to throw away and redo from scratch. I've also seen very large programs whose structure is transparent enough that it is comprehensible even at progressively more detailed views of it. And everything in between...
I think look at this question from the standpoint of an entire codebase is a good one, but it probably pays to work up from the bottom and look first at the comprehensibility of individual classes, to multi-class components, to subsystems, and finally up to an entire system. I would expect the answers at each level of detail to build on each other.
For my money, the simplest benchmark is this: Can you explain the essence of what X does in one sentence? Where X is some granularity of component, and you can assume an understanding of the levels immediately above and below the component.
When you come to need a utility method or class, and have no idea whether someone else has already implemented it or have any idea where to look for one.
Related: when several slightly different implementations of the same functionality exist, because each author was unaware of other authors' work.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
When I am stuck with a problem:
I search Google for code snippets.
I look at isolating the problem, so that I can better explain it to others in order to get answers.
What search techniques do you use to find the solution to your problem?
I started asking questions in Stack Overflow.
What other techniques or methods do you follow, to fix the problem more quickly?
Go and do something else. No, really. I've found that putting the problem away in the back of my mind helps. I can't count the number of times I thought of a great solution to something I've been working on when I was working on something else, or watching TV, or eating. It seems your brain is still working on the problem in the background.
If that fails to solve your problem, try talking to someone. You'd be surprised how often others can give solutions to your problem that are so simple you'd facepalm.
Well there's:
Google
Google
Google
Stack Overflow
Google
Google
Maybe a book if I have one.
Seriously, I started (hobby) programming in the 1980s and even into the mid 90s you had to know things and have a technical library. Then Google came along and it's easier to Google something than it is to look up (bookmarked!) API documentation (Google "java stringbuilder" will get me there faster than navigating will) let alone an actual book (electronic or paper).
Most problems you're trying to solve have been solved before. Many times.
The rest of debugging comes down to decomposition, possibly unit testing (which is related to decomposition) and verifying your assumptions.
By "decomposition", I mean that your solution is structured in such a way that small pieces can be individually tested and readily understood. If you have a 7000 line method you're (probably) doing something wrong.
Understanding what assumptions you've made is key too so you can verify them. For example, when I started with PHP I wrote a piece of code like this:
$fields = $_SESSION["fields"]; // $fields is an associative array
$fields["blah"] = "foo";
and I was scratching my head trying to figure out why it didn't work (the array wasn't being updated next time I queried $_SESSION). I came from a Java background where you might do this:
Map fields = (Map)httpSession.get("fields");
fields.put("blah", "foo");
and that would most definitely work. PHP however copies the array. A working solution is to use references:
$fields =& $_SESSION["fields"]; // $fields is an associative array
$fields["blah"] = "foo";
or simply:
$_SESSION["fields"]["blah"] = "foo";
The last thing I'll say about debugging and writing robust code in general is to understand the boundaries of your solution. By this I mean if you're implementing a linked list then the boundary conditions will revolve around when the list is empty.
Explain the problem to a colleague, or write it down to describe it. That will makes you think a different way, from a different perspective. In order to be more accurate, and to describe the context of the problem, you'll step back, get a higher level view of the problem, you may find out think you overlooked something that is actually important.
Sometimes, you even find the explanation before ending your description.
My best friend for many years has been to jump on my bike and go home. Just getting away from the keyboard has solved many problems over the years for me.
If the problem lasts to the end of the day, I try and consciously lock the problem away for solving before I go to sleep.
I realise this sounds a bit out there, but it has been very common in my experience that I'll wake up with at least an alternate approach to the problem, if not the full-on solution. The idea is not to stress about it - but actively decide to solve it over night. That way you can go to sleep without worry.
I think eating well, regular exercise and good sleep are huge contributors to the problem-solving process.
Usually I'll try nut out the problem for a few hours or so, trying different things writing it on paper, making diagrams. If none of that works I'll usually work through the following options.
Put a sticky note on my monitor and keep going with something else
Glance at the note through out the next few hours to keep the problem in the back of my mind
Google for similar problems and the methods used
Consult a co-worker or a friend
Ask on a forum such as stackoverflow
Give up and design the problem away or design a way around the problem so it can be dealt with some other time and stick a TODO note at the site of said workaround
Don't forget Google Code Search
It's often best to clear your head by doing something other than programming for a little while. See this answer for an example - I did it while struggling with a particularly thorny bug, and when I came back to the problem I solved it in about a minute.
Usually I try to solve it until I go to sleep.. Sometimes I write on paper what the code is doing and then I divide it in pieces; I try to know how the variables of the program change when it's running.
Try solving a much smaller version of the problem first and see how you get on with that.
Once you've done that the bigger problem won't look so scary.
Ask yourself: is solving this particular tricky problem really important to what you doing?
For the purposes of your application (or whatever the big picture is) is there a similar but easier problem that you could address to accomplish broadly the same thing.
Normally, I would get pen and paper and try to work out the details of the problem there. If that doesn't help, Google. Failing that, I'd do something else for a while, or ask online. Worked for me so far.
The fact you are stuck might be a 'code-smell'. Suggesting that their is something wrong with the design or approach somewhere else. Try to put your finger on what's causing this and fix this instead.
When you come back to your problem it might no longer exist.
One more time, browse thru what I think might be relevant, then take nap.
There are two other answers which mention sleeping or napping, but this deserves more emphasis. It is now known that there's SERIOUS machinery in there which goes to work when you sleep. Google (( CBS SCIENCE SLEEP )) will get you to a great free video.
If I can't figure out how to solve the real problem, I try to consider a simplified version of the problem. To take a simple example: I recently had the problem of finding a set of shipping routes to get an item from point A to point B, when there is not necessarily a direct route from A to B, but there might be an A to C and then C to B, or A to C, C to D, and then D to B. (I'm sure the airlines and railroads do this all the time.) That was fairly complex, so I tried looking first at the simple case: a direct A to B. That was easy. Then consider how I'd handle it with one stop along the way. Then consider two stops. At that point I was able to see the pattern to a solution.
Solutions to a simplified version of the problem may end up being a part of the bigger solution, with some additional complexity wrapped around them. But even if not, the exercise of solving the easier problem often gives you ideas on how to solve the real problem.
The main techniques I use (should be followed in order so that you can reuse what you have done in previous steps to be more efficient):
Define your issue: Try to clearly define what's the problem, and what's expected. See 2 to help you.
Collect data about the bug: Log everything: your attempts, the expected result, the observed result. This will avoid the need to redo several times the same tests (because your mind cannot memorize it all), and probably help you see the bigger picture.
Reduce your problem. This is true in general for any abstract modelling of natural phenomenons, but it's even more true of programming, because programs are very complex entities. You should try to reduce your code to a minimal program that reproduces your issue. Automated tools exist.
Talk to someone: several anecdotes affirm than about 2/3 of the bugs are resolved just by talking about it. See the Helpful Teddy Bear anecdote. If you have previously reduced your program to a minimal program, and have a clear definition of your issue, both will be useful to your explanation.
Reach for collaborative help: search on Google and on StackOverflow, and post if you can't find anything that answers your problem (but first see 1, you must have a clear definition of your problem).
As you can see, I put the collaborative help as the last step, because you should only ask for help after you have clearly defined your issue and tried to reduce the problem and fix it by yourself. If you reach for collaborative help before having tried the previous steps, you will either end up with a poor help or figure it out by yourself as soon as you have posted.
Also you can be interested in the Coursera Software Debugging course, which also describes several automated debugging methods.
Go to the toilet.
You move, so your brain gets oxygen.
You relax, so you focus on other things.
Peeing for innovation! :)