How to solve problems with Binary Indexed Tree? - algorithm

This question sounds very vague and needs some explanation:
I learned about Binary Indexed Tree a few weeks ago. This data structure is a brilliant design. It actually took me very long to figure out how it's built thanks to this video (I mean... this is the first time I couldn't understand a written documentation and have to watch someone drawing a BIT step by step..)
Anyway, so (I think) I know how to build a BIT and the basic idea behind the structure design.. And now, I'm excited to practise some problems that can be easily solved with BIT.. In fact, someone has gathered a list of nice problems in this Quora post. I also tried some on HackerRank.
I spent long time trying and only managed to solve two (one by myself, the other taken from other's solution).. For instance, this direct connections problem..
I realise that the problem is never about how to build a BIT. The real challenge is to conceptualise the problem and use BIT to solve it... this is really beyond my imagination.. Is there a technique that I can use to tackle such problems?
And the interesting observation is.. for each problem set, the discussion below contains some comments like:
"BIT... :)"
It's like whoever managed to solve the problem always ended up with a proud smiley face without further explanation :(
Also, are there some classical problems that are solved by BIT?
EDIT
For those who voted to close this question: please give a valid reason. I believe this question is worth a discussion here!

Related

Problem solving/ Algorithm Skill is a knack or can be developed with practice? [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
Every time I start a hard problem and if can not figure out the exact solution or can not get started, I get into this never ending discussion with myself, as below:
That problem
solving/mathematics/algorithms skills
are gifted (not that you can learn
by practicing, by practice, you only
master the kind of problems that you
already have solved before)
only those who went to good schools can do it, as they learned it early.
What are your thoughts, can one achieve awesomeness in problem solving/algorithms just by hard work or you need to have that extra-gene in you?
I spent a big part of my life wondering whether talent was something you developed or something you were born with. Then it occurred to me that the answer was irrelevant, at least if you want to achieve things yourself. Even if you have talent, it will only help you if you act as if talent only comes from practice, because you will work that much harder.
With regards to algorithms, as well as any other really difficult skill, it takes practice to get good. Whether or not you have to have some amount of talent too, I don't know. I do know for a fact, however, that people have made huge improvements in competitions like TopCoder by practicing. I myself have learned a lot from that.
If you set up a systematic training program, you will be way ahead of the pack, even if it is not perfect. I have written a few hundred programs on TopCoder by now and it has affected my thinking in a profound way. I have learned a lot of things that could only ever be learned by doing them wrong and then fixing my mistake. A friend of mine has written several thousand programs on TopCoder and he is way better than I am, even though his stats were worse when he started out than mine were. That is no coincidence.
EDIT:
I just came across this answer at math.stackexchange. I think it is one of the best explanations of how to learn algorithms I have read, even though he writes about chess and math.
1) Don't try to solve the problem in its most general abstraction.
2) Choose the right time when your mind is working at maximum.
I got the first point as an advice from a math instructor. It works! try to do different examples and scenarios of the problem. This helps greatly in identifying the edge cases which are the hardest to understand in most problems.
My favorite time for solving this kind of problems is the dawn(4-6 AM). Have a good sleep the night before, and wakeup ready to solve the problem. Silence is your friend.
I do believe that some people have extra intelligence than others, but it is not the most important factor. It is how you utilize this intelligence to solve the problem.
I took magic lessons in a group setting when I was twelve years old. The magician's name was Joe Carota. He did a magic trick one time and I blurted out, "How did you do that?" He said something that day that has stuck with me ever since.
Joe's response, "Michael, if you really want to know how that trick is done you must figure out how you would do it yourself."
Well of course that's not what I wanted to hear but it did get my mind focused on problem solving. This was problem solving from my perspective. If my first attempt at solving the problem took seventeen steps and was really klunky, the good news was I solved the problem.
Then by looking at the solution I had developed and further looking for ways to refine that solution I would learn how to streamline the end result. Later on in my computer programming life I found out that this process was called "Stepwise Refinement".
It worked back in 1971 and it still works today.
For me, i think it's a bit talent, but much more important is experience and practice. If you know many problems and the best solutions to them, you can come up more easily with a solution to a new problem.
Example from my own past: There was some programming contest (good for training, btw) and I did not find a good solution. The winner solved the problem mainly by using a KD-Tree. To come up with this, you first of all need to know what, in this case, a KD-Tree is, and where it's useful. Today, this is clear to me and if i'd encounter a similar problem again, i'd be able to solve it really quickly.
Hardwork beats talent if talent doesn't work hard.
This above statement defines what the true potential of persistence is.Any skill in this world can be developed by practice.This process is analogous to nailing a nail in the wall.It not only requires correct magnitude but also appropriate direction.
To answer the question, first we need to find the ingredients for the capability to solve an issue.
There is a so-called natural talent. This is the talent you are born with. This predetermines your potential. People born with more gray matter will tend to perform better than people with whom nature was less generous with. This means that a person having better talent has a higher probability to perform better than a person not as talented if they had the same parameters (education, personality, resistance to stress, willpower). If one observes that he or she tends to consume a great time to absorb new information until he or she is able to apply it, then the wisest decision for the person is to leave programming and prevent a life full of frustration. Naturally, one cannot expect as a beginner to be able to instantly understand the most complex phenomenon, but if a beginner is too slow to understand beginner concepts, then programming is not his or her cup of tea.
Developed talent. One has a natural talent, but that is, in itself not enough to solve problems. I have never seen newborns writing code. One has to get some education. The earlier, the better. Also, the quality of school is of high importance. We should never deny the fact that a person who did not have the chance to learn programming at a good school early, then he or she has a handicap in the race for success. However, if someone misses good schools early, then the handicap can be covered with hard work. For instance, my wife had an education in another field, but after finishing the university, she did not find proper jobs. So I started to educate her. After a month she learned how to learn and was able to solve almost any problems presented to her, but she was not yet effective. She gradually became to start learning in auto-didacting manner. After a year she was already a professional coder. She does not have a paper from a school that she can code, but she is doing a fantastic job. So, she missed early education, but was later able to neutralize the handicap. Developed talent can be described as the set of information learned and known, along with the right attitude, the scientific approach to new types of challenges.
Practice: Practice is good to increase the level of developed talent, yet, it SHOULD not be the sole source of developing talent. Along with practice, the theoretical horizons must be regularly expanded.
Working strategy: One can be extremely talented, can have a lot of knowledge. If he or she does not have a right working strategy, then he or she has a handicap. Whenever a new task is given, the right questions should be asked:
what was the closest task to this one? Can I reuse my solution to an extent?
what should I learn to be able to solve this problem?
how can I write clear and efficient code to solve the problem?
So the answer is: while it is good to have excellent education as early as possible, it is not necessary. Do not forget, that life is the best school and you can recuperate the lost opportunity later if you have talent, willpower and source of information. Practice is not only showing you the right steps to solve a problem, it also widens your horizons. For instance, if one understands number systems, then he or she will be able to understand a variety of things later, like colors in CSS, PSD, or number overflows. If one learns how to code in Java, then he or she will understand C# very quickly. So, practice is giving you knowledge about the solution to a given problem type, but also, gives new theoretical knowledge which will be useful in various areas. The core skill one has to develop is the ability to learn quickly.
There have been many examples of people having extraordinary talent with minimum success. You see such examples in sports,politics,business and also in general around you. So, I feel after a certain limit, talent is a meaningless virtue. Its mostly the hard word that rewards you with greater success. If you follow cricket, here is a link with good example.
I feel same principle applies to algorithm and problem solving. An year back I use to pick up algorithmic problems to solve and used to find myself completely lost. An year invested in reading algorithmic books, solving its exercises and also practicing some more programming problems, I am confident that now I can solve most problems ( I still have a long way to go in making myself efficient in it). But the point is smart work is enough to develop this knack of solving problems.
Talent is cheap and useless without hardwork. Talent can only take you to some extent, but with hardwork and practice anybody can reach great heights
- Josh Waitzkin, 8-time National Chess Champion, a 13-Time National and 2-time World Champion
He himself says this in his voice over in Chessmaster Grandmaster Edition

What to do when faced with a seemingly unsolvable situation with a time limit?

I am a computer sciences student and I usually have really tough programming assignments. I don't know if it is only happening to me but sometimes, particularly when deadline is approaching, I find myself in a harsh situation.
I cannot find my mistake in the code or come up with a another great idea. Then boredom comes in and the problem begins to seem unsolvable.
I would like to learn their ideas to cope with this situation. Is it better to focus on something else for a while? Or try again? Or try harder and harder and look for the solution on the net, etc?
I alway like talking about the solution with another programmer. Just talking makes me use a different part of my brain and most of the time I hear myself talk through a solution.
Sleep is good, or if not sleep then at least taking a break, going for a walk in the fresh air etc.
Brainstorming the problem with colleagues / fellow students can help. Even just explaining the issue to someone else can be enough to make the solution click in your brain.
Failing all the above, ask on Stackoverflow :-)
Try breaking down the problem into smaller, easier problems, and solve those. Don't try and tackle everything at once, and avoid trying to hack your way through.
If you're still stuck, taking a break can be good. Sometimes the answer is suddenly obvious when looking through a refreshed pair of eyes. Solutions to problems often come to me in my sleep, and I'll wake up knowing the answer.
For me, I encountered a couple of times where I took quite a bit of time (10 to 30 mins) to define the problem in writing as to submit the question on SO, and got ideas that led to the eventual solution while typing out the question.
I find that when I document your problem in a way that others can understand without having to understand the unrelated parts of the your entire application/project, I consciously break down the problem into isolated, independent parts which helps me or another developer analyze and decide the next course of action.
Just my two cents :)
In your case (school work) I would probably seek out the instructor/professor or TA. While they will certainly not "give" you the answer at the very least you might learn something else in the process.
Specifically I would explain to the the difficulty you are having, what you have done to try to solve it and any other things to show that you did work.
A lot of times while walking though this on your own you might come up with solutions. They can probably give you hints or suggestions as well.
Worst case scenario is that they tell you to go away and to leave them alone.
Others have posted sleep (#sjobe, &Vicky) and asking someone is good (#Christopher Altman). BTW, that is often referred to as "rubber-ducking".
My personal problem is wanting to see something through and getting consumed in getting to the finish, almost always to my own determent. What I've learned over the years if a little research doesn't help (< 30 minutes) and talking it through doesn't explain it and you can't or don't want to sleep on it, do something for the mind, body and spirit: Go outside!
Seriously, go for a 30-45 minute bike-ride, run, walk, swim, whatever. Try to think of something else. Tell yourself a story or mentally work on another problem if you must. Cool down and return. You'll be amazed at how refreshed you'll feel. The endorphins will help.
If you're embarking on career driving a desk, it's a great habit to get into as well.
-Cheers
The entire art of surviving or rather conquering in situations like this is about staying to have a solution oriented approach . By this I mean staying positive that even if a solution is not working it , have faith that your tries are getting you closer to it .
Yes I strongly agree that taking a break is an integral step of reaching your goals but take a break to return back with stronger spirits to solve a problem .
Involve yourself into different solution finding strategies along with a spirit to enjoy the same .
The solution finding strategies may involve :
Talking to your friends who dont understand the problem to a good level and help them understand . It will help you explore the ins and outs of the scenarios . Sometimes explaining other people helps us understand the problem in a much better scenarios .
Sit down with a paper and a pen or its better to have a diary where jot down all the ideas as they strike you . Take your diary with you always as it helps to jot down the ideas otherwise later we forget . Also sometimes the game is about connecting the dots . An idea from morning first half and evening time can be perfect mix to resolve the problem .
Go out on a brainstorming session with couple of friends and entertain all the ideas that they put on table for once and consider them . Remember no idea is an stupid idea . Either it is solution or a contributing step towards a solution .
There may be times when you need to visit an industry expert or a researcher to dig deeper into concepts of technology . Before you visit an industry expert keep all your research documents and brainstorming ideas collected . Share it with the researcher correctly . Also have a SWOT analysis of the person you are trying to meet so that you get to understand in which part is the person strong and can help you . Also carry a recorder with you such meetings because jotting down everything becomes difficult .
Dont believe in what ever is always suggested make sure to come home and do entire research on internet on whatever is shared . That will help you increase your knowledge .
Do some experiments . Some hits and tries randomly and based on results reach to conclusions .
Each of these steps plays a very important role in brainstorming and reaching to a solution . Looking forward to hear from you what were your experiences trying these out .
A similar question has already been asked here https://stackoverflow.com/questions/427532/what-do-you-do-when-youre-stuck.
Sleep is my personal favorite though, although if you're like most college students, you're probably doing a lot of last minute coding and you don't have enough time to sleep and submit your work on time [I was guilty of this too].
What I like to do when I'm stuck on a problem, I usually try to draw out my problems. I just get myself a piece of paper and write down the problems that I encouter. While doing this I like to make Class diagrams/Sequence Diagrams, just to clearify the situation in. Really helps to just get back to old skool pen and paper and not look at your screen for a while.
As a student I also face this problem from time to time. What helps me quite often is to get away from the computer, take a pencil and some paper and start to write down the code by hand. I don't know why but often it is easier for me to solve it on paper than by using an IDE/editor. Probably because your brain works differently then.

How does a programmer think? [closed]

As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
This may be a hopelessly vague question. But I am interested to hear whatever logical thought processes people go through when learning a new concept or trying to get their brain around code they might not have ever seen before.
Basically, what general steps does one take to to break down problems and what does it take to "get it"? If you were to diagram a flowchart of how your mental process works when you look at code or try to solve a problem what might it look like?
What common references, tips, and mental assumptions do you find useful in problem solving?
How is this different between different domains? For example in what ways is a web programmer's thought process similar or different from a traditional desktop app developer's process?
I'm a big believer that no matter what type of application you're looking at for the first time, may it be a web app, a desktop app, a device driver, or whatever else, there are three steps one developer usually follows in order to understand how it works:
Get the big picture :
What kind of app is this (web, desktop, ...)?
How is it layered (standalone, client-server, n-tier, ...)?
What is the app's purpose? What is it supposed to do?
Who is the app made for?
See how it works :
What language(s) is (are) used?
How is the code structured?
How is the data structured?
Understand (or at least try to) the way the app has been thought through:
Has it been thought through at all?
Is the app clearly optimized? (For performances? For readability?)
Is the app finished? Or is there room for evolutions?
Are there signs of multiple releases?
etc...
The 1st and 2nd steps are purely technical, while the 3rd MUST be as untechnical as possible... it's more about psychology and understanding how the app has been built. It obviously requires experience, but as long as you think hard enough and don't waste your brain's time with technical details, you'll eventually get it.
This whole process shouldn't require the use of a keyboard. You're only supposed to read, think, and take notes on a paper (I'm not kidding: pen and paper!).
Ho ho, good luck with this one. It's a great question and I'm sure you'll get a ton of answers. Although I have to say I cannot give a satisfactory answer to this - the last thing I would describe my thought processes as is a flow chart - I don't think there is any golden formula for this.
The only tip in problem solving I can recommend is discussing it with somebody else. In those times when you hit a brick wall, going through it with a colleague is invaluable. Quite often, as well, they will actually not even add much to the discussion - in the process of getting all your thoughts out in the open, the solution can become clear.
People are notoriously bad at examining their own thought processes, but I'll give it a whirl. I test very high for visuo-spacial ability in IQ tests, medium-to-high for verbal skills, and moderate for mathematical skills (explains my A-level Maths grade, I suppose). amd when I start to design software, I think in terms of shapes and the connections between them. When it comes to describing these thoughts to others (or clarifying them for myself), I use simple block diagrams or the object diagrams taken from Jacobson's Objectory method - NOT the over complex stuff that UML suggests. I sometimes write textual descriptions of complex things, mostly as reminders to myself, but never use numbers or maths.
Of course this is just me - I've worked with maths whizzes who were just as good or even better programmers than myself.
I don't think... I process.
This is actually less flip than it sounds. I always break down tasks into their components and then break these down further, and that doesn't just go for writing software! Much like #Mark Pim U go through things sequentially.
My wife gets really annoyed when I make dinner because I take so long to get started.
Divide & Conquer
I start by trying grasp the entire problem as it is, and then start to find patterns I can recognize, and do the same for them in a kind of recursive process, until I have a broken down solution I can implement and follow more easily.
This is one of the rare times I would answer with "it just works." I learn things by steamrolling through them. I don't have gimmicks, or devices to help me. Took me some time to learn PHP, but after that Javascript was much easier. Once you tackle one thing, the next items become cumulatively-easier.
Personally, I conduct an internal dialogue with myself 'OK so we need to loop over this list of integers.' 'But we can break when we find the value we want.' 'OK, will the list definitely be initialised when we start?'
I'd be interested to see if any psychological research had been done on problem solving techniques.
Similar to Jonathan Sampson - it kind of just works.
When I'm attacking a real problem, I try and think of the most logical way of getting through it is.
Then, when everything goes wrong (as it usually does), I have to make hundreds of sidesteps to get things done. Just keep focusing on that end goal, that logical way and you'll get there.
Eventually though, it decides to work for me and I end up with a finished product that is usually nothing like I planned it out to be. As long as the customers are happy, I am!
Personally, I see code in my head pictorally rather than textually (like Neil Butterworth) - it's a bit hard to describe since (quoting STIV) "there's no common frame of reference."
My main skill is identifying similarities between models or systems I already know about and the task at hand. The connections between some of these can seem quite abstract; the key is to spot the connections. This leads to the abstraction of common patterns and approaches which are widely applicable. Related to this, the most important thing I learnt about algorithms was that the problem is never 'come up with a smart algorithm to solve X'. It's 'model problem X such that it can be solved by existing smart algorithm Y'.

debugger's block [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
Do you ever run into a bug where you are just out of ideas and don't know what to try next, and you're getting really annoyed? Are there any general ideas on how to break out of this mode?
When I get blocked like that, the best advice I've found is leave for a while. Whether it's going for a long lunch or leave for the day. Come back fresh and take a new look. Most of the time the answer will be staring you in the face. So far a 10 minute coffee break hasn't been enough for me, but your mileage may vary.
Second, talk to your peers. Walk through everything you've tried. Just ask them to listen, and while you're talking it through, a lot of times, the answer will come to you.
Those are just the two that I use most often when I'm debugging blocked.
Five minute break
Utterly critical, lest you smash your keyboard.
Explain the problem in an email to your sister
Or your two year old son. Someone who wouldn't understand a word* unless you explained it very clearly. You don't need to send the email, you just need to be certain that you understand it inside out. You be surprised at how many times simply restating the problem suddenly makes it obvious to you where you went wrong. This is a good way of discovering what your assumptions were about the problem, and how they might not necessarily be correct.
*That link is to an excellent answer by JaredPar on a different SO question.
Talk to a colleague
By this point you've already 'emailed' the family pet, so you know that you've hopefully covered all the stupid aspects, it's time to talk to a real person. They may have experienced some obscure thing in the past that reminds them of this situation, and they'll definitly have a different perspective. Try to make sure you are clear about what the problem is, not what you think it is. You don't want to bias them. You can and should talk about what they've tried, and explain why you think the problem is in that area (you're probably right) but you don't want to close your mind to their suggestions, even if they don't seem right.
Look at the code again
By this time, you should hopefully be less violent, and you should be armed with new ideas. Start by re-verifying the bug. A simple step, but I've been frustrated for hours on a bug that was in our test script and not in our code. Once you've verified the bug again, start at the top. Verify EVERYTHING. Put a breakpoint at the last point you are 100% confident in, and then work your way forward until you find that the output is broken again. That's a huge success because you now have a hopefully smaller code block to investigate. Then, if necessary, pull in a colleague to look at the actual running code.
I usually take a break... if that doesn't work, I sleep on the problem. (Actually, to be honest, I should say that I spend a sleepless night mulling over the problem).
I almost always have a list of possible solutions ready by morning. :-)
I usually take a few steps:
look at the code that is involved in the functionality that has the bug and place logger messages in a way that they will help me narrow the problem (this allows you to look at the logger later on when you are not that annoyed anymore, and maybe find useful hints :P)
ask my senior collegues for hints
call the client to have a clear understanding of when the problem arises
I usually end up in that mood when I didn't take a structural approach from the beginning.
By iteratively narrowing the area where the problem can live and trying to write the smallest code base that can reproduce the error, I haven't failed yet.
Others have mentioned taking a break or "sleeping on it". This technique is known as (amongst other things) Incubation. Once you embrace the idea you can take it further too.
One important aspect as to really leave the problem behind, confident that you'll have answers later. The dangerous otherwise, especially if sleeping on it, is that you'll be worrying over it up to the last minute, then this becomes an endless cycle that your mind gets caught in overnight. You'll probably end up waking up not too well rested having dreamt many circular dreams. Do this too much and it's a path to depression.
I think if you run into that kind of problem it is time to do some serious refactoring...
Also it could indicate false assumptions. Try to assume programmatically all that you are 'just' assuming. See to it no method-invariants are broken of any methods that are in the stack.
Another good option is to bounce ideas off other developers/team members. Sometimes they will ask you questions you hadn't considered.
If you work alone, it's a good idea to have a few fellow developers who you can chat to to troubleshoot with some different approaches. You'd be amazed how often a fresh perspective can be the breakthrough you need.
Try David Ungars "Shower Methodology":
"If you know what to type, type. If you don't know what to type, take a shower and stay in the shower until you know what to type"
This kind of summons up what the majority likes to do. Take a break ! Doesn't matter what kind of break it is, as long as you are not thinking of the things you were doing before the break. Distraction in any form is good.
Cheers !
I usually take a dry run through the offending method/function, step by step. And don't assume the bug is coming from there, it could be coming from anywhere before it.
Use a proper debugger, don't use a series of printfs.

Techniques to follow when you are stuck programming [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 7 years ago.
Improve this question
When I am stuck with a problem:
I search Google for code snippets.
I look at isolating the problem, so that I can better explain it to others in order to get answers.
What search techniques do you use to find the solution to your problem?
I started asking questions in Stack Overflow.
What other techniques or methods do you follow, to fix the problem more quickly?
Go and do something else. No, really. I've found that putting the problem away in the back of my mind helps. I can't count the number of times I thought of a great solution to something I've been working on when I was working on something else, or watching TV, or eating. It seems your brain is still working on the problem in the background.
If that fails to solve your problem, try talking to someone. You'd be surprised how often others can give solutions to your problem that are so simple you'd facepalm.
Well there's:
Google
Google
Google
Stack Overflow
Google
Google
Maybe a book if I have one.
Seriously, I started (hobby) programming in the 1980s and even into the mid 90s you had to know things and have a technical library. Then Google came along and it's easier to Google something than it is to look up (bookmarked!) API documentation (Google "java stringbuilder" will get me there faster than navigating will) let alone an actual book (electronic or paper).
Most problems you're trying to solve have been solved before. Many times.
The rest of debugging comes down to decomposition, possibly unit testing (which is related to decomposition) and verifying your assumptions.
By "decomposition", I mean that your solution is structured in such a way that small pieces can be individually tested and readily understood. If you have a 7000 line method you're (probably) doing something wrong.
Understanding what assumptions you've made is key too so you can verify them. For example, when I started with PHP I wrote a piece of code like this:
$fields = $_SESSION["fields"]; // $fields is an associative array
$fields["blah"] = "foo";
and I was scratching my head trying to figure out why it didn't work (the array wasn't being updated next time I queried $_SESSION). I came from a Java background where you might do this:
Map fields = (Map)httpSession.get("fields");
fields.put("blah", "foo");
and that would most definitely work. PHP however copies the array. A working solution is to use references:
$fields =& $_SESSION["fields"]; // $fields is an associative array
$fields["blah"] = "foo";
or simply:
$_SESSION["fields"]["blah"] = "foo";
The last thing I'll say about debugging and writing robust code in general is to understand the boundaries of your solution. By this I mean if you're implementing a linked list then the boundary conditions will revolve around when the list is empty.
Explain the problem to a colleague, or write it down to describe it. That will makes you think a different way, from a different perspective. In order to be more accurate, and to describe the context of the problem, you'll step back, get a higher level view of the problem, you may find out think you overlooked something that is actually important.
Sometimes, you even find the explanation before ending your description.
My best friend for many years has been to jump on my bike and go home. Just getting away from the keyboard has solved many problems over the years for me.
If the problem lasts to the end of the day, I try and consciously lock the problem away for solving before I go to sleep.
I realise this sounds a bit out there, but it has been very common in my experience that I'll wake up with at least an alternate approach to the problem, if not the full-on solution. The idea is not to stress about it - but actively decide to solve it over night. That way you can go to sleep without worry.
I think eating well, regular exercise and good sleep are huge contributors to the problem-solving process.
Usually I'll try nut out the problem for a few hours or so, trying different things writing it on paper, making diagrams. If none of that works I'll usually work through the following options.
Put a sticky note on my monitor and keep going with something else
Glance at the note through out the next few hours to keep the problem in the back of my mind
Google for similar problems and the methods used
Consult a co-worker or a friend
Ask on a forum such as stackoverflow
Give up and design the problem away or design a way around the problem so it can be dealt with some other time and stick a TODO note at the site of said workaround
Don't forget Google Code Search
It's often best to clear your head by doing something other than programming for a little while. See this answer for an example - I did it while struggling with a particularly thorny bug, and when I came back to the problem I solved it in about a minute.
Usually I try to solve it until I go to sleep.. Sometimes I write on paper what the code is doing and then I divide it in pieces; I try to know how the variables of the program change when it's running.
Try solving a much smaller version of the problem first and see how you get on with that.
Once you've done that the bigger problem won't look so scary.
Ask yourself: is solving this particular tricky problem really important to what you doing?
For the purposes of your application (or whatever the big picture is) is there a similar but easier problem that you could address to accomplish broadly the same thing.
Normally, I would get pen and paper and try to work out the details of the problem there. If that doesn't help, Google. Failing that, I'd do something else for a while, or ask online. Worked for me so far.
The fact you are stuck might be a 'code-smell'. Suggesting that their is something wrong with the design or approach somewhere else. Try to put your finger on what's causing this and fix this instead.
When you come back to your problem it might no longer exist.
One more time, browse thru what I think might be relevant, then take nap.
There are two other answers which mention sleeping or napping, but this deserves more emphasis. It is now known that there's SERIOUS machinery in there which goes to work when you sleep. Google (( CBS SCIENCE SLEEP )) will get you to a great free video.
If I can't figure out how to solve the real problem, I try to consider a simplified version of the problem. To take a simple example: I recently had the problem of finding a set of shipping routes to get an item from point A to point B, when there is not necessarily a direct route from A to B, but there might be an A to C and then C to B, or A to C, C to D, and then D to B. (I'm sure the airlines and railroads do this all the time.) That was fairly complex, so I tried looking first at the simple case: a direct A to B. That was easy. Then consider how I'd handle it with one stop along the way. Then consider two stops. At that point I was able to see the pattern to a solution.
Solutions to a simplified version of the problem may end up being a part of the bigger solution, with some additional complexity wrapped around them. But even if not, the exercise of solving the easier problem often gives you ideas on how to solve the real problem.
The main techniques I use (should be followed in order so that you can reuse what you have done in previous steps to be more efficient):
Define your issue: Try to clearly define what's the problem, and what's expected. See 2 to help you.
Collect data about the bug: Log everything: your attempts, the expected result, the observed result. This will avoid the need to redo several times the same tests (because your mind cannot memorize it all), and probably help you see the bigger picture.
Reduce your problem. This is true in general for any abstract modelling of natural phenomenons, but it's even more true of programming, because programs are very complex entities. You should try to reduce your code to a minimal program that reproduces your issue. Automated tools exist.
Talk to someone: several anecdotes affirm than about 2/3 of the bugs are resolved just by talking about it. See the Helpful Teddy Bear anecdote. If you have previously reduced your program to a minimal program, and have a clear definition of your issue, both will be useful to your explanation.
Reach for collaborative help: search on Google and on StackOverflow, and post if you can't find anything that answers your problem (but first see 1, you must have a clear definition of your problem).
As you can see, I put the collaborative help as the last step, because you should only ask for help after you have clearly defined your issue and tried to reduce the problem and fix it by yourself. If you reach for collaborative help before having tried the previous steps, you will either end up with a poor help or figure it out by yourself as soon as you have posted.
Also you can be interested in the Coursera Software Debugging course, which also describes several automated debugging methods.
Go to the toilet.
You move, so your brain gets oxygen.
You relax, so you focus on other things.
Peeing for innovation! :)

Resources