I'm reading the chapter 2 and 3 of CLRS, and get stuck so often, especially in the problems provided at the end of each chapter, that I wonder if it'll ever be worthwhile for this much effort. I can't understand the solution online like this one: http://clrs.skanev.com/02/problems/01.html
I heard that this book is one of the most popular text books for university CS class, but do people skip intricate parts and just memorize important things, like insertion sort has this order of growth and merge sort has that order of growth, and go ahead?
Isn't it just enough to be familiar with many useful algorithms to have about as much understanding of computer science as people with a degree in CS do in general?
Understanding isn't about memorization. It's about being able to apply the knowledge to solve problems. The textbook problems are quite simple compared to most real-life problems. So, skipping these simply means you're not learning at all, and you certainly won't be able to apply any of it in real life. You're memorizing, but you can't use what you've memorized.
TL;DR: The proof of being able to use the knowledge is the ability to solve problems, and textbook problems are simple.‡ One doesn't go without the other.
‡ Knuth's texts are a notable exception: he also offers some borderline intractable problems, and everything in between :)
The point is that "people with a degree in CS ... in general" can work out the order of growth of an algorithm. That's why people go to the effort of learning this stuff. If you just want to be able to say "mergesort is O(n log n)", then indeed, all you need is to see and memorise that fact. If you want to be able to work out the O() of an algorithm, even when it's one you've never seen before - then you need these methods.
"Designing the right algorithm for a given application is a difficult job. It requires a major creative act, taking a problem and pulling a solution out of the ether. This is much more difficult than taking someone else's idea and modifying it or tweaking it to make it a little better. The space of choices you can make in algorithm design is enormous, enough to leave you plenty of freedom to hang yourself".
I have studied several basic design techniques of algorithms like Divide and Conquer, Dynamic Programming, greedy, backtracking etc.
But i always fail to recognize what principles to apply when i come across certain programming problems. I want to master the designing of algorithms.
So can any one suggest a best place to understand the principles of algorithm design in depth.....
I suggest Programming Pearls, 2nd edition, by Jon Bentley. He talks a lot about algorithm design techniques and provides examples of real world problems, how they were solved, and how different algorithms affected the runtime.
Throughout the book, you learn algorithm design techniques, program verification methods to ensure your algorithms are correct, and you also learn a little bit about data structures. It's a very good book and I recommend it to anyone who wants to master algorithms. Go read the reviews in amazon: http://www.amazon.com/Programming-Pearls-2nd-Edition-Bentley/dp/0201657880
You can have a look at some of the book's contents here: http://netlib.bell-labs.com/cm/cs/pearls/
Enjoy!
You can't learn algorithm design just from reading books. Certainly, books can help. Books like Programming Pearls as suggested in another answer are great because they give you problems to work. Each problem forces you to think about how to solve a particular type of problem.
The idea is that you expose yourself to many different types of problems and their solutions. In doing so, you learn how to examine a problem and see if it shares anything in common with problems you've already seen. In that regard, it's not a whole lot different than the way you learned how to solve "word problems" in math class. Granted, most algorithm problems are more complex than having to figure out where on the tracks the two trains will collide, but the way you learn how to solve the problems is the same. You learn common techniques used to solve simple problems, then combine those techniques to solve more complex problems, etc.
Read, practice, lather, rinse, repeat.
In addition to books like Programming Pearls, there are sites online that post different programming challenges that you can test yourself on. It helps if you have friends or co-workers who also are interested in algorithms, because you can bounce ideas off each other and pose interesting challenges, or work together to come up with solutions to problems.
Did I mention that it takes practice?
"Mastering" anything takes time. A long time. A popular theory is that it takes 10,000 hours of practice to become an expert at anything. There's some dispute about that for particular endeavors, but in general it's true. You don't master anything overnight. You have to study. And practice. And read what others have done. Study some more and practice some more.
A good book about algorithm design is Kleinbeg Tardos. Every design technique depends on the problem that you are going to tackle. It is very important to do the exercises in the algorithm books and have feedback from teachers about that.
If there exist a locally optimal choice taht brings the globally optimal solution you can use a greedy algorithm.
If the problem has optimal substructure, you can use dynamic programming.
at workplace, the work I do is hardly near to challenging and doing that I think I might be losing the skills to look at a completely new problem and think about different ideas to solve it.
A friend suggested TopCoder.com to me, but looking at the overwhelming number of problems I can not decide how to get started?
what I want is to sharpen my techniques ( not particular language or framework ).
The only way to get started would be to pick problems. Division I is the more difficult division, so you will probably find that the division I medium and hard problems will be somewhat interesting and challenging (unless you are quite clever.)
If you check the event calendar, you can see what algorithm competition rounds are coming up in your time zone. The competitions have the added virtue of forcing you to read and analyze other people's code in the challenge phase, so even if you would just as soon practice without a clock, you may find them interesting.
TopCoder algorithm contests are a way to develop your coding speed. Solving any of the problems in the practice arena is difficult unless you already have knowledge of various algorithms.
The problems on Project Euler suffer from the same flaw. You already have to know the algorithms to solve the problems in a reasonable time frame.
What I would suggest is to pick a project that you're interested in, and pursue it as you have time. As an example, I'm currently learning how to work with the open street map tiles in an Eclipse rich client platform.
Try whit http://projecteuler.net Problems difficulty can be assumed by number of solvers.
I prefer this page, because it is language invariant and problems are really challenging
You need the experience of solving 2 problems in any online judge (like http://www.spoj.com, http://www.lightoj.com, http://www.codeforces.com) in any programming language of your choice. That will give you an idea about how are your programs tested online.
Then you can follow this -> http://localboyfrommadurai.blogspot.in/2011/12/new-to-topcoder.html
I have a decent grasp of NP Complete problems; that's not the issue. What I don't have is a good sense of where they turn up in "real" programming. Some (like knapsack and traveling salesman) are obvious, but others don't seem obviously connected to "real" problems.
I've had the experience several times of struggling with a difficult problem only to realize it is a well known NP Complete problem that has been researched extensively. If I had recognized the connection more quickly I could have saved quite a bit of time researching existing solutions to my specific problem.
Are there any resources (online or print) that specifically connect NP Complete to real world instances?
Edit:
For example, I was working on a program that tried to divide students into groups based on age, grade, and school of origin, which is essentially a graph partitioning problem. It took me a while to realize the connection.
I have found that Computers and Intractability is the definitive reference on this topic.
Usually the connection you are talking about must be extracted with a so-called reduction, for example you reduce 3-SAT to the problem you are working with and then you can conclude that your problem has the same complexity of it.
This passage is not trivial, since you have to prove that you can turn every problem instance l of a known NP-Hard problem L into an instance c of your problem C using a deterministic polinomyal algorithms.
So, except from learning basical correlations of common NP-Hard problems using your memory, there's no way to be sure if a problem is similar to another NP-Hard without first trying to guessing and then proving it, you have to be smart.
here is a wiki link:
http://wapedia.mobi/en/List_of_NP-complete_problems
Notice it says
This list is in no way comprehensive (there are more than 3000 known NP-complete problems)
probably it would be a great task if anyone could compile such list.
A theorist should try to understand/proof an NP-Complete/Hard problem. But, a programmer doesn't have that time to. He needs a list.
Am I correct?
I think you should google it. And, read through all the links. Add any new problem found in the link to your list.
Hope it helps
PS : Don't forget to post the list when you're finished :P
For developing better intuition the book "The Algorithm Design Manual, Second Edition" by Skiena (excerpts on google books) is simply great.
List in the back with problems
(including hard problems), that
include an illustration and a
discussion (often) with a real world
example.
Covers both the theoretical
and practical side of things, often
talking about actual code.
Read excepts online here (see some examples in chapters 14):
http://books.google.dk/books?id=7XUSn0IKQEgC&printsec=frontcover#v=onepage&q&f=false
Chapter 16 (not online) discusses some hard problems, including graph partition.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I'm about to start (with fellow programmers) a programming & algorithms club in my high school. The language of choice is C++ - sorry about that, I can't change this. We can assume students have little to no experience in the aforementioned topics.
What do you think are the most basic concepts I should focus on?
I know that teaching something that's already obvious to me isn't an easy task. I realize that the very first meeting should be given an extreme attention - to not scare students away - hence I ask you.
Edit: I noticed that probably the main difference between programmers and beginners is "programmer's way of thinking" - I mean, conceptualizing problems as, you know, algorithms. I know it's just a matter of practice, but do you know any kind of exercises/concepts/things that could stimulate development in this area?
Make programming fun!
Possible things to talk about would be Programming Competitions that either your club could hold itself or it could enter in locally. I compete in programming competitions at the University (ACM) level and I know for a fact that they have them at lower levels as well.
Those kind of events can really draw out some competitive spirit and bring the club members closer.
Things don't always have to be about programming either. Perhaps suggest having a LAN party where you play games, discuss programming, etc could be a good idea as well.
In terms of actual topics to go over that are programming/algorithm related, I would suggest as a group attempting some of these programming problems in this programming competition primer "Programming Challenges": Amazon Link
They start out with fairly basic programming problems and slowly progress into problems that require various Data Structures like:
Stacks
Queues
Dictionaries
Trees
Etc
Most of the problems are given in C++.
Eventually they progress into more advanced problems involving Graph Traversal and popular Graph algorithms (Dijkstra's, etc) , Combinatrics problems, etc. Each problem is fun and given in small "story" like format. Be warned though, some of these are very hard!
Edit:
Pizza and Soda never hurts either when it comes to getting people to show up for your club meetings. Our ACM club has pizza every meeting (once a month). Even though most of us would still show up it is a nice ice breaker. Especially for new clubs or members.
Breaking it Down
To me, what's unique about programming is the need to break down tasks into small enough steps for the computer. This varies by language, but the fact that you may have to write a "for loop" just to count to 100 takes getting used to.
The "top-down" approach may help with this concept. You start by creating a master function for your program, like filterItemsByCriteria();
You have no idea how that will work, so you break it down into further steps:
(Note: I don't know C++, so this is just a generic example)
filterItemsByCritera() {
makeCriteriaList();
lookAtItems();
removeNonMatchingItems();
}
Then you break each of those down further. Pretty soon you can define all the small steps it takes to make your criteria list, etc. When all of the little functions work, the big one will work.
It's kind of like the game kids play where they keep asking "why?" after everything you say, except you have to keep asking "how?"
Linked lists - a classic interview question, and for good reason.
I would try to work with a C subset, and not try to start with the OO stuff. That can be introduced after they understand some of the basics.
Greetings!
I think you are getting WAY ahead of yourself in forcing a specific language and working on specific topics and a curriculum.. It sounds like you (and some of the responders) are confusing "advising a programming club" with "leading a programming class". They are very different things.
I would get the group together, and the group should decide what exactly they want to get out of the club. In essence, make a "charter" for the club. Then (and only then) can you make determinations such as preferred language/platform, how often to meet, what will happen at the meetings, etc.
It may turn out that the best approach is a "survey", where different languages/platforms are explored. Or it may turn out that the best approach is a "topical"one, where there topic changes (like a book club) on a regular basis (this month is pointers, next month is sorting, the following is recursion, etc.) and then examples and discussions occur in various languages.
As an aside, I would consider a "language-agnostic" orientation for the club. Encourage the kids to explore different languages and platforms.
Good luck, and great work!
Well, it's a programming club, so it should be FUN! So I would say dive into some hand on experience right away. Start with explaining what a main() method is,then have students write a hello world program. Gradually improve the hello world program so it has functions and prints out user inputs.
I would say don't go into algorithm too fast for beginners, let them play with C++ first.
Someone mentioned above, "make programming fun". It is interesting today that people don't learn for the sake of learning. Most people want instant gratification.
Teach a bit of logic using Programming. This helps with(and is) problem solving. The classing one I have in my head are guessing games.
Have them make a program that guesses at a number between 0 and 100.
Have them make a black jack clone ... I have done this in basic :-(
Make paper instructions.
Explain the "Fried eggs" story. Ask the auditory what they would do to make themselves fried eggs. Make them note the step they think about. Probably you will receive less than 5 steps algorithm. Then explain them how many steps should be written down if we want to teach a computer to fry eggs. Something like:
1) Go to the Fridge
2) Open the fridge door
3) Search for eggs
4) If there are no eggs - go to the shop to buy eggs ( this is another function ;) )
5) If there are eggs - calculate how many do you need to fry
6) Close the fridge door
7) e.t.c. :)
Start with basics of C - syntax semantics e.t.c, and in parallel with that explain the very basic algorithms like bubble sort.
After the auditory is familiar with structured programming (this could take several weeks or months, depending how often you make the lessons), you can advance to C++ and OOP.
The content in Deitel&Deitel's C++ programming is a decent introduction, and the exercises proposed at the end of each chapter are nice toy problems.
Basically, you're talking about:
- control structures
- functions
- arrays
- pointers and strings
You might want to follow up with an introduction to the STL ("ok, now that we've done it the hard way... here's a simpler option")
Start out by making them understand a problem like for instance sorting. This is very basic and they should be able to relate quite fast. Once they see the problem then present them with the tools/solution to solve it.
I remember how it felt when I first was show an example of merge-sort. I could follow all the steps but what the hell was I for? Make then crave a solution to a problem and they will understand the tool and solution much better.
start out with a simple "hello world" program. This introduces fundamentals such as variables, writing to a stream and program flow.
Then add complexity from there (linked lists, file io, getting user input, etc).
The reason I say start with hello world is because the kid will get to see a running program really quick. It's nearly immediate feedback-as they will have written a running program right from the start.
IMO, Big-O is one of the more important concepts for beginning programmers to learn.
Have a debugging contest. Provide code samples that include a bug. Have a contest to see who can find the most or fastest.
There is an excellent book, How Not to Program in C++, that you could use to start with.
You always learn best from mistakes and I prefer to learn from some else's.
It will also let those with little experience learn by see code, even if the code only almost works.
In addition to the answers to this question, there are certain important topics to cover. Here's an example of how you could structure the lessons.
First Lesson: Terminology and Syntax
Terminology to cover: variable, operator, loop (iteration), method, reserved word, data type, class
Syntax to cover: assignment, operation, if/then/else, for loop, while loop, select, input/output
Second Lesson: Basic Algorithm Construction
Cover a few simple algorithms, involving some input, maybe a for or a while loop.
Third Lesson: More Advanced Algorithm Topics
This is for things like recursion, matrix manipulation, and higher-level mathematics. You don't have to get into too complex of topics, but introduce enough complexity to be useful in a real project.
Final Lesson: Group Project
Make a project that groups can get involved in doing.
These don't have to be single day lessons. You can spread the topics across multiple days.
Pseudocode should be a very first.
Edit: If they are total programming beginners then I would make the first half just about programming. Once you get to a level where talking about algorithms would make sense then pseudocode is really important to get under the nails.
Thanks for your replies!
And how would you teach them actual problem solving?
I know a bunch of students that know C++ syntax and a few basic algorithms, but they can't apply the knowledge they know when they solve real problems - they don't know the approach, the way to transcribe their thoughts into a set of strict steps. I do not talk about 'high-level' approaches like dynamic programming, greedy etc., but about basic algorithmic mindset.
I assume it's just because of the poor learning process they were going through. In other sciences - math, for example - they are really brilliant.
Just because you are familiar with algorithms does not mean you can implement them and just because you can program does not mean you can implement an algorithm.
Start simple with each topic (keep programming separate from designing algorithms). Once they have a handle on each, slowly start to bring the two concepts together.
Wow. C++ is one of the worst possible languages to start with, in terms of the amount of unrelated crap you need to get anything working (Java would be slightly worse, I guess).
When teaching beginners in a boilerplate-heavy environment, it's usual to start with "here's a simple C program. We'll discuss what all this crap at the top of the file is for later, but for now, concentrate on the lines between 'int main(void)' and the 'return' statement, which is where all the useful work is accomplished".
Once you're past that point, basic concepts to cover include the basic data structures (arrays, linked lists, trees, and dictionaries), and the basic algorithms (sorting, searching, etc).
Have your club learn how to actually program in any language by teaching the concepts of building software. Instead of running out an buying a dozen licenses for Visual Studio, have students use compilers, make systems, source files, objects and librarys in order to turn their C code into programs. I feel this is truly the beginning and actually empowers these kids to understand how to make software on any platform, without crutches that many educational institutions like to rely on.
As for the language of choice - congratulations - you'll find C++ is very rich in making you think of mathematical shortcuts and millions of ways to make your code perform even better (or to implement fancy patterns).
To the question: When I was beggining to program I would always try to break down one real life problem into several steps and then as I see similarity between tasks or data they transform I would always try to find a lazier, easier, meanier way to implement it.
Elegance came after when learning patterns and real algorithms.
Hank: Big O??? you mean tell beginning programmers that their code is of O(n^2) and yours is of n log n ??
I could see a few different ways to take this:
1) Basic programming building blocks. What are conditional statements, e.g. switch and if/else? What are repetition statements, e.g. for and while loops? How do we combine these to get a program to be the sequence of steps we want? You could take something as easy as adding up a grocery bill or converting temperatures or distances from metric to imperial or vice versa. What are basic variable types like a string, integer, or double? Also in here you could have Boolean Algebra for an advanced idea or possibly teach how to do arithmetic in base 2 or 16 which some people may find easy and others find hard.
2) Algorithmically what are similar building blocks. Sorting is a pretty simple topic that can be widely discussed and analysed to try to figure out how to make this faster than just swapping elements that seem out of order if you learn the Bubblesort which is the most brain dead way to do.
3) Compiling and run-time elements. What is a call stack? What is a heap? How is memory handled to run a program,e.g. the code pieces and data pieces? How do we open and manipulate files? What is compiling and linking? What are make files? Some of this is simple, but it can also be eye-opening just to see how things work which may be what the club covers most of the time.
These next 2 are somewhat more challenging but could be fun:
4) Discuss various ideas behind algorithms such as: 1) Divide and conquer, 2) Dynamic programming, 3) Brute force, 4) Creation of a data structure, 5) Reducing a problem to a similar one already solved for example Fibonacci numbers is a classic recursive problem to give beginning programmers, and 6) The idea of being, "greedy," like in a making change example if you were in a country where coin denominations where a,b, and c. You could also get into some graph theory examples like a minimum weight spanning tree if you want something somewhat exotic, or the travelling salesmen for something that can be easy to describe but a pain to solve.
5) Mathematical functions. How would you program a factorial, which is the product of all numbers from 1 to n? How would you compute the sums of various Arithmetic or Geometric Series? Or compute the number of Combinations or Permutations of r elements from a set of n? Given a set of points, approximate the polynomial that meets this requirement, e.g. in a 2-dimensional plane called x and y you could give 2 points and have people figure out what are the slope and y intercept if you have solved pairs of linear equations already.
6) Lists which can be implemented using linked lists and arrays. Which is better for various cases? How do you implement basic functions such as insert, delete, find, and sort?
7) Abstract Data Structures. What are stacks and queues? How do you build and test classes?
8) Pointers. This just leads to huge amounts of topics like how to allocate/de-allocate memory, what is a memory leak?
Those are my suggestions for various starting points. I think starting a discussion may lead to some interesting places if you can get a few people together that don't mind talking on the same subject week after week in some cases as sorting may be a huge topic to cover well if you want to get into the finer points of things.
You guys could build the TinyPIM project from "C++ Standard Library from Scratch" and then, when it's working, start designing your own extensions.