I've got interviews coming up for an internship this summer, and I'd like to review common algorithms and data structures so that I won't be caught off guard by any of the technical questions.
Can anyone recommend a good resource, online or otherwise, for a refresher on common things like shuffling an array, tree traversal, linked lists, etc?
Robert Sedgewick's "Algorithms in $language" are a nice source.
http://www.cs.princeton.edu/~rs/
And of course, Cormen et. al.'s "Introduction to Algorithms" for a more abstract view
http://mitpress.mit.edu/catalog/item/default.asp?ttype=2&tid=11866
Oh, and of course, can't forget Knuth's "The Art of Computer Programming", the definitive compendium...
http://en.wikipedia.org/wiki/The_Art_of_Computer_Programming
This book has a decent review for the basics along with code samples: Programming Interviews Exposed. MIT has an open courseware algorithms class with video lectures here. That goes into way more depth, but you can skip around and pick one what you want. I'm sure just searching google for "algorithms site:.edu" will bring up lots of useful results too.
Related
I'm about to take a course in AI and I want to practice before. I'm using a book to learn the theory, but resources and concrete examples in any language to help with the practice would be amazing. Can anyone recommend me good sites or books with plenty of examples and tutorials ?
Thanks !
Edit: My course will deal with Perceptrons, Neural networks and Bayesian AI.
Really depends on what area you want to specialize on. There is the startup - resource : is
here. I learned about neural nets from their example. Can you elaborate what kind of AI it should be?
Ah and i forgot: this link is a very nice forum where you can look at problems other people have and learn from that.
Cheers.
My advice would be to learn it by trying to implement the various types of learners yourself. See if you can find yourself a dataset related to some interest you have (sports, games, health, etc.) and then try and create a learner to do some kind of classification (predicting a winner in a sports game, learning how to classify backgammon positions, detecting cancer based on patient data, etc.) using different methods. Start with Decision Trees if that's part of your future course work since they're relatively simple, then move on to neural networks.
Here is a set of sources, each one of which i recommend highly--for the quality of the explanation, for the quality of the code, and for the 'completeness' of the algorithm demo.
Least-Squares Regression
(Python)
k-means clustering (Python)
Multi-Layer Perceptron (Python)
Hopfield Network (Python)
Decision Tree (ID3 & C4.5)
In addition, the excellent textbook Elements of Statistical Learning by Hastie, et al. is actually free to download. The authors have an R package that accompanies this textbook which contains all of the code. This book includes detailed discussion of most (if not all) of the major classes of ML algorithms, with specific examples that refer to working code and 'real-world' data.
Personally I would recommend this M.Tim.Jones book on AI.
Has many many topics on AI and almost every type of AI discussion is followed by C example code. Very pragmatic book on AI indeed !!
Russel & Norvig have a good survey of the broad field.
I have been working on RTOS and Linux driver development for quite some time. Now I am interviewing at semiconductor companies and failing to answer questions about algorithms on strings, and time and space complexity. I have not studied discrete maths and algorithms during my as I have an electronics background.
How can I overcome this gap?
Start with something simple like: Algorithms in a Nutshell (good starting point for interview like questions)
Or Algorithms For Interviews
When you feel you know the above book, then you can think of diving into introduction to Algorithms.
You need to review most of the course material for a Data Structures and Algorithms class. In order to answer those types of interview questions, you don't generally need the material covered in Discrete Math.
If you want to take the long way around (and actually understand the topic), I recommend you actually work through the class material and listen to the lectures. Since UC Berkeley posts some of their lectures online, you can watch just such a class (with an excellent instructor) here:
http://webcast.berkeley.edu/course_details.php?seriesid=1906978343
Pay particular attention to the lectures on Big O notation, and the sorting and searching algorithms. Those tend to be the bits that people ask about for job interviews.
If you can't be bothered to actually spend the 30 hours watching the lectures and many more working through the problems, you should at least consult the book that course uses:
Goodrich and Tamassia's Data Structures and Algorithms in Java
Introduction to Algorithms is a great algorithms book (and also happens to be listed 6th on the great influential book question)
I would suggest you Introduction to Algorithms(by CLRS) and Algorithm design manual for algorithms and complexity(by steven skienna) for understanding the Algorithms and complexity.Other than that there are quite a few good tutorials on algorithms and complexity at top coder ( site : topcoder.com/tc ) , you can check them out too.
I started reading Intro to Algorithms by Cormen et al like 3 weeks ago on my free time. I finished the second chapter and have been trying out the exercises for quite a while. I find them a bit difficult.
Is this normal? Should I finish all the exercises before moving on? Or is it alright if I solve the ones I can and move on to the next chapters, possibly coming back to the exercises I can't figure out right now?
If anyone out there has had experience with this book, can you tell me how it was for you? I'm a bit discouraged on not being able to solve quite a few of the exercises here.
That book was hard for me too. We used it at the university I attended and I often had to refer to other sources to get simpler explanations when I found CLRS a little over my head. Once I got the Wikipedia explanation straight in my head, and a code sample working (which CLRS often lacks), I found that I was able to go back to the text and make sense of it.
Don't worry about doing all of the exercises. Even the super-elite MIT students don't have to do them all. Do what you can do and move on. If you need a concept in the next chapter that you had glossed over, it will still be there for you to backtrack to.
MIT OpenCourseWare has also made available the old lectures for Introduction to Algorithms (SMA 5503).
Good for you for diving into CLRS by yourself. You're a braver man than me. I used the book for a grad algorithms course I took last semester, and I had a hard time just finishing the problem sets assigned for the course. Completing all of the exercises would be a truly Herculean effort.
I'd recommend tackling the chapters that interest you most and those that you don't find to difficult. The beginning of the book, if I remember correctly, is one of the harder parts, diving into the mathematical background of a lot of different areas of algorithms. Chapter 5 is especially difficult unless you know a fair bit of probability theory. Also, starred sections and problems are significantly more challenging than the surrounding material (like 21.4, which contains material our professor confessed to being unable to prove in class). Finally at the end of the book, there is just a survey of miscellaneous topics; you can just look at those that interest you, since there are entire books written about each of those topics if you want to learn more about them.
I hope this helps, and most importantly, don't get too discouraged! This is the seminal book on algorithms for a reason.
It's a difficult book, used by one of the pre-eminent technical universities in the world. It's no surprise that it's challenging. There are a LOT of exercises of varying difficulty. It's a noble goal to attempt all of them.
Aren't the course materials on-line? It'd be interesting to see if students taking the course for credit do all the exercises.
I wouldn't be discouraged. Keep plugging, even if you have to pass on some of the exercises. There's nothing saying that you have to master it in one pass, either. Go through, take in what you can, and re-do if necessary. You might find that the extra context helps.
The lectures are available on iTunes if you find that helps.
The important thing is to set a deadline and make steady progress. Good luck.
The problem with not doing all the problems is that when you are self-studying, you really don't have a good gage for how much you should be able to answer.
You can look at the course assignments online, I would recommend that for figuring out problem sets to get done.
I am learning Algorithms on my own from the CLRS book in 2020. Regardless of what people tell you about solutions manuals in general, it is advisable to get "good" solutions manuals if you are self studying with the book.
The two sets of solutions I recommend are (1) The official instructor manual and (2) solutions by Rutger's university students Michelle Bodnar and Andrew Lohr. When one of those solutions is unclear, I simply refer to the other one. If you get stuck at a problem, then give yourself a few minutes to solve it. If you don't get the answer, then use the solutions manuals. You can always test yourself on the problems from other text books or leetcode to see how much you can do on your own vs just following a solutions manual.
I won't post the solutions manuals here. I suggest that you search for them online. The Rutgers one is easily available and is legal. The official one is restricted to instructors only and is hard to get. You might be able to pay obscure online sellers/hackers to get the official one for you. Use a preloaded visa or master card gift card to make that purchase. Make sure that the card is accepted in the sellers country.
Chapter 2 was doable because I used Youtube to understand algorithms and time complexity when it was not clearly explained in the CLRS book, which is quite often. The solutions manuals also helped a little bit.
Chapter 3 is hard and I don't know if I will be able to get past this one. I might have to switch to another book, perhaps the one by Tamassia. I had studied elementary algebra, set theory, functions, probability, mathematical series and calculus a few years ago. But, I remember only a few of those things. So, it is difficult to understand Chapter 3 and move ahead.
In general, it is a comprehensive and rigorous book. However, it is bad because of these:
One-based array indexing (instead of the usual 0-based) - so everytime you translate the algorithm present in the book into code you have to either +1 or -1 and / or use < instead of <= or the other way around and so on.
Bad variable naming in pseudocode - instead of lo, hi or left, right you get p and q.
The fact that is is very rigorous may get you confused in the little details and usually you can miss the overall idea of an algorithm.
It is a famous book because many scientific papers refer to this book in their references.
Otherwise, it is ok.
I've been reading things here and there for a while now about using an "ant colony" model as a heuristic approach to optimizing various types of algorithms. However, I have yet to find an article or book that discusses ant colony optimizations in an introductory manner, or even in a lot of detail. Can anyone point me at some resources where I can learn more about this idea?
On the off chance that you know German (yes, sorry …), a friend and I have written an introduction with code about this subject which I myself find quite passable. The text and code uses the example of TSP to introduce the concept.
Even if you don't know German, take a look at the code and the formulas in the text, this might still serve.
link Wikipedia actually got me started. I read the article and got to coding. I was solving a wicked variation of the traveling salesman problem. It's an amazing meta-heuristic. Basically, any type of search problem that can be put into a graph (nodes & edges, symmetric or not) can be solved with an ACO.
Look out for the difference between global and local pheromone trails. Local pheromones discourage one generation of ants from traversing the same path. They keep the model from converging. Global pheromones are attractors and should snag at least one ant per generation. They encourage optimum paths over several generations.
The best suggestion I have, is simply to play with the algorithm. Setup a basic TSP solver and some basic colony visualization. Then have some fun. Working with ants, conceptually, is way cool. You program their basic behaviors and then set them loose. I even grow fond of them. :)
ACOs are a greedier form of genetic algorithms. Play with them. Alter their communicative behaviors and pack behavior. You'll rapidly begin to see network / graph programming in an entirely different way. That's their biggest benefit, not the recipe that most people see it as.
You just gotta play with it to really understand it. Books & research papers only give a general sky-high understanding. Like a bike, you just gotta start riding. :)
ACOs, by far, are my favorite abstraction for graph problems.
National Geographic wrote an interesting article awhile back talking about some of the theories.
The best resource for these topics is Google scholar. Ive been working on Ant Colony Optimization algorithms for a while, here are some good papers:
Ant Colony Optimization - A New Metaheuristic
Ant Colony Optimization - Artificial Ants as a Computational Intelligence Technique
Just search for "Ant Colony" on google scholar.
Also, search for papers published by Marco Dorigo.
I am surprised nobody has mentioned the bible of ACO:
Marco Dorigo & Thomas Stützle: Ant Colony Optimization
This book is written by the author of ACO and it is highly readable. You can take it to the beach and have fun reading it. But it is also the most complete resource of all, great as a reference when implementing the thing.
You can read some excerpts on Google Books
Another great source of wisdom is the ACO Homepage
See for example this article on scholarpedia.
There is also discussion here in the What is the most efficient way of finding a path through a small world graph? question.
At first glance this seems to be closely related to (or prehaps a special case of) the Metropolis algorithm. So that's another possible direction for searching.
Addition: This PDF file includes a reference to the original Metropolis paper from 1953.
Well, i found the Homepage of Eric Rollins and his different Implementations (Haskell, Scala, Erlang,...) of a ACO Algorithm helpfull.
And also the Book from Enrique Alba, titled "Parallel Metaheuristics: A New Class of Algorithms" where you can find a whole chapter of explanation about ACO Algorithms and their different usages.
Hth
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I want to learn algorithms using some very basic simple tutorials. Are there any out there? I have heard of recursion and stuff and I would like to get good at it. Any help would be appreciated.
I would start out by taking a look at EternallyConfuzzled which contains great tutorials for basic Data Structures an Algorithms including linked lists and binary search trees, sorting and searching algorithms. If you want to learn more after this I would recommend the following books in order of increasing complexity, completeness, and required math knowledge:
Algorithms in C (also available in C++ and Java)
Introduction to Algorithms
The Art of Computer Programming
If you want to learn algorithms this book is the best choice.
(source: mcgraw-hill.com)
MIT's OCW has video lectures of their Algorithm course. The professor is one of the authors of the book Introduction to Algorithms, which another poster suggested.
It assumes a basic knowledge of Discrete Maths.
TopCoder has some good algorithm tutorials.
If you're interested in a tutorial, avoid the CLRS book recommend above. It takes a rigorous theoretical approach to the study of Algorithms, which is very different from a tutorial approach.
You learn Algorithms by doing them. So find a resource that provides Algorithms problems and guidance in solving them. If you want a textbook, check out the Algorithm Design Manual, which also has an online Algorithm Repository.
If you prefer an online course, Udacity offers a python-based Algorithms course, while Coursera offers general and Java-based ones.
Since the important part is practicing Algorithms, you can skip the video courses and just solve challenges. Other answers suggested sites with challenges you can practice once you're good at Algorithms. In the beginning you'll want more guidance, so find a resource that provides Algorithms challenges and help with solving them. I created Learneroo for this purpose. You can start by learning the fundamentals of Recursion with the Recursion Tutorial.
Recursion really isn't an algorithm. Since you don't have anything specific you're interested in I'd suggest you read wikipedia's List of alorithms or as others have suggested grab a book.
I would start at the Stony Brook Algorithm Repository. The site has some really good explanations of different types of algorithms, and it references what books and other resources it uses so you can get a taste of what's available.
I suggest that you start from sorting algorithms. Read the related wikipedia page, skip the O(n log n) stuff, and focus on the implementations of, say, insertion sort, merge sort, and quick sort. Familiarize with binary searching. Also, learn about some basic data structures, such as vectors, linked lists, stacks, their implementation, and what they are useful for. (More often than not, an algorithm to solve a problem goes together with a suitable data structure.) Once you are confident with different algorithms and data structures, you can dive in a more complete treatise such as the book by Cormen et al.
As for recursion, it is not an algorithm in itself. It is instead a technique that some algorithms employ to solve a problem, when the latter can be naturally split into subproblems. The technique of splitting a problem, solving the subproblems separately and then merging their solutions to obtain a solution for the original problem, is called "divide et impera", or "divide and conquer". (Recursion is also the related feature of most programming languages, where it basically means "functions that call themselves".)
The most cited, the most trivial, and the most useless example of a "recursive algorithm", is the one to compute factorials. Don't mind it. Instead, read about the Tower of Hanoi problem, which admits a simple and elegant recursive solution, and again, study some sorting algorithms, for many of them are indeed recursive.
To the various people who have commented that book xyz is not simple, I'd point out that algorithmics is not a simple topic. You need at least university entry level mathematics to understand the concepts plus the ability to reason about computation at a suitably abstract level. If you ever find an "Algorithmics for Dummies" book, don't waste your money!
my choice http://aduni.org/courses/algorithms/
Going through solutions in topcoder problems is a very good way to pick up algorithms. Reading theory alone won't help
Khan academy started an excellent interactive self paced course on algorithms - https://www.khanacademy.org/computing/computer-science/algorithms.
Recursion is a language feature, and less an "algorithm" per se. All recursion can be replaced with proper data structures (like a stack).
I'd recommend grabbing a book. The problem with algorithms is that it's a relatively progressive topic. You first need to learn simple searches before you can learn sorting, and you need sorting before you can do minimum spanning trees etc. A book will properly order these, and if the text doesn't give you enough information the internet is a great next step. Try Amazon and look at the comments for someone who is new.
Make sure you learn an implementation language before you try to go at this though, until you understand how the language works it's going to be very hard to pick out bugs in your logic vs a misunderstanding of what's happening for a given sequence of commands.
USA Computing Olympiad has a nice algorithms training site that so far anyone can sign up for and it's almost in a class like format. read a little, do an exercise, read more, do an exercise etc.
One of my favorite list of algorithm problems is Project Euler, they are pretty diverse and you can solve the same problem many times for optimizations, and you will find lots of communities (C++, C#, Python, ... etc) posting their benchmarks for every problem
It is so much fun, geek fun
Solve questions on various sites as SPOJ etc . and read books on Introduction to Algorithms, there are some online courses as well on coursera .