I am just starting out studying machine learning and currently doing Andrew Ng's course on Coursera. I am going through the course but am a bit lost. It will make studying all those algorithms/theory a lot rewarding if I can see some use cases for them.
For example, the first topic I read about was gradient descent and then linear regression and logistic regression. Are these used directly in practice or are other algorithms like k-means and kernel density used? I guess I am trying to get real world (software engineering, data mining) examples of these topics. Can some one suggest a post that might have some explanation of any machine learning algorithm(s) usage? It will be greatly helpful.
NO FREE LUNCH THEOREM states that if algorithm A outperforms algorithm B for some problem, then loosely speaking there must exist exactly as many other problems where B outperforms.
So, it is difficult to link algorithm with particular use case.
If you are looking only for use cases where you can use machine learning algorithms, visit https://www.kaggle.com/wiki/DataScienceUseCases
Update : Just now, i came across http://pkghosh.wordpress.com Check it out. (use cases with algorithms)
Given an algorithm, described in a paper of your choice with a bunch of symbols and a very special notation.
How do I learn to read such algorithm descriptions and turn them into a computer program?
The following picture describes an algorithm to calculate the incremental local outlier factor:
What is the best approach to translate that to a programming language?
Your answer can also help me by pointing to articles that describe the symbols being used, the notation in general and tutorials on how to read and understand such papers.
You seem to be asking for what is generally described as part of a course in first-order predicate calculus. A general introduction (including notation) can be found here or in your library.
Generally, notation like means "for all that are in the set ..." and would often be implemented in programming language using a for loop or similar looping structure, depending on the particular language.
Introductory books on algorithms will also likely be useful in answering the questions you have. An example would be Sedgewick's book Algorithms in C if your target computer language is C.
I think the question can only be answered in a very broad sense. The pseudocode notation in algorithmic papers does not follow a consistent standard, and sometimes no pseudocode notation for the algorithms in question. A general advice would be to study the problem covered as much as possible and to get into mathematics a bit.
I have written a translator that is able to convert some simple mathematical formulas into various programming languages. This is the input in in mathematical notation:
distance_formula(x1,y1,x2,y2) = sqrt((x1-x2)^2+(y1-y2)^2)
and this is the output in Python:
def distance_formula(x1,y1,x2,y2):
return math.sqrt(((x1-x2)**2)+((y1-y2)**2))
Similarly, there are several pseudocode translators that have been designed to convert descriptions of algorithms into equivalent programs.
I have been working on RTOS and Linux driver development for quite some time. Now I am interviewing at semiconductor companies and failing to answer questions about algorithms on strings, and time and space complexity. I have not studied discrete maths and algorithms during my as I have an electronics background.
How can I overcome this gap?
Start with something simple like: Algorithms in a Nutshell (good starting point for interview like questions)
Or Algorithms For Interviews
When you feel you know the above book, then you can think of diving into introduction to Algorithms.
You need to review most of the course material for a Data Structures and Algorithms class. In order to answer those types of interview questions, you don't generally need the material covered in Discrete Math.
If you want to take the long way around (and actually understand the topic), I recommend you actually work through the class material and listen to the lectures. Since UC Berkeley posts some of their lectures online, you can watch just such a class (with an excellent instructor) here:
http://webcast.berkeley.edu/course_details.php?seriesid=1906978343
Pay particular attention to the lectures on Big O notation, and the sorting and searching algorithms. Those tend to be the bits that people ask about for job interviews.
If you can't be bothered to actually spend the 30 hours watching the lectures and many more working through the problems, you should at least consult the book that course uses:
Goodrich and Tamassia's Data Structures and Algorithms in Java
Introduction to Algorithms is a great algorithms book (and also happens to be listed 6th on the great influential book question)
I would suggest you Introduction to Algorithms(by CLRS) and Algorithm design manual for algorithms and complexity(by steven skienna) for understanding the Algorithms and complexity.Other than that there are quite a few good tutorials on algorithms and complexity at top coder ( site : topcoder.com/tc ) , you can check them out too.
I am planning to implement spam filter using Naive Bayesian classification model.
Online I see a lot of info on Naive Bayesian classification, but the problem is its a lot of mathematical stuff, than clearly stating how its done. And the problem is I am more of a programmer than a mathematician (yes I had learnt Probability and Bayesian theorem back in school, but out of touch for a long long time, and I don't have luxury of learning it now (Have nearly 3 weeks to come-up with a working prototype)).
So if someone can explain or point me to location where its explained for programmers than a mathematician, it would be a great help.
PS: By the way I have to implement it in C, if you want to know. :(
Regards,
Microkernel
The book Programming Collective Intelligence has chapter that covers this and other methods. The chapter (#6) can be understood without reference to previous chapters, is written clearly, and discusses only the minimal mathematics necessary to get the job done.
You could try this website. It's got some source code.
I would highly recommend Andrew Moore's tutorials and I think you should start with this one.
You could also take a look at POPFile, an open source spam filter engine.
Have you looked at dspam?
http://dspam.irontec.com/faq.shtml#1.0
http://www.nuclearelephant.com/
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I want to learn algorithms using some very basic simple tutorials. Are there any out there? I have heard of recursion and stuff and I would like to get good at it. Any help would be appreciated.
I would start out by taking a look at EternallyConfuzzled which contains great tutorials for basic Data Structures an Algorithms including linked lists and binary search trees, sorting and searching algorithms. If you want to learn more after this I would recommend the following books in order of increasing complexity, completeness, and required math knowledge:
Algorithms in C (also available in C++ and Java)
Introduction to Algorithms
The Art of Computer Programming
If you want to learn algorithms this book is the best choice.
(source: mcgraw-hill.com)
MIT's OCW has video lectures of their Algorithm course. The professor is one of the authors of the book Introduction to Algorithms, which another poster suggested.
It assumes a basic knowledge of Discrete Maths.
TopCoder has some good algorithm tutorials.
If you're interested in a tutorial, avoid the CLRS book recommend above. It takes a rigorous theoretical approach to the study of Algorithms, which is very different from a tutorial approach.
You learn Algorithms by doing them. So find a resource that provides Algorithms problems and guidance in solving them. If you want a textbook, check out the Algorithm Design Manual, which also has an online Algorithm Repository.
If you prefer an online course, Udacity offers a python-based Algorithms course, while Coursera offers general and Java-based ones.
Since the important part is practicing Algorithms, you can skip the video courses and just solve challenges. Other answers suggested sites with challenges you can practice once you're good at Algorithms. In the beginning you'll want more guidance, so find a resource that provides Algorithms challenges and help with solving them. I created Learneroo for this purpose. You can start by learning the fundamentals of Recursion with the Recursion Tutorial.
Recursion really isn't an algorithm. Since you don't have anything specific you're interested in I'd suggest you read wikipedia's List of alorithms or as others have suggested grab a book.
I would start at the Stony Brook Algorithm Repository. The site has some really good explanations of different types of algorithms, and it references what books and other resources it uses so you can get a taste of what's available.
I suggest that you start from sorting algorithms. Read the related wikipedia page, skip the O(n log n) stuff, and focus on the implementations of, say, insertion sort, merge sort, and quick sort. Familiarize with binary searching. Also, learn about some basic data structures, such as vectors, linked lists, stacks, their implementation, and what they are useful for. (More often than not, an algorithm to solve a problem goes together with a suitable data structure.) Once you are confident with different algorithms and data structures, you can dive in a more complete treatise such as the book by Cormen et al.
As for recursion, it is not an algorithm in itself. It is instead a technique that some algorithms employ to solve a problem, when the latter can be naturally split into subproblems. The technique of splitting a problem, solving the subproblems separately and then merging their solutions to obtain a solution for the original problem, is called "divide et impera", or "divide and conquer". (Recursion is also the related feature of most programming languages, where it basically means "functions that call themselves".)
The most cited, the most trivial, and the most useless example of a "recursive algorithm", is the one to compute factorials. Don't mind it. Instead, read about the Tower of Hanoi problem, which admits a simple and elegant recursive solution, and again, study some sorting algorithms, for many of them are indeed recursive.
To the various people who have commented that book xyz is not simple, I'd point out that algorithmics is not a simple topic. You need at least university entry level mathematics to understand the concepts plus the ability to reason about computation at a suitably abstract level. If you ever find an "Algorithmics for Dummies" book, don't waste your money!
my choice http://aduni.org/courses/algorithms/
Going through solutions in topcoder problems is a very good way to pick up algorithms. Reading theory alone won't help
Khan academy started an excellent interactive self paced course on algorithms - https://www.khanacademy.org/computing/computer-science/algorithms.
Recursion is a language feature, and less an "algorithm" per se. All recursion can be replaced with proper data structures (like a stack).
I'd recommend grabbing a book. The problem with algorithms is that it's a relatively progressive topic. You first need to learn simple searches before you can learn sorting, and you need sorting before you can do minimum spanning trees etc. A book will properly order these, and if the text doesn't give you enough information the internet is a great next step. Try Amazon and look at the comments for someone who is new.
Make sure you learn an implementation language before you try to go at this though, until you understand how the language works it's going to be very hard to pick out bugs in your logic vs a misunderstanding of what's happening for a given sequence of commands.
USA Computing Olympiad has a nice algorithms training site that so far anyone can sign up for and it's almost in a class like format. read a little, do an exercise, read more, do an exercise etc.
One of my favorite list of algorithm problems is Project Euler, they are pretty diverse and you can solve the same problem many times for optimizations, and you will find lots of communities (C++, C#, Python, ... etc) posting their benchmarks for every problem
It is so much fun, geek fun
Solve questions on various sites as SPOJ etc . and read books on Introduction to Algorithms, there are some online courses as well on coursera .