As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have been trying to grasp the basics of Support Vector Machines, and downloaded and read many online articles. But still am not able to grasp it.
I would like to know, if there are some
nice tutorial
sample code which can be used for understanding
or something, that you can think of, and that will enable me to learn SVM Basics easily.
PS: I somehow managed to learn PCA (Principal Component Analysis).
BTW, you guys would have guessed that I am working on Machine Learning.
The standard recommendation for a tutorial in SVMs is A Tutorial on Support Vector Machines for Pattern Recognition by Christopher Burges. Another good place to learn about SVMs is the Machine Learning Course at Stanford (SVMs are covered in lectures 6-8). Both these are quite theoretical and heavy on the maths.
As for source code; SVMLight, libsvm and TinySVM are all open-source, but the code is not very easy to follow. I haven't looked at each of them very closely, but the source for TinySVM is probably the is easiest to understand. There is also a pseudo-code implementation of the SMO algorithm in this paper.
This is a very good beginner's tutorial on SVM:
SVM explained
I always thought StompChicken's recommended tutorial was a bit confusing in the way that they jump right into talking about bounds and VC statistics and trying to find the optimal machine and such. It's good if you already understand the basics, though.
Lots of video lectures on SVM:
http://videolectures.net/Top/Computer_Science/Machine_Learning/Kernel_Methods/Support_Vector_Machines/
I found the one by Colin Campbell to be very useful.
A practical guide to SVM classification for libsvm
PyML Tutorial for PyML
I think 1 is practical for use, 3 is clear for understanding.
Assuming you know the basics (eg max margin classifiers, constructing a kernel), solve Problem Set 2 (handout #5) of that stanford machine learning course. There's answer keys & he holds your hand through the whole process. Use Lecture notes 3 & video #7-8 as references.
If you don't know the basics, watch earlier videos.
I would grab a copy of R, install the e1071 package which nicely wraps libsvm, and try to get good results on your favorite data sets.
If you just figured out PCA, it might be informative to look at data with many more predictors than cases (e.g., microarray gene expression profiles, time series, spectra from analytical chemistry, etc.) and compare linear regression on the PCA'd predictors with SVM on the raw predictors.
There are a lot of great references in the other answers, but I think there's value in playing around with the black box before you read what's inside.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
(edited)
For anyone interested in music and artificial intelligence:
Do you know of any music-composing algorithm that produces really interesting, fun or intelligent music? And not something sounding like a random noise.
(Previous, too broad question:)
What are some state of the art (very good, non-boring) music composition algorithms, software, researches that you have heard of?
Feel free to post any interesting link about this subject.
P.S. I don't mean programs that assist you at playing, but primarily anything that can compose melody by itself (or with little assistance).
OR: Analyses existing music pieces and tells how much it likes them :)
One of the leading researchers in algorithmic composition is David Cope of the University of California, Santa Cruz. His approach emphasizes machine learning, the results of which were impressively demonstrated in a 2006 performance.
http://www.wired.com/wired/archive/14.09/posts.html?pg=3
A good place to start would be with his aptly named book, The Algorithmic Composer, which covers much of his approach and provides most of the software he has written for his work.
http://books.google.com/books?id=rFGH07I2KTcC
Though not specifically algorithmic composition another invaluable resource is David Temperley's book, The Cognition of Basic Musical Structures, which provides quite a few models begging to be implemented.
http://books.google.com/books?id=IDoLEvTQuewC
Those two alone a pretty time consuming for anyone with an interest in that they are concrete enough that experimenting along the way is inevitable.
Hope that helps.
One possibility would be to use a hidden Markov model: feed it samples of music, and have it generate "similar" music.
One example: http://www.cogs.susx.ac.uk/users/christ/talks/music-making-with-HiMMs.pdf
I did something similar with Shakespeare's sonnets. The results were ... interesting. Amusing, at times.
There's a search engine that lets you whistle a tune and that searches for music alike. I'm not sure whether http://www.midomi.com/ is what I originally heard of. You can for example play the music and see if it finds what you intended.
A fellow student of me created a score composer for his Master's project. The input was humming or whistling and through FFT, music theory and combinatorial algorithms (I'm not sure whether it was simulated annealing). I'm not sure how it was related, but the project had something to do with the http://www.wikifonia.org/ project.
(edit)
I heard a talk from someone who worked at http://last.fm. They analyze music (machine learning) as one of the ways to overcome the cold start problem in their recommender system. They try to predict how much a new song resembles other songs.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
I'm preparing for an exam concerning languages, grammars, parsing and compilers. It's not really my cup of tea and most resources I find use the language of mathematics to define the different terms of the trade and explain the different concepts I need to know rather than stick with English or French, which I would very much prefer. Therefore, I'm having some trouble both with finding the motivation to continue studying and with simply understanding the theory.
So here is my question: Do any of you know where I could find a "fun" way of learning all this? Or at the very least, maybe a more "concrete" and less "mathematical" way of handling this subject.
I need to cover the following so anything on these subjects is welcome!
Parsing (LR, LL, ...)
Grammars (Context-free, deterministic, ...)
Syntax analysis Static flow analysis
Impact analysis concerning software maintenance and dependency to user interfaces
Dynamic analysis
Here are some resources which could be considered "fun" (with an emphasis on the quotation marks) ways to learn about a technical subject, just to get a sense of what I'm looking for.
Why's Poignant Guide to Ruby
Try MongoDB (type Help +
Enter)
If you want to learn a lot in a short time, go learn about meta compilers from Val Schorre's 1964 (yes, you read it right) Meta II paper on how to build self-compiling metacompilers. As a freebie, they can compile conventional lanuages, too! The paper is 10 pages, describes meta compilers (as a virtual machine beleive it or not), and contains two complete compilers.
There's a mind-blowing moment you eventually arrive at when you grok how the compiler can compile itself... I learned compilers this way back in the early 70s and it is the most memorable compiler lesson I ever had. This is fun.
There is an online tutorial here which implements all the ideas in JavaScript..
The author of the tutorial is Dr. James Neighbors, the guy who invented the term "domain analysis". He used the MetaII ideas for a spectacular domain-specific code generator named Draco. Draco was a key inspiration to compiler-like tools I've been building for the last 30 years.
How long do you have to prepare? The "best" way to learn compilers is to dig into them and the best way to do that is to use the best book on compilers EVER WRITTEN: The Dragon Book It's old, but awesome. It's not cheap but it is, quite possibly, the most concrete and least mathematical way to learn about the magical compiler.
It doesn't have any flashing lights and it won't be in an awesome font like the Ruby guide, but it's in the top 10 Books Every Programmer Should Read
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
At some point in our lives we're put in the situation to learn a new language (either by job requirements or just passion). Personally, I'm trying to learn Objective-C coming from a background of several years coding php. My problem is that I'm bored with your average starting programs (most of them coming from maths; eg: Fibonacci).
What are your first algorithms and programs you write when learning a new language?
I'm thinking we could make a list of interesting problems to be solved and that would make the learning curve at least more entertaining.
Later edit:
My question is about fun things (and also useful), but not like Code Kata or other very boring math stuff (I've done many of those already)
Later edit [ 2 ]:
I found this today and the problems there seem fun to do.
Try to develop something proper, a project of some kind; solving algorithmic problems really only teaches you to apply the algorithm, not really a lot about the language your using.
Deciding on a useful project then deciding on a language and then 'doing it' will cover many of the bases you need to learn a new language.
I like this series of articles by Larry O'Brien.
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-1/
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-2-data-structures/
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-3-libraries-frameworks-and-mashups/
Have a look at these:
How do you find interesting problems to solve?
https://stackoverflow.com/questions/6327/what-are-your-programming-exercises
Just as an idea: Look at the problems of the The Computer Language Benchmarks Game.
When you've done a snippet, you can compare your solutution to the provided one.
Whenever I pick up a new language, I try using it to solve the problems at SPOJ. They basically support every mainstream language out there (as well as a few esolangs like Intercal and Whitespace). The problems are mostly from various ACM and similar format contests around the world and so can be quite hard, and perhaps be too CS/math for everyone's tastes. The tutorial problems are quite doable in any language.
I would also recommend adding that language's tag to your SO interested tags list. Many SO questions involve small snippets of code, and trying to answer them and looking at the other answers can be very enlightening.
But as soon as I've got the basics right, the strategy I choose is to try my hand at a very small project. Usually this a simple game, GUI app or a tiny webapp. This is valuable since there are some things you cannot learn by only doing small isolated blobs of code.
Try to solve these challenges using your new language. What's it all about:
In software we do our practicing on
the job, and that’s why we make
mistakes on the job. We need to find
ways of splitting the practice from
the profession. We need practice
sessions...
I sometimes use the exercises from the first few chapters of Structure and Interpretation of Computer Programs.
You can also try to understand or improve existing open source projects as suggested in this so question. Even though the answers are cocoa biased it should be a good starting point.
This way you can dive in to the level of your choice (understand, debug, improve) and in direction of your interest (UI, specific library, etc, etc...).
EDIT: When I said starting point, I meant that you can browse repositories for objective-c code, for example sourceforge
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
After reading an introductory book on algorithms and data structures I am now craving for examples on how to combine these for optimal efficiency.
For instance, you can combine hashmaps with specific sorting algorithms to create a simple text search program.
Is there any good book or online resource for this?
(I have already ordered Programming Pearls, which looks great, but I want to learn more about this.)
Any good algorithms book is going to have a chapter or two on the importance of choosing the right data structures. I recommend the following books:
Algorithms in a Nutshell
Introduction to Algorithms
The Algorithm Design Manual
I also recommend you check out the Stony Brook Algorithm Repository, particularly the lectures.
Go to these websites and try out the problems:
acm.uva.es
topcoder.com/tc
SPOJ
Codechef
They have problems to satisfy you for at least the next 3-4 years.
Good book (worked for me):
Data Structures and Algorithm Analysis in Java (Second Edition)
Published by Addison-Wesley, 2007
ISBN: 0-321-37013-9
You're basically asking for what a programmer does all day: Flow data through certain algorithms in such a way that you get the result you want in a timely manner. I suggest that you download Python and work through the tutorials. Python allows you to test your ideas very quickly.
To answer my own question, it seems I just have to read up on a lot of algorithms and real world use cases.
As for books, I've ordered
Programming Pearls to read about how one solves real world problems using a combination of algorithms.
The Algorithm Design Manual by Steven Skiena, which contains a catalogue of alrogithms and war stories on how real world problems were solved.
http://www.amazon.com/Structure-Interpretation-Computer-Programs-Second/dp/0070004846/ref=sr_1_1?ie=UTF8&qid=1301994609&sr=8-1
I can warmly recommend this book. It is rather abstract with examples in Scheme (a Lisp dialect) but it will really change the way you think about programs, data and algorithms.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
My question, his question, but blogs as resources to be specific.
I find blogs great to keep up to date... refresh material...
So do you know any blogs who tackle math-related programming problems...
Not exactly what you're asking for, but Project Euler freshens up my math skills.
Following MIT Open Courseware is another good computer-based way to learn and practice these skills.
Good Math, Bad Math is my favorite regular math blog.
Steve Yegge's post Math for Programmers gives a pretty decent rundown of what math is important for programmers to understand.
I also like to keep an eye on the math subreddit.
Better Explained has several good articles.
This blog has some interesting math-related things. Some of them are pretty high-level. You've been warned.
Not exactly a blog, but: Notices of the American Mathematical Society
I blog about programming and math, especially probability and statistics, at The Endeavour.
If you want highly lucid explanations and discussions of mathematics related to computer science, then this blog, Developing for Developers, is superb. [No posts for a while, but the previous posts are great.]
It sometimes goes off on tangents but this guy blogs about math software a lot
www.walkingrandomly.com
More along the lines of Project Euler than a blog, William Wu has quite a number of math and CS challenges.
MathPuzzle is one of my favorite sites on math. It may seem not directly related to problem-solving, but games are an excellent way of learning.
This Weeks Finds in mathematical physics is well worth a look, though whethers it's maths depends on where you draw the borderline.
good question, I'm surprised to see that nobody mentioned wikipedia so far.. However, many articles, especially about higher math are written by experts and are overloaded with details, which is not ideal if you just want to learn a bit. But still wikipedia is something that I use regularly to look up math questions.
When it comes to a specific math subject you may also want to read a book.. ;)
Found this one with the help of stackoverflow... :)
link
not before you have your first coffee I think...