As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
I hope this is the right place to ask[1], but I've read a lot of good comments on other topics here so I'll just ask. At the moment I'm searching for a topic for my dissertation (Ph.d in non-german countries I think), which must have to do something with parallelism or concurrency, etc. but otherwise I'm quite free to chose what I'm interested in. Also Everything with GPU's is not reasonable, because a colleague of me does already research on this topic and we'd like to have something else for me :)
So, the magic questions is: What would you say are interesting topics in this area? Personally I'm interested in parallel functional programming languages and virtual machines in general but I'd say that a lot of work has been already done there or is actively researched (e.g. in the Haskell community).
I'd greatly appreciate any help in pointing me to other interesting topics.
Best regards,
Michael
PS: I've already looked at https://stackoverflow.com/questions/212253/what-are-the-developments-going-on-in-all-languages-in-parallel-programming-area but there weren't a lot of answers.
[1] I've already asked at http://lambda-the-ultimate.org but the response was unfortunately not as much as expected.
Software transactional memory?
Another reseach area is automated parallelization. That is to say, given a sequence of instructions S0..Sn, come up with multiple sequences that perform the same work in less steps.
Erlang programming!
From the top of my head:
Load balancing & how to achieve the best level of parallelization. I think it can be very good starting point for PhD because here you can propose new methodology and compare it with real values in hand (number of steps - that was already mentioned, CPU usage, memory usage etc) in general or for particular algorithm or set of tasks (like image processing for example).
Parallel Garbage Collection. There are lot of algorithms for collection, there are a lot algorithms to present objects in a memory. For example, there is a recent work from Haskell comunity about Parallel GC: http://research.microsoft.com/en-us/um/people/simonpj/papers/parallel-gc/index.htm
Then again, there is a good way to present your resutls and compare it with others and it gives you the flexibility in the end - you can focus on concurrent data structures later, or synchronizaiton primitives or algorithms etc.
Probably you already have your PhD by now ;) . Anyways: Fault tolerance on massive parallel systems comes into my mind.
Parallel processing and rules engines are both high-visibility topics in the commercial/industrial computing world. So, how about looking at parallel implementations of the Rete algorithm (introductory descriptions here and here), the foundation under many commercial busines rules engines? Are there techniques for building Rete networks
that are better suited for parallelization? Could a "vanilla" Rete network be refactored into multiple networks that could be executed more effectively in parallel? Etc.
Parallelism friendly common-application features. Right now, parallelism has been heavily focused on scientific computing and programming languages, but not so much on consumer applications or consumer-application friendly features/data structures/design patterns, and these will be very important in a multi-core world.
You mentioned Haskell and you surely has stumbled upon Data Parallel Haskell. Since Big Data Analysis is a big word lately and given that the Map/Reduce niche is overcrowded, I think that DPH is a good area of research.
Related
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 9 years ago.
I'm a team-leader/feature-architect that emerged from a developer position, so I have coding experience and a lot of the things being evolved now a days were implemented by me in the first place. Now to the point: Reviewing some code for the sake of refactoring (and some nostalgia) I've found a bunch of places that could be optimized, so as an exercise I gave myself 2 days to explore and improved a lot of stuff. After running a benchmark I found out that the general module performance had improved about 5%.
So I aproached some colleagues (and the team I run) and presented my changes. I was surprised by the general impression of "micro-optimization". If you do look at every single optimization, then yes, they are micro, but if you look at the big picture...
So my question here is: When is an optimization no longer considered "micro"?
Whether an optimization is micro or not is usually not important. The important stuff is whether it gives you any bang for the buck.
You wrote you spend two whole working days for a 5% performance increase. Did you spend those days wisely? Was those things you fixed the "most slow" part of your application, or at least those most easy to fix performance issues? Did your changes made you reach your performance target (that you didn't do before)? Does 5% performance matter at all in your case? Usually you want something like 100% or 1000% increase if you figure out that you need to improve your performance.
Could you perform your optimizations without disturbing readability and/or maintainability of the code?
Besides, what other costs did those optimizations render? How much regression test were you required to perform? How many new bugs did you create?
I know, this looks more like questions than an answer, but those are the kind of questions that should rule your decision to make an optimization or not.
Personally, I would differentiate between changes that lead to a reduction in algorithmic time or space complexity (from O(N^2) to O(N), for example) and changes that speed up the code or reduce its memory requirements but keep the overall complexities the same. I'd call the latter micro optimizations.
However, keep in mind that while this is a precise definition it should not be the only criterion for deciding whether a change is worth keeping: Reduced code complexity (as in difficulty to understand) is often more important, especially if speed and memory requirements are not a major cause of concern.
Ultimately, the answer will depend on your project: For software running on embedded devices the rules are different than for stuff running on an Hadoop cluster.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
(edited)
For anyone interested in music and artificial intelligence:
Do you know of any music-composing algorithm that produces really interesting, fun or intelligent music? And not something sounding like a random noise.
(Previous, too broad question:)
What are some state of the art (very good, non-boring) music composition algorithms, software, researches that you have heard of?
Feel free to post any interesting link about this subject.
P.S. I don't mean programs that assist you at playing, but primarily anything that can compose melody by itself (or with little assistance).
OR: Analyses existing music pieces and tells how much it likes them :)
One of the leading researchers in algorithmic composition is David Cope of the University of California, Santa Cruz. His approach emphasizes machine learning, the results of which were impressively demonstrated in a 2006 performance.
http://www.wired.com/wired/archive/14.09/posts.html?pg=3
A good place to start would be with his aptly named book, The Algorithmic Composer, which covers much of his approach and provides most of the software he has written for his work.
http://books.google.com/books?id=rFGH07I2KTcC
Though not specifically algorithmic composition another invaluable resource is David Temperley's book, The Cognition of Basic Musical Structures, which provides quite a few models begging to be implemented.
http://books.google.com/books?id=IDoLEvTQuewC
Those two alone a pretty time consuming for anyone with an interest in that they are concrete enough that experimenting along the way is inevitable.
Hope that helps.
One possibility would be to use a hidden Markov model: feed it samples of music, and have it generate "similar" music.
One example: http://www.cogs.susx.ac.uk/users/christ/talks/music-making-with-HiMMs.pdf
I did something similar with Shakespeare's sonnets. The results were ... interesting. Amusing, at times.
There's a search engine that lets you whistle a tune and that searches for music alike. I'm not sure whether http://www.midomi.com/ is what I originally heard of. You can for example play the music and see if it finds what you intended.
A fellow student of me created a score composer for his Master's project. The input was humming or whistling and through FFT, music theory and combinatorial algorithms (I'm not sure whether it was simulated annealing). I'm not sure how it was related, but the project had something to do with the http://www.wikifonia.org/ project.
(edit)
I heard a talk from someone who worked at http://last.fm. They analyze music (machine learning) as one of the ways to overcome the cold start problem in their recommender system. They try to predict how much a new song resembles other songs.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
At some point in our lives we're put in the situation to learn a new language (either by job requirements or just passion). Personally, I'm trying to learn Objective-C coming from a background of several years coding php. My problem is that I'm bored with your average starting programs (most of them coming from maths; eg: Fibonacci).
What are your first algorithms and programs you write when learning a new language?
I'm thinking we could make a list of interesting problems to be solved and that would make the learning curve at least more entertaining.
Later edit:
My question is about fun things (and also useful), but not like Code Kata or other very boring math stuff (I've done many of those already)
Later edit [ 2 ]:
I found this today and the problems there seem fun to do.
Try to develop something proper, a project of some kind; solving algorithmic problems really only teaches you to apply the algorithm, not really a lot about the language your using.
Deciding on a useful project then deciding on a language and then 'doing it' will cover many of the bases you need to learn a new language.
I like this series of articles by Larry O'Brien.
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-1/
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-2-data-structures/
http://www.knowing.net/index.php/2006/06/16/15-exercises-to-know-a-programming-language-part-3-libraries-frameworks-and-mashups/
Have a look at these:
How do you find interesting problems to solve?
https://stackoverflow.com/questions/6327/what-are-your-programming-exercises
Just as an idea: Look at the problems of the The Computer Language Benchmarks Game.
When you've done a snippet, you can compare your solutution to the provided one.
Whenever I pick up a new language, I try using it to solve the problems at SPOJ. They basically support every mainstream language out there (as well as a few esolangs like Intercal and Whitespace). The problems are mostly from various ACM and similar format contests around the world and so can be quite hard, and perhaps be too CS/math for everyone's tastes. The tutorial problems are quite doable in any language.
I would also recommend adding that language's tag to your SO interested tags list. Many SO questions involve small snippets of code, and trying to answer them and looking at the other answers can be very enlightening.
But as soon as I've got the basics right, the strategy I choose is to try my hand at a very small project. Usually this a simple game, GUI app or a tiny webapp. This is valuable since there are some things you cannot learn by only doing small isolated blobs of code.
Try to solve these challenges using your new language. What's it all about:
In software we do our practicing on
the job, and that’s why we make
mistakes on the job. We need to find
ways of splitting the practice from
the profession. We need practice
sessions...
I sometimes use the exercises from the first few chapters of Structure and Interpretation of Computer Programs.
You can also try to understand or improve existing open source projects as suggested in this so question. Even though the answers are cocoa biased it should be a good starting point.
This way you can dive in to the level of your choice (understand, debug, improve) and in direction of your interest (UI, specific library, etc, etc...).
EDIT: When I said starting point, I meant that you can browse repositories for objective-c code, for example sourceforge
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
I have been trying to grasp the basics of Support Vector Machines, and downloaded and read many online articles. But still am not able to grasp it.
I would like to know, if there are some
nice tutorial
sample code which can be used for understanding
or something, that you can think of, and that will enable me to learn SVM Basics easily.
PS: I somehow managed to learn PCA (Principal Component Analysis).
BTW, you guys would have guessed that I am working on Machine Learning.
The standard recommendation for a tutorial in SVMs is A Tutorial on Support Vector Machines for Pattern Recognition by Christopher Burges. Another good place to learn about SVMs is the Machine Learning Course at Stanford (SVMs are covered in lectures 6-8). Both these are quite theoretical and heavy on the maths.
As for source code; SVMLight, libsvm and TinySVM are all open-source, but the code is not very easy to follow. I haven't looked at each of them very closely, but the source for TinySVM is probably the is easiest to understand. There is also a pseudo-code implementation of the SMO algorithm in this paper.
This is a very good beginner's tutorial on SVM:
SVM explained
I always thought StompChicken's recommended tutorial was a bit confusing in the way that they jump right into talking about bounds and VC statistics and trying to find the optimal machine and such. It's good if you already understand the basics, though.
Lots of video lectures on SVM:
http://videolectures.net/Top/Computer_Science/Machine_Learning/Kernel_Methods/Support_Vector_Machines/
I found the one by Colin Campbell to be very useful.
A practical guide to SVM classification for libsvm
PyML Tutorial for PyML
I think 1 is practical for use, 3 is clear for understanding.
Assuming you know the basics (eg max margin classifiers, constructing a kernel), solve Problem Set 2 (handout #5) of that stanford machine learning course. There's answer keys & he holds your hand through the whole process. Use Lecture notes 3 & video #7-8 as references.
If you don't know the basics, watch earlier videos.
I would grab a copy of R, install the e1071 package which nicely wraps libsvm, and try to get good results on your favorite data sets.
If you just figured out PCA, it might be informative to look at data with many more predictors than cases (e.g., microarray gene expression profiles, time series, spectra from analytical chemistry, etc.) and compare linear regression on the PCA'd predictors with SVM on the raw predictors.
There are a lot of great references in the other answers, but I think there's value in playing around with the black box before you read what's inside.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
After reading an introductory book on algorithms and data structures I am now craving for examples on how to combine these for optimal efficiency.
For instance, you can combine hashmaps with specific sorting algorithms to create a simple text search program.
Is there any good book or online resource for this?
(I have already ordered Programming Pearls, which looks great, but I want to learn more about this.)
Any good algorithms book is going to have a chapter or two on the importance of choosing the right data structures. I recommend the following books:
Algorithms in a Nutshell
Introduction to Algorithms
The Algorithm Design Manual
I also recommend you check out the Stony Brook Algorithm Repository, particularly the lectures.
Go to these websites and try out the problems:
acm.uva.es
topcoder.com/tc
SPOJ
Codechef
They have problems to satisfy you for at least the next 3-4 years.
Good book (worked for me):
Data Structures and Algorithm Analysis in Java (Second Edition)
Published by Addison-Wesley, 2007
ISBN: 0-321-37013-9
You're basically asking for what a programmer does all day: Flow data through certain algorithms in such a way that you get the result you want in a timely manner. I suggest that you download Python and work through the tutorials. Python allows you to test your ideas very quickly.
To answer my own question, it seems I just have to read up on a lot of algorithms and real world use cases.
As for books, I've ordered
Programming Pearls to read about how one solves real world problems using a combination of algorithms.
The Algorithm Design Manual by Steven Skiena, which contains a catalogue of alrogithms and war stories on how real world problems were solved.
http://www.amazon.com/Structure-Interpretation-Computer-Programs-Second/dp/0070004846/ref=sr_1_1?ie=UTF8&qid=1301994609&sr=8-1
I can warmly recommend this book. It is rather abstract with examples in Scheme (a Lisp dialect) but it will really change the way you think about programs, data and algorithms.