Related
For my research project in biology for my final year I need to present a project in the field of Biotechnology. Being passionate about programming I immediately thought of Evolutionary Algorithms! However I am not sure if Evolutionary Algorithms would fall into the category of Biotechnology, hence I would rather confirm with the best and most passionate programming experts on the world.
Unfortunately no, a genetic algorithm (ga) is just an optimization technique that is inspired from various evolutionary processes like mutation or crossover. They belong to the area of evolutionary computing and artificial intelligence and not biotechnology.
Please follow this link for a brief introduction to genetic algorithms.
Biotechnology from the other hand has to do with actual organisms that are used in some way to make a product or an application. It sounds kind of broad but that is only because the particular field is in itself very broad. We use forms of biotechnology for thousands of years now in many common and not so common ways. This is not bad though as it gives you a lot of freedom regarding your project. Choose anything from food production to medicine and you will still be relevant to the subject.
Maybe the links provided will give you some inspiration.
Link one
Link two
Until you're implementing your evolutionary algorithms with organic material, no.
They are, of course, inspired from the way modern organisms have come to exist. But there's no biology in what you're doing.
No. It's just an example of a biological algorithm adapted for computational purposes.
Other examples include Ant-Colony Optimization, Flocking behavior, etc.
IIRC, Biotechnology requires the use of actual biology (i.e., living things or parts of them) adapted for technological purposes, not just an algorithmic emulation or modelling of their processes.
"Designing the right algorithm for a given application is a difficult job. It requires a major creative act, taking a problem and pulling a solution out of the ether. This is much more difficult than taking someone else's idea and modifying it or tweaking it to make it a little better. The space of choices you can make in algorithm design is enormous, enough to leave you plenty of freedom to hang yourself".
I have studied several basic design techniques of algorithms like Divide and Conquer, Dynamic Programming, greedy, backtracking etc.
But i always fail to recognize what principles to apply when i come across certain programming problems. I want to master the designing of algorithms.
So can any one suggest a best place to understand the principles of algorithm design in depth.....
I suggest Programming Pearls, 2nd edition, by Jon Bentley. He talks a lot about algorithm design techniques and provides examples of real world problems, how they were solved, and how different algorithms affected the runtime.
Throughout the book, you learn algorithm design techniques, program verification methods to ensure your algorithms are correct, and you also learn a little bit about data structures. It's a very good book and I recommend it to anyone who wants to master algorithms. Go read the reviews in amazon: http://www.amazon.com/Programming-Pearls-2nd-Edition-Bentley/dp/0201657880
You can have a look at some of the book's contents here: http://netlib.bell-labs.com/cm/cs/pearls/
Enjoy!
You can't learn algorithm design just from reading books. Certainly, books can help. Books like Programming Pearls as suggested in another answer are great because they give you problems to work. Each problem forces you to think about how to solve a particular type of problem.
The idea is that you expose yourself to many different types of problems and their solutions. In doing so, you learn how to examine a problem and see if it shares anything in common with problems you've already seen. In that regard, it's not a whole lot different than the way you learned how to solve "word problems" in math class. Granted, most algorithm problems are more complex than having to figure out where on the tracks the two trains will collide, but the way you learn how to solve the problems is the same. You learn common techniques used to solve simple problems, then combine those techniques to solve more complex problems, etc.
Read, practice, lather, rinse, repeat.
In addition to books like Programming Pearls, there are sites online that post different programming challenges that you can test yourself on. It helps if you have friends or co-workers who also are interested in algorithms, because you can bounce ideas off each other and pose interesting challenges, or work together to come up with solutions to problems.
Did I mention that it takes practice?
"Mastering" anything takes time. A long time. A popular theory is that it takes 10,000 hours of practice to become an expert at anything. There's some dispute about that for particular endeavors, but in general it's true. You don't master anything overnight. You have to study. And practice. And read what others have done. Study some more and practice some more.
A good book about algorithm design is Kleinbeg Tardos. Every design technique depends on the problem that you are going to tackle. It is very important to do the exercises in the algorithm books and have feedback from teachers about that.
If there exist a locally optimal choice taht brings the globally optimal solution you can use a greedy algorithm.
If the problem has optimal substructure, you can use dynamic programming.
I'm about to take a course in AI and I want to practice before. I'm using a book to learn the theory, but resources and concrete examples in any language to help with the practice would be amazing. Can anyone recommend me good sites or books with plenty of examples and tutorials ?
Thanks !
Edit: My course will deal with Perceptrons, Neural networks and Bayesian AI.
Really depends on what area you want to specialize on. There is the startup - resource : is
here. I learned about neural nets from their example. Can you elaborate what kind of AI it should be?
Ah and i forgot: this link is a very nice forum where you can look at problems other people have and learn from that.
Cheers.
My advice would be to learn it by trying to implement the various types of learners yourself. See if you can find yourself a dataset related to some interest you have (sports, games, health, etc.) and then try and create a learner to do some kind of classification (predicting a winner in a sports game, learning how to classify backgammon positions, detecting cancer based on patient data, etc.) using different methods. Start with Decision Trees if that's part of your future course work since they're relatively simple, then move on to neural networks.
Here is a set of sources, each one of which i recommend highly--for the quality of the explanation, for the quality of the code, and for the 'completeness' of the algorithm demo.
Least-Squares Regression
(Python)
k-means clustering (Python)
Multi-Layer Perceptron (Python)
Hopfield Network (Python)
Decision Tree (ID3 & C4.5)
In addition, the excellent textbook Elements of Statistical Learning by Hastie, et al. is actually free to download. The authors have an R package that accompanies this textbook which contains all of the code. This book includes detailed discussion of most (if not all) of the major classes of ML algorithms, with specific examples that refer to working code and 'real-world' data.
Personally I would recommend this M.Tim.Jones book on AI.
Has many many topics on AI and almost every type of AI discussion is followed by C example code. Very pragmatic book on AI indeed !!
Russel & Norvig have a good survey of the broad field.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I want to learn algorithms using some very basic simple tutorials. Are there any out there? I have heard of recursion and stuff and I would like to get good at it. Any help would be appreciated.
I would start out by taking a look at EternallyConfuzzled which contains great tutorials for basic Data Structures an Algorithms including linked lists and binary search trees, sorting and searching algorithms. If you want to learn more after this I would recommend the following books in order of increasing complexity, completeness, and required math knowledge:
Algorithms in C (also available in C++ and Java)
Introduction to Algorithms
The Art of Computer Programming
If you want to learn algorithms this book is the best choice.
(source: mcgraw-hill.com)
MIT's OCW has video lectures of their Algorithm course. The professor is one of the authors of the book Introduction to Algorithms, which another poster suggested.
It assumes a basic knowledge of Discrete Maths.
TopCoder has some good algorithm tutorials.
If you're interested in a tutorial, avoid the CLRS book recommend above. It takes a rigorous theoretical approach to the study of Algorithms, which is very different from a tutorial approach.
You learn Algorithms by doing them. So find a resource that provides Algorithms problems and guidance in solving them. If you want a textbook, check out the Algorithm Design Manual, which also has an online Algorithm Repository.
If you prefer an online course, Udacity offers a python-based Algorithms course, while Coursera offers general and Java-based ones.
Since the important part is practicing Algorithms, you can skip the video courses and just solve challenges. Other answers suggested sites with challenges you can practice once you're good at Algorithms. In the beginning you'll want more guidance, so find a resource that provides Algorithms challenges and help with solving them. I created Learneroo for this purpose. You can start by learning the fundamentals of Recursion with the Recursion Tutorial.
Recursion really isn't an algorithm. Since you don't have anything specific you're interested in I'd suggest you read wikipedia's List of alorithms or as others have suggested grab a book.
I would start at the Stony Brook Algorithm Repository. The site has some really good explanations of different types of algorithms, and it references what books and other resources it uses so you can get a taste of what's available.
I suggest that you start from sorting algorithms. Read the related wikipedia page, skip the O(n log n) stuff, and focus on the implementations of, say, insertion sort, merge sort, and quick sort. Familiarize with binary searching. Also, learn about some basic data structures, such as vectors, linked lists, stacks, their implementation, and what they are useful for. (More often than not, an algorithm to solve a problem goes together with a suitable data structure.) Once you are confident with different algorithms and data structures, you can dive in a more complete treatise such as the book by Cormen et al.
As for recursion, it is not an algorithm in itself. It is instead a technique that some algorithms employ to solve a problem, when the latter can be naturally split into subproblems. The technique of splitting a problem, solving the subproblems separately and then merging their solutions to obtain a solution for the original problem, is called "divide et impera", or "divide and conquer". (Recursion is also the related feature of most programming languages, where it basically means "functions that call themselves".)
The most cited, the most trivial, and the most useless example of a "recursive algorithm", is the one to compute factorials. Don't mind it. Instead, read about the Tower of Hanoi problem, which admits a simple and elegant recursive solution, and again, study some sorting algorithms, for many of them are indeed recursive.
To the various people who have commented that book xyz is not simple, I'd point out that algorithmics is not a simple topic. You need at least university entry level mathematics to understand the concepts plus the ability to reason about computation at a suitably abstract level. If you ever find an "Algorithmics for Dummies" book, don't waste your money!
my choice http://aduni.org/courses/algorithms/
Going through solutions in topcoder problems is a very good way to pick up algorithms. Reading theory alone won't help
Khan academy started an excellent interactive self paced course on algorithms - https://www.khanacademy.org/computing/computer-science/algorithms.
Recursion is a language feature, and less an "algorithm" per se. All recursion can be replaced with proper data structures (like a stack).
I'd recommend grabbing a book. The problem with algorithms is that it's a relatively progressive topic. You first need to learn simple searches before you can learn sorting, and you need sorting before you can do minimum spanning trees etc. A book will properly order these, and if the text doesn't give you enough information the internet is a great next step. Try Amazon and look at the comments for someone who is new.
Make sure you learn an implementation language before you try to go at this though, until you understand how the language works it's going to be very hard to pick out bugs in your logic vs a misunderstanding of what's happening for a given sequence of commands.
USA Computing Olympiad has a nice algorithms training site that so far anyone can sign up for and it's almost in a class like format. read a little, do an exercise, read more, do an exercise etc.
One of my favorite list of algorithm problems is Project Euler, they are pretty diverse and you can solve the same problem many times for optimizations, and you will find lots of communities (C++, C#, Python, ... etc) posting their benchmarks for every problem
It is so much fun, geek fun
Solve questions on various sites as SPOJ etc . and read books on Introduction to Algorithms, there are some online courses as well on coursera .
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Since Graduating from a very small school in 2006 with a badly shaped & outdated program (I'm a foreigner & didn't know any better school at the time) I've come to realize that I missed a lot of basic concepts from a mathematical & software perspective that are mostly the foundations of other higher concepts.
I.e. I tried to listen/watch the open courseware from MIT on Introduction to Algorithms but quickly realized I was missing several mathematical concepts to better understand the course.
So what are the core mathematical concepts a good software engineer should know? And what are the possible books/sites you will recommend me?
Math for Programmers. A good read.
Boolean algebra is fundamental to understanding control structures and refactoring. For example, I've seen many bugs caused by programmers who didn't know (or couldn't use) deMorgan's law. As another example, how many programmers immediately recognize that
if (condition-1) {
if (condition-2) {
action-1
} else {
action-2
} else {
action-2
}
can be rewritten as
if (condition-1 and condition-2) {
action-1
} else {
action-2
}
Discrete mathematics and combinatorics are tremendously helpful in understanding the performance of various algorithms and data structures.
As mentioned by Baltimark, mathematical induction is very useful in reasoning about loops and recursion.
Set theory is the basis of relational databases and SQL.
By way of analogy, let me point out that carpenters routinely use a variety of rule-of-thumb techniques in constructing things like roofs and stairs. However, a knowledge of geometry allows you to solve problems for which you don't have a "canned" rule of thumb. It's like learning to read via phonetics versus sight-recognition of a basic vocabulary. 90+% of the time there's not much difference. But when you run into an unfamiliar situation, it's VERY nice to have the tools to work out the solution yourself.
Finally, the rigor/precision required by mathematics is very useful preparation for programming, regardless of specific technique. Again, many of the bugs in programming (or even specifications) that I've seen in my career have sloppy thinking at their root cause.
I would go with the fields that Landon stated:
Discrete Math, Linear Algebra,
Combinatorics, Probability and
Statistics, Graph Theory
and add mathematical logic.
This would give you a grip on most fields of CS. If you want to go into special fields, you have to dive into some areas especially:
Computer graphics -> Linear Algebra
Gaming -> Linear Algebra, Physics
Computer Linguistics -> Statistics, Graph Theory
AI -> Statistics, Stochastics, Logic, Graph Theory
In order of importance:
Counting (needed for loops)
Addition, subtraction, multiplication, division.
Algebra (only really required to understand the use of variables).
Boolean algebra, boolean logic and binary.
Exponents and logarithms (i.e. understand O(n) notation).
Anything more advanced than that is usually algorithm-specific or domain-specific. Depending on which areas you are interested in, the following may also be relevant:
Linear algebra and trigonometry (3D visualization)
Discrete mathematics and set theory (database design, algorithm design, compiler design).
Statistics (well, for statistical and/or scientific/economic applications. possibly also useful for algorithm design).
Physics (for simulations).
Understanding functions is also useful (don't remember what the mathematical term is for that area), but if you know how to program you probably already do.
My point being: A ten year old should know enough mathematics to be able to understand programming. There isn't really much math required for the basic understanding of things. It's all about the logic, really.
"Proof by induction" is a core mathematical concept for programmers to know.
Big O notation in general algorithm analysis, and in relation to standard collections (sorting, retrieval insertion and deletion)
For discrete math, here is an awesome set of 20 lectures from Arsdigita University. Each is about an hour and twenty minutes long.
Start with what we CS folks call "discrete math". Calculus and linear algebra can come in quite handy too because they get your foot in the door to a lot of application domains. Once you've mastered those three, go for probability theory. Those 4 will get you to competency in 95% (I made that up) of application domains.
Concrete Mathematics covers most of the major topics. A good book on Discrete Math, like Rosen's Discrete Mathematics and Its Applications, will fill in any gaps.
I think it depends on your focus. A few years ago I purchased the set of Art of Computer Programming by Donald Knuth. After looking at the books I realized pretty much everything is calculus proofs. If you're interested in developing your own generic algorithms and proofs for them, then I recommend being able to understand the above books since its what you'd be dealing with in that world. On the other hand if you only want/need to use various sorting/searching/tree/etc... routines then big O notation at a minimum, boolean math, and general algebra will be fine. If you're dealing with 3D then geometry and trig as well.
I tend to be more on the using side than making proofs, and while I'd like to think I've done some clever things over the years I've never sat down and developed a new sorting routine. The best advice I can give is learn what you need for your field, but expose yourself to higher levels so you know it exists and how much more there is to learn, you won't get much growth otherwise.
I would say boolean logic. AND, OR, XOR, NOT.
I found as programmer we use this more often than the rest of math concepts.
Basic Algebra and Statistics are good starting points, and the foundation for a lot of other fields.
Here is a simple one that baffles me when I see developers that don't understand it:
- Order of Operations
Chapter 1 of "The Art of Computer Programming" aims to provide exactly this.
There was a book that was recommended...the title was something like Concrete Mathematics. It was recommended in a few questions.
Back in school, on of my instructors said for business applications, all you need to know know add, subtract, multiply, and divide. All other formulas the requester will know and inform you what is needed. Now realize that this is for financing reporting and application focused school. To this day, this has held true for me. I have never once needed to know more than that.
Check the book Foundations of Computer Science
This book is authored by: Al Aho and Jeff Ullman and the entire book is available online.
This is what the authors say in their Preface about the goal of this book:
"Foundations of Computer Science covers subjects that are often found split
between a discrete mathematics course and a sophomore-level sequence in computer
science in data structures. It has been our intention to select the mathematical
foundations with an eye toward what the computer user really needs, rather than
what a mathematician might choose."
a site for brushing up on Math:
http://www.khanacademy.org/
My math background is really poor (Geologist by training), but I took a discrete math class in high school and I use the concepts every day as a programmer. It is probably the most valuable class I took in all of my education as it relates to my current profession.
Discrete Math
Linear Algebra
Combinatorics
Probability and Statistics
Graph Theory
Boolean Algebra
Set Theory
Discrete Math
Well, that depends on what you goal is. As someone said, Linear Algebra, Combinatorics, Probability and Statistics and Graph Theory are important if you're into solving hard problems. Asymptotic growth of functions (bit-Oh notation) is very important. You will also need to master summations and series if you need to work on analyzing some more complex algorithms (see the appendix on Cormen&others Intro to Algorithms).
Even if you're into "Java for the enterprise" or "server-side PHP", you will find some Statistics and Algorithm Complexity (hence combinatorics, induction, summations, series, etc) useful when your boss wants you to get the server to work faster, and adding new hardware doesn't seem to help. :-) I've been through that once.
Boolean Algebra
Set Theory
Why is everybody including probability and statistics in the gold list without mentioning calculus? One cannot understand what probability and statistics are about without at least a working knowledge of limits, derivatives, integrals and series. And all in all, calculus (together with linear algebra) is the workhorse of all mathematics.
I think algorithms and theory are of great importance. Being able to come up with a fast, and correct solution is what differentiates good programmers from the rest. Also, being able to prove your algorithm (using standard proof techniques-- induction, contradiction, etc) is equally important.
Yeah, I would say a basic understanding of induction helps so that you understand what n represents in algorithms. Also some Logic and Discrete Structures is helpful.
Probability and Statistics are very helpful if you ever have to do anything resembling machine learning.
I cover the basics in my "Computing Your Skill" blog post where I discuss how Xbox Live's TrueSkill ranking and matchmaking algorithm works.