Related
As the question describe itself "What is the core difference between algorithm and pseudocode?".
algorithm
An algorithm is a procedure for solving a problem in terms of the actions to be executed and the order in which those actions are to be executed. An algorithm is merely the sequence of steps taken to solve a problem. The steps are normally "sequence," "selection, " "iteration," and a case-type statement.
Pseudocode
Pseudocode is an artificial and informal language that helps programmers develop algorithms. Pseudocode is a "text-based" detail (algorithmic) design tool.
The rules of Pseudocode are reasonably straightforward. All statements showing "dependency" are to be indented. These include while, do, for, if, switch. Examples below will illustrate this notion.
I think all the other answers give useful explanations and definitions, but I'm going to give mine.
An algorithm is the idea of how to obtain some result from some input. It is an abstract concept; an algorithm is not something material by itself, but more something like an imagination or a deduction, a thing that only exists in the mind. In the broad sense, any sequence of steps that give you some thing(s) from other thing(s) could be called an algorithm. For example, if the screen of your computer is dirty, "spraying some glass cleaner on it and wipe it with a cloth" could be said to be an algorithm to solve the problem of how to obtain a clean screen form a dirty screen. It is important to note the difference between the problem itself (getting a clean screen) and the algorithm (wiping it with a cloth and cleaner); generally, several different algorithms are possible to solve the same problem. The idea of complexity is inherent to the algorithms itself, not the problem or the particular implementation or execution of the algorithm.
Pseudocode is a language to express algorithms. Since, as said before, algorithms are only concepts, we need to use something to express them and explain them to other people. Pseudocode is a convenient way for many computer science algorithms, because it is usually unambiguous, easy to read and somewhat similar to many programming languages. However, a specific programming language like C or Java can also be used to express and algorithm (it's just less convenient to those not familiarized with that language). In other cases, pseudocode may not be the best way to express an algorithm; for example, many graph and tree algorithms can be explained more easily with drawings or diagrams. In the previous example, the algorithm to get your screen cleaned is probably better expressed in a natural language like English, because it is simple and specific enough for that case.
Obviously, terms are frequently used loosely and exchanged depending on the context, and there's no need to be nitpicky about it, but I think it is important to have the difference clear. An algorithm doesn't stop being an algorithm just because it is written in Python instead of pseudocode. Pseudocode is just a convenient and widespread communication tool to express them.
An algorithm is something (a sequence of steps) you can do. Pseudocode is a notation to describe an algorithm.
Algorithm is something which is represented in mathematical terms. It includes, analysis, complexity considerations(best, average and worstcase analysis etc.).Pseudo code is a human readable representation of a program.
From Wikipedia :
Starting from an initial state and initial input (perhaps empty), the instructions describe a computation that, when executed, proceeds through a finite number of well-defined successive states, eventually producing "output" and terminating at a final ending state.
With a pseudo language one can implement an algorithm without using a programming language such as C.
An example of pseudo language is Flow Charts.
I want to learn more about the internals of Prolog and understand how this works.
I know how to use it. But not how it works internally. What are the names of the algorithms and concepts used in Prolog?
Probably it builds some kind of tree structure or directed object graph, and then upon queries it traveres that graph with a sophisticated algorithm. A Depth First Search maybe. There might be some source code around but it would be great to read about it from a high level perspective first.
I'm really new to AI and understanding Prolog seems to be a great way to start, imho. My idea is to try to rebuild something similar and skipping the parser part completely. I need to know the directions in which I have to do my research efforts.
What are the names of the algorithms and concepts used in Prolog?
Logic programming
Depth-first, backtracking search
Unification
See Sterling & Shapiro, The Art of Prolog (MIT Press) for the theory behind Prolog.
Probably it builds some kind of tree structure or directed object graph, and then upon queries it traveres that graph with a sophisticated algorithm. A Depth First Search maybe.
It doesn't build the graph explicitly, that wouldn't even be possible with infinite search spaces. Check out the first chapters of Russell & Norvig for the concept of state-space search. Yes, it does depth-first search with backtracking, but no, that isn't very sophisticated. It's just very convenient and programming alternative search strategies isn't terribly hard in Prolog.
understanding Prolog seems to be a great way to start, imho.
Depends on what you want to do, but knowing Prolog certainly doesn't hurt. It's a very different way of looking at programming. Knowing Prolog helped me understand functional programming very quickly.
My idea is to try to rebuild something similar and skipping the parser part completely
You mean skipping the Prolog syntax? If you happen to be familiar with Scheme or Lisp, then check out section 4.4 of Abelson & Sussman where they explain how to implement a logic programming variant of Scheme, in Scheme.
AI is a wide field, Prolog only touches symbolic AI. As for Prolog, the inner workings are too complex to explain here, but googling will give you plenty of resources. E.g. http://www.amzi.com/articles/prolog_under_the_hood.htm .
Check also Wikipedia articles to learn about the other areas of AI.
You might also want to read about the Warren Abstract Machine
typically, prolog code is translated to WAM instructions and then executed more efficiently.
I would add:
Programming Languages: An interpreter based approach by Samuel N. Kamin. The book is out of print, but you may find it in a University Library. It contains a Prolog implementation in Pascal.
Tim Budd's "The Kamin Interpreters in C++" (in postscript)
The book by Sterling and Shapiro, mentioned by larsmans, actually contains an execution model of Prolog. It's quite nice and explains clearly "how Prolog works". And it's an excellent book!
There are also other sources you could try. Most notably, some Lisp books build pedagogically-oriented Prolog interpreters:
On Lisp by paul Graham (in Common Lisp, using -- and perhaps abusing -- macros)
Paradigms of Artificial Intelligence Programming by Peter Norvig (in Common Lisp)
Structure and Interpretation of Computer Programs by Abelson and Sussman (in Scheme).
Of these, the last one is the clearest (in my humble opinion). However, you'd need to learn some Lisp (either Common Lisp or Scheme) to understand those.
The ISO core standard for Prolog also contains an execution model. The execution model is of interest since it gives a good model of control constructs such as cut !/0, if-then-else (->)/2, catch/3 and throw/1. It also explains how to conformantly deal with naked variables.
The presentation in the ISO core standard is not that bad. Each control construct is described in a form of a prose use case with a reference to an abstract Prolog machine consisting of a stack, etc.. Then there are pictures that show the stack before and after execution of the control construct.
The cheapest source is ANSI:
http://webstore.ansi.org/RecordDetail.aspx?sku=INCITS%2FISO%2FIEC+13211-1-1995+%28R2007%29
In addition to the many good answers already posted, I add some historical facts on Prolog.
Wikipedia on Prolog: Prolog was created around 1972 by Alain Colmerauer with Philippe Roussel, based on Robert Kowalski's procedural interpretation of Horn clauses.
Alain was a French computer scientist and professor at Aix-Marseille University from 1970 to 1995. Retired in 2006, he remained active until he died in 2017. He was named Chevalier de la Legion d’Honneur by the French government in 1986.
The inner works of Prolog can best be explained by its inventor in this article Prolog in 10 figures. It was published in Communications of the ACM, vol. 28, num. 12, December. 1985.
Prolog uses a subset of first order predicate logic, called Horn logic. The algorithm used to derive answers is called SLD resolution.
First of all, is this only possible on algorithms which have no side effects?
Secondly, where could I learn about this process, any good books, articles, etc?
COQ is a proof assistant that produces correct ocaml output. It's pretty complicated though. I never got around to looking at it, but my coworker started and then stopped using it after two months. It was mostly because he wanted to get things done quicker, but if you need to verify an algorithm this might be a good idea.
Here is a course that uses COQ and talks about proving algorithms.
And here is a tutorial about writing academic papers in COQ.
It's generally a lot easier to verify/prove correctness when no side effects are involved, but it's not an absolute requirement.
You might want to look at some of the documentation for a formal specification language like Z. A formal specification isn't a proof itself, but is often the basis for one.
I think that verifying the correctness of an algorithm would be validating its conformance with a specification. There is a branch of theoretical Computer Science called Formal Methods which may be what you are looking for if you need to get as close to proof as you can. From wikipedia,
Formal Methods are a particular kind
of mathematically-based techniques for
the specification, development and
verification of software and hardware
systems
You will be able to find many learning resources and tools from the multitude of links on the linked Wikipedia page and from the Formal Methods wiki.
Usually proofs of correctness are very specific to the algorithm at hand.
However, there are several well known tricks that are used and re-used again. For example, with recursive algorithms you can use loop invariants.
Another common trick is reducing the original problem to a problem for which your algorithm's proof of correctness is easier to show, then either generalizing the easier problem or showing that the easier problem can be translated to a solution to the original problem. Here is a description.
If you have a particular algorithm in mind, you may do better in asking how to construct a proof for that algorithm rather than a general answer.
Buy these books: http://www.amazon.com/Science-Programming-Monographs-Computer/dp/0387964800
The Gries book, Scientific Programming is great stuff. Patient, thorough, complete.
Logic in Computer Science, by Huth and Ryan, gives a reasonably readable overview of modern systems for verifying systems. Once upon a time people talked about proving programs correct - with programming languages which may or may not have side effects. The impression I get from this book and elsewhere is that real applications are different - for instance proving that a protocol is correct, or that a chip's floating point unit can divide correctly, or that a lock-free routine for manipulating linked lists is correct.
ACM Computing Surveys Vol 41 Issue 4 (October 2009) is a special issue on software verification. It looks like you can get to at least one of the papers without an ACM account by searching for "Formal Methods: Practice and Experience".
The tool Frama-C, for which Elazar suggests a demo video in the comments, gives you a specification language, ACSL, for writing function contracts and various analyzers for verifying that a C function satisfies its contract and safety properties such as the absence of run-time errors.
An extended tutorial, ACSL by example, shows examples of actual C algorithms being specified and verified, and separates the side-effect-free functions from the effectful ones (the side-effect-free ones are considered easier and come first in the tutorial). This document is also interesting in that it was not written by the designers of the tools it describe, so it gives a fresher and more didactic look at these techniques.
If you are familiar with LISP then you should definitely check out ACL2: http://www.cs.utexas.edu/~moore/acl2/acl2-doc.html
Dijkstra's Discipline of Programming and his EWDs lay the foundation for formal verification as a science in programming. A simpler work is Wirth's Systematic Programming, which begins with the simple approach to using verification. Wirth uses pre-ISO Pascal for the language; Dijkstra uses an Algol-68-like formalism called Guarded (GCL). Formal verification has matured since Dijkstra and Hoare, but these older texts may still be a good starting point.
PVS tool developed by Stanford guys is a specification and verification system. I worked on it and found it very useful for Theoram Proving.
WRT (1), you will probably have to create a model of the algorithm in a way that "captures" the side-effects of the algorithm in a program variable intended to model such state-based side-effects.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 6 years ago.
Improve this question
I want to learn algorithms using some very basic simple tutorials. Are there any out there? I have heard of recursion and stuff and I would like to get good at it. Any help would be appreciated.
I would start out by taking a look at EternallyConfuzzled which contains great tutorials for basic Data Structures an Algorithms including linked lists and binary search trees, sorting and searching algorithms. If you want to learn more after this I would recommend the following books in order of increasing complexity, completeness, and required math knowledge:
Algorithms in C (also available in C++ and Java)
Introduction to Algorithms
The Art of Computer Programming
If you want to learn algorithms this book is the best choice.
(source: mcgraw-hill.com)
MIT's OCW has video lectures of their Algorithm course. The professor is one of the authors of the book Introduction to Algorithms, which another poster suggested.
It assumes a basic knowledge of Discrete Maths.
TopCoder has some good algorithm tutorials.
If you're interested in a tutorial, avoid the CLRS book recommend above. It takes a rigorous theoretical approach to the study of Algorithms, which is very different from a tutorial approach.
You learn Algorithms by doing them. So find a resource that provides Algorithms problems and guidance in solving them. If you want a textbook, check out the Algorithm Design Manual, which also has an online Algorithm Repository.
If you prefer an online course, Udacity offers a python-based Algorithms course, while Coursera offers general and Java-based ones.
Since the important part is practicing Algorithms, you can skip the video courses and just solve challenges. Other answers suggested sites with challenges you can practice once you're good at Algorithms. In the beginning you'll want more guidance, so find a resource that provides Algorithms challenges and help with solving them. I created Learneroo for this purpose. You can start by learning the fundamentals of Recursion with the Recursion Tutorial.
Recursion really isn't an algorithm. Since you don't have anything specific you're interested in I'd suggest you read wikipedia's List of alorithms or as others have suggested grab a book.
I would start at the Stony Brook Algorithm Repository. The site has some really good explanations of different types of algorithms, and it references what books and other resources it uses so you can get a taste of what's available.
I suggest that you start from sorting algorithms. Read the related wikipedia page, skip the O(n log n) stuff, and focus on the implementations of, say, insertion sort, merge sort, and quick sort. Familiarize with binary searching. Also, learn about some basic data structures, such as vectors, linked lists, stacks, their implementation, and what they are useful for. (More often than not, an algorithm to solve a problem goes together with a suitable data structure.) Once you are confident with different algorithms and data structures, you can dive in a more complete treatise such as the book by Cormen et al.
As for recursion, it is not an algorithm in itself. It is instead a technique that some algorithms employ to solve a problem, when the latter can be naturally split into subproblems. The technique of splitting a problem, solving the subproblems separately and then merging their solutions to obtain a solution for the original problem, is called "divide et impera", or "divide and conquer". (Recursion is also the related feature of most programming languages, where it basically means "functions that call themselves".)
The most cited, the most trivial, and the most useless example of a "recursive algorithm", is the one to compute factorials. Don't mind it. Instead, read about the Tower of Hanoi problem, which admits a simple and elegant recursive solution, and again, study some sorting algorithms, for many of them are indeed recursive.
To the various people who have commented that book xyz is not simple, I'd point out that algorithmics is not a simple topic. You need at least university entry level mathematics to understand the concepts plus the ability to reason about computation at a suitably abstract level. If you ever find an "Algorithmics for Dummies" book, don't waste your money!
my choice http://aduni.org/courses/algorithms/
Going through solutions in topcoder problems is a very good way to pick up algorithms. Reading theory alone won't help
Khan academy started an excellent interactive self paced course on algorithms - https://www.khanacademy.org/computing/computer-science/algorithms.
Recursion is a language feature, and less an "algorithm" per se. All recursion can be replaced with proper data structures (like a stack).
I'd recommend grabbing a book. The problem with algorithms is that it's a relatively progressive topic. You first need to learn simple searches before you can learn sorting, and you need sorting before you can do minimum spanning trees etc. A book will properly order these, and if the text doesn't give you enough information the internet is a great next step. Try Amazon and look at the comments for someone who is new.
Make sure you learn an implementation language before you try to go at this though, until you understand how the language works it's going to be very hard to pick out bugs in your logic vs a misunderstanding of what's happening for a given sequence of commands.
USA Computing Olympiad has a nice algorithms training site that so far anyone can sign up for and it's almost in a class like format. read a little, do an exercise, read more, do an exercise etc.
One of my favorite list of algorithm problems is Project Euler, they are pretty diverse and you can solve the same problem many times for optimizations, and you will find lots of communities (C++, C#, Python, ... etc) posting their benchmarks for every problem
It is so much fun, geek fun
Solve questions on various sites as SPOJ etc . and read books on Introduction to Algorithms, there are some online courses as well on coursera .
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Since Graduating from a very small school in 2006 with a badly shaped & outdated program (I'm a foreigner & didn't know any better school at the time) I've come to realize that I missed a lot of basic concepts from a mathematical & software perspective that are mostly the foundations of other higher concepts.
I.e. I tried to listen/watch the open courseware from MIT on Introduction to Algorithms but quickly realized I was missing several mathematical concepts to better understand the course.
So what are the core mathematical concepts a good software engineer should know? And what are the possible books/sites you will recommend me?
Math for Programmers. A good read.
Boolean algebra is fundamental to understanding control structures and refactoring. For example, I've seen many bugs caused by programmers who didn't know (or couldn't use) deMorgan's law. As another example, how many programmers immediately recognize that
if (condition-1) {
if (condition-2) {
action-1
} else {
action-2
} else {
action-2
}
can be rewritten as
if (condition-1 and condition-2) {
action-1
} else {
action-2
}
Discrete mathematics and combinatorics are tremendously helpful in understanding the performance of various algorithms and data structures.
As mentioned by Baltimark, mathematical induction is very useful in reasoning about loops and recursion.
Set theory is the basis of relational databases and SQL.
By way of analogy, let me point out that carpenters routinely use a variety of rule-of-thumb techniques in constructing things like roofs and stairs. However, a knowledge of geometry allows you to solve problems for which you don't have a "canned" rule of thumb. It's like learning to read via phonetics versus sight-recognition of a basic vocabulary. 90+% of the time there's not much difference. But when you run into an unfamiliar situation, it's VERY nice to have the tools to work out the solution yourself.
Finally, the rigor/precision required by mathematics is very useful preparation for programming, regardless of specific technique. Again, many of the bugs in programming (or even specifications) that I've seen in my career have sloppy thinking at their root cause.
I would go with the fields that Landon stated:
Discrete Math, Linear Algebra,
Combinatorics, Probability and
Statistics, Graph Theory
and add mathematical logic.
This would give you a grip on most fields of CS. If you want to go into special fields, you have to dive into some areas especially:
Computer graphics -> Linear Algebra
Gaming -> Linear Algebra, Physics
Computer Linguistics -> Statistics, Graph Theory
AI -> Statistics, Stochastics, Logic, Graph Theory
In order of importance:
Counting (needed for loops)
Addition, subtraction, multiplication, division.
Algebra (only really required to understand the use of variables).
Boolean algebra, boolean logic and binary.
Exponents and logarithms (i.e. understand O(n) notation).
Anything more advanced than that is usually algorithm-specific or domain-specific. Depending on which areas you are interested in, the following may also be relevant:
Linear algebra and trigonometry (3D visualization)
Discrete mathematics and set theory (database design, algorithm design, compiler design).
Statistics (well, for statistical and/or scientific/economic applications. possibly also useful for algorithm design).
Physics (for simulations).
Understanding functions is also useful (don't remember what the mathematical term is for that area), but if you know how to program you probably already do.
My point being: A ten year old should know enough mathematics to be able to understand programming. There isn't really much math required for the basic understanding of things. It's all about the logic, really.
"Proof by induction" is a core mathematical concept for programmers to know.
Big O notation in general algorithm analysis, and in relation to standard collections (sorting, retrieval insertion and deletion)
For discrete math, here is an awesome set of 20 lectures from Arsdigita University. Each is about an hour and twenty minutes long.
Start with what we CS folks call "discrete math". Calculus and linear algebra can come in quite handy too because they get your foot in the door to a lot of application domains. Once you've mastered those three, go for probability theory. Those 4 will get you to competency in 95% (I made that up) of application domains.
Concrete Mathematics covers most of the major topics. A good book on Discrete Math, like Rosen's Discrete Mathematics and Its Applications, will fill in any gaps.
I think it depends on your focus. A few years ago I purchased the set of Art of Computer Programming by Donald Knuth. After looking at the books I realized pretty much everything is calculus proofs. If you're interested in developing your own generic algorithms and proofs for them, then I recommend being able to understand the above books since its what you'd be dealing with in that world. On the other hand if you only want/need to use various sorting/searching/tree/etc... routines then big O notation at a minimum, boolean math, and general algebra will be fine. If you're dealing with 3D then geometry and trig as well.
I tend to be more on the using side than making proofs, and while I'd like to think I've done some clever things over the years I've never sat down and developed a new sorting routine. The best advice I can give is learn what you need for your field, but expose yourself to higher levels so you know it exists and how much more there is to learn, you won't get much growth otherwise.
I would say boolean logic. AND, OR, XOR, NOT.
I found as programmer we use this more often than the rest of math concepts.
Basic Algebra and Statistics are good starting points, and the foundation for a lot of other fields.
Here is a simple one that baffles me when I see developers that don't understand it:
- Order of Operations
Chapter 1 of "The Art of Computer Programming" aims to provide exactly this.
There was a book that was recommended...the title was something like Concrete Mathematics. It was recommended in a few questions.
Back in school, on of my instructors said for business applications, all you need to know know add, subtract, multiply, and divide. All other formulas the requester will know and inform you what is needed. Now realize that this is for financing reporting and application focused school. To this day, this has held true for me. I have never once needed to know more than that.
Check the book Foundations of Computer Science
This book is authored by: Al Aho and Jeff Ullman and the entire book is available online.
This is what the authors say in their Preface about the goal of this book:
"Foundations of Computer Science covers subjects that are often found split
between a discrete mathematics course and a sophomore-level sequence in computer
science in data structures. It has been our intention to select the mathematical
foundations with an eye toward what the computer user really needs, rather than
what a mathematician might choose."
a site for brushing up on Math:
http://www.khanacademy.org/
My math background is really poor (Geologist by training), but I took a discrete math class in high school and I use the concepts every day as a programmer. It is probably the most valuable class I took in all of my education as it relates to my current profession.
Discrete Math
Linear Algebra
Combinatorics
Probability and Statistics
Graph Theory
Boolean Algebra
Set Theory
Discrete Math
Well, that depends on what you goal is. As someone said, Linear Algebra, Combinatorics, Probability and Statistics and Graph Theory are important if you're into solving hard problems. Asymptotic growth of functions (bit-Oh notation) is very important. You will also need to master summations and series if you need to work on analyzing some more complex algorithms (see the appendix on Cormen&others Intro to Algorithms).
Even if you're into "Java for the enterprise" or "server-side PHP", you will find some Statistics and Algorithm Complexity (hence combinatorics, induction, summations, series, etc) useful when your boss wants you to get the server to work faster, and adding new hardware doesn't seem to help. :-) I've been through that once.
Boolean Algebra
Set Theory
Why is everybody including probability and statistics in the gold list without mentioning calculus? One cannot understand what probability and statistics are about without at least a working knowledge of limits, derivatives, integrals and series. And all in all, calculus (together with linear algebra) is the workhorse of all mathematics.
I think algorithms and theory are of great importance. Being able to come up with a fast, and correct solution is what differentiates good programmers from the rest. Also, being able to prove your algorithm (using standard proof techniques-- induction, contradiction, etc) is equally important.
Yeah, I would say a basic understanding of induction helps so that you understand what n represents in algorithms. Also some Logic and Discrete Structures is helpful.
Probability and Statistics are very helpful if you ever have to do anything resembling machine learning.
I cover the basics in my "Computing Your Skill" blog post where I discuss how Xbox Live's TrueSkill ranking and matchmaking algorithm works.