Related
Evolutionary programming seems to be a great way to solve many optimization problems. The idea is very easy and the implementation does not make problems.
I was wondering if there is any way to evolutionarily create a program in ruby/python script (or any other language)?
The idea is simple:
Create a population of programs
Perform genetic operations (roulette-wheel selection or any other selection), create new programs with inheritance from best programs, etc.
Loop point 2 until program that will satisfy our condition is found
But there are still few problems:
How will chromosomes be represented? For example, should one cell of chromosome be one line of code?
How will chromosomes be generated? If they will be lines of code, how do we generate them to ensure that they are syntactically correct, etc.?
Example of a program that could be generated:
Create script that takes N numbers as input and returns their mean as output.
If there were any attempts to create such algorithms I'll be glad to see any links/sources.
If you are sure you want to do this, you want genetic programming, rather than a genetic algorithm. GP allows you to evolve tree-structured programs. What you would do would be to give it a bunch of primitive operations (while($register), read($register), increment($register), decrement($register), divide($result $numerator $denominator), print, progn2 (this is GP speak for "execute two commands sequentially")).
You could produce something like this:
progn2(
progn2(
read($1)
while($1
progn2(
while($1
progn2( #add the input to the total
increment($2)
decrement($1)
)
)
progn2( #increment number of values entered, read again
increment($3)
read($1)
)
)
)
)
progn2( #calculate result
divide($1 $2 $3)
print($1)
)
)
You would use, as your fitness function, how close it is to the real solution. And therein lies the catch, that you have to calculate that traditionally anyway*. And then have something that translates that into code in (your language of choice). Note that, as you've got a potential infinite loop in there, you'll have to cut off execution after a while (there's no way around the halting problem), and it probably won't work. Shucks. Note also, that my provided code will attempt to divide by zero.
*There are ways around this, but generally not terribly far around it.
It can be done, but works very badly for most kinds of applications.
Genetic algorithms only work when the fitness function is continuous, i.e. you can determine which candidates in your current population are closer to the solution than others, because only then you'll get improvements from one generation to the next. I learned this the hard way when I had a genetic algorithm with one strongly-weighted non-continuous component in my fitness function. It dominated all others and because it was non-continuous, there was no gradual advancement towards greater fitness because candidates that were almost correct in that aspect were not considered more fit than ones that were completely incorrect.
Unfortunately, program correctness is utterly non-continuous. Is a program that stops with error X on line A better than one that stops with error Y on line B? Your program could be one character away from being correct, and still abort with an error, while one that returns a constant hardcoded result can at least pass one test.
And that's not even touching on the matter of the code itself being non-continuous under modifications...
Well this is very possible and #Jivlain correctly points out in his (nice) answer that genetic Programming is what you are looking for (and not simple Genetic Algorithms).
Genetic Programming is a field that has not reached a broad audience yet, partially because of some of the complications #MichaelBorgwardt indicates in his answer. But those are mere complications, it is far from true that this is impossible to do. Research on the topic has been going on for more than 20 years.
Andre Koza is one of the leading researchers on this (have a look at his 1992 work) and he demonstrated as early as 1996 how genetic programming can in some cases outperform naive GAs on some classic computational problems (such as evolving programs for Cellular Automata synchronization).
Here's a good Genetic Programming tutorial from Koza and Poli dated 2003.
For a recent reference you might wanna have a look at A field guide to genetic programming (2008).
Since this question was asked, the field of genetic programming has advanced a bit, and there have been some additional attempts to evolve code in configurations other than the tree structures of traditional genetic programming. Here are just a few of them:
PushGP - designed with the goal of evolving modular functions like human coders use, programs in this system store all variables and code on different stacks (one for each variable type). Programs are written by pushing and popping commands and data off of the stacks.
FINCH - a system that evolves Java byte-code. This has been used to great effect to evolve game-playing agents.
Various algorithms have started evolving C++ code, often with a step in which compiler errors are corrected. This has had mixed, but not altogether unpromising results. Here's an example.
Avida - a system in which agents evolve programs (mostly boolean logic tasks) using a very simple assembly code. Based off of the older (and less versatile) Tierra.
The language isn't an issue. Regardless of the language, you have to define some higher-level of mutation, otherwise it will take forever to learn.
For example, since any Ruby language can be defined in terms of a text string, you could just randomly generate text strings and optimize that. Better would be to generate only legal Ruby programs. However, it would also take forever.
If you were trying to build a sorting program and you had high level operations like "swap", "move", etc. then you would have a much higher chance of success.
In theory, a bunch of monkeys banging on a typewriter for an infinite amount of time will output all the works of Shakespeare. In practice, it isn't a practical way to write literature. Just because genetic algorithms can solve optimization problems doesn't mean that it's easy or even necessarily a good way to do it.
The biggest selling point of genetic algorithms, as you say, is that they are dirt simple. They don't have the best performance or mathematical background, but even if you have no idea how to solve your problem, as long as you can define it as an optimization problem you will be able to turn it into a GA.
Programs aren't really suited for GA's precisely because code isn't good chromossome material. I have seen someone who did something similar with (simpler) machine code instead of Python (although it was more of an ecossystem simulation then a GA per se) and you might have better luck if you codify your programs using automata / LISP or something like that.
On the other hand, given how alluring GA's are and how basically everyone who looks at them asks this same question, I'm pretty sure there are already people who tried this somewhere - I just have no idea if any of them succeeded.
Good luck with that.
Sure, you could write a "mutation" program that reads a program and randomly adds, deletes, or changes some number of characters. Then you could compile the result and see if the output is better than the original program. (However we define and measure "better".) Of course 99.9% of the time the result would be compile errors: syntax errors, undefined variables, etc. And surely most of the rest would be wildly incorrect.
Try some very simple problem. Say, start with a program that reads in two numbers, adds them together, and outputs the sum. Let's say that the goal is a program that reads in three numbers and calculates the sum. Just how long and complex such a program would be of course depends on the language. Let's say we have some very high level language that lets us read or write a number with just one line of code. Then the starting program is just 4 lines:
read x
read y
total=x+y
write total
The simplest program to meet the desired goal would be something like
read x
read y
read z
total=x+y+z
write total
So through a random mutation, we have to add "read z" and "+z", a total of 9 characters including the space and the new-line. Let's make it easy on our mutation program and say it always inserts exactly 9 random characters, that they're guaranteed to be in the right places, and that it chooses from a character set of just 26 letters plus 10 digits plus 14 special characters = 50 characters. What are the odds that it will pick the correct 9 characters? 1 in 50^9 = 1 in 2.0e15. (Okay, the program would work if instead of "read z" and "+z" it inserted "read w" and "+w", but then I'm making it easy by assuming it magically inserts exactly the right number of characters and always inserts them in the right places. So I think this estimate is still generous.)
1 in 2.0e15 is a pretty small probability. Even if the program runs a thousand times a second, and you can test the output that quickly, the chance is still just 1 in 2.0e12 per second, or 1 in 5.4e8 per hour, 1 in 2.3e7 per day. Keep it running for a year and the chance of success is still only 1 in 62,000.
Even a moderately competent programmer should be able to make such a change in, what, ten minutes?
Note that changes must come in at least "packets" that are correct. That is, if a mutation generates "reax z", that's only one character away from "read z", but it would still produce compile errors, and so would fail.
Likewise adding "read z" but changing the calculation to "total=x+y+w" is not going to work. Depending on the language, you'll either get errors for the undefined variable or at best it will have some default value, like zero, and give incorrect results.
You could, I suppose, theorize incremental solutions. Maybe one mutation adds the new read statement, then a future mutation updates the calculation. But without the calculation, the additional read is worthless. How will the program be evaluated to determine that the additional read is "a step in the right direction"? The only way I see to do that is to have an intelligent being read the code after each mutation and see if the change is making progress toward the desired goal. And if you have an intelligent designer who can do that, that must mean that he knows what the desired goal is and how to achieve it. At which point, it would be far more efficient to just make the desired change rather than waiting for it to happen randomly.
And this is an exceedingly trivial program in a very easy language. Most programs are, what, hundreds or thousands of lines, all of which must work together. The odds against any random process writing a working program are astronomical.
There might be ways to do something that resembles this in some very specialized application, where you are not really making random mutations, but rather making incremental modifications to the parameters of a solution. Like, we have a formula with some constants whose values we don't know. We know what the correct results are for some small set of inputs. So we make random changes to the constants, and if the result is closer to the right answer, change from there, if not, go back to the previous value. But even at that, I think it would rarely be productive to make random changes. It would likely be more helpful to try changing the constants according to a strict formula, like start with changing by 1000's, then 100's then 10's, etc.
I want to just give you a suggestion. I don't know how successful you'd be, but perhaps you could try to evolve a core war bot with genetic programming. Your fitness function is easy: just let the bots compete in a game. You could start with well known bots and perhaps a few random ones then wait and see what happens.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 10 years ago.
First of all, sorry, but my English is not very good.
I'm at the third semester at a programming course and I can't "get it".
I passed previous exams because I've studied three times the amount my colleagues did. And, now in my Data Structures class when the professor asks us to make list algorithms ( for example ), my colleagues just start to write the code while me ( or would be "I"? ), even after seeing it done, can't understand.
I find it so hard it's painful. It takes me 1 hour to understand simple algorithms.
So the question is: can anyone recommend me books or something, that will help me think more like a programmer? I mean, "normal" people, after a given problem, seem to immediately make the picture in their head and star to write the code.
you might want to take a philosophy class on intro to logic. it's exactly the same thing as you're doing with computers, but in a different context.
I think there are two ways to learn to write algorithms:
1. Try it yourself (and then review it with the "right" answer). Of course you should start with the simple ones.
2. Implement other's algorithms (to code). Going from algorithms to code and the other way around gives you a great "feeling" of how algorithms are created.
But the best advice I can give you (which you can use the two ways above to acquire) - try being very good at some algorithms.
If you could remember, write and really understand even just a few basic algorithms - you're on the right way.
Anyways, I think learning algorithms is mainly practice and less "abstract knowledge", so get ready to get your hands dirty..
Best of luck!!
I found that the most important skill for mastering data structures is to "visualize" them. If you have no clear picture of your data structure, everything stays fuzzy and vague. E.g. take a stack (single linked list). It may help to visualize it as a stack of plates or people in a polonaise (everyone has the hands on the shoulders of the person in front of him).
Or a double linked list: Imagine a row of people grabbing the belt of the left and right neighbor. Now you can imagine what you must do to remove one person: Its left neighbor needs to grab the belt of the right neighbor and the right neighbor the one of the left neighbor. Then you can "delete" that person (or the garbage collector does this for you in Java etc)
The second important skill is to understand recursion. You need always a base case, usually involving an empty list or empty node. When you follow the algorithm in the recursive method, check that it is correct, but don't follow recursive calls inside that method. That will drive you mad and lead to nothing. Just assume that recursive calls always "do the right thing". If your current method is correct and the base case as well, you are done. Once you understood this, you understand recursion.
If it helps, you might want to try books in your native language. If there is any gap in your understanding that comes from your English comprehension, that would add additional difficulty for you.
Second, an algorithm, is just a list of steps for achieving something. Try to focus on something simple like sorting algorithms, they are fun and easy to learn. Hope that helps.
I do not know how far this problem is concerned, but it might help.
Algorithms and data structures are way cleaner for me if I think of them just as informal ideas/boxes/shapes/forms/connections/... instead of abstract things.
That is, everytime I come across a new data structure I try to visualize it somehow in order to actually beeing able to "see" what the single operations do with the structure.
I had also sometimes the problem that I didn't actually know instantly how to solve something algorithmically, so I just went ahead and started drawing.
Let's take - as an example - the problem of converting a binary tree into a list. So, I draw a tree (of arbitrary size and holding arbitrary elements, but beeing a "good" example). Then I think how I would convert the tree into a list manually:
1
2 3
4 5 6 7
So I think: I want to have the result [4,2,5,1,6,3,7]. How can I do this step-by-step?
Ok, first of all, I find the leftmost element (4); how can I do this? Ok, I start just by the root and go left until there's nothing more. Then I remove this element and go on with the remaining tree:
1
2 3
. 5 6 7
Ok, now I would pick the 2. How can I reach the 2? Ok, I can either start again at the root and go left until there's nothing more or i just go back from the deleted node.
Then I go on and look for recurring patterns, how one can generalize steps etc.
The outcome is, it might often be helpful to have a visual representation of what the algorithm is doing and then you can implement it.
I think it is a good practice to use diagrams before implementing anything. You can make diagrams or write a pseudo code, they are helpful before you start to design or implement the actual code.
I had a similar problem when I was first introduced to programming languages. I missed a lot of lectures because it was my first year of college! For me there was no books or lecturers that I found that knew how to help you think like a programmer. I always found the people teaching didn't know how 'not' to think like a programmer anymore and as a result they assume you know the simple concepts. So finally by the end of my first year I had to cram majorly to catch up and and had to fill in the gaps myself...! This is how I think about programming problems now:
OBJECTS: For Object-Oriented programming objects are the key to the whole thing. If you think about what it is your program needs to be able to do then you can break up the program into smaller chunks. For example, if you imagine making a cup of tea, the objects you need to make the cup of tea are:
1 -> A cup
2 -> A tea bag
3 -> Water
4 -> A kettle
5 -> A spoon
6 -> Milk
7 -> Sugar
So now your program has 7 objects which will interact in some way to make a cup of tea. Objects are always declared as their own class and will have constructor methods which when called will create a copy (instantiation) of your object that can then be used in your program. All of the method that are inside your class will define what functionality your object can provide.
Kettle kettle = new Kettle();
kettle.boilWater();
So now that you have your objects you should think about your algorithm.
ALGORITHMS: In all programming languages an algorithm is basically just a list of steps that you take that will help you achieve your end goal. In our case our end goal is to make a cup of tea.
The steps you will take in your algorithm have to come one after the other in a logical fashion i.e. You cannot pour the milk into the kettle, or pour cold water into the cup and boil the sugar etc.
So our algorithm can be as follows:
Step 1: Pour water into Kettle
Step 2: Turn kettle on - to boil the water
Step 3: Put tea-bag into cup
Step 4: "IF" water is boiled -> pour into cup
"ELSE" wait until water has boiled
Step 5: Stir teabag with spoon
Step 6: Pour milk into cup
Step 7: Put sugar into cup
Step 8: Stir
There are always a few different ways you can arrange the steps in an algorithms that will still work but always remember to have a logical order or else you will make a mess!!
The same principle can be applied to even the most complex problems. The most important thing to do is try to break the problem down into the simplest steps and arrange the steps in a common sense way.
When it comes to more complex tasks it is obviously very important to know what tools you have available to you i.e. know what functionality APIs provide you with and be familiar with the syntax. But as people have mentioned to you already before practice makes perfect. It is the only way that you will begin to understand it and believe me you will get it eventually... One day it WILL all make sense to you it is just about thinking a certain way. Break everything down to small simple steps and then order the steps in a logical way. Do this and it will start to make sense to you. I PROMISE!!
If you want to jump in the deep end and are willing to work hard, there is "Structure and Interpretation of Computer Programs". It is out of print but available on the net.
It goes deep into conceptual thinking processes behind programming and converting them to code. It uses scheme, but the principles are applicable everywhere and it is great training for flexing the abstraction muscles.
Don't do it in one go... take your time with it, come back regularly to it, and have fun!
Try to think about how you approach everyday cognitive problems, and formalize the steps you take to solve them, on paper.
For example, if you've ever had to reorder a deck of 52 jumbled cards, you probably know Insertion sort or some variant of Merge Sort already. If you're asked to sort five 2-digit numbers in your head, you're probably using Selection Sort. If you've had to find a specific card from a jumbled deck, you probably used Linear Search. If you've ever seen the 'High Low Game' on 'The Price is Right', you probably know Binary Search. If you're asked to solve the 8-Queens problem as a 'puzzle', you will almost certainly use the classic backtracking technique.
Don't let jargon like 'loop invariant' or 'asymptotic time complexity' intimidate you straight away. Think about how an algorithm is dealing with the problem, and you will find yourself 'rediscovering' these terms yourself when you want to reason about its correctness or efficiency.
I'd suggest you could try out some of the websites which contain a number of algorithmic problems to solve. Usually you could find a number of very easy problems to start with, and forums to discuss them. Then it's up to you and how much time you spend on them. Just try to go into as much detail as possible and question yourself what exactly is it that you don't understand, then how could you figure that out (maybe someone already had figured that out and you just need to find the answer). For the easiest problems Google is more than enough to find the answers you're looking for.
And here is the list of websites you could try:
uva.onlinejudge.org - Huge list of problems related to different algorithms.
www.topcoder.com/tc - Live competitions and lots of tutorials to get started.
www.algorithmist.com - Constains a list of links collected over time by problem-solvers.
The other day I thought I'd attempt creating the Fibonacci algorithm in my code, but I've never been good at maths.
I ended up writing my own method with a loop but it seemed inefficient or not 'the proper way'.
Does anyone have any recommendations/reading material on implementing algorithms in code?
I find Project Euler useful for this kind of thing. It forces you to think about an algorithm and then implement it. Many of the questions then have extensive discussions on how to solve the problem (from the naive solutions to some pretty ingenious ones) that you can use to see what you did right and wrong.
In the discussion threads you'll find various implementations from other people in many different languages too. Coming up with a solution yourself and then comparing it to that from other people is (imho) a good way to learn.
Both of these introductory books have good information about this sort of thing:
How To Design Programs and moreso Structure and Interpretation of Computer Programs
Both are somewhat funcitonal (and scheme) oriented, but that's a natural fit for these sorts of problems.
On top of that, you might get quite a bit out of Project Euler
Derive your algorithm test-driven. I've been able to write much more complex algorithms correctly by using TDD than I was before.
Go on youtube and look at some of the lectures on Introduction to Algorithms. There are some really, really good lectures that break down some of the most common algorithms such as the Fibonacci series and how to optimize them.
Start reading about O notation so you can understand how your algorithm grows with variable size input and how to classifiy the run-time of the algorithm you have.
Start with this video series which I found excellent material on the subject:
Algorithms Lecture
If you can't translate pseudo code for a fibonacci function to your language, then you should go and find a basic tutorial for your language, since it seems that you have not yet grasped its basic idioms.
If you have a working solution, but feel insecure about it, show it to others for review.
Back in college, only the use of pseudo code was evangelized more than OOP in my curriculum. Just like commenting (and other preached 'best practices'), I found that in crunch time psuedocode was often neglected. So my question is...who actually uses it a lot of the time? Or do you only use it when an algorithm is really hard to conceptualize entirely in your head? I'm interested in responses from everyone: wet-behind-the-ears junior developers to grizzled vets who were around back in the punch card days.
As for me personally, I mostly only use it for the difficult stuff.
I use it all the time. Any time I have to explain a design decision, I'll use it. Talking to non-technical staff, I'll use it. It has application not only for programming, but for explaining how anything is done.
Working with a team on multiple platforms (Java front-end with a COBOL backend, in this case) it's much easier to explain how a bit of code works using pseudocode than it is to show real code.
During design stage, pseudocode is especially useful because it helps you see the solution and whether or not it's feasible. I've seen some designs that looked very elegant, only to try to implement them and realize I couldn't even generate pseudocode. Turned out, the designer had never tried thinking about a theoretical implementation. Had he tried to write up some pseudocode representing his solution, I never would have had to waste 2 weeks trying to figure out why I couldn't get it to work.
I use pseudocode when away from a computer and only have paper and pen. It doesn't make much sense to worry about syntax for code that won't compile (can't compile paper).
I almost always use it nowadays when creating any non-trivial routines. I create the pseudo code as comments, and continue to expand it until I get to the point that I can just write the equivalent code below it. I have found this significantly speeds up development, reduces the "just write code" syndrome that often requires rewrites for things that weren't originally considered as it forces you to think through the entire process before writing actual code, and serves as good base for code documentation after it is written.
I and the other developers on my team use it all the time. In emails, whiteboard, or just in confersation. Psuedocode is tought to help you think the way you need to, to be able to program. If you really unstand psuedocode you can catch on to almost any programming language because the main difference between them all is syntax.
If I'm working out something complex, I use it a lot, but I use it as comments. For instance, I'll stub out the procedure, and put in each step I think I need to do. As I then write the code, I'll leave the comments: it says what I was trying to do.
procedure GetTextFromValidIndex (input int indexValue, output string textValue)
// initialize
// check to see if indexValue is within the acceptable range
// get min, max from db
// if indexValuenot between min and max
// then return with an error
// find corresponding text in db based on indexValue
// return textValue
return "Not Written";
end procedure;
I've never, not even once, needed to write the pseudocode of a program before writing it.
However, occasionally I've had to write pseudocode after writing code, which usually happens when I'm trying to describe the high-level implementation of a program to get someone up to speed with new code in a short amount of time. And by "high-level implementation", I mean one line of pseudocode describes 50 or so lines of C#, for example:
Core dumps a bunch of XML files to a folder and runs the process.exe
executable with a few commandline parameters.
The process.exe reads each file
Each file is read line by line
Unique words are pulled out of the file stored in a database
File is deleted when its finished processing
That kind of pseudocode is good enough to describe roughly 1000 lines of code, and good enough to accurately inform a newbie what the program is actually doing.
On many occasions when I don't know how to solve a problem, I actually find myself drawing my modules on a whiteboard in very high level terms to get a clear picture of how their interacting, drawing a prototype of a database schema, drawing a datastructure (especially trees, graphs, arrays, etc) to get a good handle on how to traverse and process it, etc.
I use it when explaining concepts. It helps to trim out the unnecessary bits of language so that examples only have the details pertinent to the question being asked.
I use it a fair amount on StackOverflow.
I don't use pseudocode as it is taught in school, and haven't in a very long time.
I do use english descriptions of algorithms when the logic is complex enough to warrant it; they're called "comments". ;-)
when explaining things to others, or working things out on paper, i use diagrams as much as possible - the simpler the better
Steve McConnel's Code Complete, in its chapter 9, "The Pseudocode Programming Process" proposes an interesting approach: when writing a function longer than a few lines, use simple pseudocode (in the form of comments) to outline what the function/procedure needs to do before writing the actual code that does it. The pseudocode comments can then become actual comments in the body of the function.
I tend to use this for any function that does more than what can be quickly understood by looking at a screenful (max) of code. It works specially well if you are already used to separate your function body in code "paragraphs" - units of semantically related code separated by a blank line. Then the "pseudocode comments" work like "headers" to these paragraphs.
PS: Some people may argue that "you shouldn't comment what, but why, and only when it's not trivial to understand for a reader who knows the language in question better then you". I generally agree with this, but I do make an exception for the PPP. The criteria for the presence and form of a comment shouldn't be set in stone, but ultimately governed by wise, well-thought application of common sense anyway. If you find yourself refusing to try out a slight bent to a subjective "rule" just for the sake of it, you might need to step back and realize if you're not facing it critically enough.
Mostly use it for nutting out really complex code, or when explaining code to either other developers or non developers who understand the system.
I also flow diagrams or uml type diagrams when trying to do above also...
I generally use it when developing multiple if else statements that are nested which can be confusing.
This way I don't need to go back and document it since its already been done.
Fairly rarely, although I often document a method before writing the body of it.
However, If I'm helping another developer with how to approach a problem, I'll often write an email with a pseudocode solution.
I don't use pseudocode at all.
I'm more comfortable with the syntax of C style languages than I am with Pseudocode.
What I do do quite frequently for design purposes is essentially a functional decomposition style of coding.
public void doBigJob( params )
{
doTask1( params);
doTask2( params);
doTask3( params);
}
private void doTask1( params)
{
doSubTask1_1(params);
...
}
Which, in an ideal world, would eventually turn into working code as methods become more and more trivial. However, in real life, there is a heck of a lot of refactoring and rethinking of design.
We find this works well enough, as rarely do we come across an algorithm that is both: Incredibly complex and hard to code and not better solved using UML or other modelling technique.
I never use or used it.
I always try to prototype in a real language when I need to do something complex, usually writting unit tests first to figure out what the code needs to do.
It's difficult to tell what is being asked here. This question is ambiguous, vague, incomplete, overly broad, or rhetorical and cannot be reasonably answered in its current form. For help clarifying this question so that it can be reopened, visit the help center.
Closed 10 years ago.
I'm about to start (with fellow programmers) a programming & algorithms club in my high school. The language of choice is C++ - sorry about that, I can't change this. We can assume students have little to no experience in the aforementioned topics.
What do you think are the most basic concepts I should focus on?
I know that teaching something that's already obvious to me isn't an easy task. I realize that the very first meeting should be given an extreme attention - to not scare students away - hence I ask you.
Edit: I noticed that probably the main difference between programmers and beginners is "programmer's way of thinking" - I mean, conceptualizing problems as, you know, algorithms. I know it's just a matter of practice, but do you know any kind of exercises/concepts/things that could stimulate development in this area?
Make programming fun!
Possible things to talk about would be Programming Competitions that either your club could hold itself or it could enter in locally. I compete in programming competitions at the University (ACM) level and I know for a fact that they have them at lower levels as well.
Those kind of events can really draw out some competitive spirit and bring the club members closer.
Things don't always have to be about programming either. Perhaps suggest having a LAN party where you play games, discuss programming, etc could be a good idea as well.
In terms of actual topics to go over that are programming/algorithm related, I would suggest as a group attempting some of these programming problems in this programming competition primer "Programming Challenges": Amazon Link
They start out with fairly basic programming problems and slowly progress into problems that require various Data Structures like:
Stacks
Queues
Dictionaries
Trees
Etc
Most of the problems are given in C++.
Eventually they progress into more advanced problems involving Graph Traversal and popular Graph algorithms (Dijkstra's, etc) , Combinatrics problems, etc. Each problem is fun and given in small "story" like format. Be warned though, some of these are very hard!
Edit:
Pizza and Soda never hurts either when it comes to getting people to show up for your club meetings. Our ACM club has pizza every meeting (once a month). Even though most of us would still show up it is a nice ice breaker. Especially for new clubs or members.
Breaking it Down
To me, what's unique about programming is the need to break down tasks into small enough steps for the computer. This varies by language, but the fact that you may have to write a "for loop" just to count to 100 takes getting used to.
The "top-down" approach may help with this concept. You start by creating a master function for your program, like filterItemsByCriteria();
You have no idea how that will work, so you break it down into further steps:
(Note: I don't know C++, so this is just a generic example)
filterItemsByCritera() {
makeCriteriaList();
lookAtItems();
removeNonMatchingItems();
}
Then you break each of those down further. Pretty soon you can define all the small steps it takes to make your criteria list, etc. When all of the little functions work, the big one will work.
It's kind of like the game kids play where they keep asking "why?" after everything you say, except you have to keep asking "how?"
Linked lists - a classic interview question, and for good reason.
I would try to work with a C subset, and not try to start with the OO stuff. That can be introduced after they understand some of the basics.
Greetings!
I think you are getting WAY ahead of yourself in forcing a specific language and working on specific topics and a curriculum.. It sounds like you (and some of the responders) are confusing "advising a programming club" with "leading a programming class". They are very different things.
I would get the group together, and the group should decide what exactly they want to get out of the club. In essence, make a "charter" for the club. Then (and only then) can you make determinations such as preferred language/platform, how often to meet, what will happen at the meetings, etc.
It may turn out that the best approach is a "survey", where different languages/platforms are explored. Or it may turn out that the best approach is a "topical"one, where there topic changes (like a book club) on a regular basis (this month is pointers, next month is sorting, the following is recursion, etc.) and then examples and discussions occur in various languages.
As an aside, I would consider a "language-agnostic" orientation for the club. Encourage the kids to explore different languages and platforms.
Good luck, and great work!
Well, it's a programming club, so it should be FUN! So I would say dive into some hand on experience right away. Start with explaining what a main() method is,then have students write a hello world program. Gradually improve the hello world program so it has functions and prints out user inputs.
I would say don't go into algorithm too fast for beginners, let them play with C++ first.
Someone mentioned above, "make programming fun". It is interesting today that people don't learn for the sake of learning. Most people want instant gratification.
Teach a bit of logic using Programming. This helps with(and is) problem solving. The classing one I have in my head are guessing games.
Have them make a program that guesses at a number between 0 and 100.
Have them make a black jack clone ... I have done this in basic :-(
Make paper instructions.
Explain the "Fried eggs" story. Ask the auditory what they would do to make themselves fried eggs. Make them note the step they think about. Probably you will receive less than 5 steps algorithm. Then explain them how many steps should be written down if we want to teach a computer to fry eggs. Something like:
1) Go to the Fridge
2) Open the fridge door
3) Search for eggs
4) If there are no eggs - go to the shop to buy eggs ( this is another function ;) )
5) If there are eggs - calculate how many do you need to fry
6) Close the fridge door
7) e.t.c. :)
Start with basics of C - syntax semantics e.t.c, and in parallel with that explain the very basic algorithms like bubble sort.
After the auditory is familiar with structured programming (this could take several weeks or months, depending how often you make the lessons), you can advance to C++ and OOP.
The content in Deitel&Deitel's C++ programming is a decent introduction, and the exercises proposed at the end of each chapter are nice toy problems.
Basically, you're talking about:
- control structures
- functions
- arrays
- pointers and strings
You might want to follow up with an introduction to the STL ("ok, now that we've done it the hard way... here's a simpler option")
Start out by making them understand a problem like for instance sorting. This is very basic and they should be able to relate quite fast. Once they see the problem then present them with the tools/solution to solve it.
I remember how it felt when I first was show an example of merge-sort. I could follow all the steps but what the hell was I for? Make then crave a solution to a problem and they will understand the tool and solution much better.
start out with a simple "hello world" program. This introduces fundamentals such as variables, writing to a stream and program flow.
Then add complexity from there (linked lists, file io, getting user input, etc).
The reason I say start with hello world is because the kid will get to see a running program really quick. It's nearly immediate feedback-as they will have written a running program right from the start.
IMO, Big-O is one of the more important concepts for beginning programmers to learn.
Have a debugging contest. Provide code samples that include a bug. Have a contest to see who can find the most or fastest.
There is an excellent book, How Not to Program in C++, that you could use to start with.
You always learn best from mistakes and I prefer to learn from some else's.
It will also let those with little experience learn by see code, even if the code only almost works.
In addition to the answers to this question, there are certain important topics to cover. Here's an example of how you could structure the lessons.
First Lesson: Terminology and Syntax
Terminology to cover: variable, operator, loop (iteration), method, reserved word, data type, class
Syntax to cover: assignment, operation, if/then/else, for loop, while loop, select, input/output
Second Lesson: Basic Algorithm Construction
Cover a few simple algorithms, involving some input, maybe a for or a while loop.
Third Lesson: More Advanced Algorithm Topics
This is for things like recursion, matrix manipulation, and higher-level mathematics. You don't have to get into too complex of topics, but introduce enough complexity to be useful in a real project.
Final Lesson: Group Project
Make a project that groups can get involved in doing.
These don't have to be single day lessons. You can spread the topics across multiple days.
Pseudocode should be a very first.
Edit: If they are total programming beginners then I would make the first half just about programming. Once you get to a level where talking about algorithms would make sense then pseudocode is really important to get under the nails.
Thanks for your replies!
And how would you teach them actual problem solving?
I know a bunch of students that know C++ syntax and a few basic algorithms, but they can't apply the knowledge they know when they solve real problems - they don't know the approach, the way to transcribe their thoughts into a set of strict steps. I do not talk about 'high-level' approaches like dynamic programming, greedy etc., but about basic algorithmic mindset.
I assume it's just because of the poor learning process they were going through. In other sciences - math, for example - they are really brilliant.
Just because you are familiar with algorithms does not mean you can implement them and just because you can program does not mean you can implement an algorithm.
Start simple with each topic (keep programming separate from designing algorithms). Once they have a handle on each, slowly start to bring the two concepts together.
Wow. C++ is one of the worst possible languages to start with, in terms of the amount of unrelated crap you need to get anything working (Java would be slightly worse, I guess).
When teaching beginners in a boilerplate-heavy environment, it's usual to start with "here's a simple C program. We'll discuss what all this crap at the top of the file is for later, but for now, concentrate on the lines between 'int main(void)' and the 'return' statement, which is where all the useful work is accomplished".
Once you're past that point, basic concepts to cover include the basic data structures (arrays, linked lists, trees, and dictionaries), and the basic algorithms (sorting, searching, etc).
Have your club learn how to actually program in any language by teaching the concepts of building software. Instead of running out an buying a dozen licenses for Visual Studio, have students use compilers, make systems, source files, objects and librarys in order to turn their C code into programs. I feel this is truly the beginning and actually empowers these kids to understand how to make software on any platform, without crutches that many educational institutions like to rely on.
As for the language of choice - congratulations - you'll find C++ is very rich in making you think of mathematical shortcuts and millions of ways to make your code perform even better (or to implement fancy patterns).
To the question: When I was beggining to program I would always try to break down one real life problem into several steps and then as I see similarity between tasks or data they transform I would always try to find a lazier, easier, meanier way to implement it.
Elegance came after when learning patterns and real algorithms.
Hank: Big O??? you mean tell beginning programmers that their code is of O(n^2) and yours is of n log n ??
I could see a few different ways to take this:
1) Basic programming building blocks. What are conditional statements, e.g. switch and if/else? What are repetition statements, e.g. for and while loops? How do we combine these to get a program to be the sequence of steps we want? You could take something as easy as adding up a grocery bill or converting temperatures or distances from metric to imperial or vice versa. What are basic variable types like a string, integer, or double? Also in here you could have Boolean Algebra for an advanced idea or possibly teach how to do arithmetic in base 2 or 16 which some people may find easy and others find hard.
2) Algorithmically what are similar building blocks. Sorting is a pretty simple topic that can be widely discussed and analysed to try to figure out how to make this faster than just swapping elements that seem out of order if you learn the Bubblesort which is the most brain dead way to do.
3) Compiling and run-time elements. What is a call stack? What is a heap? How is memory handled to run a program,e.g. the code pieces and data pieces? How do we open and manipulate files? What is compiling and linking? What are make files? Some of this is simple, but it can also be eye-opening just to see how things work which may be what the club covers most of the time.
These next 2 are somewhat more challenging but could be fun:
4) Discuss various ideas behind algorithms such as: 1) Divide and conquer, 2) Dynamic programming, 3) Brute force, 4) Creation of a data structure, 5) Reducing a problem to a similar one already solved for example Fibonacci numbers is a classic recursive problem to give beginning programmers, and 6) The idea of being, "greedy," like in a making change example if you were in a country where coin denominations where a,b, and c. You could also get into some graph theory examples like a minimum weight spanning tree if you want something somewhat exotic, or the travelling salesmen for something that can be easy to describe but a pain to solve.
5) Mathematical functions. How would you program a factorial, which is the product of all numbers from 1 to n? How would you compute the sums of various Arithmetic or Geometric Series? Or compute the number of Combinations or Permutations of r elements from a set of n? Given a set of points, approximate the polynomial that meets this requirement, e.g. in a 2-dimensional plane called x and y you could give 2 points and have people figure out what are the slope and y intercept if you have solved pairs of linear equations already.
6) Lists which can be implemented using linked lists and arrays. Which is better for various cases? How do you implement basic functions such as insert, delete, find, and sort?
7) Abstract Data Structures. What are stacks and queues? How do you build and test classes?
8) Pointers. This just leads to huge amounts of topics like how to allocate/de-allocate memory, what is a memory leak?
Those are my suggestions for various starting points. I think starting a discussion may lead to some interesting places if you can get a few people together that don't mind talking on the same subject week after week in some cases as sorting may be a huge topic to cover well if you want to get into the finer points of things.
You guys could build the TinyPIM project from "C++ Standard Library from Scratch" and then, when it's working, start designing your own extensions.