How to write an interpreter? - ruby

I have decided to write a small interpreter as my next project, in Ruby. What knowledge/skills will I need to have to be successful?
I haven't decided on the language to interpret yet, but I am looking for something that is not a toy language, but would be relatively easy to write an interpreter for.
Thanks in advance.

You will have to learn at least:
lexical analysis (grouping characters into tokens)
parsing (grouping tokens together into structure)
abstract syntax trees (representing program structure in a data structure)
data representation (assuming your language will have variables)
an evaluation loop that "runs" your program
An excellent introduction to some of these topics can be found in the introductory text Structure and Interpretation of Computer Programs. The language used in that book is Scheme, which is a robust, well-specified language that is ideally suited for your first interpreter implementation. Highly recommended.

I haven't decided on the language to interpret yet, but I am looking for
something that is not a toy language, but would be relatively easy to write an
interpreter for. Thanks in advance.
Try some dialect of Lisp like Scheme or Clojure. (Now there's an idea: Clojure-in-Ruby, which integrates with Ruby as well as Clojure does with Java.)
With Lisp, there is no need to bother with idiosyncracies of syntax, as Lisp's syntax is much closer to the abstract syntax tree.

This SICP chapter shows how to write a Lisp interpreter in Lisp (a metacircular evaluator). In my opinion this is the best place to start. Then you can move on to Lisp in Small Pieces to learn how to write advanced interpreters and compilers for Lisp. The advantage of implementing a language like Lisp (in Lisp itself!) is that you get the lexical analyzer, parser, AST, data/program representation and REPL for free. You can concentrate on the task of getting your great language working!

There is Tree top project wich can be helpful for you http://treetop.rubyforge.org/

You can checkout Ruby Draft Specification http://ruby-std.netlab.jp/

I had a similar idea a couple of days ago. LISP is by far the easiest to implement because the syntax is so simple, and the data structures that the language manipulates are the same structures that the code is written in. Hence you need only a minimal implementation, and can define the rest in terms of itself.
However, if you are trying to learn about parsing, you may want to do a more complex language with Abstract Syntax Trees, etc.
If you want to check out my (literally two days old) Java implementation of lisp, check out mylisp.googlecode.com. I'm still working on it but it is incredible how short a time it took to get the existing stuff working.

It's not sooo hard. here's a LISP interpreter in ruby and the source is so small you are supposed to copy/paste it. but are you gonna learn LISP now? hehe.

If you're just doing this for fun, make up your own, simple language and just try it. My recommendation would be something like a really simple classic BASIC (no visual basic or object oriented stuff). With line numbers, GOTO, INPUT and PRINT and that's it. You get to do the basics, and you get a better understanding of how things work.
The knowledge you'll need?
Tokenizing (turning that huge chunk of characters into something more efficiently readable, effectively splitting it up into 'words')
Parsing (going over the tokens and building a data structure from it)
Interpreting (looping over the data structure and executing each command)
And for that last one you'll also need a way to keep around variables. Usually you'd just implement a "stack", one huge block of data where you can mark off an area at the end.

It's not implemented in Lisp, but I found Write Yourself A Scheme in 48 Hours to be a very useful document while I was starting out with Haskell (though I didn't get anywhere near finishing it after 48 hours; YMMV). It also gives you a lot of insight into interpreters in general.

I can recommend this book. It discusses patterns for writing parsers and interpreters and more:
http://www.amazon.co.uk/s/ref=nb_sb_noss?url=search-alias%3Daps&field-keywords=language+implementation+patterns&x=0&y=0

Related

Pseudocode interpreter?

Like lots of you guys on SO, I often write in several languages. And when it comes to planning stuff, (or even answering some SO questions), I actually think and write in some unspecified hybrid language. Although I used to be taught to do this using flow diagrams or UML-like diagrams, in retrospect, I find "my" pseudocode language has components of C, Python, Java, bash, Matlab, perl, Basic. I seem to unconsciously select the idiom best suited to expressing the concept/algorithm.
Common idioms might include Java-like braces for scope, pythonic list comprehensions or indentation, C++like inheritance, C#-style lambdas, matlab-like slices and matrix operations.
I noticed that it's actually quite easy for people to recognise exactly what I'm triying to do, and quite easy for people to intelligently translate into other languages. Of course, that step involves considering the corner cases, and the moments where each language behaves idiosyncratically.
But in reality, most of these languages share a subset of keywords and library functions which generally behave identically - maths functions, type names, while/for/if etc. Clearly I'd have to exclude many 'odd' languages like lisp, APL derivatives, but...
So my questions are,
Does code already exist that recognises the programming language of a text file? (Surely this must be a less complicated task than eclipse's syntax trees or than google translate's language guessing feature, right?) In fact, does the SO syntax highlighter do anything like this?
Is it theoretically possible to create a single interpreter or compiler that recognises what language idiom you're using at any moment and (maybe "intelligently") executes or translates to a runnable form. And flags the corner cases where my syntax is ambiguous with regards to behaviour. Immediate difficulties I see include: knowing when to switch between indentation-dependent and brace-dependent modes, recognising funny operators (like *pointer vs *kwargs) and knowing when to use list vs array-like representations.
Is there any language or interpreter in existence, that can manage this kind of flexible interpreting?
Have I missed an obvious obstacle to this being possible?
edit
Thanks all for your answers and ideas. I am planning to write a constraint-based heuristic translator that could, potentially, "solve" code for the intended meaning and translate into real python code. It will notice keywords from many common languages, and will use syntactic clues to disambiguate the human's intentions - like spacing, brackets, optional helper words like let or then, context of how variables are previously used etc, plus knowledge of common conventions (like capital names, i for iteration, and some simplistic limited understanding of naming of variables/methods e.g containing the word get, asynchronous, count, last, previous, my etc). In real pseudocode, variable naming is as informative as the operations themselves!
Using these clues it will create assumptions as to the implementation of each operation (like 0/1 based indexing, when should exceptions be caught or ignored, what variables ought to be const/global/local, where to start and end execution, and what bits should be in separate threads, notice when numerical units match / need converting). Each assumption will have a given certainty - and the program will list the assumptions on each statement, as it coaxes what you write into something executable!
For each assumption, you can 'clarify' your code if you don't like the initial interpretation. The libraries issue is very interesting. My translator, like some IDE's, will read all definitions available from all modules, use some statistics about which classes/methods are used most frequently and in what contexts, and just guess! (adding a note to the program to say why it guessed as such...) I guess it should attempt to execute everything, and warn you about what it doesn't like. It should allow anything, but let you know what the several alternative interpretations are, if you're being ambiguous.
It will certainly be some time before it can manage such unusual examples like #Albin Sunnanbo's ImportantCustomer example. But I'll let you know how I get on!
I think that is quite useless for everything but toy examples and strict mathematical algorithms. For everything else the language is not just the language. There are lots of standard libraries and whole environments around the languages. I think I write almost as many lines of library calls as I write "actual code".
In C# you have .NET Framework, in C++ you have STL, in Java you have some Java libraries, etc.
The difference between those libraries are too big to be just syntactic nuances.
<subjective>
There has been attempts at unifying language constructs of different languages to a "unified syntax". That was called 4GL language and never really took of.
</subjective>
As a side note I have seen a code example about a page long that was valid as c#, Java and Java script code. That can serve as an example of where it is impossible to determine the actual language used.
Edit:
Besides, the whole purpose of pseudocode is that it does not need to compile in any way. The reason you write pseudocode is to create a "sketch", however sloppy you like.
foreach c in ImportantCustomers{== OrderValue >=$1M}
SendMailInviteToSpecialEvent(c)
Now tell me what language it is and write an interpreter for that.
To detect what programming language is used: Detecting programming language from a snippet
I think it should be possible. The approach in 1. could be leveraged to do this, I think. I would try to do it iteratively: detect the syntax used in the first line/clause of code, "compile" it to intermediate form based on that detection, along with any important syntax (e.g. begin/end wrappers). Then the next line/clause etc. Basically write a parser that attempts to recognize each "chunk". Ambiguity could be flagged by the same algorithm.
I doubt that this has been done ... seems like the cognitive load of learning to write e.g. python-compatible pseudocode would be much easier than trying to debug the cases where your interpreter fails.
a. I think the biggest problem is that most pseudocode is invalid in any language. For example, I might completely skip object initialization in a block of pseudocode because for a human reader it is almost always straightforward to infer. But for your case it might be completely invalid in the language syntax of choice, and it might be impossible to automatically determine e.g. the class of the object (it might not even exist). Etc.
b. I think the best you can hope for is an interpreter that "works" (subject to 4a) for your pseudocode only, no-one else's.
Note that I don't think that 4a,4b are necessarily obstacles to it being possible. I just think it won't be useful for any practical purpose.
Recognizing what language a program is in is really not that big a deal. Recognizing the language of a snippet is more difficult, and recognizing snippets that aren't clearly delimited (what do you do if four lines are Python and the next one is C or Java?) is going to be really difficult.
Assuming you got the lines assigned to the right language, doing any sort of compilation would require specialized compilers for all languages that would cooperate. This is a tremendous job in itself.
Moreover, when you write pseudo-code you aren't worrying about the syntax. (If you are, you're doing it wrong.) You'll wind up with code that simply can't be compiled because it's incomplete or even contradictory.
And, assuming you overcame all these obstacles, how certain would you be that the pseudo-code was being interpreted the way you were thinking?
What you would have would be a new computer language, that you would have to write correct programs in. It would be a sprawling and ambiguous language, very difficult to work with properly. It would require great care in its use. It would be almost exactly what you don't want in pseudo-code. The value of pseudo-code is that you can quickly sketch out your algorithms, without worrying about the details. That would be completely lost.
If you want an easy-to-write language, learn one. Python is a good choice. Use pseudo-code for sketching out how processing is supposed to occur, not as a compilable language.
An interesting approach would be a "type-as-you-go" pseudocode interpreter. That is, you would set the language to be used up front, and then it would attempt to convert the pseudo code to real code, in real time, as you typed. An interactive facility could be used to clarify ambiguous stuff and allow corrections. Part of the mechanism could be a library of code which the converter tried to match. Over time, it could learn and adapt its translation based on the habits of a particular user.
People who program all the time will probably prefer to just use the language in most cases. However, I could see the above being a great boon to learners, "non-programmer programmers" such as scientists, and for use in brainstorming sessions with programmers of various languages and skill levels.
-Neil
Programs interpreting human input need to be given the option of saying "I don't know." The language PL/I is a famous example of a system designed to find a reasonable interpretation of anything resembling a computer program that could cause havoc when it guessed wrong: see http://horningtales.blogspot.com/2006/10/my-first-pli-program.html
Note that in the later language C++, when it resolves possible ambiguities it limits the scope of the type coercions it tries, and that it will flag an error if there is not a unique best interpretation.
I have a feeling that the answer to 2. is NO. All I need to prove it false is a code snippet that can be interpreted in more than one way by a competent programmer.
Does code already exist that
recognises the programming language
of a text file?
Yes, the Unix file command.
(Surely this must be a less
complicated task than eclipse's syntax
trees or than google translate's
language guessing feature, right?) In
fact, does the SO syntax highlighter
do anything like this?
As far as I can tell, SO has a one-size-fits-all syntax highlighter that tries to combine the keywords and comment syntax of every major language. Sometimes it gets it wrong:
def median(seq):
"""Returns the median of a list."""
seq_sorted = sorted(seq)
if len(seq) & 1:
# For an odd-length list, return the middle item
return seq_sorted[len(seq) // 2]
else:
# For an even-length list, return the mean of the 2 middle items
return (seq_sorted[len(seq) // 2 - 1] + seq_sorted[len(seq) // 2]) / 2
Note that SO's highlighter assumes that // starts a C++-style comment, but in Python it's the integer division operator.
This is going to be a major problem if you try to combine multiple languages into one. What do you do if the same token has different meanings in different languages? Similar situations are:
Is ^ exponentiation like in BASIC, or bitwise XOR like in C?
Is || logical OR like in C, or string concatenation like in SQL?
What is 1 + "2"? Is the number converted to a string (giving "12"), or is the string converted to a number (giving 3)?
Is there any language or interpreter
in existence, that can manage this
kind of flexible interpreting?
On another forum, I heard a story of a compiler (IIRC, for FORTRAN) that would compile any program regardless of syntax errors. If you had the line
= Y + Z
The compiler would recognize that a variable was missing and automatically convert the statement to X = Y + Z, regardless of whether you had an X in your program or not.
This programmer had a convention of starting comment blocks with a line of hyphens, like this:
C ----------------------------------------
But one day, they forgot the leading C, and the compiler choked trying to add dozens of variables between what it thought was subtraction operators.
"Flexible parsing" is not always a good thing.
To create a "pseudocode interpreter," it might be necessary to design a programming language that allows user-defined extensions to its syntax. There already are several programming languages with this feature, such as Coq, Seed7, Agda, and Lever. A particularly interesting example is the Inform programming language, since its syntax is essentially "structured English."
The Coq programming language allows "syntax extensions", so the language can be extended to parse new operators:
Notation "A /\ B" := (and A B).
Similarly, the Seed7 programming language can be extended to parse "pseudocode" using "structured syntax definitions." The while loop in Seed7 is defined in this way:
syntax expr: .while.().do.().end.while is -> 25;
Alternatively, it might be possible to "train" a statistical machine translation system to translate pseudocode into a real programming language, though this would require a large corpus of parallel texts.

Which will serve a budding programmer better: A classic book in scheme or a modern language like python?

I'm really interested in becoming a serious programmer, the type that people admire for hacker chops, as opposed to a corporate drone who can't even complete FizzBuzz.
Currently I've dabbled in a few languages, most of my experience is in Perl and Shell, and I've dabbled slightly in Ruby.
However, I can't help but feel that although I know bits and pieces of languages, I don't know how to program.
I'm really in no huge rush to immediately learn a language that can land me a job (though I'd like to do it soon), and I'm considering using PLT Scheme (now called Racket) to work through How to Design Programs or Structure and Interpretation of Computer Programs, essentially, one of the Scheme classics, because I have always heard that they teach people how to write high-quality, usable, readable code.
However, even MIT changed its introductory course from using SICP and Scheme to one in Python.
So, I ask for the sage advice of the many experienced programmers here regarding the following:
Does Scheme (and do those books) really teach one how to program well? If so, which of the two books do you recommend?
Is this approach to learning still relevant and applicable? Am I on the right track?
Am I better off spending my time learning a more practical/common language like Python?
Is Scheme (or lisp in general) really a language that one learns, only to never use? Or do those of you who know a lisp code in it often?
Thanks, and sorry for the rambling.
If you want to learn to really program, start doing it. Quit dabbling and write code. Pick a language and write code. Solve problems and release applications. Work with experienced programmers on open source projects, but get doing. A lot.
Does Scheme (and do those books) really teach one how to program well? If so, which of the two books do you recommend?
Probably. Probably better than any of the Learn X in Y Timespan books.
Is this approach to learning still relevant and applicable? Am I on the right track?
Yes.
Am I better off spending my time learning a more practical/common language like Python?
Only if you plan to get a job in it. Scheme will give you a better foundation though.
Is Scheme (or lisp in general) really a language that one learns, only to never use? Or do those of you who know a lisp code in it often?
I do emacs elisp fiddling to adjust my emacs. I also work with functional languages on the side to try to keep my mind flexible.
My personal opinion is that there are essentially two tracks that need to be walked before the student can claim to know something about programming. Track one is the machine itself, the computer. You should start with assembly here and learn how the computer works. After some work and understanding there - don't skimp - you should learn C and then C++; really getting the understanding of resource management and what really happens. Track two is the very high level language track - Scheme, Prolog, Haskell, Perl, Python, C#, Java, and others that execute on a VM or interpreter lie in this area. These, too, need to be studied to learn how problems can be abstracted and thought about in different ways that do not involve the fiddly bits of a real computer.
However, what will not work is being a language dilettante when learning to program. You will need to find a language - Scheme is acceptable, although I'd recommend starting at the low level first - and then stick with that language for a good year at least.
The most important parts of Scheme are the programming-language concepts you can pick up that modern languages are now just adopting or adding support for.
Lisp and Scheme have supported, before most other languages, features that were often revolutionary for the time: closures and first-order functions, continuations, hygienic macros, and others. C has none of these.
But they're appearing more and more often in programming languages that Get Stuff Done today. Why can you just declare functions seemingly anywhere in JavaScript? What happens to outside variables you reference from within a function? What are these new "closures" that PHP 5.3 is just now getting? What are "side effects" and why can they be bad for parallel computing? What are "continuations" in Ruby? How do LINQ functions work? What's a "lambda" in Python? What's the big deal with F#?
These are all questions that learning Scheme will answer but C won't.
I'd say it depends on what you want to do.
If you want to get into programming, Python is probably better. It's an excellent first language, resembles most common programming languages, and is widely available. You'll find more libraries handy, and will be able to make things more easily.
If you want to get into computer science, I'd recommend Scheme along with SICP.
In either case, I'd recommend learning several very different languages eventually, to give you more ways to look at and solve problems. Getting reasonably proficient in Common Lisp, for example, will make you a better Java programmer. I'd take them one at a time, though.
The best languages to start with are probably:
a language you want to play/learn in
a language you want to work in
And probably in that order, too, unless the most urgent need is to feed yourself.
Here's the thing: the way to learn to program is to do it a lot. In order to do it a lot, you're going to need a lot of patience and more than a little bit of enthusiasm. This is more important than the specific language you pick.... but picking a language that you like working in (whether because you like the features or because you feel it'll teach you something) can be a big boost.
That said, here's a couple of comments on Scheme:
Does Scheme (and do those books)
really teach one how to program well?
The thing about Scheme (or something like it) is that if you learn it, it'll teach you some very useful abstractions that a lot of programmers who don't ever really come to grips with a functional programming language never learn. You'll think differently The substance of programming languages and computing will look more fluid to you. You'll have a better idea of how to compose your own quasi-primitives out of a very small set of primitives rather than relying on the generally static set of primitives offered in some other languages.
The problem is that a lot of what I'm saying might not mean much to you at the moment, and it's a bit more of a mind-bending road than coming into a common dynamic language like Perl, Python, or Ruby... or even a language like C which is close to the Von Neumann mechanics of the machine.
This doesn't mean it's necessarily a bad idea to start there: I've been part of an experiment where we taught Prolog of all things to first-time programmers, and it worked surprisingly well. Sometimes beginner's mind actually helps. :) But Scheme as a first language is definitely an unconventional path. I suspect Ruby or Python would be a gentler road.
Is Scheme (or lisp in general) really
a language that one learns, only to
never use?
It's a language that you're unlikely to be hired to program in. However, while you're learning to program, and after you've learned and are doing it in your free time, you can write code in whatever you want, and because of the Internet, you'll probably be able to find people working on open source projects in whatever language you want. :)
I hate to tell ya, but nobody admires programmers for their "hacker chops". There's people who get shit done, then there's everyone else. A great many of the former types are the "corporate drones" you appear to hold in contempt.
Now, for your question, I personally love Lisp (and Scheme), but if you want something you're more likely to use in industry "Beginning Python" might be better material for you as Python is found more often in the wild. Or if you enjoy Ruby, find some good Ruby material and start producing working solutions (same with Java or .Net or whatever).
Really, either route will serve you well. The trick is to stick with it until you've internalized the concepts being taught.
Asking whether an approach to learning is relevant and applicable is tricky - there are many different learning styles, and it's a matter of finding out which ones apply to you personally. Bear in mind that the style you like best might not be the one that actually works best for you :-)
You've got plenty of time and it sounds like you have enthusiasm to spare, so it's not a matter of which language you should learn, but which one you should learn first. personally, I'd look at what you've learnt so far, what types of languages and paradigms you've got under your belt, and then go off on a wild tangent and chose one completely different.
I started programming at a very very young age. When I was in high school, I thought I was a good programmer. That's when I started learning about HOW and WHY the languages work rather than just the syntax.
Before learning the how and why, switching to a new language would have been hell. I had learned a language, but I hadn't learned to program. Now that I know the fundamental concepts well, I can apply them to virtually any language and pick it up with ease.
I would highly recommend a book (or even a school coarse, if you can afford it) that takes you through the processes of coding without relying on a specific language.
Unfortunately I don't have any books to recommend, but if others agree with me and know of any, maybe they can offer a suggestion.
//Edit: After re-reading your question, I realize that I may have not actually answered any of them... Sorry about that. I think picking up a book that will take you in-depth with best-practices would be extremely helpful, regardless of the language you choose.
There are basic programming concepts (logic flow, data structures), which are easily taught by using languages like Python. However, there are much more complex programming concepts (design patterns, optimization, threading, etc.) which the classic languages don't abstract away for you.
If your search for knowledge leans more toward algorithm development and the science of programming, start with C. If your search is more for a practical ends, I hear Ruby is a good starting point.
I agree with gruszczy. I'd start programming with C.
It may be kind of scary at first (at least for me :S ) but in the long run you'd be grateful it. I mean I love Python, but because I learned C first, the learning curve for other languages wasn't very steep at all.
Start with C and make it so.
Just remember to practice, because you'll never improve at something by doing nothing. ;)
To a specific point in your question, the "classics" you mention will help you with exactly what the titles say. SICP is about the structure and interpretation of computer programs. It is not about learning Scheme (though you will learn Scheme). HtDP is about how to design programs, it is not about learning Scheme (though you will learn Scheme).
Scheme, in principle, is a very small and concise language with almost no gotchas. This makes it excellent for moving on to learning how to structure and interpret programs, or how to design them. More traditional "practical" languages like C, C++, Python, or Java do not have this quality. They are rife with syntax. Learning with these languages means you must simultaneously learn syntactical quirks while learning to think like a programmer. In my opinion, this is unfortunate. In some cases the quirks are good, in others they are accidents of history, but in all cases it is unfortunate.
Start coding in C. It should be a horror for you at first, but this teaches you most important stuff like: pointers, recurrence, memory management. Try reading some classic books about programming like The Art of Computer Programming by Donald Knuth. After you master that, you can think about learning object oriented programming or functional programming. First basics. If fou manage to learn them, nothing will be hard for you ever again.

if i like Ruby a lot, is there a reason I should learn another language now, such as Lua or Erlang?

if i like Ruby a lot, is there a reason I should learn another language now, such as Lua or Erlang?
New programming languages, much like spoken languages, can open up new perspectives. Learning new languages -- especially ones rather different from what you're used to (and Erlang will probably fit that bill) -- can teach you a lot of different things you didn't even know you didn't know about programming. So yes, I think you absolutely should, even if you just learn enough to tinker with it and get a feel for the new language.
Learning a functional language in particular can be extremely beneficial. Becoming familiar with the functional style of programming is a surefire step toward becoming a better programmer. Lisp (or its derivatives) in particular is a good language to study. Here's a list of past thread on SO that might offer you some insight along these lines:
Why do people think functional programming will catch on?
What’s a good Functional language to learn?
Benefits of learning scheme?
Leaving aside the (excellent) general reasons to want to learn another language, if you like Ruby a lot you might want to
Learn Smalltalk, which is a language very, very similar to Ruby but in purer form.
Learn a language that is very, very different—say something that is based on algebraic data types and functions rather than objects and methods, and something with a static type system rather than a dynamic type system—but something that, like Ruby, will support powerful methods of program composition and generic programming. Good candidates would include Standard ML and Haskell.
Learn a language that is very, very different—say something that makes you control every bit, address, and word in memory—something that forces you to understand and take control of the hardware. In other words, learn C.
Regarding the other languages you mention,
Lua is small and very elegantly designed and implemented. That may appeal to the Rubyist in you. But unlike Ruby it does not impose much of a worldview; it is more of a collection of piece parts. I would suggest you're more likely to appreciate and enjoy Lua after you've worked in three or four other languages first.
Erlang is interesting, but I have a gut feel it's either too different (purely functional, distributed) or not different enough (dynamic type system). But if it appeals to you, go for it.
On the other hand, there's something to be said for really knowing a language well. You'll be able to do a lot more with in-depth knowledge of a single language than you will with surface knowledge of a dozen.
If you like Ruby a lot you should definitely learn another language... one without sigils if possible.
Seems to me that a professional learns the tools he needs to use. Frameworks, containers, languages, all are fair game. I started out in Pascal, went to C and then C++. Then converted to Java. These days its mostly Java with a lot of Javascript and some PHP. Easy enough right? Well, I also need to learn Bash scripting and Perl. Never mind all the other crap I need to get on top of (if you say you understand all of web authentication I will call you a liar). There's a lot of stuff out there. Jump in. Be willing to try different things.
I always enjoy learning new languages for the mere challenge of it. It keeps my brain fit. I've also found it makes for good job interview fodder to be able to say "I'm flexible. I'm adaptable to whatever your needs may be in the future. And I can prove it with my long list of languages."
My main language is PHP. I am a script language fan, nevertheless I have dived into C#, Java, Python, Ruby and even OO JavaScript books to find new mechanisms, ways of thinking. I have found pretty many stunts in Java for example, that I could implement in my all day work. So learning or just studying new languages can widen your perspective.

Dynamic languages - which one should I choose?

Dynamic languages are on the rise and there are plenty of them: e.g. Ruby, Groovy, Jython, Scala (static, but has the look and feel of a dynamic language) etc etc.
My background is in Java SE and EE programming and I want to extend my knowledge into one of these dynamic languages to be better prepared for the future.
But which dynamic language should I focus on learning and why? Which of these will be the preferred language in the near future?
Learning Ruby or Python (and Scala to a lesser extent) means you'll have very transferrable skills - you could use the Java version, the native version or the .NET version (IronRuby/IronPython). Groovy is nice but JVM-specific.
Being "better prepared for the future" is tricky unless you envisage specific scenarios. What kind of thing do you want to work on? Do you have a project which you could usefully implement in a dynamic language? Is it small enough to try on a couple of them, to get a feeling of how they differ?
Scala is not a dynamic language at all. Type inference doesn't mean that its untyped. However, Its a very nice language that has nice mixture of OOPs and functional programming. The only problem is some gotchas that you encounter along the way.
Since you are already an experienced Java programmer, it will fit nicely into your skillset. Now, if you want to go all the way dynamic both Ruby or Python are awesome languages. There is demand for both the languages.
I would personally recommend Clojure. Clojure is an awesome new language that is going in popularity faster than anything I've ever seen. Clojure is a powerful, simple, and fast Lisp implemented on the JVM. It has access to all Java libraries of course, just like Scala. It has a book written about it already, it's matured to version 1.0, and it has three IDE plugins in development, with all three very usable.
I would take a look at Scala. Why ?
it's a JVM language, so you can leverage off your current Java skills
it now has a lot of tooling/IDE support (e.g. Intellij will handle Scala projects)
it has a functional aspect to it. Functional languages seem to be getting a lot of traction at the moment, and I think it's a paradigm worth learning for the future
My (entirely subjective) view is that Scala seems to be getting a lot of the attention that Groovy got a year or two ago. I'm not trying to be contentious here, or suggest that makes it a better language, but it seems to be the new JVM language de jour.
As an aside, a language that has some dynamic attributes is Microsoft's F#. I'm currently looking at this (and ignoring my own advice re. points 1 and 2 above!). It's a functional language with objects, built on .Net, and is picking up a lot of attention at the moment.
In the game industry Lua, if you're an Adobe based designer Lua is also good, if you're an embedded programmer Lua is practically the only light-weight solution, but if you are looking into Web development and General tool scripting Python would be more practical
I found Groovy to be a relatively easy jump from an extensive Java background -- it's sort of a more convenient version of Java. It integrates really nicely with existing Java code as well, if you need to do that sort of thing.
I'd recommend Python. It has a huge community and has a mature implementation (along with several promising not-so-mature-just-yet ones). Perl is as far as I've seen loosing a lot of traction compared to the newer languages, presumably due to its "non-intuitiveness" (no, don't get me started on that).
When you've done a project or two in Python, go on to something else to get some broader perspective. If you've done a few non-trivial things in two different dynamic languages, you won't have any problems assimilating any other language.
JScript is quite usefull, and its certainly a dynamic language...
If you want a language with a good number of modules (for almost anything!), go for Perl. With its CPAN, you will always find what you want without reinventing the wheel.
Well keeping in mind your background, i would recommend a language where the semantics are similar to what you are aware of. Hence a language like Scala, Fan, Groovy would be a good starting point.Once you get a hang of the basic semantics of using a functional language(as well as start loving it), you can move onto a language like Ruby. The turn around time for you in this way gets reduced as well as the fact that you can move towards being a polyglot programmer.
i would vote +1 for Groovy (and Grails). You can type with Java style or Groovy still (you can also mix both and have no worry about that). Also you can use Java libs.
As a general rule, avoid dynamically typed languages. The loss of compile time checking and the self-documenting nature of strong, static typing is well worth the necessity of putting type information into your source code. If the extra typing you need to do when writing your code is too great an effort, then a language with type inference (Scala, Haskell) might be of interest.
Having type information makes code much more readable, and readability should be your #1 criteria in coding. It is expensive for a person to read code, anything that inhibits clear, accurate understanding by the reader is a bad thing. In OO languages it is even worse, because you are always making new types. A reader just getting familiar will flounder because they do not know the types that are being passed around and modified. In Groovy, for example, the following is legal def accountHistoryReport(in, out) Reading that, I have no idea what in and out are. When you are looking at 20 different report methods that look just like that, you can quickly go completely homicidal.
If you really think you have to have non-static typing, then a language like Clojure is a good compromise. Lisp-like languages are built on a small set of key abstractions and massive amount of capability on each of the abstractions. So in Clojure, I will create a map (hash) that has the attributes of my object. It is a bit reductionalist, but I will not have to look through the whole code base for the implementation of some un-named class.
My rule of thumb is that I write scripts in dynamic languages, and systems in compiled, statically typed languages.

Benefits of learning scheme?

I've just started one of my courses, as classes just began 2 weeks ago, and we are learning Scheme right now in one for I assume some reason later on, but so far from what he is teaching is basically how to write in scheme. As I sit here trying to stay awake I'm just trying to grasp why I would want to know this, and why anyone uses it. What does it excel at? Next week I plan to ask him, whats the goal to learn here other than just how to write stuff in scheme.
It's a functional programming language and will do well broaden your experience.
Even if you don't use it in the real world doesn't mean it doesn't have any value. It will help you master things like recursion and help to force you to think of problems in different ways than you normally would.
I wish my school forced us to learn a functional programming language.
Languages like LISP (and the very closely related Scheme) are to programming what Latin is to English.
You may never speak Latin a day in your normal life again after taking a course, but simply learning a language like Latin will improve your ability to use English.
The same is true for Scheme.
I see all these people here saying that while they would never actually use Scheme again it's nevertheless been a worthwhile language to learn because it forces a certain way of thinking. While this can be true, I would hope that you would learn Scheme because you eventually will find it useful and not simply as an exercise in learning.
Though it's not blazingly fast like a compiled language, nor is it particularly useful at serving websites or parsing text, I've found that Scheme (and other lisps by extension) has no parallel when it comes to simplicity, elegance, and powerful functional manipulation of complex data structures. To be honest, I think in Scheme. It's the language I solve problems in. Don't give up on or merely tolerate Scheme - give it a chance and it won't disappoint you.
By the way, the best IDE for Scheme is DrScheme, and it contains language extensions to do anything you can do in another language, and if you find something it can't you can just use the C FFI and write it yourself.
I would suggest to keep an open mind when learning. Most of the time in school we don't fully comprehend what/why we are learning a particular subject. But as I've experienced about a million times in life, it turns out to be very useful and at the very least being aware of it helps you. Scheme, believe it or not, will make you a better programmer.
Some people say Scheme's greatest strength is as a teaching language. While it is very beneficial to learn functional programming (it's an entirely new way of thinking) another benefit in learning scheme is that it is also "pure". Sure it can't do a ton of stuff like java, but that's also what's great about it, it's a language made entirely of parentheses, alphanumeric characters, and a mere handful other punctuations.
In my intro course, we are taught Java, and I see lots of my friends struggling with 'public static void main' even though that's not the point of the program and how the profs have no choice but to 'handwave' it until they're more advanced. You don't see that in Scheme.
If you really want to learn what Scheme can do in a piece of cake that is really hard to implement in languages like Java, I suggest looking at this: http://mitpress.mit.edu/sicp/full-text/book/book-Z-H-12.html#%_sec_1.3
This is probably the best book written on Scheme.
Scheme was used by NASA to program some of the Mars rovers. It's usage in the marketplace is pretty specific, but like I'm sure your teachers are telling you, the things you learn in Scheme will carry over to programming in general.
Try not to get caught up on details like the parenthesis, and car/cdr. Most of what you're learning translates to other languages in one way or another. Don't worry about whether or not you can take Scheme to the market place, chances are you'll be learning some other more marketable languages in other classes. What you are learning here is more important.
If you are learning scheme, you can learn all about how object systems are implemented (hint: an object system isn't always about a type that has methods and instance variables bound to it...). While this kind of knowledge won't help in 95% of your daily work, for 5% of your work you will depend on that knowledge.
Additionally, you can learn about completely different styles of computation, such as streams/lazy evaluation, or even logic programming. You could also learn more about how computer programs in general are interpreted; from the basics in how program code is evaluated, to more deeper aspects like making your own interpreter and compiler). Knowing this kind of information is what separates a good programmer from a great programmer.
Scheme is not really a Functional language, it's more method agnostic then that. Perhaps more to the point, Scheme is an excellent language to choose if you want to explore with different methods of computation. As an example, a highly parallel functional language "Termite" was built on top of Scheme.
In short, the point in learning scheme is so that you can learn the fundamentals of programming.
If you need some help in making programming in scheme more enjoyable, don't be afraid to ask. A lot of programmers get hung up on (for instance) the parenthesis, when there are perfectly great ways to work with scheme source code that makes parenthesis something to cherish, rather then hate. As an example, emacs with paredit-mode,some kind of scheme interaction mode and highlight-parenthesis-mode is pretty awesome.
My problem was when learning this we learned clisp right along with it. I couldn't keep the two strait to save my life.
What I did learn from them though was how to write better c and java code. This is simply because of the different programming style I learned. I have adapted more of the functional style into some of my programming and It has helped me in some cases.
I would never want to program in scheme or lisp again if I didn't have to, but I am glad that I at least did a little in them just to learn the different way to program.
Functional languages like Scheme have great application to mathematics, artificial intelligence, linguistics, and other highly theoretical areas of computer science (machine learning, natural language processing, etc). This is due to the purity of functional programming languages, which have no side effects, as well as their ability to navigate higher-order procedures with ease. A strong knowledge of functional programming languages is critical for solving many of the questions which hover just beyond the frontier of computer science. As a bonus, you'll get great with higher-order procedures and recursion.

Resources