I would like to build an application that uses Genetic Programming to figure out what exactly the user is asking. It's a programming application for non-programmers. Basically the user feeds the application a bunch of examples, and from the examples the application will derive the rules required to build a new program for the user's own use/distribution.
I've built prototypes using linear regression but it could only solve simple problems. This week I experimented genetic programming using pyevolve and it worked much more brilliantly than I expected! However, I suspect it being written in pure python made it require dozens of seconds to solve an example, whereas in my application I only have at most a couple of seconds time.
I've been trying to find a more performant library that was as easy to use as pyevolve but cannot find a suitable one. I tried openBeagle but after getting an example running, and hours of poring through the documentation later, I still cannot find a way to actually pick an individual out of the "Vivarium". I've seen people recommend GAUL but that is a GPL library and will limit how I can license my future application. I've tried to download lil-gp but the ftp download links are locked by a university's login screen.
Since the application will be a Mac OS X cocoa application, I did not consider Java, C# or Matlab GP libraries.
As a developer of Open BEAGLE I still recommend you to use that library if you seek a fast GP library. Retrieving your best individual would actually be done by running a second program that parses the XML file that is logged at the end of the evolution. Otherwise, you can access it through the Vivarium.getHallOfFame() method and then sort it and access the first element with the HallOfFame.operator[]. The Member you'll get is a struct of the individual with the generation it was recorded and in what deme it was.
That way you can get access to the best individual that ever lived in your evolution.
If you have specific questions on Open BEAGLE I recommend you to ask them directly to the developer list, we usually answer very quickly.
Although, if you wish to try a very different library in Python I recommend you DEAP that allows a lot more flexibility than Pyevolve. Some GP examples run much faster under PyPy than Python.
If you ask the key developer of the GAUL project for permission to use an alternative license agreement, then he* is quite likely to agree.
*"he" is me.
Related
I am looking into developing a commercial application using PHP. Since I have experience in CodeIgniter and it has been working well for me, I decided to use it. Now, if ever my application comes to a point that it needs to grow and have to have custom modifications in the platform, is it possible to modify the CodeIgniter source code to the point that it's far from the original?
I'd strongly recommend not modifying the source code as this is the back bone of the application will make updating to futures releases of codeigniter impossible.
Instead you should be creating your own classes/libraries that extend the core bases. This is best practice.
The codeigniter users guide has some fantastic information regarding this, http://codeigniter.com/user_guide/general/creating_libraries.html
Good luck.
Don't modify it, extend it. Ellislab is good about releasing bug fixes and patches, and the last thing you want is to have to re-engineer all the updates into your new hybrid everytime they release updated source code. Plus, different features and bug fixes are released at different paces for the core and reactor, so unless you plan to lock your framework in at the current version, extending is the way to go.
The good news is CI is built for and encourages extensions of the source. The system folder and the application folder separation are a clear indication of how you should segregate your enhancements from the base libraries.
CodeIgniter is written in PHP. You can completely rewrite the framework anyway you like. So the answer is "yes".
There are many big websites that were written in a different language or build on a framework when they started and have been rewritten many times since. I think a lot of developers (mostly freelancers) are over-thinking when starting out. When I start a project, then I try to get it done and get it out in the web asap. If a website starts taking of - and most of the times start making you money - then you can always take the time to rewrite it.
I think it is a lot more important to spend time when you plan and design your database and tables. I think it is a lot harder to redo parts of your database later on then it is to rewrite the code that uses the data.
Just my 2 cents.
If you have used CI, then you should know that being a PHP framework that all of the source-code is there in the download, furthermore opening the system and application folders and looking through there will tell you a lot. Yes all of the source is there in plain English (plain programming English) and not only is it in plain English but has been extensively documented inside and out (literally that is in the source and in the user-guide). CI gained initial fame from that simple fact, that all code is extensively and meticulously documented.
Beyond all that, the question itself raises concerns that maybe you should study CI a bit further before writing commercial applications using it. Ci is a powerful and very easy to use PHP framework, but it is not a WYSIWYG. In my opinion, a coder should know his tools inside and out in order to be able to create a solid secure and trusted application. The first measure of which is to read the user manual, you should know at the very least everything in it, and since there are 12 sections in it that cover everything from extending CI libraries to creating your own libraries, and everything in between I would say you need to spend a little more time with it.
I want to say though, I am not being rude or trying to shame you in any way I am simply saying that you should learn the framework a bit more before venturing into a commercial application using it.
In the early days of PHP people realized how amazingly easy it was to use and how fast you could write an application with it. At the time the major options where very difficult for new and hobby programmers to use or involved expensive software to run, PHP was free, easy to learn and most of all ran on a free OS. It also took hardly any setup to get going, you could download PHP and essentially be programming in minutes. All of these factors lead to the almost destruction of the language.
Entry level programmers were destroying it with bits of code taken from other applications, never knowing (or caring) what the code actually did beyond the simple fact that it did what they wanted at the time, never considering or even investigating if the code might be harmful. Because of this practice PHP applications that had grown to Goliath sized websites, taking thousands of hits an hour were:
beginning to crash
being hacked to reveal sensitive customer/client data
generally crumbling all around the web
All because since the language was so easy to use that people had taken advantage of it and failed to take time to learn it. PHP was becoming a joke to other professional programmers and wasn't even thought of as a viable application language by many who had dubbed it "the copy and paste" language.
So my advice to you, please take the time to know your tools inside and out, what makes them tick, if they have any gotchas and where they are vulnerable. I understand that in order to learn a language to a professional level you have to build with it so I suggest that you take it slow with CI stick to the core for now. Trust me when I say that even in its purest form CI is an amazing and powerful tool that in the right hands can create awesome powerful web application, but in the uneducated/inexperienced hands it can create havoc and destruction.
So (stepping off of the soap box) I simply ask that if you are serious about creating commercial applications period that you take your time and learn your tools/language become as close to an expert on them as possible. I gurentee that if you do that you will always have work when you need it and you will spend less hours beating your head against the table or worse explaining to a client why their site is down.
I truly wish you good luck, just slow down and learn your trade and you will do just fine.
Yes, Codeigniter is an open source framework. However, I would advise against modifying the core of Codeigniter, as most files can be extended and rewritten safely without modifying the core files which will cause you headaches if you ever decide to update.
To extend a core class by default you would do this in Codeigniter. We'll extended the parser class for this example, but this applies to all classes pretty much. This link in the comprehensive user guide will give you all the information you need to extended and overload methods inside of a Codeigniter core class: http://codeigniter.com/user_guide/general/core_classes.html
What resources are available that use benchmarks for comparing programming languages?
I am interested in both
How quickly a program in a given language can execute a given benchmark?
How many lines of code are required in a given language to implement a given benchmark?
There is a long-standing web site called the Computer Language Benchmarks Game, originally created by Doug Bagley as the "Great Computer Language Shootout". (You can view a little history at Portland Patterns Repository.)
Is anyone aware of other resources that enable programmers to compare performance and size of programs written in different languages?
Alternatives
After a quick google search, I found a couple other sites where benchmarks for various languages have been done. Some other sites mention the programming language shootout site that is currently down.
There is a CPAN module for Perl that uses the same code found on that site.
Google has a directory where pages on this topic can be found. I have not found any yet that are as comprehensive as the page you speak of, but there are certainly other resources out there for comparisons.
Archived / Cached Page
If you're only seeking some information there, you can view archived pages of the site using the Wayback Machine or Google's cached version. Try searching Google with "site: shootout.alioth.debian.org" and click on the "Cached" links for the pages you find.
Find the Author?
Perhaps the best option is to try to contact the owner of the old site and find out what happened. The author mentioned in the BSD licence on this page is "Brent Fulgham". He may or may not be the one to contact.
Wait until Alioth is Fixed
As #ioguy found out, Debian's Alioth server that hosts the site in question is currently under maintenance. I would suggest subscribing to the debian-devel-announce mailing list for updates, and an idea of when it may be fully functional again.
If you find problems in the future, you can probably post to the debian-user list.
Each year there are two or three
isolated blog posts that claim to
compare performance and size of one
or two programs written in different
languages.
As a resource the blog posts fail for obvious
reasons, most obviously:
not updated with newer versions of the language implementation
not updated with better programs
Every couple of years someone
dissatisfied with something about
the benchmarks game (often some
detail about the code repository or
website technology) starts a project that will
fix everything they dislike about the benchmarks game.
As a resource the most obvious problem with those
projects is that they never seem to get
close to publishing performance
data.
Every year some group of programmers
campaigns to have language X
included in the benchmarks game,
while some other group demands that
some program is included (or
excluded).
Sadly, they rarely accept that among
the resources provided by the
benchmarks game are
scripts they can use to make and publish language performance
measurements
examples of which basic information (language version, build
commands, run commands, measurement
techniques, ...) is required to provide context for the measurements.
They rarely accept that they are
empowered to create what they wish
to see.
The benchmarks game website is now back to normal!
From Friday 20 May 2011 through Monday 23 May 2011, ALL alioth.debian.org subdomains were down - because the alioth admins were upgrading "in every way we can find: kernel, Debian release, FusionForge software, hardware, and so on."
In addition, making the benchmarks game website work again required:
installation of the GD library on the new server, for chart generation
basic information about changes to ssh use on the new servers
basic information about the project cvs repository on the new servers
basic information about the project /htdocs location on the new servers
replacement of the long deprecated
$HTTP_GET_VARS by $_GET in a couple
of dozen PHP scripts
Since the performance benchmark site
for Programming Languages (aka
Programming Language "Shootout" &
shootout.alioth.debian.org) is
permanently down ...
The original question was predicated on a false premise.
What would you suggest would be a good challenge for a programmer to show us her/his skills? I'm thinking of a small demo implementation of a GUI program which would not take too much time to do.
Here are the circumstances: (this should not imply the intention to find programmers here, I think there'd be other forums to do that)
We are planning a project which has a tight time frame but apparently we are short on resources so we want to pull in external developers. The project is targetted to be Qt based (although this is not yet finally set) on the Windows platform. We'd prefer Qt as this allows to use own resources later when features need to be added to the software and we are familiar with the Qt platform.
The project needs to interface with HID USB hardware (writing some data blocks out, reading back the result, within to be guaranteed time frames) and a GUI showing graphs of the analyses.
The main intention however is not to find a Qt programmer (although we would prefer that) but a capable programmer - thus the important part of this question is about the challenge.
Don't ask programmers to write something from scratch as an interview task. It's far too suspect.
Think of the qualities that you want in a developer and then write an application that has all of those things done wrong, and ask them to fix it. For example, if you want an Object Oriented developer, give them an application with the data tables directly bound to the UI and ask them to make it OO - it means they can show you in a few minutes that they have OO skills.
By starting with a sample application that is "fixed up" with all the problems, it makes it really easy to compare the results and it will be a much faster test than if you ask people to write something from scratch.
Don't forget to make the test measurable. Score each thing you are testing as well as how long it takes.
I did a numeric method as my diploma thesis and coded it in java. It needs a lot of computational time when adequately executed. So I looked for an alternative and found BOINC. Unfortunately I didn't have time for doing my method in BOINC, because I'm an Aerospace student and not a programmer and I decided to keep my priority on my java program. Now it's finished an I still would like to port this to BOINC environment.
Unfortunately I'm learning in re-doing examples and I couldn't find any, neither on the official site http://boinc.berkeley.edu nor in the internet.
So do you know a good and easy example or do you have any experience in BOINC and would like to start a new platform for such a boinc project?
I'm realistic about my method, that it wouldn't run 24/7, because there aren't as many work units as for seti or folding projects. So I would like to have a platform for more than just my project so that another platform project can be worked on, when one part of the project does not have any work units at that moment.
But to start this, I would keep it simple and just want to know how to code it and use it in the client and server system. It doesn't matter what the example projects will work on, as long as it is simple enough, that I can understand it and extending it for my method.
Thank you in advance, Andreas! :)
PS: I know that BOINC supports JAVA as a programming language, and my method is coded in JAVA.
As far as I know, JavaApps is just an idea; I don't know if anyone actually tried it in a real BOINC project. And it's Windows-only. And it seems to be a bit of a pain to redistribute the entire JRE as part of the BOINC application (both technically and legally).
Also, I generally dislike using that kind of “wrapper” where the science app (using the BOINC API) starts another process that then does the real computation. It's usually unreliable. There are lots of things that could go wrong with the wrapper, especially related to controlling the child process (eg. if something kills the wrapper, the child process has to quit too).
However, I just found something pretty interesting that may let me do a better Java wrapper for BOINC... Stay tuned! (but don't hold your breath either; it's the holidays!)
Meanwhile, I suggest you start by reading BOINC wiki and setting up a server with a “hello world” application; and if you have any trouble, ask a specific question about your trouble either here or in the boinc_projects mailing list.
(Of course, payin’ me to install the server for you is also an option ;) but I can't guarantee anything; not even my mere availability at this time of the year)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
What successful conversion/rewrite have you done of software you were involved with? What where the languages and framework involved in the process? How large was the software in question? Finally what is the top one or two thing you learned from being involved with the process.
This is related to this question
I'm going for "most abstruse" here:
Ported an 8080 simulator written in
FORTRAN 77 from a DECSystem-10 running TOPS-10 to an
IBM 4381 mainframe running VM/CMS.
I rewrote 20,000 lines of Perl to use "use strict" in every file. I had to add "my" everywhere it was needed and I had to fix the bugs that were uncovered during the process.
The biggest thing I learned from doing this is, "It always takes longer than you think."
I had to get it done all at once overnight so that the other coders would not be writing new, unfixed code at the same time. I thought it would go quickly, but it didn't, and I was still hacking on it at 6 AM the next morning.
I did get it complete and checked in before everyone else started work though!
I rewrote a large java web application to an ASP.Net application for a realty company for various reasons.
The biggest thing I learned is that, no matter how trivial the feature the original system had, if it's not in the second system, the client thinks the rewrite is a failure. Expectation management is everything when writing the new system.
This is the biggest reason rewrites are so hard: it seems so easy to the client ("Just re-do what I already have and add a few things.").
The coolest one for me, I think, was the port of MAME to the iPod. It was a great learning experience with embedded hardware, and I got to work with a lot of great people. Official site.
I am doing a rewrite of an Inhouse Project managment system to a more standard MVC model. Its in the LAMP stack (PHP) and i am close to the 1st milestone.
The things i have learned from that currently is how simple the program feels at the beginning and i tried to not add complexity until i have to.
Example is that i programmed all the functionality first (like i was an admin user) and then when that is sorted out, add the complexity of having restrictions (user levels etc)
I ported/redesigned/rewrote a 30,000-line MS-DOS C++ program into a similar-length but much more fully-featured and usable Java Swing program.
I learned never to take another job involving C++ or Java.
I ported a client server Powerbuilder app, a couple of hundred screens worth, into an ASP.NET app (C#).
Due to performance and maintainability issues, I had over the previous year moved a ton of embedded SQL out of Powerbuilder scripts and into stored procedures.
Although this would make a lot of you wince, having a lot of business logic in the database, it mean the Powerbuilder app was relatively "light" and when we built the .Net front end, it could take advantage of the SQL codebase and have a lot of functionality already built and tested.
Not saying I'd recommend building apps that way, but it certainly worked to our advantage in this instance.
We had a code generation tool in our application framework that was used to read in text-based data files, About 20 other applications made use of it.
We wanted to make use of XML data files instead of structured text-based files. The original code was quite outdated and difficult to maintain. We replaced this tool by a combination of XSLT scripts and a utility library. For the utility library we could make use of some code in the old tool.
The result was that all 20 applications could now make use of either the obsolete text based file format or the new XML based format. We also delivered a conversion-generation tool that converted old data files to new XML data files.
After bringing out one or two release we have now decided that we will no longer support the old text based format and everybody is able to convert their data to XML.
We did hardly have to do manual conversions,
Converted the main company app from pre-standard C++ to standard C++. We had a multimillion dollar sale contingent on making it work on AIX, and after looking at it we decided that converting to standard C++ was going to be just as easy as converting to IBM's traditional C++.
I don't know the line count, but the source code ran to hundreds of megabytes.
We used standard Unix tools to do this, including vi and the assorted compilers.
It took a few months. Most of the fixes were simple ones, caught by the compiler and almost mechanically fixed. Some of them were much more complicated.
I think my main takeaway was: Don't get too awfully clever with code in a language that hasn't been standardized yet, or is likely to have things change in unexpected ways. We had to do a lot of digging in some of the ingenious adaptations/abuses of C++ streams.
Ten years ago I managed a team that converted a CAD system from DOS into Windows. The DOS version used home-brew libraries for graphics drawing, the Windows version used MFC. The software was about 70.000 lines of C code at the time of the conversion. The most important thing we learned in the process is the power of abstraction. All device-specific non-portable routines were isolated in a few files. It was therefore relatively easy to substitute the calls to the DOS-based library that would draw by directly accessing the frame buffer with Windows API calls. Similarly, for input we just substituted the event loop that checked for keyboard and mouse events, with the corresponding Windows event loop. We continued our policy of isolating the non-portable (this time Windows) code from the rest of the system, but we have not yet found this particularly useful. Perhaps one day we will port the system to Mac OS X and be thankful again.
Several. But I mention one.
It was a performance modeling tool. Part delphi 1, part turbo pascal. It needed a rewrite else it was not going to survive. So we started as a team of 2, but only me survived to the end. And I was ready before the deadline ;-).
Several things we did:
Make it multimodel. The original had lots of globals. I removed them all and multi model was easy to adapt.
Extended error messages. Click on a message and get the help.
Lots of graphs and diagrams. All clickable to drill down.
Simulation. Change parameters over time and see how long the current configuration was enough.
We really made this one clean and it paid back heavily in the end. Such a big learning experience.
Re-wrote a system for a company that processes legal invoices - the original system was a VB monstrosity that had no idea of good OO principles - everything was mixed together. The HTML did SQL, and the SQL wrote HTML. A large part of it was a custom rules engine that used something like XML for the rules.
Two teams did the re-write, which took about 9 months. One team did the web front end and the backend workflow, while the other team (that I was on) re-wrote the rules engine. The new system was written in C#, and was done test-first. Adding new rules to the system when we were done was dirt simple, and it was all testable. Along the way we did things like convert the company from VSS to SVN, implement continuous integration, automate the deployment, and teach the other developers how to do TDD and other Scrum/XP practices.
Managing expectations was crucial through the project. Having a customer that was savvy about software was very helpful.
Having a mix of large scale (end-to-end) tests along with comprehensive unit and integration tests helped tons.
Converted vBulletin which is written in PHP into C#/Asp.NET. I'm pretty familiar with both languages, but PHP is the hands down the winner for building that software. The biggest pain in the rear was needing to do a C# equivalent of PHP's eval() for calling the templates.
It was my first challenge in trying to do a conversion. I learned that I need more experience with C# and that writing it from scratch is just the easier route sometimes.
I converted a dynamical build-process completely written in Perl to a C#/.Net solution using a workflow-engine a co-worker had developed (which was still in beta - so I had to do some refinements). That gave me the oppertunity to add fail-safe and fail-over functionality to the build process.
Before you ask - no - the microsoft workflow-foundation could not be used since you cannot dynamically change a process during its runtime.
What I learned:
to hate the Perl-developer
process-optimization using a wf-engine
fail-safe and fail-over strategies
some C# tweaks ;)
In the end it covered about 5k - 6k (including the wf-engine) LoC origin from 3 200 LoC Perl-files. But it was fun - and far better in the end ;)
Converting theoretically portable C code into theoretically portable C code across architectures to support a hardware change that saves the company X dollars per unit.
The size varies - this is a common need, and I've done small and large projects.
I learned to write more portable C code. Elegance is great, but when it comes right down to it the compiler takes care of performance, and the code should be as simple and portable as possible.
Ported a simulation written in Fortran 77 (despite being written in the 90s) to C/Java because the original only worked on small data sets. I learned to love big O notation after several times of explaining why just moving the entire data table into memory at the start of the program was not going to scale.
Migrating the B-2 Stealth Bomber mission software from JOVIAL to C. 100% fully automated conversion. Seriously!
Main lesson: using configurable automated conversion tools is a huge win.
See DMS Software Reengineering Toolkit.