I'm trying to perform a genetic programming system which solves an equation (basically first and second degree polynom equations) symboliclly.
It means that for a*x+b=0 it must give me a tree representation of -b/a. What i mean by symbolically is that i'm not giving numbers for 'a' and 'b' ... basically what Generic Programming are made for.
I'm actually stuck in finding the fitness function. For a given gene (potential solution), how can i predict how far is this one from the right solution.
My doubt is that there's no way to do such a thing unless i convert the problem into a numerical one.
I've been searching in the net, but all i found was related to genetic algorithms and so numerical resolution for such problem.
a link exposing such topic is found here: Generic Programming but it's not explaining a detailed approach about the problem.
Any ideas ?
Thanks in advance.
I've finally found a solution.
I've found a powerfull library which is designed for Genetic Algorithms for java developpers, and have a module specialized in Genetic Programming (GP).
Library's link: Jenetics
My implementation of the solution is available on my github at this link: Symbolic Equation Solver
It's basically a Maven based project, which makes use of Java 11 and the above library to solve Linear and Quadratic equations and gives symbolic (and not numeric) result.
Related
Both NN and Greedy Search algorithms have a Greed nature, and both have tendency towards the lowest cost/distance (my understanding may be incorrect though). But what makes them different in a way that each one can be classified into a distinct algorithm group is somehow unclear to me.
For instance, if I can solve a particular problem using NN, I can surely solve it with Greedy Search algorithm as well specially if minimization is the case. I came to this conclusion because when I start coding them I come across very similar implementations in code although the general concept behind both might be different. Sometimes I can't even tell if the implementation follows NN or Greedy Search.
I have done my homework well and searched enough on Google, but couldn't find a decent explanation on what distinguishes them from one another. Any such explanation is indeed appreciated.
Hmm, at a very high level they both driven by heuristics in order to evaluate a given solution against an ideal solution. But, whilst a greedy search algo outputs a solution for a given input, the NN trains a model that will generate solutions for given inputs. So at a very very high level, you can think that the NN generates a solution finder, whereas the greedy search is a harcoded solution finder.
In other words, the NN will generate "code" (i.e. the model (aka the weights)) that finds solutions to the problem when provided to the same network topology. The greedy search is you actually writing the code that finds the solution to the problem. This is quite wishy washy though, I'm sure there is a much more concise, academically sound way of expressing what I've just said
All of what I've just said in based on the assumption that by "Greedy search" you meant the algorithms to solve problems such as travelling sales man.
Another way to think of it is:
In greedy search, you write an algorithm that solves a search problem (find me the graph that best describes the relationship, based on provided heuristic(s), between data point A and data point B).
When you write a neural network, you declare a network topology, provide some initially "random" weights and some heuristics to measure output errors and then train the networks weights via a plethora of different methods (back prop, GAN etc). These weights can then be used as a solver for novel problems.
For what it's worth, I don't think an NN would be a good approach to generate a solver for travelling sales man problem. You would be far better off just using a common graph search algorithm..
I am a student from Germany, currently doing the master thesis. In my master thesis, I am writing a Fortran code in code blocks. In my code, I am using some of the LAPACK functions.
I want help regarding adding LAPACK library in code block software. I searched a lot on the internet but I couldn't find anything. It'll be better if you provide me all the source links extension of the previous question.
In my code, I need to solve the following system of equation, {K}{p} = {m}
Where
{K} = Matrix
{p} = vector
{m} = vector
I have all the elements of vector {m} and matrix {K} computed and I have some known values of vector {p}. It's boundary value problem.
Now I want to find out only unknown values of elements of a vector {p}.
Which function should I use?
I went through the LAPACK manual available online but couldn't find.
Well you did not go through the documentation so thoroughly :) You are looking for solvers of a linear equation.
http://www.netlib.org/lapack/lug/node38.html
Please specify a little more about complex/real double/float. ill posed? Over or under determined or quadratic? If quadratic then symmetric or upper triangular? Banded?
There is a hord of different algorithms one would consider. Runtime / stability / convergence behaviour are very different dependent on K. The most stable would be [x]gelsd. It's a divide and conquer algorithm via SVD and gives you with proper conditioning the moore penrose generalised inverse. But it is by far the slowest algorithm too.
btw, http://www.netlib.org/lapack/lug/node27.html, outlines all general solvers.
If you already have some values of your p, you would like to go a different route than a straight forward inversion. You are a lot better off, if you use an iterative method like conjugate gradient least squares problem with regularization. This is discussed in length in Stoer, Bullirsch, Introduction to Numerical Analysis, Chapter 8.7 (The Conjugate-gradient Method of Heestens and Stiefel).
There are multiple implementations online. One you will find in a library I wrote during my PhD: https://github.com/kvahed/codeare/blob/master/src/optimisation/CGLS.hpp
I am working on optimization using evolutionary algorithm, specifically SPEA-II. I was curious if there is any algorithm named SPEA-III. I tried googling but I could only find a proposed modification of the SPEA-II using reference search direction. Is it same as SPEA-III, in case of NSGA-III, this is the probably only difference from NSGA-II.
I was able find a reference paper on SPEA3, the authors have proposed what they called a generalization of SPEA2 algorithm and name it as SPEA3.
You might have to read this twice so that my idea becomes clear. Please be patient.
I'm looking for existing work for exhausive search for algorithms for given problems. Exhaustive search is also known as brute force search, or simply brute force.
Other exhaustive search algorithms search for a solution for a given problem. Usually, a solution for such a problem is some data which fulfills some requirements.
Exhaustive search example:
You want the solution for a KNAPSACK problem. That is the objects which can be packed into the bag so that there is no other combination of objects which fit into the bag and which would sum up to a bigger value than your result combination.
You can solve this by going over all possible combinations (exhaustive) and search for the one which fits into the bag and which is the most valuable one of those combinations.
What I'm looking for is just a special case of exhaustive search: Exhaustive search which searches for an algorithm as a solution. So in the end, I'm looking for an algorithm which searches for an algorithm which solves some given problem.
You might say: Go google it. Well, yes I've done that. The difficulty I'm facing here is that googling for "algorithm which searches for another algorithm" results in all the same as "another search algorithm". Obviously, this has too many unwanted results, so I'm stuck here.
How can I find existing work related to exhaustive search for algorithms?
More specifically: Has there any software been written for this? Can you point me to any links or algorithm names/better keywords in relation to this topic?
Update:
The purpose for which I'm looking for such an algorithm search is solving problems for which no good heuristics are known, e.g. prooving-algorithms or trying to finding other solution algorithms for problems which might or might not be NP-complete problems (thus proving that the problem is not NP-complete if a faster algorithm can be found; without any human interaction).
You seem to be looking for "program synthesis", which can work in some limited instances, provided you can correctly and formally specify what your algorithm is supposed to do (without giving an implementation).
Synthesis is an effective approach to build gate-level circuits, but applying synthesis to software is so far still more of a research avenue than practical.
Still, here are a couple of references on the subject,
(Some of the most advanced work in the area in my opinion, has a tool)
program sketching by Armando Solar-Lezama
Check out Microsoft research page on the topic, they think it's hot topic: http://research.microsoft.com/en-us/um/people/sumitg/pubs/synthesis.html
Some other similar stuff I've seen :
Model Checking-Based Genetic Programming with an Application to Mutual Exclusion. (Katz & Peled # TACAS '08), they have a more recent version on ArXiv : http://arxiv.org/abs/1402.6785
Essentially the search space is explored (exhaustively) using a model-checker.
I wouldn't think it would be possible (at least without some restriction on the class of algorithms) and, in any event, the search space would be so large that it would make ordinary brute-force tame by comparison. You could, for example, enumerate algorithms for a specific model of computation (e.g. Turing machines) but then the Halting problem rears its head -- how can you tell if it solves your problem? If you have a family of algorithms which depend on discrete parameters then you could of course brute force the choice of the parameters.
There is an extensive literature on Genetic Programming (see https://en.wikipedia.org/wiki/Genetic_programming ). That might give you something to go on. In particular -- the tree data structures that are often used in this context (essentially expression trees or more generally abstract syntax trees) could admit a brute force enumeration.
Take a look at Neural Turing Machines by Alex Graves, Greg Wayne and Ivo Danihelka from Google DeepMind.
Abstract:
We extend the capabilities of neural networks by coupling them to
external memory resources, which they can interact with by attentional
processes. The combined system is analogous to a Turing Machine or Von
Neumann architecture but is differentiable end-to-end, allowing it to
be efficiently trained with gradient descent. Preliminary results
demonstrate that Neural Turing Machines can infer simple algorithms
such as copying, sorting, and associative recall from input and output
examples.
This paper, entitled "Systematic search for lambda expressions" performs exhaustive enumeration in the space of small type-safe functional programs expressed in the lambda calculus.
I used Genetic Programming for evolving/generating heuristics for the Travelling Salesman Problem. The evolved heuristic is better than the Nearest Neighbor on the tested problems (random graphs and other graphs taken from TSPLIB). If you want the source code, you can download it from here: http://www.tcreate.org/evolve_heuristics.html
I am currently following an algorithms class and we have to solve a sudoku.
I have already a working solution with naive backtracking but I'm much more interest in solving this puzzle problem with a tree data structure.
My problem is that I don't quite understand how it works. Is anyone can explain to me the basic of puzzle solving with tree?
I don't seek optimization. I looking for explanation on algorithms like the Genetic algorithm or something similar. My purpose only to learn at this point. I have hard time to take what I read in scientific articles and translate it on real implementation.
I Hope, I've made my question more clear.
Thank you very much!
EDIT: I edit the post to be more precise.
I don't know how to solve Sudoku "with a tree", but you're trying to mark as many cells as possible before trying to guess and using backtrace. Therefore check out this question.
Like in most search problems, also in Sudoku you may associate a tree with the problem. In this case nodes are partial assignments to the variables and branches correspond to extensions of the assignment in the parent node. However it is not practical (or even feasible) to store these trees in memory, you can think of them as a conceptual tool. Backtracking actually navigates such a tree using variable ordering to avoid exploring hopeless branches. A better (faster) solution can be obtained by constraint propagation techniques since you know that all rows, columns, and 3x3 cells must contain different values. A simple algorithm for that is AC3 and it is covered in most introductory AI books including Russell & Norvig 2010.