I have two confusions when I use machine learning algorithm. At first, I have to say that I just use it.
There are two categories A and B, if I want to pick as many as A from their mixture, what kind of algorithm should I use ( no need to consider the number of samples) . At first I thought it should be a classification algorithm. And I use for example boost decision tree in a package TMVA, but someone told me that BDT is a regression algorithm indeed.
I find when I have coarse data. If I analysis it ( do some combinations ...) before I throw it to BDT, the result is better than I throw the coarse data into BDT. Since the coarse data contains every information, why do I need analysis it myself?
Is you are not clear, please just add a comment. And hope you can give me any advise.
For 2, you have to perform some manipulation on data and feed it to perform better because from it is not built into algorithm to analyze. It only looks at data and classifies. The problem of analysis as you put it is called feature selection or feature engineering and it has to be done by hand (of course unless you are using some kind of technique that learns features eg. deep learning). In machine learning, it has been seen a lot of times that manipulated/engineered features perform better than raw features.
For 1, I think BDT can be used for regression as well as classification. This looks like a classification problem (to choose or not to choose). Hence you should use a classification algorithm
Are you sure ML is the approach for your problem? In case it is, some classification algorithms would be:
logistic regression, neural networks, support vector machines,desicion trees just to name a few.
In the context of Multi-Class Classification (MCC) problem,
a common approach is to build final solution from multiple binary classifiers.
Two composition strategy typically mentioned are one-vs-all and one-vs-one.
In order to distinguish the approach,
it is clearer to look at what each binary classifier attempt to do.
One-vs-all's primitive classifier attempt to separate just one class from the rest.
Whereas one-vs-one's primitive attempts to separate one against
One-vs-one is also, quite confusingly, called all-vs-all and all-pairs.
I want to investigate this rather simple idea of building
MCC classifier by composing binary classifier in
binary-decision-tree-like manner.
For an illustrative example:
has wings?
/ \
quack? nyan?
/ \ / \
duck bird cat dog
As you can see the has wings? does a 2-vs-2 classification,
so I am calling the approach many-vs-many.
The problem is, I don't know where to start reading.
Is there a good paper you would recommend?
To give a bit more context,
I'm considering using a multilevel evolutionary algorithm (MLEA) to build the tree.
So if there is an even more direct answer, it would be most welcomed.
Edit: For more context (and perhaps you might find it useful),
I read this paper which is one of the GECCO 2011 best paper winners;
It uses MLEA to compose MCC in one-vs-all manner.
This is what inspired me to look for a way to modify it as decision tree builder.
What you want looks very much like Decision Trees.
From wiki:
Decision tree learning, used in statistics, data mining and machine learning, uses a decision tree as a predictive model which maps observations about an item to conclusions about the item's target value. More descriptive names for such tree models are classification trees or regression trees. In these tree structures, leaves represent class labels and branches represent conjunctions of features that lead to those class labels.
Sailesh's answer is correct in that what you intend to build is a decision tree. There are many algorithms already for learning such trees such as e.g. Random Forests. You could e.g. try weka and see what is available there.
If you're more interested in evolutionary algorithms, I want to mention Genetic Programming. You can try for example our implementation in HeuristicLab. It can deal with numeric classes and attempts to find a formula (tree) that maps each row to its respective class using e.g. mean squared error (MSE) as fitness function.
There are also instance-based classification methods like nearest neighbor or kernel-based methods like support vector machines. Instance-based method also support multiple classes, but with kernel-methods you have to use one of the approaches you mentioned.
I am currently working on a project based on graph and I am searching for an algorithm for slicing an dynamic graph. I have already done some research but most algorithms that I have found works only for a static graph. In my environment, the graph is dynamic, it means that users add/delete elements, create/delete dependences at runtime.
(In reality I am working with UML models but UML models can be also represented by typed graphs, wich are composed of typed Vertices and edges)
I also search for the terms graph fragmentation but I did not find anything. And I would like to know if exist such algorithm for slicing a dynamic graph?
[UPDATE]
Sorry for not being clear and I am updating my question.Let me first expose the context.
In MDE (Model Driven Engineering), large-scale industrial systems involve nowadays hundreds of developpers working on hundreds of models representing pars of the whole system specification. In a such context, the approach commonly adopted is to use a central repository. The solution I provide for my project (I am currently working on a research lab), is a solution which is peer-to-peer oriented, that means that every developper has his own replication of the system specification.
My main problem is how to replicate this data, the models.
For instance, imagine Alice and Bob working on this UML diagram and Alice has the whole diagram in his repository. Bob wants to have the elements {FeedOrEntry, Entry}, how can I slice this diagram UML?
I search for the terms of "model Slicing".I have found one paper which gives an approach for slicing UML Class Diagrams but the problem with this algorithm is it only works for a static graph. In our context, developpers add/update/remove elements constantly and the shared elements should be consistent with the other replicas.
Since UML Models can also be seen as a graph, I also search for the terms for "graph slicing" or "graph fragment" but I have found nothing useful.
And I would like to know if exist such algorithm for slicing a dynamic graph
If you make slicing atomic, I see no problem with using algorithm shown in paper you linked.
However, for your consistency constraints, I believe that your p2p approach is incompatible. Alternative is merge operation, but I have no idea how would that operation work. It probably, at least partially, would have to be done manually.
Sounds like maybe you need a NoSQL graph database such as Neo4J or FlockDB. They can store billions of vertexes and edges.
What about to normalize the graph to an adjacent tree model? Then you can use a DFS or BFS to slice the graph?
I am trying to store a large list of strings in a concise manner so that they can be very quickly analyzed/searched through.
A directed acyclic word graph (DAWG) suits this purpose wonderfully. However, I do not have a list of the strings to include in the first place, so it must be incrementally buildable. Additionally, when I search through it for a string, I need to bring back data associated with the result (not just a boolean saying if it was present).
I have found information on a modification of the DAWG for string data tracking here: http://www.pathcom.com/~vadco/adtdawg.html It looks extremely, extremely complex and I am not sure I am capable of writing it.
I have also found a few research papers describing incremental building algorithms, though I've found that research papers in general are not very helpful.
I don't think I am advanced enough to be able to combine both of these algorithms myself. Is there documentation of an algorithm already that features these, or an alternative algorithm with good memory use & speed?
I wrote the ADTDAWG web page. Adding words after construction is not an option. The structure is nothing more than 4 arrays of unsigned integer types. It was designed to be immutable for total CPU cache inclusion, and minimal multi-thread access complexity.
The structure is an automaton that forms a minimal and perfect hash function. It was built for speed while traversing recursively using an explicit stack.
As published, it supports up to 18 characters. Including all 26 English chars will require further augmentation.
My advice is to use a standard Trie, with an array index stored in each node. Ya, it is going to seem infantile, but each END_OF_WORD node represents only one word. The ADTDAWG is a solution to each END_OF_WORD node in a traditional DAWG representing many, many words.
Minimal and perfect hash tables are not the sort of thing that you can just put together on the fly.
I am looking for something else to work on, or a job, so contact me, and I'll do what I can. For now, all I can say is that it is unrealistic to use heavy optimization on a structure that is subject to being changed frequently.
Java
For graph problems which require persistence, I'd take a look at the Neo4j graph DB project. Neo4j is designed to store large graphs and allow incremental building and modification of the data, which seems to meet the criteria you describe.
They have some good examples to get you going quickly and there's usually example code to get you started with most problems.
They have a DAG example with a link at the bottom to the full source code.
C++
If you're using C++, a common solution to graph building/analysis is to use the Boost graph library. To persist your graph you could maintain a file based version of the graph in GraphML (for example) and read and write to that file as your graph changes.
You may also want to look at a trie structure for this (potentially building a radix-tree). It seems like a decent 'simple' alternative structure.
I'm suggesting this for a few reasons:
I really don't have a full understanding of your result.
Definitely incremental to build.
Leaf nodes can contain any data you wish.
Subjectively, a simple algorithm.
As it currently stands, this question is not a good fit for our Q&A format. We expect answers to be supported by facts, references, or expertise, but this question will likely solicit debate, arguments, polling, or extended discussion. If you feel that this question can be improved and possibly reopened, visit the help center for guidance.
Closed 11 years ago.
After reading Stevey Yegge's Get That Job At Google article, I found this little quote interesting:
Whenever someone gives you a problem, think graphs. They are the most fundamental and flexible way of representing any kind of a relationship, so it's about a 50–50 shot that any interesting design problem has a graph involved in it. Make absolutely sure you can't think of a way to solve it using graphs before moving on to other solution types. This tip is important!
What are some examples of problems that are best represented and/or solved by graph data structures/algorithms?
One example I can think of: navigation units (ala Garmin, TomTom), that supply road directions from your current location to another, utilize graphs and advanced pathing algorithms.
What are some others?
Computer Networks: Graphs model intuitively model computer networks and the Internet. Often nodes will represent end-systems or routers, while edges represent connections between these systems.
Data Structures: Any data structure that makes use of pointers to link data together is making use of a graph of some kind. This includes tree structures and linked lists which are used all the time.
Pathing and Maps: Trying to find shortest or longest paths from some location to a destination makes use of graphs. This can include pathing like you see in an application like Google maps, or calculating paths for AI characters to take in a video game, and many other similar problems.
Constraint Satisfaction: A common problem in AI is to find some goal that satisfies a list of constraints. For example, for a University to set it's course schedules, it needs to make sure that certain courses don't conflict, that a professor isn't teaching two courses at the same time, that the lectures occur during certain timeslots, and so on. Constraint satisfaction problems like this are often modeled and solved using graphs.
Molecules: Graphs can be used to model atoms and molecules for studying their interaction and structure among other things.
I am very very interested in graph theory and ive used it solved so many different kinds of problem. You can solve a lot of Path related problem, matching problem, structure problems using graph.
Path problems have a lot of applications.
This was in a career cup's interview question.
Say you want to find the longest sum of a sub array. For example, [1, 2, 3, -1] has the longest sum of 6. Model it as a Directed Acyclic Graph (DAG), add a dummy source, dummy destination. Connect each node with an edge which has a weight corresponding to the number. Now use the Longest Path algorithm in the DAG to solve this problem.
Similarly, Arbitrage problems in financial world or even geometry problems of finding the longest overlapping structure is a similar path problem.
Some obvious ones would be the network problems (where your network could have computers people, organisation charts, etc).
You can glean a lot of structural information like
which point breaks the graph into two pieces
what is the best way to connect them
what is the best way to reach one place to another
is there a way to reach one place from another, etc.
I've solved a lot of project management related problems using graphs. A sequence of events can be pictured as a directed graph (if you don't have cycles then thats even better). So, now you can
sort the events according to their priority
you can find the event that is the most crucial (that is would free a lot of other projects)
you can find the duration needed to solve the total project (path problem), etc.
A lot of matching problems can be solved by graph. For example, if you need to match processors to the work load or match workers to their jobs. In my final exam, I had to match people to tables in restaurants. It follows the same principle (bipartite matching -> network flow algorithms). Its simple yet powerful.
A special graph, a tree, has numerous applications in the computer science world. For example, in the syntax of a programming language, or in a database indexing structure.
Most recently, I also used graphs in compiler optimization problems. I am using Morgan's Book, which is teaching me fascinating techniques.
The list really goes on and on and on. Graphs are a beautiful math abstraction for relation. You really can do wonders, if you can model it correctly. And since the graph theory has found so many applications, there are many active researches in the field. And because of numerous researches, we are seeing even more applications which is fuelling back researches.
If you want to get started on graph theory, get a good beginner discrete math book (Rosen comes to my mind), and you can buy books from authors like Fould or Even. CLRS also has good graph algorithms.
Your source code is tree structured, and a tree is a type of graph. Whenever you hear people talking about an AST (Abstract Syntax Tree), they're talking about a kind of graph.
Pointers form graph structures. Anything that walks pointers is doing some kind of graph manipulation.
The web is a huge directed graph. Google's key insight, that led them to dominate in search, is that the graph structure of the web is of comparable or greater importance than the textual content of the pages.
State machines are graphs. State machines are used in network protocols, regular expressions, games, and all kinds of other fields.
It's rather hard to think of anything you do that does not involve some sort of graph structure.
An example most people are familiar: build systems. Make is the typical example, but almost any good build system relies on a Directed Acyclic Graph. The basic idea is that the direction models the dependency between a source and a target, and you should "walk" the graph in a certain order to build things correctly -> this is an example of topological sort.
Another example is source control system: again based on a DAG. It is used for merging, for example, to find common parent.
Well, many program optimization algorithms that compilers use are based on graphs (e.g., figure out call graph, flow control, lots of static analysis).
Many optimization problems are based on graph. Since many problems are reducable to graph colouring and similar problems, then many other problems are also graph based.
I'm not sure I agree that graphs are the best way to represent every relation and I certainly try to avoid these "got a nail, let's find a hammer" approaches. Graphs often have poor memory representations and many algorithms are actually more efficient (in practice) when implemented with matrices, bitsets, and other things.
OCR. Picture a page of text scanned at an angle, with some noise in the image, where you must find the space between lines of text. One way is to make a graph of pixels, and find the shortest path from one side of the page to the other, where the difference in brightness is the distance between pixels.
This example is from the Algorithm Design Manual, which has lots of other real world examples of graph problems.
One popular example is garbage collection.
The collector starts with a set of references, then traverses all the objects they reference, then all the objects referenced there and so on. Everything it finds is added into a graph of reachable objects. All other objects are unreachable and collected.
To find out if two molecules can fit together. When developing drugs one is often interested in seeing if the drug molecules can fit into larger molecules in the body. The problem with determining whether this is possible is that molecules are not static. Different parts of the molecule can rotate around their chemical bindings so that a molecule can change into quite a lot of different shapes.
Each shape can be said to represent a point in a space consisting of shapes. Solving this problem involves finding a path through this space. You can do that by creating a roadmap through space, which is essentially a graph consisting of legal shapes and saying which shape a shape can turn into. By using a A* graph search algorithm through this roadmap you can find a solution.
Okay that was a lot of babble that perhaps wasn't very understandable or clear. But my point was that graphs pop up in all kinds of problems.
Graphs are not data structures. They are mathematical representation of relations. Yes, you can think and theoretize about problems using graphs, and there is a large body of theory about it. But when you need to implement an algorithm, you are choosing data structures to best represent the problem, not graphs. There are many data structures that represent general graphs, and even more for special kinds of graphs.
In your question, you mix these two things. The same theoretical solution may be in terms of graph, but practical solutions may use different data structures to represent the graph.
The following are based on graph theory:
Binary trees and other trees such as Red-black trees, splay trees, etc.
Linked lists
Anything that's modelled as a state machine (GUIs, network stacks, CPUs, etc)
Decision trees (used in AI and other applications)
Complex class inheritance
IMHO most of the domain models we use in normal applications are in some respect graphs. Already if you look at the UML diagrams you would notice that with a directed, labeled graph you could easily translate them directly into a persistence model. There are some examples of that over at Neo4j
Cheers
/peter
Social connections between people make an interesting graph example. I've tried to model these connections at the database level using a traditional RDMS but found it way too hard. I ended up choosing a graph database and it was a great choice because it makes it easy to follow connections (edges) between people (nodes).
Graphs are great for managing dependencies.
I recently started to use the Castle Windsor Container, after inspecting the Kernel I found a GraphNodes property. Castle Windsor uses a graph to represent the dependencies between objects so that injection will work correctly. Check out this article.
I have also used simple graph theory to develop a plugin framework, each graph node represent a plugin, once the dependencies have been defined I can traverse the graph to create a plugin load order.
I am planning on changing the algorithm to implement Dijkstra's algorithm so that each plugin is weighted with a specific version, thus a simple change will only load the latest version of the plugin.
I with I had discovered this sooner. I like that quote "Whenever someone gives you a problem, think graphs." I definitely think that's true.
Profiling and/or Benchmarking algorithms and implementations in code.
Anything that can be modelled as a foreign key in a relational database is essentially an edge and nodes in a graph.
Maybe that will help you think of examples, since most things are readily modelled in a RDBMS.
You could take a look at some of the examples in the Neo4j wiki,
http://wiki.neo4j.org/content/Domain_Modeling_Gallery
and the projects that Neo4j is used in (the known ones)
http://wiki.neo4j.org/content/Neo4j_In_The_Wild .
Otherwise, Recommender Algorithms are a good use for Graphs, see for instance PageRank, and other stuff at
https://github.com/tinkerpop/gremlin/wiki/pagerank
Analysing transaction serialisability in database theory.
You can utilise graphs anywhere you can define the problem domain objects into nodes and the solution as the flow of control and/or data amongst the nodes.
Considering the fact that trees are indeed connected-acyclic graphs, there are even more areas you can use the graph theory.
Basically nearlly all common data structures like trees, lists, queues, etc, can be thought as type of graph, some with different type of constraint.
To my experiences, I have used graph intensively in network flow problems, which is used in lots of areas like telecommunication network routing and optimisation, workload assignment, matching, supply chain optimisation and public transport planning.
Another interesting area is social network modelling as previous answer mentioned.
There are far more, like integrated circuit optimisation, etc.