I'm trying to use ent to build a recommendation engine (https://entgo.io/docs/schema-def/) by essentially treating ent over MySQL the same a Gremlin over a GraphDB.
I cannot figure out if there is a canonical way to add edge weights to the pre-generated schemas in ent. I think I could roll my own by modifying the generated code, but I feel like there must be a builtin way to generate edge weights, its not exactly an uncommon requirement.
How can I add weights or, generally, value data to edges in ent?
Related
I've read many blog posts, articles, presentation and videos, even inspected V8's source code, both the bytecode generator, the sea-of-nodes graph generator and the optimization phases, and still couldn't find an answer.
V8's optimizing compiler, TurboFan, uses an IR of type "sea of nodes". All of the academic articles I found about it says that it's basically a CFG combined with a data-flow graph, and as such has two type of edges to connect nodes: data edges and control edges. Basically, if you take only the data edges you form a data-flow graph while if you choose the control edges you get a control flow graph.
However, TurboFan has one more edge type: "effect edges" (and effect phis). I suppose that this is what this slide means when it says that this is not "sea" of nodes but "soup" of nodes, because I couldn't find this term anywhere else. From what I understood, effect edges help the compiler keep the structure of statements/expressions that if reordered will have a visible side-effect. The example everyone uses is o.f = o.f + 1: the load has to come before the store (or we'll read the new value), and the addition has to come before the store, too (or otherwise we'll store the old value and uselessly increment the result).
But I cannot understand: isn't this the goal of control edges? From searching through the code I can see that almost every node has an effect edge and a control edge. Their uses isn't obvious though. From what I understand, in sea of nodes you use control edges to constrain evaluation order. So why do we need both effect and control edges? Probably I'm missing something fundamental, but I can't find it.
TL;DR: What's the use of effect edges and EffectPhi nodes, and how they're different from control edges.
Great thanks.
The idea of a sea-of-nodes compiler is that IR nodes have maximum freedom to move around. That's why in a snippet like o.f = o.f + 1, the sequence load-add-store is not considered "control flow". Only conditions and branches are. So if we slightly extend the example:
if (condition) {
o.f = o.f + 1;
} else {
// something else
}
then, as you describe in the question, effect dependencies ensure that load-add-store are scheduled in this order, and control dependencies ensure that all three operations are only performed if condition is true (technically, if the true-branch of the if-condition is taken). Note that this is important even for the load; for instance, o might be an invalid object if condition is false, and attempting to load its f property might cause a segfault in that case.
I am working on a system, when given a bank of different types of elements will create a directed acyclic graph connecting some or all the elements. Each element has some input A and an output B. When building the Graph, the system will need to make sure, the output of the previous node, matches the input of the current one.
The input and output of the nodes are to make sure only certain types of elements are connected
The elements would look like this
ElementName : Input -> Output
Possibly with multiple inputs/output, or with no outputs(See below).
One : X -> Y
Two : Y -> Z,F
Three : Y, Z -> W
Four : Z -> F
Five : F -> NULL
Note:
We are talking about a lot of different elements, 30 or so now, but the plan is to add more as time goes on.
This is part of a project to do a procedural generated narrative. The nodes are individual quests. The inputs are what you need to start the quest. The outputs are how the story state is effected.
Problem:
I have seen several different approaches to generating a random DAG, not one for making a DAG from some preset connection requirements(with rules on connecting them).
I also want some way of limiting complexity of the graph. i.e limit the number of branches they can have.
Idea of what I want:
You have a bunch of different types of legos in a bin, say 30. You have rules on connecting the Legos.
Blue -> Red
Blue -> White
Red -> Yellow
Yellow -> Green/Brown
Brown -> Blue
As you all know, in addition to a color each lego had a shape.So 2 blue legos may not be the same type of lego. So The goal is to build a large structure that fits our rules. Even with our rules, we can still connect the legos in a bunch of different structures.
P.S. I am hoping this is not to general of a question. If it is, please make a note and I will try to make it more specific.
It sounds like an L-system (aka Lindenmayer system) approach would work:
Your collection of Legos is analogous to an alphabet of symbols
Your connection rules correspond to a collection of production rules that expand each symbol into some larger string of symbols
Your starting Lego represents the the initial "axiom" string from which to begin construction
The resulting geometric structures is your DAG
The simplest approach would be something like: given a Lego, randomly select a valid connection rule & add a new Lego to the DAG. From there you could add in more complexity as needed. If you need to skew the random selection to favor certain rules, you're essentially building a stochastic grammar. If the selection of a rule depends on previously generated parts of the DAG it's a type of context sensitive grammar.
Graph rewriting, algorithmically creating a new graph out of base graph, might be a more literal solution, but I personally find that L-systems easier to internalize & that researching them yields results that are not overly academic/theoretical in nature.
L-systems themselves are a category of formal grammars. It might be worth checking into some of those related ideas, but it's pretty easy (for me at least) to get side tracked by theoretical stuff at the expense of core development.
I have a grid of "blocks" (in the form of a 2D array, could be 5*5, 17*17 or whatever) where I can add or remove blocks at will, except for the one at the center that should always remain there.
I can place blocks if they have a local neighbour : on their right/left/up/down (at least one of them).
By removing some blocks, it may leave other blocks isolated with no "connection" to the center-block, and I want to avoid this.
I'm looking for a quick solution to check if all my blocks have a connection to the center, the simplest possible (in terms of coding, I can accept to have a non optimal solution since this is supposed to be on executed on very small data and not so often). The first thing that came to my mind was to implement this as a path search but that seems overkill.
I'm using C++ but that should not make any difference.
You need to find the connected components using DFS/BFS.Construct the initial graph and as you add new blocks, you can add new edges, or when you remove blocks you can remove edges.When you remove a block, temporarily delete those edges in the graph and check if it causes two pieces of the graph to disconnect.This is simple, carry out DFS again.If it does not disconnect you can remove that block.
DFS is only about 8 lines to implement, and for small data sets this is elegant.
I currently try implementing the D* Lite Algorithm for path-planning (see also here) to get a grasp on it. I found two implementations on the web, both for C/C++, but somehow couldn't entirely follow the ideas since they seem to differ more than expected from the pseudo code in the whitepapers. I especially use the following two papers:
Koening,S.;Likhachev,M. - D* Lite, 2002
Koenig & Likkachev, Fast replanning for Navigation in Unknown Terrain, IEEE Transactions on Robotics, Vol. 21, No. 3, June 2005
I tried implementing the optimized version of D* Lite from the first whitepaper (p.5,Fig.4) and for "debugging" I use the example as shown and explained in the second whitepaper (p.6,Fig.6 and Fig.7). All work is done in MatLab (easier for exchanging code with others).
So far I got the code running to find the initial shortest path by running computeShortestPath() once. But now I am stuck at lines {36''} and {37''} of the pseudo-code:
{36”} Scan graph for changed edge costs;
{37”} if any edge costs changed
How do I detect those changes? I somehow don't seem to have a grasp on how this is being detected? In my implementation so far, I mainly used 3 matrices.
One matrix of same size as the grid map containing all rhs-values. One matrix of same size containing similarly all g-values. And one matrix with variable row count for the priority queue with the first two columns being the priority keys and the third and fourth row being the x- and y-coordinates.
Comparing my results, I get the same result for the first run of computeShortestPath() in Step5 as seen in the second whitepaper, p.6 Fig.6. Moving the robot one step also isn't that a problem. But I really have no clue how the next step of scanning for changed edge costs should be implemented.
Thanks for any hint, advice and/or help!!!
The following was pointed out to me by someone else:
In real-world code, you almost never have to "scan the graph for
changes." Your graph only changes when you change it in the code, so
you already know exactly when and where it can change!
One common way of implementing this is to have a OnGraphChanged
callback in your Graph class, which can be setup to call the
OnGraphChanged method in your PathFinder class. Then anywhere the
graph changes in your Graph class, make sure the OnGraphChanged
callback is called.
I personally implemented it by using a "true map" and a "known map" and after every move letting the robot check/scan all next possible successors and comparing them on the true map and the known map.
I was wondering if someone could help me understand this problem. I prepared a small diagram because it is much easier to explain it visually.
alt text http://img179.imageshack.us/img179/4315/pon.jpg
Problem I am trying to solve:
1. Constructing the dependency graph
Given the connectivity of the graph and a metric that determines how well a node depends on the other, order the dependencies. For instance, I could put in a few rules saying that
node 3 depends on node 4
node 2 depends on node 3
node 3 depends on node 5
But because the final rule is not "valuable" (again based on the same metric), I will not add the rule to my system.
2. Execute the request order
Once I built a dependency graph, execute the list in an order that maximizes the final connectivity. I am not sure if this is a really a problem but I somehow have a feeling that there might exist more than one order in which case, it is required to choose the best order.
First and foremost, I am wondering if I constructed the problem correctly and if I should be aware of any corner cases. Secondly, is there a closely related algorithm that I can look at? Currently, I am thinking of something like Feedback Arc Set or the Secretary Problem but I am a little confused at the moment. Any suggestions?
PS: I am a little confused about the problem myself so please don't flame on me for that. If any clarifications are needed, I will try to update the question.
It looks like you are trying to determine an ordering on requests you send to nodes with dependencies (or "partial ordering" for google) between nodes.
If you google "partial order dependency graph", you get a link to here, which should give you enough information to figure out a good solution.
In general, you want to sort the nodes in such a way that nodes come after their dependencies; AKA topological sort.
I'm a bit confused by your ordering constraints vs. the graphs that you picture: nothing matches up. That said, it sounds like you have soft ordering constraints (A should come before B, but doesn't have to) with costs for violating the constraint. An optimal algorithm for scheduling that is NP-hard, but I bet you could get a pretty good schedule using a DFS biased towards large-weight edges, then deleting all the back edges.
If you know in advance the dependencies of each node, you can easily build layers.
It's amusing, but I faced the very same problem when organizing... the compilation of the different modules of my application :)
The idea is simple:
def buildLayers(nodes):
layers = []
n = nodes[:] # copy the list
while not len(n) == 0:
layer = _buildRec(layers, n)
if len(layer) == 0: raise RuntimeError('Cyclic Dependency')
for l in layer: n.remove(l)
layers.append(layer)
return layers
def _buildRec(layers, nodes):
"""Build the next layer by selecting nodes whose dependencies
already appear in `layers`
"""
result = []
for n in nodes:
if n.dependencies in flatten(layers): result.append(n) # not truly python
return result
Then you can pop the layers one at a time, and each time you'll be able to send the request to each of the nodes of this layer in parallel.
If you keep a set of the already selected nodes and the dependencies are also represented as a set the check is more efficient. Other implementations would use event propagations to avoid all those nested loops...
Notice in the worst case you have O(n3), but I only had some thirty components and there are not THAT related :p