Example
A intergalactic space explorer wants to visit one planet from each solar system.
The travel time between solar systems is different dependent on what planet you travel from/to due to different technologies. It may be quicker to travel to another planet in the current solar system to catch a transfer. The explorer has to travel to at least one planet from each solar system.
Here is a graphic example
In the graphic we will need to visit at least one from each colour, and can begin/end anywhere. Is there an algorithm to complete this? It's similar to Dijkstra but not quite. I currently have my data in Neo4j but am able to change this.
From the graphic example I can see how to navigate the network but the real life example has close to 1000 groupings and each group has at least 8 nodes.
Related
let me start saying that I'm new to neural networks, Machine Learning etc, and so far I have just made few very simple experiments to learn so please be patient with me also If I ask very naive or long questions.
My favorite coding language is java and I'm looking at playing with Weka. To play with this API that looks like to me
being very clear and complete, this time, to have something more than a set of ideal data to train and check the success rate, I started not from just the software but from a real world problem that I created for myself.
The problem I created and that I'd like to solve with neural networks, is a four legged spider shaped robot controlled by a Raspberry PI with some ADC and servo Hats. This sort of strange robot has 4 legs, each leg being made
of 3 parts, each part is moved by a servo motor. In total I have so 4 legs * 3 leg parts = 12 servo motors. On each servo motor I have a 3 axis analog accellerometer attached (12 in total) I can read from the Raspberry PI.
From each of these accellerometers I read 2 axis to determine the position of each servo, that is the position of each leg segment for each leg. In addition, the "sole of the foot" of each of the 4 legs, has a button to
determine if the leg has reached the floor and is sustaining the spider. The spider construction blog is here, for those interested: https://thestrangespider.blogspot.com/
The purpose of this experiment is to make the spider able to put itself in balance and set itself horizontally starting from any condition.
The spider should be able in few words, to level horizontally its body regardless of the fact that I have put the spider on a horizontal or oblique surface. The hardware platform is ready, I miss few details but let's assume
I can read all the signals I need from a java Spring boot application using the PI4J APIs to interface hardware (24 values from ADC for 12 servos and 4 digital inputs (true/false) from the buttons on
the soles of the spider feets). The intent it to solve the problem of moving the legs servo motors using neural networks built with Weka, reading the various input signals until the system reaches the
success condition (static balance and body in a horizontal position). The main problem is how to use all the data I have to build a data set in the best way, what neural networks to use to put in place
the adaptive corrective feedback, until the spider reaches the success condition of having its body finally horizontal and in static balance.
Going deeper into this topic, let me provide my analysis.
Each leg should perform this steps:
Can move independently one servo per time starting from a random position (in the range of the angle allowed by the leg portion)
Re-read after each move, all the legs segments position, waiting for the next static condition
Determine if last move has brought an advantage or a disadvantage to the system as a whole.
If no change happened because of the last move, keep the latest change/move and continue
If any max angle has been reached, change the direction for that angle next move
Check if system success is closer now respect to previous step and determine next action type
The problem to me is a mix of open questions where one is how to represent this system from a data perspective:
Data coming from acceletometers all belonging from servos in the same legs are candidates for a cluster of data or a robot sub-system ?
Can each servo and its position measurement be a subsystem of the main system instead ?
Can these be multiple problems instead of one really ?
How would you approach this problem from a neural network perspective ?
after studying Weka classification and regression, I finally got to the conclusion that the processing should be a regression problem. That ended as well in a too complex problem that would have need regression with multi label output that apart from Weka would have required another framework on top of that called Meka.
Finally I got to a conclusion that now I will try that is, using first Weka to classify data and detect first on which length the inclination is, the solve the single specific problem of acting on that leg eventually with a different Weka model working only on a single leg per time.
Problem resolution so, will have these steps:
Use a legs Weka classification model first to detect which of the four legs has the higher inclination.
Use a leg specific Weka classification to correct for the leg detected above in first step, the actions to be taken by the three leg segments in order to correct the problem.
Stefano
My question isn't related to how to code a particular thing or any technical programming question. I need help in developing a logic of an algorithm that I am working on which I will be explaining in a bit. I am here because I couldnt think of a better place than stackoverflow to help me out as it has proved the best option in the past. I would like to thank you guys for helping your me out and giving me your precious time.
So here goes my brain teasing algorithm:
Aim :
I am developing an algorithm that aims to equally distribute traffic(cars) over a network. I will be simulating this algorithm to check how it works on some traffic simulator.
To put it visually what I am planning to do I have included this image:
Plotting the traffic network which shows that some roads are populated than the others. Thicker lines indicate roads that have more traffic as compared to others.**
Here are some points to note :
The image represents 8*8 junctions(intersection of roads) and the roads themselves. Not good with my graphic skills so just wanted to make clear that 4 roads meet up to make a junction(not clearly shown in diagram, but you get what I mean).
Notice the directed arrows, those indicate the direction in which the traffic will be moving. For example, the first row indicates the traffic will be moving from right to left. Second row indicates the traffic will move from left to right. Similary, the first column represents that traffic will more from bottom to top etc. If you observe closely, you will notice that I have kept an alternate pattern ie. right to left,then left to right, then again right to left etc. Similar is the case for columns.
No turning of vehicles is allowed. Since, this is the initial stages of my algorithm I have put a restriction that vehicles are not allowed to turn and continue in the direction of the arrow.
This is a loop-around type network. Meaning, the cars after reaching the extremes of the network do not vanish away, they loop around and are fed again into the network. Keep in mind, the cars are fed in the direction of arrow. For example in first row, after the cars leave from the left-most junction, the cars will join in again into the right-most junction. Simply to say, it is some sort of wrapping around or mesh kind of thing.
The traffic is created randomly and as the image suggests thicker the lines, the more traffic on that road and vice-versa.
Important: When at a junction, if we are to allow cars to pass from left to right (horizontally) all the cars moving in vertical direction will have to be stopped (obvious so that they don't crash into each other) and vice-versa.
What I need help in:
Pseudocode/Logic for the algorithm which allows me to equally distribute the traffic in the network (graphically lines of almost the same thickness as per my image). Mathematically, the thickness of the line will be average number of cars that should be on each road so that the whole network is balanced. Roads, should be equally populated, no road should have more traffic than the other (fairness policy).
I wanted to solve the problem using a divide and conquer approach. Meaning, in real world the network will be very big so it is almost impossible to have the information of the whole network with us at any given moment of time.
I want to solve the problems locally and round-wise.
For example, Round 1: Solve 4 junctions at a time and try to equalize the traffic at them.
Round 2: Some other 4 junctions
and so on...
Tip: The effect should propagate as the rounds advances reaching to some optimal solution where the traffic is equally distributed and then we stop the algorithm.
I know this is a long post and I really appreciate anyone who even tried to understand my objective and gave a thought about it.
This idea popped into my head giving because of the growing traffic situations and that there are some roads that are less populated than the others and so equally distribute the cars and take more advantage of the available infrastructure.
Background: I want to create a weather service, and since most available APIs limit the number of daily calls, I want to divide the planet in a thousand or so areas.
Obviously, internet users are not uniformly distributed, so the sampling should be finer around densely populated regions.
How should I go about implementing this?
Where can I find data regarding geographical internet user density?
The algorithm will probably be something similar to k-means. However, implementing it on a sphere with oceans may be a bit tricky. Any insight?
Finally, maybe there is a way I can avoid doing all of this?
Very similar to k-mean is the centroidal Voronoi diagram (it is the continuous version of k-means). However, this would produce a uniform tesselation of your sphere that does not account for user density as you wish.
So a similar solution is the same technique but used with a Power Diagram : a Power Diagram is a Voronoi Diagram that accounts for a density (by assigning a weight to each Voronoi seed). Such diagram can be computed using an embedding in a 3D space (instead of 2D) that consists of the first two (x,y) coordinates plus a third one which is the square root of [any large positive constant minus the weight for the given point].
Using that, you can obtain a tesselation of your domain accounting for a user density.
You don't care about internet user density in general. You care about the density of users using your service - and you don't care where those users are, you care where they ask about. So once your site has been going for more than a day you can use the locations people ask about the previous day to work out what the areas should be for the next day.
Dynamic programming on a tree is easy. What I would do for an algorithm is to build a tree of successively more finely divided cells. More cells mean a smaller error, because people get predictions for points closer to them, and you can work out the error, or at least the relative error between more cells and fewer cells. Starting from the bottom up work out the smallest possible total error contributed by each subtree, allowing it to be divided in up to 1,2,3,..N. ways. You can work out the best possible division and smallest possible error for each k=1..N for a node by looking at the smallest possible error you have already calculated for each of its descendants, and working out how best to share out the available k divisions between them.
I would try to avoid doing this by thinking of a different idea. Depending on the way you look at life, there are at least two disadvantages of this:
1) You don't seem to be adding anything to the party. It looks like you are interposing yourself between organizations that actually make weather forecasts and their clients. Organizations lose direct contact with their clients, which might for instance lose them advertising revenue. Customers get a poorer weather forecast.
2) Most sites have legal terms of service, which must clients can ignore without worrying. My guess is that you would be breaking those terms of service, and if your service gets popular enough to be noticed they will be enforced against you.
I have a server that receives GPS data from a mobile device (iphone) and then finds the distinct cities from which the data comes for each distinct day. The app syncs every three hours. Since the granularity I need is not that small -I'm not interested in anything "smaller" than cities- I'd like to be able to say "this person was in this or that city during this or that date". The problem is, GPS warm-up, bad accuracy and the sheer amount of data received (the device collects data every ten minutes and syncs with the server every 3 hours) sometimes yield false positives or bad data -I have a user that lives near the border of nyc/nj and I keep getting alternate locations from one or the other place, even though he spends most of his actual day far from the border, so those times when he's home shouldn't matter).
My question is: what algorithms should I consider, what papers should I read, or, even, what terms should I google to find an approach that will help me get rid of the noise and false positives for data that's being synced every n hours and that needn't be more granular than a certain level (city, in this case) and which is significant for a certain period of time? (think about it as the fact that I'm counting the visits to distinct cities, states or countries for distinct dates). I was thinking of something like "clustering" or "dissolving" the data, but I don't know anything about geo algorithms, yet ;)
I use android instead of iphone so I'll answer hoping iphone has similar information available in its stream of fixes.
First, you should only use locations found from GPS and not from wifi or cell tower locations for this effort. In android, the source of the location fix is "GPS" "Network" or "cell", the GPS are the most accurate. Using cell towers, especially near a river or on a hill, you often pick up towers pretty distant from you as you move around, and it sounds like that's what's happening with your ny/nj problem.
2nd, if a guy is in a certain city he should be there for lengths of time without moving too far. You could write something that only declares a location if it receives a bunch of locations in a row that are within that location, that essentially filters out bad results by noting that it is not realistic to bounce back and forth quickly between two locations that are more than a 500 m apart or something like that.
I'm designing a city building game and got into a problem.
Imagine Sierra's Caesar III game mechanics: you have many city districts with one market each. There are several granaries over the distance connected with a directed weighted graph. The difference: people (here cars) are units that form traffic jams (here goes the graph weights).
Note: in Ceasar game series, people harvested food and stockpiled it in several big granaries, whereas many markets (small shops) took food from the granaries and delivered it to the citizens.
The task: tell each district where they should be getting their food from while taking least time and minimizing congestions on the city's roads.
Map example
Suppose that yellow districts need 7, 7 and 4 apples accordingly.
Bluish granaries have 7 and 11 apples accordingly.
Suppose edges weights to be proportional to their length. Then, the solution should be something like the gray numbers indicated on the edges. Eg, first district gets 4 apples from the 1st and 3 apples from the 2nd granary, while the last district gets 4 apples from only the 2nd granary.
Here, vertical roads are first occupied to the max, and then the remaining workers are sent to the diagonal paths.
Question
What practical and very fast algorithm should I use? I was looking at some papers (Congestion Games: Optimization in Competition etc.) describing congestion games, but could not get the big picture.
You want to look into the Max-flow problem. Seems like in this case it is a bipartite graph, which should make things easier to visualize.
This is a Multi-source Multi-sink Maximum Flow Problem which can easily be converted into a simple Maximum Flow Problem by creating a super source and a super sink as described in the link. There are many efficient solutions to Maximum Flow Problems.
One thing you could do, which would address the incremental update problem discussed in another answer and which might also be cheaper to computer, is forget about a globally optimal solution. Let each villager participate in something like ant colony optimization.
Consider preventing the people on the bottom-right-hand yellow node in your example from squeezing out those on the far-right-hand yellow node by allowing the people at the far-right-hand yellow node to bid up the "price" of buying resources from the right-hand blue node, which would encourage some of those from the bottom-right-hand yellow node to take the slightly longer walk to the left-hand blue node.
I agree with Larry and mathmike, it certainly seems like this problem is a specialization of network flow.
On another note, the problem may get easier if your final algorithm finds a spanning tree for each market to its resources (granaries), consumes those resources greedily based on shortest path first, then moves onto the next resource pile.
It may help to think about it in terms of using a road to max capacity first (maximizing road efficiency), rather than trying to minimize congestion.
This goes to the root of the problem - in general, it's easier to find close to optimal solutions in graph problems and in terms of game dev, close to optimal is probably good enough.
Edit: Wanted to also point out that mathmike's link to Wikipedia also talks about Maximum Flow Problem with Vertex Capacities where each of your granaries can be thought of as vertices with finite capacity.
Something you have to note, is that your game is continuous. If you have a solution X at time t, and some small change occurs (e.g: the player builds another road, or one of the cities gain more population), the solution that the Max Flow algorithms give you may change drastically, but you'd probably want the solution at t+1 to be similar to X. A totally different solution at each time step is unrealistic (1 new road is built at the southern end of the map, and all routes are automatically re-calculated).
I would use some algorithm to calculate initial solution (or when a major change happens, like an earthquake destroys 25% of the roads), but most of the time only update it incrementally: meaning, define some form of valid transformation on a solution (e.g. 1 city tries to get 1 food unit from a different granary than it does now) - you try the update (simulate the expected congestion), and keep the updated solution if its better than the existing solution. Run this step N times after each game turn or some unit of time.
Its both efficient computationally (don't need to run full Max Flow every second) and will get you more realistic, smooth changes in behavior.
It might be more fun to have a dynamic that models a behavior resulting in a good reasonable solution, rather than finding an ideal solution to drive the behavior. Suppose you plan each trip individually. If you're a driver and you need to get from point A to point B, how would you get there? You might consider a few things:
I know about typical traffic conditions at this hour and I'll try to find ways around roads that are usually busy. You might model this as an averaged traffic value at different times, as the motorists don't necessarily have perfect information about the current traffic, but may learn and identify trends over time.
I don't like long, confusing routes with a lot of turns. When planning a trip, you might penalize those with many edges.
If speed limits and traffic lights are included in your model, I'd want to avoid long stretches with low speed limits and/or a lot of traffic lights. I'd prefer freeways or highways for longer trips, even if they have more traffic.
There may be other interesting dynamics that evolve from considering the problem behaviorally rather than as a pure optimization. In real life, traffic rarely converges on optimal solutions, so a big part of the challenge in transportation engineering is coming up with incentives, penalties and designs that encourage a better solution from the natural dynamics playing out in the drivers' decisions.