let me start saying that I'm new to neural networks, Machine Learning etc, and so far I have just made few very simple experiments to learn so please be patient with me also If I ask very naive or long questions.
My favorite coding language is java and I'm looking at playing with Weka. To play with this API that looks like to me
being very clear and complete, this time, to have something more than a set of ideal data to train and check the success rate, I started not from just the software but from a real world problem that I created for myself.
The problem I created and that I'd like to solve with neural networks, is a four legged spider shaped robot controlled by a Raspberry PI with some ADC and servo Hats. This sort of strange robot has 4 legs, each leg being made
of 3 parts, each part is moved by a servo motor. In total I have so 4 legs * 3 leg parts = 12 servo motors. On each servo motor I have a 3 axis analog accellerometer attached (12 in total) I can read from the Raspberry PI.
From each of these accellerometers I read 2 axis to determine the position of each servo, that is the position of each leg segment for each leg. In addition, the "sole of the foot" of each of the 4 legs, has a button to
determine if the leg has reached the floor and is sustaining the spider. The spider construction blog is here, for those interested: https://thestrangespider.blogspot.com/
The purpose of this experiment is to make the spider able to put itself in balance and set itself horizontally starting from any condition.
The spider should be able in few words, to level horizontally its body regardless of the fact that I have put the spider on a horizontal or oblique surface. The hardware platform is ready, I miss few details but let's assume
I can read all the signals I need from a java Spring boot application using the PI4J APIs to interface hardware (24 values from ADC for 12 servos and 4 digital inputs (true/false) from the buttons on
the soles of the spider feets). The intent it to solve the problem of moving the legs servo motors using neural networks built with Weka, reading the various input signals until the system reaches the
success condition (static balance and body in a horizontal position). The main problem is how to use all the data I have to build a data set in the best way, what neural networks to use to put in place
the adaptive corrective feedback, until the spider reaches the success condition of having its body finally horizontal and in static balance.
Going deeper into this topic, let me provide my analysis.
Each leg should perform this steps:
Can move independently one servo per time starting from a random position (in the range of the angle allowed by the leg portion)
Re-read after each move, all the legs segments position, waiting for the next static condition
Determine if last move has brought an advantage or a disadvantage to the system as a whole.
If no change happened because of the last move, keep the latest change/move and continue
If any max angle has been reached, change the direction for that angle next move
Check if system success is closer now respect to previous step and determine next action type
The problem to me is a mix of open questions where one is how to represent this system from a data perspective:
Data coming from acceletometers all belonging from servos in the same legs are candidates for a cluster of data or a robot sub-system ?
Can each servo and its position measurement be a subsystem of the main system instead ?
Can these be multiple problems instead of one really ?
How would you approach this problem from a neural network perspective ?
after studying Weka classification and regression, I finally got to the conclusion that the processing should be a regression problem. That ended as well in a too complex problem that would have need regression with multi label output that apart from Weka would have required another framework on top of that called Meka.
Finally I got to a conclusion that now I will try that is, using first Weka to classify data and detect first on which length the inclination is, the solve the single specific problem of acting on that leg eventually with a different Weka model working only on a single leg per time.
Problem resolution so, will have these steps:
Use a legs Weka classification model first to detect which of the four legs has the higher inclination.
Use a leg specific Weka classification to correct for the leg detected above in first step, the actions to be taken by the three leg segments in order to correct the problem.
Stefano
Related
I'm interested in developing a semi-autonomous RC lawnmower.
That is, the operator would decide when to stop, turn, etc., but could request "slightly overlap previous cut" and the mower would automatically do so. (Having operated high-end RC mowers at trade shows, this is the tedious part. Overcoming that, plus the high cost -- which I believe is possible -- would make a commercial success.)
This feature would require accurate horizontal positioning. I have investigated ultrasonic, laser, optical, and GPS. Each has its problems in this application. (I'll resist the temptation to go off on these tangents here.)
So... my question...
I know GPS horizontal accuracy is only 3-4m. Not good enough, but:
I don't need to know where I am on the planet. I only need to know where I am relative to where I was a minute ago.
So, my question is, is the inaccuracy consistent in the short term? if so, I think it would work for me. If it varies wildly by +- 1.5m from one second to the next, then it will not work.
I have tried to find this information but have had no success (possibly because of the ubiquity of other GPS-accuracy discussion), so I appreciate any guidance.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Edit ~~~~~~~~~~~~~~~~~~~~~~
It's looking to me like GPS is not just skewed but granular. I'd be interested in hearing from anyone who can give better insight into this, but for now I'm going to explore other options.
I realized that even though my intended application is "outdoor", this question is technically in the field of "indoor positioning systems" so I am adding that tag.
My latest thinking is to have 3 "intelligent" high-dB ultrasonic (US) speaker units. The mower emits RF requests for a tone from each speaker in rapid sequence, measuring the time it takes to "hear" each unit's response, thereby calculating distance to each of these fixed point and using trilateration to get position. if the fixed-point speakers are 300' away from the mower, the mower may have moved several feet between the 1st and 3rd response, so this would have to be allowed for in the software. If it is possible to differentiate 3 different US frequencies, they could be requested/received "simultaneously". Though you still run into issues when you're close to one fixed unit and far from another. So some software correction may still be necessary. If we can assume the mower is moving in a straight line, this isn't too complicated.
Another variation is the mower does not request the tones. The fixed units send RF "here comes tone from unit A" etc., and the mower unit just monitors both RF info and US tones. This may simplify things somewhat, but it seems it really requires the ability to determine which speaker a tone is coming from.
This seems like the kind of thing you could (and should) measure empirically. Just set a GPS of your liking down in the middle of a field on a clear day and wait an hour. Then come back and see what you find.
Because I'm in a city, I can't run out and do this for you. However, I found a paper entitled iGeoTrans – A novel iOS application for GPS positioning in geosciences.
That includes this figure which duplicates the test I propose. You'll note that both the iPhone4 and Garmin eTrex10 perform pretty poorly versus the accuracy you say you need.
But the authors do some Math Magic™ to reduce the uncertainty in the position, presumably by using some kind of averaging. That gets them to a 3.53m RMSE measure.
If you have real-time differential GPS, you can do better. But this requires relatively expensive hardware and software.
Even aside from the above, you have the potential issue of GPS reflection and multipath error. What if your mower has to go under a deck, or thick trees, or near the wall of a house? These common yard features will likely break the assumptions needed to make a good averaging algorithm work and even frustrate attempts at DGPS by blocking critical signals.
To my mind, this seems like a computer vision problem. And not just because that'll give you more accurate row overlaps... you definitely don't want to run over a dog!
In my opinion a standard GPS is no way accurate enough for this application. A typical consumer grade receiver that I have used has a position accuracy defined as a CEP of 2.5 metres. This means that for a stationary receiver in a "perfect" sky view environment over time 50% of the position fixes will lie within a circle with a radius of 2.5 metres. If you look at the position that the receiver reports it appears to wander at random around the true position sometimes moving a number of metres away from its true location. When I have monitored the position data from a number of stationary units that I have used they could appear to be moving at speeds of up to 0.5 metres per second. In your application this would mean that the lawnmower could be out of position by some not insignificant distance (with disastrous consequences for your prized flowerbeds).
There is a way that this can be done, as has been proved by the tractor manufacturers who can position the seed drills and agricultural sprayers to millimetre accuracy. These systems use Differential GPS where there is a fixed reference station positioned in the neighbourhood of the tractor being controlled. This reference station transmits error corrections to the mobile unit allowing it to correct its reported position to a high degree of accuracy. Unfortunately this sort of positioning system is very expensive.
I have to implement, as part of an assignment, a learning method that enables minesweepers to avoid colliding with mines. I have been given the choice to choose between a supervised/unsupervised/reinforcement learning algorithm.
I remember in one of my lectures, the lecturer mentioned ALVIN. He was teaching artificial neural networks.
Since the behaviour I'm looking for is more or less similar to ALVINN's, I want to implement an ANN. I've implemented an ANN before for solving the 3-parity xor problem, here's my solution. I've never really understood the intuition behind ANNs.
I was wondering, what could the inputs be to my ANN? In the case of the 3parity xor problem it was obvious.
When it comes to frameworks for ANN, each person will have their own preferences. I recently used Encog framework for implementing an image processing project and found it very easy to implement.
Now, coming to your problem statement, "a learning method that enables minesweepers to avoid colliding with mines" is a very wide scope. What is indeed going to be your input to the ANN? You will have to decide your input based on whether it is going to be implemented on a real robot or in a simulation environment.
It can be clearly inferred that an unsupervised learning can be ruled out if you are trying to implement something like the ALVIN.
In a simulation environment, the best option is if you can somehow form a grid map of the environment based on the simulated sensor data. Then the occupancy grid surrounding the robot can form a good input to the robot's ANN.
If you can't form a grid map (if the data is insufficient), then you should try to feed all the available and relevant sensor data to the ANN. However, they might have to be pre-processed, depending on the modelled sensor noise given by your simulation environment. If you have a camera feed (like the ALVIN model), then you may directly follow their footsteps and train your ANN likewise.
If this is a real robot, then the choices vary considerably, depending upon the robustness and accuracy requirements. I really hope you do not want to build a robust and field-ready minesweeper single-handedly. :) For a small, controlled environment, your options will be very similar to that of a simulated environment, however sensor noise would be nastier and you would have to figure in various special cases into your mission planner. Still, it would be advisable to fuse a few other sensors (LRF, ultrasound etc.) with vision sensors and use it as an input to your planner. If nothing else is available, copy paste the ALVIN system with only a front camera input.
The ANN training methodology will be similar (if using only vision). The output will be right/left/straight etc. Try with 5-7 hidden layer nodes first, since that is what ALVIN uses. Increase it up to 8-10 max. Should work. Use activation functions properly.
Given its success in the real world, ALVIN seems like a good system to base yours off of! As the page you linked to discusses, ALVIN essentially receives an image of the road ahead as its input. On a low level, this is achieved through 960 input nodes representing a 30X32 pixel image. The input value for each node is the color saturation of the pixel that that node represents (with 0 being a completely white image and 1 being a completely black image, or something along those lines) (I'm pretty sure the picture is greyscale, although maybe they're using color now, which could be achieved, for instance, by using three input nodes per pixel, one representing red saturation, one representing green, and one blue). Is there a reason that you don't think that this would be a good input for your system too?
For more low level details, see the original paper.
What's the best way to implement real time operant conditioning (supervised reward/punishment-based learning) for an agent? Should I use a neural network (and what type)? Or something else?
I want the agent to be able to be trained to follow commands like a dog. The commands would be in the form of gestures on a touchscreen. I want the agent to be able to be trained to follow a path (in continuous 2D space), make behavioral changes on command (modeled by FSM state transitions), and perform sequences of actions.
The agent would be in a simulated physical environment.
Reinforcement Learning is a good machine learning algorithm for your problem.
The basic reinforcement learning model consists of:
a set of environment states S (you have a 2d space discretized in some way, which is the dog's current position, if you want to do continuous 2d-space, you might need a neural network to serve as the value function mapper.)
a set of actions A ( you mentioned the dog performs sequences of actions, e.g., move, rotate)
rules of transitioning between states ( your dog's position transition can be modeled by FSM)
rules that determine the scalar immediate reward r of a transition (When reaching the target position, you might want to give the dog a big reward, while small rewards are also welcomed at intermediate milestones)
rules that describe what the agent observes. (the dog might have a limited view, for example, only the 4 or 8 neighboring cells are viewable, below figure is an example showing the dog's current position P and the 4 neighboring cells that are viewable to the dog.)
To find the optimal policy, you can start with the model-free technique - q-learning.
I wrote a delphi program generating a gpx file as input for a "poor man's guidance system" for aerial spray by means of ultralight plane.
By and large, it produces route (parallel swaths) using gpx file as output.
The route's engine is based on the "Vincenty" algorithm which works fine for any wgs84 computation but
I can't get the accuracy of grid generated by ExpertGPS of Topografix (requirement).
I assume a 2D computation on the ellipsoïd :
1) From the start rtept (route point), compute the next rtept given a bearing and an arbitrary distance (swath length).
2) Compute the next rtept respective respective to previous bearing (90° turn) and another arbitrary distance (swath distance).
3) Redo 1) with the last rtept as starting point but in the opposite direction, and so on.
What's wrong with it ?
You do not describe your Pascal implementation of Vincenty's earth ellipsoid model so the following is speculation:
The model makes use of numerous geometrical trig functions-- ATAN2,
COS, SIN etc. Depending whether you use internal Delphi functions
or your own versions, there is the possibility of lack of precision
in calculations. The precision in the value of pi used in your
calculations could affect the precision you require.
Floating point arithmetic can cause decimal place errors. It will
make a difference whether you use single, double or real. I
believe some of the internal Delphi functions have changed with
different versions so possibly the version of Delphi you are
using will affect how the internal function is implemented.
If implemented accurately, Vincenty’s formula is supposed to be
accurate to within 0.5mm. Amazing accuracy. If there are rounding
errors or lack of precision in your Delphi implemention, the positional
errors can be significantly larger.
Consider the accuracy of your GPS information. Depending on how
many satellites are being used by the GPS receiver at any one time,
the accuracy of the positional information changes. Errors on
the order of 50 feet or more is possible. Additionally, the refresh
of positional information on the GPS receiver is not necessarily
instantaneous; therefore if the swath 'turns' occur rapidly, you
will have to ensure the GPS has updated at the turning point.
Your procedure to calculate the pattern seems reasonable so look
at your implementation of Vincenty's algorithm in your Delphi code.
This list is not exhaustive, I imagine others can improve it
dramatically. What I mention is based on my experience with GPS and
various versions of Delphi and what I could recall off the top of my head.
Something you might try is compare your calculations of
distance/bearing using your implementation of the algorithm with
examples provided on the Internet. There are several online
calculators. If you have not been there, the Aviation Formulary
is an excellent place to find examples of other navigational tricks.
http://williams.best.vwh.net/avform.htm . A comparison will
allow you to gain confidence in the precision of the Delphi
implementation of Vincenty's algorithm with data calculated by
mathematicians. Simply, your implementation of Vincenty may not be
precise. Then again, the error may be elsewhere.
I am doing farm GPS guidance similar for ground rig just with Android. Great for second tractor to help follow previous A B tracks especially when they disappear for a bit .
GPS accuracy repeat ability from one day to next will give larger distance. Expensive system's use dGPS2cm-10cm.5-30metres different without dGPS. Simple solution is recalibrate at known location. Cheaper light bars use this method.
Drift As above except relates to movement during job. Mostly unnoticeable <20cm 3hrs. Can jump 1-2metres rarely. I think when satellite connect or disconnect. Again recalibrate regularly at known coordinates ,i.e. spray fill point
GPS accuracy. Most phone update speed 1hz. 3? seconds between fixes at say 50km/hr , 41.66m between fixes. On ground rig 18km hrs but will be tracks after first run. Try a Bluetooth GPS 10hz check update speed and as mentioned fast turns a problem.
Accuracy of inputs and whether your guidance uses dGPS will make huge difference.
Once you are off your line say 5 metres at 100metres till next point, then at 50 metres your still 2.5 metres off unless your guidance takes you back to the route not the next coordinates.
I am not using Vincenty as I can 'bump'back onto line manually and over 1km across difference <30cm according to only reference I saw however I am taking 2 points and create parrallel points across.
Hope these ideas help your situation.
I'm making a simple RTS game. I want it to run very fast because it should work with thousands of units and 8 players.
Everything seems to work flawlessly but it seems the line of sight calculation is a bottleneck. It's simple: if an enemy unit is closer than any of my unit's LOS range it will be visible.
Currently I use a quite naive algorithm: for every enemy units I check whether any of my units is see him. It's O(n^2)
So If there are 8 players and they have 3000 units each that would mean 3000*21000=63000000 tests per player at the worst case. Which is quite slow.
More details: it's a stupid simple 2D space RTS: no grid, units are moving along a straight lines everywhere and there is no collision so they can move through each other. So even hundreds of units can be at the same spot.
I want to speed up this LOS algorithm somehow. Any ideas?
EDIT:
So additional details:
I meant one player can have even 3000 units.
my units have radars so they towards all directions equally.
Use a spatial data structure to efficiently look up units by location.
Additionally, if you only care whether a unit is visible, but not which unit spotted it, you can do
for each unit
mark the regions this unit sees in your spatial data structure
and have:
isVisible(unit) = isVisible(region(unit))
A very simple spatial data structure is the grid: You overlay a coarse grid over the playing field. The regions are this grid's cells. You allocate an array of regions, and for each region keep of list of units presently in this region.
You may also find Muki Haklay's demonstration of spatial indexes useful.
One of the most fundamental rules in gamedev is to optimize the bejeebers out of your algorithms by exploiting all possible constraints your gameplay defines - this is the main reason that you don't see wildly different games built on top of any given companies game engine, they've exploited their constraints so efficiently that they can't deal with anything that isn't within these constraints.
That said, you said that units move in straight lines - and you say that players can 3000 units - even if I assume that's 3000 units for eight players, that's 375 units per player, so I think I'm safe in assuming that on each step of game play (and I am assuming that each step involves the calculation you describe above) that more units will not change their direction than units that will change direction.
So, if this is true, then you want to divide all your pieces into two groups - those that did change direction in the last step, and those that did not.
For those that did, you need to do a bit of calulating - for units of any two opposing forces, you want to ask 'when will unit A see unit B given that neither unit A nor unit B change direction or speed ?(you can deal with accelleration/decelleration, but then it gets more complicated) - to calculate this you need first to determine if the vectors that unit A and unit B are travelling on will intersect (simple 2D line intersection calculation, combined with a calculation that tells you when each unit hits this intersection) - if they don't, and they can't see each other now, then they never will see each other unless at least one of them changes direction. If they do intersect, then you need to calculate the time differential between when the first and second unit pass through the point of intersection - if this distance is greater than the LOS range, then these units will never see each other unless one changes direction - if this differential is less than the LOS range then a few more (wave hands vigorously) calculations will tell you when this blessed event will take place.
Now, what you have is a collection of information bifurcated into elements that never will see each other and elements that will see each other at some time t in the future - each step, you simply deal with the units that have changed direction and compute their interactions with the rest of the units. (Oh, and deal with those units that previous calculations told you would come into view of each other - remember to keep these in an insertable ordered structure) What you've effectively done is exploited the linear behavior of the system to change your question from 'Does unit A see unit B' to 'When will unit A see unit B'
Now, all of that said, this isn't to discount the spatial data structure answer - it's a good answer - however, it is also capable of dealing with units in random motion, so you want to consider how to optimize this process further - you also need to be careful about dealing with cross region visibility, i.e. units at the borders of two different regions may be able to see each other - if you have pieces that tend to clump up, using a spatial data structure with variable dimensions might be the answer, where pieces that are not in the same region are guaranteed not to be able to see each other.
I'd do this with a grid. I think that's how commercial RTS games solve the problem.
Discretize the game world for the visibility tracker. (Square grid is easiest. Experiment with the coarseness to see what value works best.)
Record the present units in each area. (Update whenever a unit moves.)
Record the areas each player sees. (This has to be updated as units move. The unit could just poll to determine its visible tiles. Or you could analyze the map before the game starts..)
Make a list (or whatever structure is fitting) for the enemy units seen by each player.
Now whenever a unit goes from one area of visibility to another, perform a check:
Went from an unseen to a seen area - add the unit to the player's visibility tracker.
Went from a seen to an unseen area - remove the unit from the player's visibility tracker.
In the other two cases no visibility change occurred.
This is fast but takes some memory. However, with BitArrays and Lists of pointers, the memory usage shouldn't be that bad.
There was an article about this in one of the Game Programming Gems books (one of the first three, I think.)