I am new to the field of Q-learning (QL) and I am trying to implement a small task using QL in MATLAB. The task is : Say there is one transmitter, one receiver and between them there are 10 relays. The main part is that I want to choose one of the relay using QL that will carry the signal from transmitter to receiver successfully.
So, as per QL theory, we need to define state, action, reward. Hence I had chosen them as:
State : [P1,...,P10] where P1 is power from 1st relay to receiver. Like wise P10 is power from 10th relay to receiver.
action : [1,...,10] where action is nothing but choosing that relay which has highest power at that time.
My query is I am not getting how should I choose reward in this case ?
Any help in this regard will be highly appreciated.
There is only one state (i.e., this is actually a multi-armed bandit problem).
There are ten actions, one per relay.
The reward of each action is the power of the corresponding relay.
When the radio is of Ieee80211DimensionalRadio radio type, why doesn't the INET 4.4 have a communicationRange parameter?
If not, please help how to set this parameter.
The reason is the same, why a WiFi card in a real computer does not have a setting where you can set how far the WiFi card can communicate. You do not set range in a real system. You set the transmission power and the effective range depends on that.
In short, you have to calculate the required power backwards from the range. Obviously, that is extremely complicated assuming that there are error correction, directional antennas, non-freespace propagation, etc..
The best you can do is to manually change the TX power and experiment to set it to a level suitable to you.
(additionally: There is no such thing as communication range in real world. There is always a probability that a packet is received for any distance. The question is, where you set your cutoff probability. This is always subjective).
I started (since about one week) using veins (4.4) under omnet++ (5.0).
My current task is to let vehicles adjust their transmission range according to a specific context. I did read a lot of asked questions like these ones (and in other topics/forums):
Dynamical transmission range in the ieee802.11p module
Vehicles Receive Beacon Messages outside RSU Range
How coverage distance and interference distance are affected by each other
Maximum transmission range vs maximum interference distance
Reduce the coverage area between vehicles
how to set the transmission range of a node under Veins 2.0?
My Question:
How to -really- change the transmission range of just some nodes?
From the links above, I knew that the term "transmission range", technically, is related to the received power, noise,sensitivity threshold, etc. which defines the probability of reception.
Since I am new to veins (and omnet++ as well), I did few tests and I concluded the following:
"TraCIMobility" module can adjust the nodes' parameters (for each vehicle, there is an instance) such as the ID, speed, etc.
I could, also, instantiate the "Mac1609_4" (for each vehicle) and changed some of its parameters like the "txPower" during simulation run-time but it had no effect on the real communication range.
I could not instantiate (because it was global) the "connection manager" module which was the only responsible of (and does override) the effective communication range. this module can be configured in the ".ini" file but I want different transmission powers and most importantly "can be changed during run-time".
The formula to calculate the transmission range is in the attached links, I got it, but it must be a way to define or change these parameters in one of the layers (even if it is in the phy layer, i.e., something like the attached signal strength...)
Again, maybe there is some wrong ideas in what I have said, I just want to know what/how to change this transmission range.
Best regards,
You were right to increase the mac1609_4.txPower parameter to have a node send with more power (hence, the signal being decodable further away). Note, however, that (for Veins 4.4) you will also need to increase connectionManager.pMax then, as this value is used to determine the maximum distance (away from a transmitting simulation module) that a receiving simulation module will be informed about an ongoing transmission. Any receiving simulation module further away will not be influenced by the transmission (in the sense of it being a candidate for decoding, but also in the sense of it contributing to interference).
Also note that transmissions on an (otherwise) perfectly idle channel will reach much further than transmissions on a typically-loaded channel. If you want to obtain a good measurement of how far a transmission reaches, have some nodes create interference (by transmitting broadcasts of their own), then look at how the Frame Delivery Rate (FDR) drops as distance between sender and receiver increases.
Finally, note that both 1) the noise floor and 2) the minimum power level necessary for the simulation module of a receiver to attempt decoding a frame need to be calibrated to the WLAN card you want to simulate. The values chosen in the Veins 4.4 tutorial example are very useful for demonstrating the concepts of Veins, whereas the values of more recent versions of Veins come closer to what you would expect from a "typical" WLAN card used in some of the more recent field tests. See the paper Bastian Bloessl and Aisling O'Driscoll, "A Case for Good Defaults: Pitfalls in VANET Physical Layer Simulations," Proceedings of IFIP Wireless Days Conference 2019, Manchester, UK, April 2019 for a more detailed discussion of these parameters.
I am just giving my opinion in case someone was already in my situation:
In veins (the old version that I am using is 4.4), the "connection manager" is the responsible for evaluating a "potential" exchange of packets, thus, its transmission power is almost always set to the upper-bound.
I was been confused after I changed the vehicles "Mac1609_4" transmission power and "graphically", the connection manager was still showing me that the packets are received by some far nodes which in fact was not the case, it was just evaluating whether it is properly received or not (via the formula discussed in the links above).
Thus: changing the "TxPower" of each vehicle had really an effect beside graphically (the messages were not mounted to the upper layers).
In sum, to make a transmission range aware scheme, this is what must be done:
In the sender node (vehicle), and similarly to the pointer "traci" which deals with the mobility features, a pointer to the "mac1609" must be created and pointed to it as follows:
In "tracidemo11p.h" add ->
#include "veins/modules/mac/ieee80211p/Mac1609_4.h"//added
#include "veins/base/utils/FindModule.h"//added
and as a protected variable in the class of "tracidemo11p" in the same ".h" file ->
Mac1609_4* mac;//added
In "tracidemo11p.cc" add ->
mac = FindModule<Mac1609_4*>::findSubModule(getParentModule());
now you can manipulate "mac" as in "traci", the appropriate methods are in the "modules/mac/ieee80211p/Mac1609_4.cc & .h"
for our work, the method will be:
mac->setTxPower(10);//for example
This will have an impact on the simulation in real-time for each node instance.
It may had described it with basic concepts because I am new to omnet-veins, these was done in less than one week (and will be provided for new users as well).
I hope it will be helpful (and correct)
For my simulation I want to find out the count of lines in a single road. I search sumo and viens and there is nothing there. I can retrieves all lanes count in map but what about lanes(lines) in a single road?
You can simply parse the sumo network. It is a quite straightforward XML file. For your specific question there is even support in the sumolib which is a python library coming with sumo.
net = sumolib.net.readNet("netfile")
edge = net.getEdge("edgeID")
numLanes = edge.getLaneNumber()
You can also look here: http://sumo.dlr.de/wiki/Tools/Sumolib
how to construct tree in ns2 and how to write routing protocol in ns2?
clustered nodes to be arranged in tree structure and code for deflection routing in ns2.
I also need to know how to increase the energy of a node and how to make a node inactive.
For simulating the energy of nodes the example in tcl/ex/wireless-newnode-energy.tcl
Also, to see the available command for changing the energy of a node during simulation, see see source files in mac/wireless-phy.cc, method int WirelessPhy::command(int argc, const char*const* argv)
To implement your own routing protocol in ns2, read this article
I did not fully understand the part about the tree structure and the deflection.