I am trying to collect the following data using Veins towards each single lane, including throughput, density, mean speed, delay and collision. I know TraCI has the Simulation Value Retrieval, which can provide some information that I need. Also, the Lane Value Retrieval can help. But I have no clue where should I put the customized codes, so that the statistics can be recorded properly. For example, I want to collect the density and the mean speed of each lane every minute of the simulation time, which class should I put my codes to? TraCISenarioManager?
Any suggestion is appreciated.
I think putting the code in TraCIScenarioManager is entirely reasonable. If you want per-vehicle statistics I'd recommend putting them in the vehicles' application code, the way VEINS already collects some statistics out of the box.
Related
I'm gathering data from load sensors at about 50Hz. I might have 2-10 sensors running at a time. This data is stored locally but after a period of about a month it needs to be uploaded to the cloud. The data during this one second can vary quite significantly and is quite dynamic.
It's too much data to send because its going over GSM and signal will not always be great.
The most simplistic approach I can think of is to look at the 50 data points in 1 sec and reduce it to just enough data to make a box and whisker graph. Then, the data stored in the cloud could be used to create dashboards that look similar to how you look at stocks. This would at least show me the max, min, average and give some idea around the distribution of the load during that second.
This is probably over simplified though so I was wondering if there was a common approach to this problem in data science... take a dense set of data and reduce it to still capture the highlights and not lose its meaning.
Any help or ideas appreciated
I have setup my environment using omnet++, sumo and veins in ubuntu. I want to reduce packet loss in an emergency situation among vehicles and improve packet delivery time and cost. My project is about choosing the suitable processing position among cluster head (nodes), road side unit (rsu) and cloud. I want to achieve certain tasks that is need to implement my veins project. I have configured 50 nodes and 4 rsu and provide data rate about 6mbps and assign the packet size upto 2MB.
Therefore, how can I change the behavior of vehicles (nodes), road side unit (rsu) and cloud in order to implement the following parameters?
processing rate of clusters (nodes) = 3 Mbps.
processing rate of RSUs = 7 Mbps.
processing rate of cloud = 10 Mbps.
the range of clusters (nodes) = 60 m.
the range of RSU = 120 m.
the range of cloud = 500 m.
If you could help with building these parameters I will appreciate it.
Thank you
If you are talking about transsmision rate, then you can set the bit rate in the ini file (check veins example) but if you meant processing delay then it is usually simulated by scheduling self messages (check tictoc example). In terms of transsmsion range, veins uses Free Space Propagation model and the related parameters are set in the ini file so you can change them to decide the required range. Finally, I recommand to read more about veins and how it deal with the parameters you asked about. There are alot of answered questions on StackOverFlow about your questions.
What is your question?
I am trying to implement a metric which needs access to whole data. So instead of updating the metric in *_step() methods, I am trying to collect the outputs in the *_epoch_end() methods. However, the outputs contain only the output of the partition of the data each device gets. Basically if there are n devices, then each device is getting 1/n of the total outputs.
What's your environment?
OS: ubuntu
Packaging: conda
Version [1.0.4
Pytorch: 1.6.0
See the pytorch-lightningmanual. I think you are looking for training_step_end/validation_step_end (assuming you are using DP/DDP2).
...So, when Lightning calls any of the training_step, validation_step, test_step you will only be operating on one of those pieces. (...) For most metrics, this doesn’t really matter. However, if you want to add something to your computational graph (like softmax) using all batch parts you can use the training_step_end step.
When using the DDP backend, there's a separate process running for every GPU. They don't have access to each other's data, but there are a few special operations (reduce, all_reduce, gather, all_gather) that make the processes synchronize. When you use such operations on a tensor, the processes will wait for each other to reach the same point and combine their values in some way, for example take the sum from every process.
In theory it's possible to gather all data from all processes and then calculate the metric in one process, but this is slow and prone to problems, so you want to minimize the data that you transfer. The easiest approach is to calculate the metric in pieces and then for example take the average. self.log() calls will do this automatically when you use sync_dist=True.
If you don't want to take the average over the GPU processes, it's also possible to update some state variables at each step, and after the epoch synchronize the state variables and calculate your metric from those values. The recommended way is to create a class that uses the Metrics API, which recently moved from PyTorch Lightning to the TorchMetrics project.
If it's not enough to store a set of state variables, you can try to make your metric gather all data from all the processes. Derive your own metric from the Metric base class, overriding the update() and compute() methods. Use add_state("data", default=[], dist_reduce_fx="cat") to create a list where you collect the data that you need for calculating the metric. dist_reduce_fx="cat" will cause the data from different processes to be combined with torch.cat(). Internally it uses torch.distributed.all_gather. The tricky part here is that it assumes that all processes create identically-sized tensors. If the sizes don't match, syncing will hang indefinitely.
When I was reading a query plan about timeline information, I found that the information about "activeUnits" conflicted with that of "totalSlotMs".
I think that each "unit" in timeline information consumes a single slot. Therefore totalSlotMs is equal to the product of "activeUnits" and elapsed time. However, in my timeline information, totalSlotMs was more than twice as much as the product.
The timeline information is like the following
{"activeUnits": "84", "completedUnits": "2673", "elapsedMs": "102776", "pendingUnits": "81", "totalSlotMs": "46827346"},
{"activeUnits": "84", "completedUnits": "2673", "elapsedMs": "103776", "pendingUnits": "81", "totalSlotMs": "47040505"},
The elapsed time between two sample is 1000ms, and totalSlotMs is increased 213159ms.
If an unit consumed a single slot, totalSlotMs was equal to 84000ms, which is much less than 213159ms.
Does a single slot consume multiple slots?
A unit of work is just a concept that means "a piece of work" and its not really useful for measuring something.
In the documentation its said
Within the query plan, the terms work units and workers are used to
convey information specifically about parallelism. Elsewhere within
BigQuery, you may encounter the term slot", which is an abstracted
representation of multiple facets of query execution, including
compute, memory, and I/O resources. Top level job statistics provide
the estimate of individual query cost using the totalSlotMs estimate
of the query using this abstracted accounting.
So I understand that a work unity can use many slots which explains your doubt.
Hello StackEx community.
I am implementing certain scenarios in Veins 3.0 and I wish to collect certain traffic statistics such as the Average Waiting Time, the Average Energy consumption, etc from my simulation.
Please help on how to generate and interpret these information.
Thanks
TraCIMobility already records some statistics that you can directly use or build on. See, for example totalCO2Emission. Other statistics you might have to implement yourself, e.g., after detecting that a car was stopped for a certain time. See the OMNeT++ user manual pages on result recording and analysis for general information on how to do that and the Tic Toc tutorial for a concrete example.