JFreeChart Performance - performance

I'm trying to plot some graphs simultaneously:
Each representing an attribute and displays the results for several objects each containing its own data items series.
I encounter very bad performance using either add(...) or addOrUpdate(...) methods of the TimeSeries - time for plotting ~16,000 items is about 60 seconds.
I read about the performance issue - http://www.jfree.org/phpBB2/viewtopic.php?t=12130&start=0 - but it seems to me like it is much worse in my case for some reason.
I'd like to understand whether this is truly the performance that I may squeeze out of the module (2.5GHz machine running windows - I doubt that).
How can I get my application accelerated with this respect?
Here is a basic version of the code (note that it is all done in a dedicated thread):
/* attribute -> (Object -> graph values) */
protected HashMap<String,HashMap<Object,Vector<TimeSeriesDataItem>>> m_data =
new HashMap<String,HashMap<Object,Vector<TimeSeriesDataItem>>>();
public void loadGraph() {
int items = 0;
for (String attr : m_data.keySet())
for (Object obj : m_data.get(attr).keySet())
for (TimeSeriesDataItem dataItem : m_data.get(attr).get(obj))
items++;
long before = System.currentTimeMillis();
// plot each graph
for (String attr : m_data.keySet()) {
GraphXYPlot plot = m_plots.get(attr);
plot.addToObservation(m_data.get(attr));
}
System.err.printf("Time for plotting %d items is: %d ms", items, System.currentTimeMillis()-before);
// => Time for plotting 16540 items is: 59910 ms
}
public void addToObservation(HashMap<Object, Vector<TimeSeriesDataItem>> plotData) {
for (Object obj : plotData.keySet()) {
SeriesHandler handler = m_series.get(obj);
if (handler != null) {
TimeSeries fullSeries = handler.getFullSeries();
TimeSeries periodSeries = handler.getPeriodseries();
for (TimeSeriesDataItem dataItem : plotData.get(obj)) {
fullSeries.add(dataItem);
periodSeries.add(dataItem);
}
}
}
}
Thanks a lot !
Guy

Absent more details, any of several general optimizations should be considered:
Invoke setNotify(false), as suggested here.
Cache already calculated values, as discussed here.
Adopt a paging strategy, as shown here.
Chart a summary of average/time-unit values; based on the ChartEntity seen in a ChartMouseListener, show an expanded subset in an adjacent panel.

Related

"Zooming" a brushX domain?

I've made a timeline tool based off of Mike Bostock's old Brush & Zoom example. It works great when the date range is fairly simple but becomes unworkable when there are clusters of events (e.g. hourly) within a longer time range (e.g. days or weeks). The brush becomes too thin to be usable and the user is left trying to fiddle with zooming and panning in order to see the data (as in the example below).
As a first attempt at a solution I created a context menu for the brush and use the brush extent to redefine/filter the data based on the brush range (I may be using the wrong terms here). It 'sort of' works though it is a clunky and imprecise "one shot" method. Results and code below.
I am thinking that if I could "zoom" the brush (or "brush the brush") that would be a more interactive and user friendly way of working with this type of data situation. I've searched around for d3 examples and haven't found any. I am also concerned that my "subtimeline" approach won't be performative interactively since it redefines the date set and rebuilds the timeline.
I am interested in any ideas about how to handle this sort of data situation and/or if this "brushing the brush" is a dead end. Is there a better d3 way to handle this?
(edit: the display date for the last event above reads 10:50 – that is wrong, it should be 11:50 which is what is in the data)
// code edited for clarity
function createSubtimeline() {
subtimelineDates.push(moment(x.domain()[0], "L LT"));
subtimelineDates.push(moment(x.domain()[1], "L LT"));
updateData()
}
function updateData() {
var activeData
if (subtimelineDates.length != 0) {
var firstDate = subtimelineDates[0];
var lastDate = subtimelineDates[1];
activeData = timelineJson.events.filter(function (e) {
var startDate = moment(e.startDate, "L LT");
if (startDate.isAfter(firstDate) && startDate.isBefore(lastDate)) {
if (e.eventHidden == false) {
return true
} else {
return false
}
} else {
return false
}
});
} else {
activeData = timelineJson.events.filter(event => event.eventHidden == false);
}
var tStart = moment(activeData[0].startDate, "MM/D/YYYY h:mm:ss a");
var tEnd = moment(activeData[activeData.length - 1].startDate, "MM/D/YYYY h:mm:ss a");
// update timeline range
x.domain([tStart, tEnd]);
x2.domain([tStart, tEnd]);
}

How to add Kalman filter to YOLO v2

I would like to make people detection smooth by adding Kalman filter to YOLO.
In another post I noticed* (see code below in image.c file) how to get the coordinates of the bounding box in YOLO.
if(bot > im.h-1) bot = im.h-1;
// Print bounding box values
printf("Bounding Box: Left=%d, Top=%d, Right=%d, Bottom=%d\n", left, top, right, bot);
draw_box_width(im, left, top, right, bot, width, red, green, blue);
The purpose of using Kalman filter is for object tracking (not sure for smoothing). If C++ implementation is okay, You probably want to use this popular github repository https://github.com/AlexeyAB/darknet
If you read the documentation, it has C++ API which you can use darknet as library (so you can use your yolo model) and load it to your C++ program. Take a look at C++ program example that use darknet library here https://github.com/AlexeyAB/darknet/blob/master/src/yolo_console_dll.cpp .
In that C++ code, the author gives 3 option to do object tracking, 1 of them is using kalman filter :
Track Optical Flow algorithm, but it only works for live detection, not for video. You can use the algorithm by uncomment this line //#define TRACK_OPTFLOW . Take a look at line 508 ~ 522
#ifdef TRACK_OPTFLOW
if (detection_data.new_detection) {
tracker_flow.update_tracking_flow(detection_data.cap_frame, detection_data.result_vec);
while (track_optflow_queue.size() > 0) {
draw_frame = track_optflow_queue.back();
result_vec = tracker_flow.tracking_flow(track_optflow_queue.front(), false);
track_optflow_queue.pop();
}
}
else {
track_optflow_queue.push(cap_frame);
result_vec = tracker_flow.tracking_flow(cap_frame, false);
}
detection_data.new_detection = true; // to correct kalman filter
#endif //TRACK_OPTFLOW
Kalman Filter, not really reccommended because it's not really accurate, but might work for CCTV or stationary camera. To use kalman filter change this value to true bool const use_kalman_filter = false;. Take a look at line 524 ~ 532
// track ID by using kalman filter
if (use_kalman_filter) {
if (detection_data.new_detection) {
result_vec = track_kalman.correct(result_vec);
}
else {
result_vec = track_kalman.predict();
}
}
Custom Object Tracker, I used this custom function and it performs better than kalman filter in my case, it will give you track ID for each object.
// track ID by using custom function
else {
int frame_story = std::max(5, current_fps_cap.load());
result_vec = detector.tracking_id(result_vec, true, frame_story, 40);
}

Performance drop loop vs iterator

I am using kotlin in combination with lwjgl. So far I had the following code that ran several thousand times per second:
// val textureMap = HashMap<Int, Texture>()
fun bind() {
var index = 0
for(entry in textureMap) {
glActiveTexture(GL_TEXTURE0 + index)
entry.value.bind()
program.setInt(entry.key, index)
++index
}
}
So while this was running absolutely fast and consumed virtually 0 of my frame time as expected I had to replace it because it created an Iterator in every call, eventually leading to tens of thousands of those objects eventually getting garbage collected and halting my program for a few milliseconds which is of course not usable in my application.
So I went ahead and changed it to the following code:
// textures = ArrayList<Texture>()
// indices = ArrayList<Int>()
fun bind() {
var index = 0
while(index < textures.size) {
val uniform = indices[index]
val texture = textures[index]
glActiveTexture(GL_TEXTURE0 + index)
texture.bind()
program.setInt(uniform, index)
++index
}
}
Now for some reason I am noticing a massive drop in performance, namely the function now uses several seconds per frame. Using jvisualvm I was able to determine that all that time is spent in glActiveTexture in the native part as well as the native function in program.setInt(...). I am absolutely stumped why this is the case, especially after comparing the byte code of the two.
This is the decompiled class file for the first (fast) version:
public final void bind()
{
int index = 0;
Map localMap = (Map)this.textureMap;
for (Map.Entry entry : localMap.entrySet())
{
GL13.glActiveTexture(33984 + index);
((Texture)entry.getValue()).bind(); Program
tmp66_63 = this.program;
if (tmp66_63 == null) {
Intrinsics.throwUninitializedPropertyAccessException("program");
}
tmp66_63.setInt(((Number)entry.getKey()).intValue(), index);
index++;
}
}
And that is the byte code of the slow version:
public final void bind()
{
int index = 0;
while (index < this.textures.size())
{
Integer uniform = (Integer)this.indices.get(index);
Texture texture = (Texture)this.textures.get(index);
GL13.glActiveTexture(33984 + index);
texture.bind(); Program
tmp52_49 = this.program;
if (tmp52_49 == null) {
Intrinsics.throwUninitializedPropertyAccessException("program");
}
Integer tmp62_61 = uniform;Intrinsics.checkExpressionValueIsNotNull(tmp62_61, "uniform");tmp52_49.setInt(tmp62_61.intValue(), index);
index++;
}
}
I am extremely confused what is going on here. In both versions the call to glActiveTexture is GL_TEXTURE0 + <an int value>, yet one takes so much more time thatn the other.
Does anyone have an idea what I am missing here?
Basically my entire question can be removed. I should have debugged and not only profiled. The problem was the code that populated the lists, and it didnt remove the old values so the lists grew larger and larger and the loop just ran so many more times over time...
In case anyone was wondering how I fixed my problem with the allocations I essentially created two collections, one is containing the uniforms and one is mapping them to textures. And then I can iterate over the uniforms and then get the respective texture. So no pointless Iterator objects are created but I am also not having any duplicates :)

Bad relocalization after motion tracking loss

With my team we want to implement area learning for relocalization purposes in our projects.
I added this functionnality and it seems to work well. But when a drift disaster happens (motion tracking lost) and that the main camera is instantaneously projected in "the other side of the universe" the program doesn't succeed in relocalizing it : the camera is 2 meters below, or 3 meters beside than where it should be.
Is it an area description error (because it has got not enough point of interests) ?
Or I still have not understood how to use area learning ?
Thanks a lot.
P.S.:
I use the Unity SDK.
public void Update()
{
TangoPoseData pose = new TangoPoseData ();
TangoCoordinateFramePair pair;
if(poseLocalized)
{
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_AREA_DESCRIPTION;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
}
else
{
pair.baseFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_START_OF_SERVICE;
pair.targetFrame = TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_DEVICE;
}
double timestamp = VideoOverlayProvider.RenderLatestFrame(TangoEnums.TangoCameraId.TANGO_CAMERA_COLOR);
PoseProvider.GetPoseAtTime (pose, timestamp, pair);
m_status = pose.status_code;
if (pose.status_code == TangoEnums.TangoPoseStatusType.TANGO_POSE_VALID)
{
// it does not differ with the pair base frame
Matrix4x4 ssTd = UpdateTransform(pose);
m_uwTuc = m_uwTss * ssTd * m_dTuc;
}
}
public void OnTangoPoseAvailable(TangoPoseData pose)
{
if (pose == null)
{
return;
}
// Relocalization signal
if (pose.framePair.baseFrame == TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_AREA_DESCRIPTION &&
pose.framePair.targetFrame == TangoEnums.TangoCoordinateFrameType.TANGO_COORDINATE_FRAME_START_OF_SERVICE)
{
poseLocalized = true;
}
// If pose status is not valid, nothing is valid
if (!(pose.status_code == TangoEnums.TangoPoseStatusType.TANGO_POSE_VALID))
{
poseLocalized = false;
// Do I forget something here ?
}
}
I've regularly observed that the localization and re-localization of the Area Learning can produce x,y Pose coordinates off by a few meters.
Coordinates can be more accurate if I take more care in recording an area well before moving to a new area.
Upon re-localization the coordinate accuracy is improved if the tablet is able to observe the area using slow, consistent movements before traveling to a new area.
If I learn a new area I always return to a well known area for better accuracy as described by drift correction:
I have two Tango tablets using a Java app that is autonomously navigating an iRobot in my home. I've setup a grid test site using 1 meter tape marks to make the observations.

Get a random item from a collection that does not already exist in another collection - LINQ?

I am trying to learn LINQ but it is quite confusing at first!
I have a collection of items that have a color property (MyColor). I have another collection of all colors (called AvailableColors - lets say 10 for example).
I want to get a random color from the AvailableColors that does not already exist in my collection.
My current C# code just gets a random color but I would like to rewrite this in LINQ to take in the current color collection and exclude those from the possible options:
public MyColor GetRandomColour()
{
return AvailableColors[new Random().Next(0, AvailableColors.Count)];
}
so it would take in the existing collection:
public MyColor GetRandomColour(ListOfSomethingWithColorProperty)
Thanks for any pointers!
Excluding already-used colors implies saving of state. You might be better off writing an iterator and using yield return to return the next random color in the sequence. This allows you to "remember" which colors have already been used.
Once you have that, you can call it using Take(1) from Linq, if you wish.
// assumes Random object is available, preferrably a re-used instance
Color color = AvailableColors
.Except(myItems.Select(item => item.Color).Distinct())
.OrderBy(c => random.Next())
.FirstOrDefault();
Probably not terribly efficient, but also probably not a concern given a small number of items.
Another approach is to randomly order the available colors once beforehand, therefore you can go in order. Use a List<Color> so you can remove elements as you use them, or save the current index with each pull. Once the list is depleted or the index exceeds the length of the array, notify your user that you're all out of colors.
var rnd = new Random(); // don't keep recreating a Random object.
public MyColor GetRandomColour(List<Something> coll)
{
var len = rnd.Next(0, AvailableColors.Count- coll.Count);
return AvailableColors.Except(coll.Select(s=>s.MyColor)).Skip(len).First();
}
I'm going to suggest that you be Linq-minded and create a good, general purpose IEnumerable<T> extension method that does the heavy lifting you require and then your GetRandomColor functions are simpler and you can use the extension method for other similar tasks.
So, first, define this extension method:
public static IEnumerable<T> SelectRandom<T>(this IEnumerable<T> #this, int take)
{
if (#this == null)
{
return null;
}
var count = #this.Count();
if (count == 0)
{
return Enumerable.Empty<T>();
}
var rnd = new Random();
return from _ in Enumerable.Range(0, take)
let index = rnd.Next(0, count)
select #this.ElementAt(index);
}
This function allows you to select zero or more randomly chosen elements from any IEnumerable<T>.
Now your GetRandomColor functions are as follows:
public static MyColor GetRandomColour()
{
return AvailableColors.SelectRandom(1).First();
}
public static MyColor GetRandomColour(IEnumerable<MyColor> except)
{
return AvailableColors.Except(except).SelectRandom(1).First();
}
The second function accepts an IEnumerable<MyColor> to exclude from your available colors so to call this function you need to select the MyColor property from your collection of items. Since you did not specify the type of this collection I felt it was better to use IEnumerable<MyColor> rather than to make up a type or to define an unnecessary interface.
So, the calling code looks like this now:
var myRandomColor = GetRandomColour(collectionOfItems.Select(o => o.MyColor));
Alternatively, you could just directly rely on Linq and the newly created extension method and do this:
var myRandomColor =
AvailableColors
.Except(collectionOfItems.Select(o => o.MyColor))
.SelectRandom(1)
.First();
This alternative is more readable and understandable and will aid maintainability of your code. Enjoy.
There's a nifty way to select a random element from a sequence. Here it's implemented as an extention method:
public static T Random<T>(this IEnumerable<T> enumerable)
{
var rng = new Random(Guid.NewGuid().GetHashCode());
int totalCount = 0;
T selected = default(T);
foreach (var data in enumerable)
{
int r = rng.Next(totalCount + 1);
if (r >= totalCount)
selected = data;
totalCount++;
}
return selected;
}
This method uses the fact that probability to choose n-th element over m-th when iterating is 1/n.
With this method, you can select your colour in one line:
var color = AvailableColors.Except(UsedColors).Random();

Resources