How to prevent "animated" bokeh plot from stuttering - animation

I'm trying to create a kind of dashboard for monitoring ECG data and relevant features. When "animating" the ECG by constantly updating two ColumnDataSources, very soon the plot starts stuttering and growing slow. At the moment, I'm running this in a notebook, but it's the same with a bokeh server.
The ECG itself shows about the latest second, some 400 data points in a line. Left on its own, this runs almost smooth, but this is not really helpful, dashboard-wise. Another plot shows the heart rate variability, maybe a dozen circles but derived from the last minute of data, thus the two sources.
It seems that the more is gathered in the data sources and the faster the updates happen, the slower the plotting becomes. The rollover parameter of ColumnDataSource.stream() cuts the overall length of what is collected but ultimately does not prevent the stuttering.
There is probably some rookie mistake in my code, as I feel bokeh should be well-equipped for visualizing such an amount of data. So, here is what I do for plotting:
dashboard_source = ColumnDataSource(record[:1]) # initialize with first row
ecg_source = ColumnDataSource(record[:1]) # initialize with first row
# some options
time_window = 12 # seconds to keep in view
ecg_length = 1 # seconds to keep in view in ECG plot
update_rate = 1000 / sampling_rate # number of milliseconds between each plot update
# update function in which source data is fed from record dataframe
current_record_pos = 1
def update_dashboard_source():
global current_record_pos
new_row = record.iloc[current_record_pos]
dashboard_source.stream(new_row, rollover = sampling_rate * time_window)
ecg_source.stream(new_row, rollover = sampling_rate * ecg_length)
current_record_pos += 1
def ecg_dashboard(doc):
# dashboard element: ECG plot/s ---- ------- ---- ------- ---- ------- ---- -------
ecg_plot = figure(width=800, height=400, title='ECG', x_axis_label='time in ms', y_range=(-1, 1.5))
# plot ECG channels
for record_channel, color in zip(record_channels, ['green', 'blue']):
ecg_plot.line(source=ecg_source, x='time_ms', y=record_channel, alpha=.3, legend=record_channel+' ', color=color)
# dashboard element: heart rate variability ---- ------- ---- ------- ---- ------- ---- -------
hrv_plot = figure(width=400, height=400, title='heart rate variability', x_axis_label="r'r''", y_axis_label="r''r'''")
hrv_plot.circle(source=dashboard_source, x='r_diff_1', y='r_diff_2', size=10, alpha=.23)
# gather everything in a dashboard element and add it to the document
ecg_row = row(ecg_plot)
feature_row = row(hrv_plot)
dashboard = column(ecg_row, feature_row)
doc.add_root(dashboard)
doc.add_periodic_callback(update_dashboard_source, update_rate)
show(ecg_dashboard)
I did not find bokeh's user guide very helpful regarding updated plotting. Is there maybe a collection of best practices somewhere?

Concluding the conversation in comments: Each time a single point is added to the plot the entire canvas area in the browser is being re-drawn. This is how browsers work. Having a sampling_rate of 250 results in 250 plot updates per second that is one update per 4ms. This will make the browser running slower and slower as the number of points to be rendered (re-redrawn) every 4ms will increase.
I would advice to increase the update period from 4ms to about 100ms (possibly with larger data packets)

Related

Algorithm for packing images in increments of quality

I wish to pack an image as a low-quality image along with packets of "quality increments", such that patching the low-quality image with the "quality increments" increases its quality and brings it closer to the original image.
To put it more clearly,
I want to pack an image as a "base-image" (original image in poor quality, say 10%), and packets q1, q2, q3, .... qn
such that
base-image + q1 = original image at quality 20%
base-image + q1 + q2 = original image at quality 30%
...
base-image + q1 +q2 + .... qn = original image at quality 100%
My requirement is to pack an image and send it via a Single Board Computer (Raspberry Pi). I need to reduce the file size as much as possible, but the image should not be pixelized so much that it is unclear. Using this "Image quality in increments" approach, my idea is to get an image of low quality, and receive only a few increments (say till q3) and deem it "acceptable" so I can stop sending/receiving any more data packets.
Please guide me on how to approach this.
Here's a quick toy example of what I mentioned in the comments
Original:
Rebuilt at 0.2 compression
import cv2
import numpy as np
# load image
img = cv2.imread("image.jpg", cv2.IMREAD_GRAYSCALE);
height, width = img.shape[:2];
max_rank = min([width, height]);
# do svd
columns, diags, rows = np.linalg.svd(img, full_matrices = False);
# rebuild image with reduced rank
rank = int(max_rank * 0.2);
rebuilt = np.dot(columns[:,:rank] * diags[:rank], rows[:rank, :]);
rebuilt = rebuilt.astype(np.uint8)
# show image
cv2.imshow("Image", img);
cv2.imshow("Rebuilt", rebuilt);
cv2.waitKey(0);
The idea here is that you can send each column, diagonal, and row one at a time. The number of full sets you have whenever you decide to stop waiting is the rank that you'll use to reconstruct the image.
A more complete example
import cv2
import numpy as np
# receiver
client = [[], [], []]; # columns, diags, rows
def receive(column, value, row):
# grab global
global client;
# add new data
client[0].append(column);
client[1].append(value);
client[2].append(row);
# load image
img = cv2.imread("image.jpg", cv2.IMREAD_GRAYSCALE);
height, width = img.shape[:2];
max_rank = min([width, height]);
# do svd
columns, diags, rows = np.linalg.svd(img, full_matrices = False);
# "send" data to client one rank at a time
for a in range(max_rank):
# check progress
print("Total Ranks Sent: " + str(a + 1));
# get a single rank
column = columns[:,a];
value = diags[a];
row = rows[a,:];
# "send" to client
receive(column, value, row);
# rebuild image with current client side data
client_cols, client_diags, client_rows = client;
# convert to numpy
client_cols = np.array(client_cols);
client_diags = np.array(client_diags);
client_rows = np.array(client_rows);
client_cols = np.transpose(client_cols);
# rebuild
rebuilt = np.dot(client_cols * client_diags, client_rows);
rebuilt = rebuilt.astype(np.uint8);
# show
cv2.imshow("Rebuilt", rebuilt);
key = cv2.waitKey(0);
# early quit
if key == ord('q'):
break;

My step amcharts disappears with too many data points

I'm using amcharts v3 AmGraph to display some changing thresholds across time. Currently using the step type with noStepRisers set to true.
var thresholdGraph = new AmCharts.AmGraph();
thresholdGraph.balloonText = "[[category]] <b><span style='font-size:14px;'>Activity Thresholds: [[Threshold]] L/h</span></b>";
thresholdGraph.gapPeriod = 0;
thresholdGraph.id = "Threshold";
thresholdGraph.lineColor = "#CC0000";
thresholdGraph.markerType = "line";
thresholdGraph.minDistance = 0;
thresholdGraph.noStepRisers = true;
thresholdGraph.stepDirection = "right";
thresholdGraph.title = "Activity Thresholds";
thresholdGraph.type = "step";
thresholdGraph.valueField = "Threshold";
FC_consumption_by_TimeSeries.addGraph(thresholdGraph);
Our data points is every 5 minutes. When we select 1.5 days of data (432 data points), the threshold "disappears" on a 13" monitor [demo 1] (the tooltip still displays), but it displays on a 22.9" monitor [demo 2]. When we select 7 days of data (2016 data points), the threshold does not show on the 22.9" monitor either [demo 3]. But when we zoom using the amcharts function, the threshold reappears [demo 4], but the chart becomes "dashed" because of the noStepRisers?
Also, if noStepRisers is set to false, the threshold always appears with the vertical lines.
[demo 5]
I suspect there is a minimum pixel width to the stepRisers which is preventing the steps from showing.
The end goal is to have threshold lines always with no "stepRisers".
My 1st post in stackoverflow, please let me know if more details are required. Any ideas/known issues/workarounds/previous discussion links welcome. Please help!

Motion History Image (MHI) in Matlab

My project is to detect human activity through stored video clips.
I am successfully able to do the following:
Get the Motion History Image (MHI) from a video using OpenCV
Train and classify the set of images using Matlab
However, I want to use Matlab in order to get the Motion History Image (MHI). Is it possible, and if yes can someone guide me? Thank you.
I have attached a sample Motion History Image (MHI)
I have used the following code for MHI:
http://www.ece.iastate.edu/~alexs/classes/2007_Fall_401/code/09_MotionHistory/motempl.c
MHI is just a ways of implementing motion detection (and uses silhouettes as the basis of it).
Let suppose that the silhouette of the most recent object has been created. It also uses a timestamp to identify if the current silhouette is recent or not. The older silhouettes have to be compared with the current silhouette in order to achieve movement detection. Hence, earlier silhouettes are also saved in the image, with an earlier timestamp.
MHI describes the changes of some moving objects over the image sequence. Basically, you should only maintain an image where every pixel encodes a time information - whether the silhouette is recent or not or where the movement occurs at a given time.
Therefore the implementation of MHI is very simple e.g.:
function MHI = MHI(fg)
% Initialize the output, MHI a.k.a. H(x,y,t,T)
MHI = fg;
% Define MHI parameter T
T = 15; % # of frames being considered; maximal value of MHI.
% Load the first frame
frame1 = fg{1};
% Get dimensions of the frames
[y_max x_max] = size(frame1);
% Compute H(x,y,1,T) (the first MHI)
MHI{1} = fg{1} .* T;
% Start global loop for each frame
for frameIndex = 2:length(fg)
%Load current frame from image cell
frame = fg{frameIndex};
% Begin looping through each point
for y = 1:y_max
for x = 1:x_max
if (frame(y,x) == 255)
MHI{frameIndex}(y,x) = T;
else
if (MHI{frameIndex-1}(y,x) > 1)
MHI{frameIndex}(y,x) = MHI{frameIndex-1}(y,x) - 1;
else
MHI{frameIndex}(y,x) = 0;
end
end
end
end
end
Code from: https://searchcode.com/codesearch/view/8509149/
Update #1:
Try to draw it as follows:
% showMHI.m
% Input frame number and motion history vector to display normalized MHI
% at the specified frame.
function showMHI(n, motion_history)
frameDisp = motion_history{n};
frameDisp = double(frameDisp);
frameDisp = frameDisp ./ 15;
figure, imshow(frameDisp)
title('MHI Image');

Viewing Part of a figure

I made a simulation of 10000 times and want to view part of simulation between 5000-5200. I am able to view it with the code below, but the x-axis says 0-250. I want the x-axis to display the exact figure of 5000-5200. Also there seems to be a small gap at the end of the figure as the axis runs up to 250 for some reason. I just want to view the figure in for this set time with the x-axis showing the exact labels and without the gap at the end.
Thanks
N=10000;%Number of simulation
P=0.02;
Q = zeros(N,1); %current value of queue
X=zeros(N,1);%simulation data
Ci=0;
L=0.9;
Bu=zeros(N,1);
Bs=30;
Bd1=50;
Bd2=270;
Ti=0;
for Ti=2:N
U=rand(1);
a=log10(U);
b=log10(1-P);
c=(a/b);
d=1+c;
X(Ti)=round(d);
Ci=Ci+1;
if X(Ti)< (L)*(Bs)
Bu(Ti)=Bs;
else if X(Ti) < (L)*(Bs+Bd1)
Bu(Ti)=Bs+Bd1;
else
Bu(Ti)=Bs+Bd1+Bd2;
end
end
Ti=Ti+1;
end
plot(X(5000:5200,1),'r');
set (gca,'ylim',[0 400]);
hold on;
plot(Bu(5000:5200,1),'b');
set (gca,'ylim',[0 400]);
hold off
Plot expects two inputs, the first depicting the horizontal axis and the second depicting the vertical axis. When you do not supply two inputs, then it computes the length of the single input (in this case that length is 5200-5000 = 200), and it just uses 1 through that length (1:200 in this case) as if it is the values for the horizontal axis variable.
I think you want to issue the command:
plot(5000:5200, X(5000:5200,1), 'r')
Often Matlab will adjust plot axes for better default views, so it's probably showing the axis out to the index 250 just by virtue of some default plotting convention. You can similarly use set(gca, 'xlim', [5000 5200]) if you wish.

Algorithm to layout a table of contents on a fixed-sized page (multi-column)

I'm writing a table of contents to a standard 8.5in x 11in page. Orientation (landscape versus portrait) is a variable. I can write my TOC to an inner region when x-inch margins are applied to the page (where x is variable).
The raw data is a table with two columns: Topic and Page (i.e. { "Animals" , 1 } , { "Big Plants" , 2 } , { "Small Plants" , 2 } ). This is not a nested TOC - there are no "subtopics". All topics are at the same level of importance and font size is fixed for all text.
I want to allow for 1 or more columns of TOC per page and I'm allowing multiple pages of TOC if needed. The layout is completely dependent on text in the Topic column. If the topics are short, you can imagine putting 2 columns in portrait orientation, or 3 columns in landscape orentation. If there are long topic names, then maybe only 1 column will fit (if very long, then its ok to use multiple lines for a topic). If there are many topics, then I might spill over to multiple pages. The goal is to put as much TOC info as possible on each page.
I realize this is a hard problem. There's a number of details that I haven't explored (i.e. do all pages have to have the same number of columns?). I'm just looking for a start...something simple enough to implement in an hour or two that does the job. Anything semi-intelligent is better than forcing a 1-column TOC with character counts to determine how many rows to place on a page.
First you need a few variables:
Line_hight (inches per TOC line)
Max_TOC_width (inches of the longest TOC name)
Max_pagenum_width (inches width of the largest page number when printed)
Left, Right, Top, Bottom _border (inches of the border around the page)
Then it's pretty easy to calculated.
Lines_Per_Page = Floor( (Page_Height - Top_Border - Bottom_Border) / Line_Height )
Columns_Per_Page = Floor( (Page_Width - Left_Border - Right_Border) / (Max_TOC_Width + Max_PageNum_Width) )
Total_TOC_Per_Page = Lines_Per_Page * Columns_Per_Page
FYI: Floor( ) means round down to the nearest integer. Floor(5.9) = 5, Floor(0.1) = 0
Assuming you want all columns to be the same width:
Do one pass through the table finding the (printed) width of the longest TOC entry.
Divide the page width minus margins and take the floor to figure out how many columns you can fit.
Divide the page height minus margins by the height of a line and take the floor to figure out how many lines per column.
Repeat steps 2 and 3 in the other page orientation (e.g., landscape).
Choose the one that gives the most slots (rows times columns).
The math is slightly more complicated, since you need to account for a "gutter" between the columns. This is easily accomplished by padding your widest value.

Resources