In Kubeflow, how can I log the progress of the for loop to see if it is progressing?
Moreover, in the graph view, the graph renderer is having a hard time to draw 1000s of tasks. Can it be grouped?
Related
I am running a simulation of capacity about different parking lots located according to GIS map tool in AnyLogic. I added a bar chart for Parking1.statsUtilization.max()and histogram for time measure end timeTotal.distribution. But when simulation is running I can not see the bar chart and histogram chart into the simulation. I am not so sure if the problem is because I am using GIS location tool
I've recently built a smaller version of a prototype data explorer incorporating crossfilter, dc.js, and leaflet.markerCluster. The small version, (prototype dashboard), works properly. The problem I am having is when I try to scale it up to 20k points or more.
The charts still render correctly, and the map works to update the charts smoothly when zooming or panning, but when I interact with one of the charts, the transitions between the other charts are no longer smooth. They jump to their next position rather than smoothly transitioning.
I tried removing the map and this restored the transitions between the other charts to a nice smooth transition again.
I'm wondering if the re-rendering process is getting caught up with the 20k points each time an interaction occurs.
If anyone has any suggestions about where I might look for a solution I'd be grateful.
Thanks for posting a block, that makes things easier to test.
I simulated a lot more points by generating 200 rows for each of yours ~ 46k rows. I saw only a little stuttering at 100x ~ 23k rows (2017 iMac with plenty of RAM).
Leaflet.markercluster is known to be slow with more than 10K points. With 46k rows it took about 475ms for Leaflet.markercluster to clear and add the Leaflet layers:
Since there is only one thread in JavaScript (unless you use workers), D3 needs to get timeouts (actually requestAnimationFrame) every 16ms or so in order to produce fluid animation.
One workaround is to delay the map redrawing 500ms until the others have done:
dc.override(mapChart, 'redraw', function() {
window.setTimeout(() => mapChart._redraw(), 500);
});
Fork of your block with workaround.
Of course, this also makes the map take 500ms longer to redraw. And if you click around fast enough, the last map redraw will still be running when it's trying to draw the charts.
You could also try the chunked addLayers options but I think you would have to set the chunkedInterval so low that it would also slow down the markerclusters.
Processing this much data efficiently is possible in JavaScript - obviously crossfilter has no problem here. I don’t know if the cloistering algorithm is inherently too expensive. Someone on the issue suggested pre-aggregating the points, but I think this would mean you wouldn't be able to see individual points.
I'm currently rendering geological data, and have done so successfully with good results. To clarify, I input a matrix of elevations and output a single static mesh. I do this by creating a single plane for each elevation point, then, after creating all of these individual planes, merge them into a single mesh.
I've been running at 60 FPS even on a Macbook Air, but I want to push the limits. Is using a single PlaneGeometry as a heightmap as described in other terrain examples more efficient, or is it ultimately the same product at the end of the process?
Sorry for a general question without code examples, but I think this is a specific enough topic to warrant a question.
From my own tests that I spent a couple of days running, creating individual custom geometries and merging them as they are created is wildly inefficient, not just in the render loop, but also during the loading process.
The process I am using now is creating a single BufferGeometry with enough width and height segments to contain the elevation data, as described in this article.
I have started looking into the client side performance of my app using Chrome's Timeline tools. However, whilst I have found many articles on how to use them, information on how to interpret the results is more sparse and often vague.
Currently I am looking at scroll performance and attempting to hit 60FPS.
This screenshot show's the results of my most recent timeline recording.
As can be seen most frames are over 60 FPS and several are over 30 FPS.
If I zoom in on one particular frame - the one with duration 67.076ms I can see a few things:
the duration of the frame is 67ms, but aggregated time is 204ms
201ms of this time is spent painting BUT the two paint events in
this frame are of duration 1.327 ms and 0.106 ms
The total duration for the JS event, update layer tree and paint events is
only 2.4 ms
There is a long green hollow bar (rasterize Paint)
which lasts the duration of the frame and in fact starts before and
continues after it.
I have a few questions on this:
the aggregated time is far longer than the frame time - is it
correct to assume that these are parrallel processes?
the paint time for the frame (204ms) far exceeds the time for the two paint events (1.433ms) - is this because it includes the
rasterize paint events
why does the rasterize paint event span multiple frames?
where would one start optimizing this?
Finally can someone point me to some good resources on understanding this?
This is somewhat unfortunate result of the way the 'classic' waterfall Timeline view coalesces multiple events. If you expand that long "Rasterize Paint" event, you'll see a bunch of individual events, which are going to be somewhat shorter. You may really want to switch to Flame Chart mode for troubleshooting rendering performance, where the rasterize events are shown on appropriate threads.
I'm trying to make recurring transitions in D3 that will keep repeating indefinitely. Specifically, I'm working with a map and I want the background stars to occasionally flicker. The problem with transitions is that it appears they're all run ahead of time, so it will try to do the infinite recursion ahead of time and the page will never load. I found a related example (recursive d3 animation issue) that isn't infinite. My only other idea is to somehow use the d3 timer, but I'm not entirely sure how to go about that either. Any tips are appreciated.
Right, you can’t schedule an infinite number of transitions ahead of time. :) However, you can repeatedly schedule a new transition when the old transition ends (or starts), using transition.each to listen for end (or start) events.
Take a look at the chained transitions example for an infinitely-repeating animation. Whenever a circle transition starts, it also schedules an identical following transition, allowing the transition to repeat indefinitely.
Alternatively, you could use setInterval or setTimeout to create transitions repeatedly, as in the concurrent transitions example. Unlike the chained transitions example I linked, this approach won’t guarantee exact synchronization of chained transitions, but if all you want is an occasional background flicker, it might be a slightly simpler approach.