I need to render quite alot (tens of thousands) images off-screen using OpenGL.
I am running under Windows and using QT as a framework. the solution can be windows only, it doesn't really matter.
From what I've found using Google there are a number of options for doing this
This article which seems rather dated suggest a few ways, out of which the relevant ones are:
Windows specific - Use CreateDIBSection and somehow bind the texture to it.
Use the pbuffers extension which I seem to be supported on my card.
This thread (Message 6) suggests a QT specific way of doing this using QGLWidget::renderPixmap
My question is - which one would be the fastest way? pbuffers seems to be the safest bet because it is guaranteed to be performed on the hardware but isn't using the CreateDIB method also goes through the hardware? What about the QT method? there seem to be some context-creation issue with this one. surely I would not want to create a new context for every image I create.
Does any one has some good experience with this?
EDIT: Answering the comment -
I have a constant scene which doesn't change at all and I'm rendering it from many different viewpoints. For now the images go back to the user and will be processed by the CPU. Possibly in the future they are going to be processed on the GPU.
Use FBO. It's fast, portable and much nicer to use than pbuffers.
EDIT: For best performance, alternate rendering between two different FBOs:
Render to A
Render to B
Read back from A and process it
Render to A
Read back from B
Goto 2
This way you can be reading back one FBO and then processing it while the GPU renders to the other one in parallel.
Related
Allow me to explain, so that this doesn't just get marked as an "opinion-based" question.
I'm learning processing.js right now, and I can't help but notice many of the similarities in functionality with what already exists in the Canvas API of Vanilla-JS. Perhaps writing a set of large-scale animations is much more complicated in plain old Canvas than it is in processing?
I'm asking this because, as I continue to learn more about the vanilla APIs, I'm seeing a lot of new functionality added in JS over the years that is starting to (VERY SLOWLY) make certain aspects of popular frameworks, no longer necessary (jQuery being a great example). I'm curious as to whether or not this is the case with Canvas and processing.js as well.
Personally, I'm trying to determine whether or not I should still be spending a lot of time in processing.js (I'm not asking you to make that decision for me though, but I just want some information that can help me decide what's best for me).
Stackoverflow allows specific non-coding questions about programming tools-like ProcessingJS, but your question seems likely to be closed as too broad.
Even so, here are my thoughts...
Native Canvas versus ProcessingJS
Html5 canvas was born with a rich set of possibilities rivaling Photoshop itself. However, native canvas is a relatively low-level tool where you must handle structuring, eventing, serialization and animation with your own code.
ProcessingJS adds structure, eventing, serialization, animation & many (amazing!) mathematical functions to native canvas. IMHO, ProcessingJS is a higher-level tool that's well worth learning.
Extending native canvas into a higher level tool instead of a low-level tool
With about 500 lines of javascript, you can add a reusable framework to native canvas that adds these features in within a higher level structure: eventing (including drag/drop, scaling, rotating, hit testing, etc), serialization / deserialization.
With about 100 more lines you can add a reusable framework to native canvas that does animation with easing.
Even though native canvas was born with most of the capabilities needed to present even complex content, a PathObject is sorely needed in native canvas. The PathObject would serialize paths to make them reusable. With about 50 lines you can create a reusable PathObject.
Here's a fairly useless recommendation :-p
Try to use the right tool for the job (yeah, not specifically helpful).
Learning native canvas alone will let you do, maybe 70% of pixel display tasks.
Coding the extensions (above) will get you to 90%.
Using a tool like ProcessingJS will get you to 98%.
Yes, there are always about 2% edge cases where you either "can't get there" or must reduce your design requirements to accommodate coding limitations.
A slightly more specific recommendation
Since ProcessingJS merely extends native canvas, IMHO it's well worthwhile to take a few days and learn native canvas. This knowledge will let you determine the right tool for the job.
I have 2 reshape functions and 2 display function. Each to show something that is completely independent to the other, but they must show what they are meant to show together (two independent things showing at the same time); each on it's own window.
How can I manage those 2 windows? Can anyone explain or indicate some site that explain it (I couldn't find any)? I need something really simple, if possible.
Thanks and sory for the bad English.
I take it, that you're referring to the use of GLUT. First things first: GLUT is not part of OpenGL! (I stopped counting how many times I had to write this into an answer). GLUT operates on the premise, that you register callbacks for each window; each window also has its own OpenGL context.
For a practical use of this you might want to look at my frustum codesample, which uses multiple GLUT windows to render the "same" scene from different angles: https://github.com/datenwolf/codesamples/blob/master/samples/OpenGL/frustum/frustum.c
Say I wanted to build a system that functions like git, but for images - where would I start?
For instance, say I wanted to just have 1 image (the original) stored on disk + the diff. When the second image needs to be viewed, I rebuild it based on the original + the diff (that way I don't store two images on disk at the same time).
Can I do that in Ruby and where would I start?
Anyone can provide a nice overview, I would appreciate it. Or even some links on where to get started.
Thanks.
P.S. Assume that I have a solid grasp of Ruby (or can learn). Are there other languages I would need to know, if so...which would work best assuming that I want my solution to be OS-agnostic and work seamlessly on at least Windows & Mac.
Take a look at Version Control for Graphics I would start looking at the source code for the projects mentioned and learn from them. The issue is that some formats will shift bytes around even if you made a small change in the image, this results in a situation that is less than ideal for VCS due to the fact that even though you might still have the same image, the program sees a 90 percent change and stores useless data.
The first question that comes to my mind is: will the image size increase in the future? (or will my image change in a sensible way?) If no you could just track the colour of the pixels.
If the image is going to change its size you should think to create a more complex scenario that behaves differently.
Searching on the internet I've also found this library: it could be useful to manipulate images and/or get information from them.
I'm working on a music visualization plugin for libvisual. It's an AVS clone -- AVS being from Winamp. Right now I have a superscope plugin. This element has 4 scripts, and "point" is run at every pixel. You can imagine that it has to be rather fast. The original libvisual avs clone had a JIT compiler that was really fast, but it had some bugs and wasn't fully implemented, so I decided to try v8. Well, v8 is too slow running the compiled script at every pixel. Is there any other script engine that would be pretty fast for this purpose?
If you are running your updates on a per-pixel level, I would suggest having an off-screen in-memory representation of the screen, and update the screen as a whole, not each individual pixel. I know that this is a common issue for bitmap updates in general, not V8 per-se. I don't know enough about the specific environment you are working in to be much help, only that as I said, it's a common performance issue to try to update individual pixels against a UI canvas one at a time. If you can do an offline/offscreen representation of your canvas/ui surface then update it all at once, your performance will be much better.
Also, there will be some dependencies on how your event model is worked out. If this doesn't work well, you may need to bring this logic into a compiled COM object or something, but on a per-pixel update scheme, you will have similar issues when trying to do per-pixel updates. Not saying you are, just noting again this is the most common issue with this type of problem.
sounds like you need to use native code, or maybe a Java Applet (Not that I recommend a Java Applet, use it only if you are in full control over the client environment).
All you Stackoverflowers,
I was wondering why GUI code is responsible for sucking away many, many cpu cycles. In principle, the graphical rendering is far less complex than Doom (although most corporate GUIs will introduce lots of window dressing). The event handling layer is also seemingly a heavy cost, however, it seems that a well-written implementation should switch between contexts efficiently on modern processors with a lot of memory/cache.
If anybody has run a profiler on their big GUI application, or a common API itself, I'm interested in where the bottlenecks lie.
Possible explanations (that I imagine) may be:
High levels of abstraction between hardware and application interface
Lots of levels of indirection to the correct code to execute
Low priority (compared to other processes)
Misbehaving applications flooding API with calls
Excessive object orientation?
Complete poor design choices in API (not just issues, but design philosophy)
Some GUI frameworks are much better than others, so I'd like to hear varied perspectives. For example, the Unix/X11 system is much different than Windows and even than WinForms.
Edit: Now a community wiki - go for it. I have one more thing to add -- I'm an algorithms guy in school and would be interested if there are inefficient algorithms in GUI code and which they are. Then again, it's probably just the implementation overhead.
I've no idea generally, but I'd like to add another item to your list - font rendering and calculations. Finding vector glyphs in a font and converting them to bitmap representations with anti-aliasing is no small task. And often it needs to be done twice - first to calculate the width/height of the text for positioning, and then actually drawing the text at the right coordinates.
Also, most drawing code today relies on clipping mechanisms to update just a part of the GUI. So, if just one part needs to be redrawn, the code actually redraws the whole window behind the scenes, and then takes just the needed part to actually update.
Added:
In the comments I found this:
I'm also very interested in this. It can't be that the gui is rendered using only the cpu because if you don't have proper drivers for your gfx-card, desktop graphics render incredibly slow. If you have gfx-drivers however desktop-gfx go kinda fast but never as fast as a directx/opengl app.
Here's the deal as I understand it: every graphic card out there today supports a generic interface for drawing. I'm not sure if it's called "VESA", "SVGA", or if those are just old names from the past. Anyway, this interface involves doing everything through interrupts. For every pixel there is an interrupt call. Or something like that. The proper VGA driver however is able to take advantage of DMA and other enhancements that make the whole process WAY less CPU-intensive.
Added 2: Ah, and for OpenGL/DirectX - that's another feature of today's graphics cards. They are optimized for 3D operations in exclusive mode. That's why the speed. The normal GUI just utilizes basic 2D drawing procedures. So it gets to send the contents of the whole screen every time it wants an update. 3D applications however send a bunch of textures and triangle definitions to the VRAM (video-RAM) and then just reuse them for drawing. They just say something like "take the triangle set #38 with the texture set #25 and draw them". All these things are cached in the VRAM so this is again way faster.
I'm not sure, but I would suspect that the modern 3D-accelerated GUIs (Vista Aero, compiz on Linux, etc.) also might take advantage of this. They could send common bitmaps to the VGA up front and then just reuse them directly from the VRAM. Any application-drawn surfaces however would still need to be sent directly every time for updates.
Added 3: More ideas. :) The modern GUI's for Windows, Linux, etc. are widget-oriented (that's control-oriented for Windows speakers). The problem with this is that each widget has its own drawing code and associated drawing surface (more or less). When the window needs to get redrawn, it calls the drawing code for all its child-widgets, who in turn call the drawing code for their child-widgets, etc.. Every widget redraws its whole surface, even though some of it is obscured by other widgets. With above mentioned clipping techniques some of this drawn information is immediately discarded to reduce flickering and other artifacts. But still it's lots of manual drawing code that includes bitmap blitting, stretching, skewing, drawing lines, text, flood-filling, etc.. And all this gets translated to a series of putpixel calls that get filtered through clipping filters/masks and other stuff. Ah, yes, and alpha blending has also become popular today for nice effects which means even more work. So... yes, you could say this is because of lots of abstraction and indirection. But... could you really do it any better? I don't think so. Only 3D techniques might help, because they take advantage of GPU for alpha-calculations and clipping.
Let's begin by saying that writing libraries is much harder than writing a stand-alone code. The requirement that your abstraction be reusable in as many contexts as possible, including contexts which you haven't though of yet, makes the task challenging even for experienced programmers.
Amongst libraries, writing a GUI toolkit library is a famously difficult problem. This is because the programs which use GUI libraries range over a very wide variety of domains with very different needs. Mr Why and Martin DeMollo discussed the requirements placed of GUI libraries a little while ago.
Writing GUI widgets themselves is difficult because computer users are very sensitive minute details of the behavior of the interface. Non-native widget never feel right, don't they? In order to get non-native widget right -- in order to get any widget right, in fact -- you need to spend an inordinate amount of time tweaking the details of the behavior.
So, GUI are slow because of the inefficiencies introduced by the abstraction mechanisms used to create highly-reusable components, that added to shortness of time available to optimize the code once so much time has been spent just getting the behavior right.
Uhm, that's quite a lot.
The most simple but probably obvious answer is that the programmers behind these GUI apps, are really bad programmers. You can go along way in writing code which does the most bizarre things and it will be faster but few people seem to care how to do this or they deem it to be an expensive non-profitable time wasted effort.
To set things straight off-loading computations to the GPU won't necessarily fix any problems. The GPU is just like the CPU except it's less general purpose and more a data paralleled processor. It can do graphics computations exceptionally well. Whatever graphics API/OS and driver combination you have doesn't really matter that much... well OK, with Vista as an example, they changed the desktop composition engine. This engine is far better composting only that which has changed, and since the number one bottle neck for GUI apps is redrawing is a neat optimization strategy. This idea of virtualizing your computational needs and only update the smallest change every time.
Win32 sends WM_PAINT messages to windows when they need to be redrawn, this can be a result of windows occluding each other. However it's up to the window itself to figure out whats actually changed. More than so nothing did change or the change that was made was trivial enough so that it could have been just preformed on top of what ever top most surface you had.
This kind of graphics handling doesn't necessarily exist today. I would say that people have refrained from writing really efficient and virtualizing rendering solutions because the benefit/cost ration is rather low/high (bad).
Something Windows Presentation Foundation (WPF) does, which I think is far superior to most other GUI API is that it splits layout updates and rendering updates into two separate passes. And while WPF is managed code the rendering engine is not. What happens with rendering is that the managed WPF rendering engine builds a command queue (this is what DirectX and OpenGL does) which is then handed of to the native rendering engine. What's a bit more elegant here is that WPF will then try to retain any computation which didn't change the visual state. A trick if you may, where you avoid costly rendering calls for things that doesn't have to be rendered (virtualizing).
In contrast to WM_PAINT which tells a Win32 window to repaint itself a WPF app would check what parts of that window requires repainting and only repaint the smallest change.
Now WPF is not supreme, it's a solid effort from Microsoft but it's not the holy grail yet... the code which runs the pipeline could still be improved and the memory footprint of any managed app is still more than I would want. But I hope this is the kind of answer you are looking for.
WPF is able to do some things asynchronously rather decent, which is a huge deal if you wanna make a really responsive low-latency/low-cpu UI. Asynchronous operations is more than off-loading work on a different thread.
To summarize things slow and expensive GUI means too much repainting and the kind of repainting which is very expensive i.e. the entire surface area.
I does to some degree depend on the language. You might have noticed that Java and RealBasic applications are a fair bit slower than their C-based (C++, C#, Objective-C) counterparts.
However GUI applications are much more complex than command line apps. The Terminal window needs only to draw a simple window that doesn't support buttons.
There are also multiple loops for extra inputs and features.
I think that you can find some interesting thoughts on this topic in "Window System Design: If I had it to do over again in 2002" by James Gosling (the Java guy, also known for his work on pre-X11 windowing systems). Available online here[pdf].
The article focuses on the positive side (how to make it fast), not on the negative side (what's making it slow), but it is still a good read on the topic.