SDL2: How to draw rectangles as quickly as possible? - performance

Background
I am working on a rendering client that draws graphical information it receives from a server. The server sends packets containing non-overlapping rectangles with different solid colors at a frame rate variably defined on the server. I currently have it configured so that the size of the screen being transmitted by the server is different than the size of the window onto which the client is drawing, so scaling is done. I need the client to draw these rectangles as quickly as possible to not fall behind the server's stream.
Currently, I am using SDL 2.0. I am using the streaming texture technique described in the SDL 2 Migration Guide to draw the rectangles onto an SDL_Surface. When the time to display a frame arrives, I calll SDL_UpdateTexture() to overwrite the pixel data for an SDL_Texture, and then I use SDL_RenderCopyEx() to copy the texture to the renderer. I need this function instead of SDL_RenderCopy() so I can specify SDL_FLIP_VERTICAL to account for the fact that the coordinates passed are bitmap-style.
Question
My current approach does not render the rectangles quickly enough. To get the client to be able to keep up with the server, I currently have to reduce the server's upload rate from 30+ FPS to 15- FPS. Even then, I have to make the socket's buffer dangerously large, and I end up getting to watch the client's rendering slowly fall behind and eventually result in packet loss.
What is the fastest way to get SDL to render these rectangles? If I am currently using the fastest method, what other APIs would others recommend to make a client that can keep up?
I have included a stripped-down version of my source code so others can look for improvements/mistakes.
Technical Details
I am using C++11, MinGW32, and SDL2 with Eclipse Kepler CDT and GCC 4.8.2 on Window 7 64-bit.
Stripped Code
int main(int argc, char** args) {
// omitted initialization code
SDL_Init(SDL_INIT_VIDEO);
SDL_Window* window = SDL_CreateWindow(
"RTSC",
SDL_WINDOWPOS_CENTERED,
SDL_WINDOWPOS_CENTERED,
windowWidth,
windowHeight,
SDL_WINDOW_SHOWN | SDL_WINDOW_RESIZABLE
);
SDL_Renderer* renderer = SDL_CreateRenderer(window, -1, 0);
SDL_Surface* surface = SDL_CreateRGBSurface(
0,
sourceWidth,
sourceHeight,
24,
0xFF << 16,
0xFF << 8,
0xFF,
0
);
SDL_FillRect(surface, nullptr, 0);
SDL_Texture* texture = SDL_CreateTexture(
renderer,
surface->format->format,
SDL_TEXTUREACCESS_STREAMING,
sourceWidth,
sourceHeight
);
bool running {true};
while (running) {
SDL_Event event;
while (SDL_PollEvent(&event)) {
switch (event.type) {
case SDL_QUIT:
running = false;
break;
case SDL_WINDOWEVENT:
switch (event.windowevent.event) {
case SDL_WINDOWEVENT_CLOSE:
running = false;
break;
default:
break;
}
break;
default:
break;
}
}
// omitted packet reception and interpretation code
for (uint32_t i {0}; i < receivedRegions; ++i) {
Region& region = regions[i];
SDL_Rect rect {
(int) region.x,
(int) region.y,
(int) region.width,
(int) region.height
};
uint32_t color =
(region.red << 16) +
(region.green << 8) +
region.blue;
SDL_FillRect(surface, &rect, color);
}
// omitted logic for determining whether to present the frame
SDL_RenderClear(renderer);
SDL_UpdateTexture(texture, nullptr, surface->pixels, surface->pitch);
SDL_RenderCopyEx(
renderer,
texture,
nullptr,
nullptr,
0,
nullptr,
SDL_FLIP_VERTICAL
);
SDL_RenderPresent(renderer);
SDL_FillRect(surface, nullptr, 0);
}
// omitted clean-up and return code
}

This is embarassing. Because of earlier instrumentation I had done on my server, I assumed all the problem was with the SDL rendering client. However, it turns out the client slows only when the server does. It has nothing to do with SDL at all. Sorry.

Related

Windows - GDI - Scaling a screen DC to a printer DC without modifying the draw functions

I'm writing a Windows application showing a document to the user. The content is painted using the GDI functions, and all appears as expected on the screen.
Now I want to print this document. I get a printer device context, and I do the exact same drawing as I do on the screen. Of course the printed content appears tiny on the top of the printed page. The reason of this behavior is clear for me, and is fully explained here:
https://www.codeproject.com/Articles/764057/GDI-Drawing-and-Printing
So I need to add a scaled viewport on my printer DC, and there are several functions to achieve that in the GDI. However I'm a little puzzled about HOW to configure these functions. I tried various examples found on the internet, but none of them worked for me.
My screen resolution is 1920x1080 pixels, and I'm trying to print on an A4 portrait page. I tested various configurations, and I found that the best approximation to fit on my printed page is the following:
::SetMapMode(hDC, MM_ISOTROPIC);
::SetWindowExtEx(hDC, 1, 1, NULL);
::SetViewportExtEx(hDC, 5, 5, NULL);
::SetViewportOrgEx(hDC, -10200, 0, NULL);
As the screen and print configurations may, of course, change on other PC, I need to know how the above values may be calculated, but I cannot find a formula that works in my case. Especially I don't know why I need to scale my canvas origin using the SetViewportOrgEx() function, nobody mentioned that on the documents I read.
So what is the correct manner to calculate my print DC viewport, considering that:
The exactly same painting functions will be used for both the screen and printer drawing, and I will NEVER write different functions to print on the screen and the printer
The screen and printer devices may be entirely configured by the user, but the printed result should always fit the document on both the screen and the printer
And as an additional question, it would be better to use a metafile to do this kind of job?
In order to map the screen coordinates to paper coordinates, we need the width and length of the paper. This information is available in GetDeviceCaps(hdc, PHYSICALWIDTH) and GetDeviceCaps(hdc, PHYSICALHEIGHT), where hdc is printer's device context. We already have the screen coordinates somewhere.
The printer cannot print on the edges of the paper. We can get that information from PHYSICALOFFSETX and PHYSICALOFFSETY.
The example below uses uses a common function paint which does all the painting. print doesn't do any painting, it calls paint instead.
This assumes that rc.left and rc.right is (0,0) in screen coordinates.
void paint(HDC hdc, RECT rc)
{
HBRUSH brush = GetSysColorBrush(COLOR_WINDOWTEXT);
InflateRect(&rc, -10, -10);
FrameRect(hdc, &rc, brush);
DrawText(hdc, L"hello world", -1, &rc, 0);
}
void print(HWND hWnd, RECT rc)
{
PRINTDLG pd = { sizeof(pd) };
pd.hwndOwner = hWnd;
pd.Flags = PD_RETURNDC;
if(!PrintDlg(&pd))
return;
HDC hdc = pd.hDC;
DOCINFO doc = { sizeof(doc) };
StartDoc(hdc, &doc);
StartPage(hdc);
SetMapMode(hdc, MM_ISOTROPIC);
SetWindowExtEx(hdc, rc.right, rc.bottom, NULL);
SetViewportExtEx(hdc,
GetDeviceCaps(hdc, PHYSICALWIDTH), GetDeviceCaps(hdc, PHYSICALHEIGHT), NULL);
SetViewportOrgEx(hdc,
-GetDeviceCaps(hdc, PHYSICALOFFSETX), -GetDeviceCaps(hdc, PHYSICALOFFSETY), NULL);
paint(hdc, rc);
EndPage(hdc);
EndDoc(hdc);
DeleteObject(hdc);
}
Testing:
case WM_PAINT:
{
PAINTSTRUCT ps;
HDC hdc = BeginPaint(hwnd, &ps);
RECT rc;
GetClientRect(hwnd, &rc);
paint(hdc, rc);
EndPaint(hwnd, &ps);
break;
}
case WM_LBUTTONDOWN:
{
RECT rc;
GetClientRect(hwnd, &rc);
print(hwnd, rc);
break;
}

In WinAPI, how to make a stretchable OpenGL window with correct mouse, trapped properly for games?

I have read a lot of Stack Overflow over the years when struggling with making sense of Microsoft Windows' strange world of CreateWindowEx() .. etc. This question, when originally asked was "What is the best way to create a fluidly resizable OpenGL window in WinAPI?"
I've been struggling with getting WinAPI to make a window that:
Has an OpenGL context
Is properly centered on the main monitor (or any monitor determined by command line signal) in both multi-monitor and single-monitor displays when in "Windowed" mode or in "Fullscreen" mode
Has a fixed internal client screen size (viewport 2d)
Doesn't allow you to click outside causing it to lose focus at the wrong times or in special cases for multi-monitor
Can be resized fluidly, but doesn't change internal "client size" (meaning that it stretches the OpenGL content which is a fixed size to the new screen size) ... the idea here is to add a layer of virtualization, so that all pixels are expressed in the same 1920x1080 (1080p) coordinate system. This part is no problem for me.
Correctly handles mouse event translation from screen_size -> client_size equivalent via the screen->client ratio
In my homegrown App framework, I have to set the display size, and even then, Windows doesn't give me the right sized window. (Sometimes the title bar is subtracted, sometimes the scrollbars are subtracted, but the context draws under the title bar, for example.)
Also, recently when moving from 2010 EE (Win32 / Windows 7) to 2015 (win32 / Windows 10), I had to change the parameters to recenter the view because it was off-centered on the main display. Now, only sometimes are these values correct or incorrect. If I go "fullscreen" for example, the same values will draw above the top of the screen such that there is an area at the bottom of the screen that shows the "gl clear color" (in my case, orange)
I can play with these things by providing the following command line parameters:
-bordered (default, and has no effect really, is the default windowed mode with the title bar and such)
-borderless (seems to go into fullscreen mode, with the app off-center where win 0,0 is actually in screen center)
-windowed (or -window)
If I don't provide -window, it defaults to "full screen" resolution-adjusted (but only if supported I assume, otherwise it might throw an error).
Anyway, all of this is very bad because
a) I have to write a bajillion cases for each resolution I'm working in, rather than write everything for 1080p and have it adjust to display size, which is what i want because it handles most new displays on laptops and desktops (this is Windows remember) (and only slightly squishes things in those corner cases)
b) I cannot resize the window fluidly, also i have to trap the mouse at center and recalculate the mouse position, so I record only the deltas -- this is to avoid the mouse leaving the window and clicking the desktop, or floating off the monitor to some other monitor, even when it is hidden. I also have to make the mouse cursor invisible so the user doesn't see this, then show a simulated mouse cursor.
c) Users who don't support specifically 1920x1080 won't be able to use full screen mode
Someone pointed this article out in another question (window border width and height in Win32 - how do I get it?):
https://web.archive.org/web/20120716062211/http://suite101.com/article/client-area-size-with-movewindow-a17846
And I've read through this, learning that AdjustWindowRectEx() has some issues:
AdjustWindowRectEx() and GetWindowRect() give wrong size with WS_OVERLAPPED
I don't use WS_OVERLAPPED, so this was only moderately helpful:
AdjustWindowRectEx() and GetWindowRect() give wrong size with WS_OVERLAPPED
Here's how I do it now:
display.Resized(display.w,display.h);
// Fill in the window class structure for testing display type.
winclass.cbSize = sizeof(WNDCLASSEX);
winclass.style = CS_DBLCLKS | CS_OWNDC | CS_HREDRAW | CS_VREDRAW;
winclass.lpfnWndProc = WinProc;
winclass.cbClsExtra = 0;
winclass.cbWndExtra = 0;
winclass.hInstance = hinstance;
winclass.hIcon = LoadIcon(NULL, IDI_APPLICATION);
winclass.hIconSm = LoadIcon(NULL, IDI_APPLICATION);
winclass.hCursor = LoadCursor(NULL, IDC_ARROW);
winclass.hbrBackground = (HBRUSH)GetStockObject(BLACK_BRUSH);
winclass.lpszMenuName = NULL;
winclass.lpszClassName = WINDOW_CLASS_NAME;
// Save the game instance handle
display.hinstance = game_instance = hinstance;
// Register the window class
if (!RegisterClassEx(&winclass)) return(0);
if (!gl.Init(hinstance, display.bits)) {
return(0);
}
// Detect the display size and create the final display profile
DWORD winStyle=
WS_EX_APPWINDOW |
WS_EX_TOPMOST /*|
WS_EX_ACCEPTFILES*/ ;
// Adjust Window, Account For Window Borders
int xPos = GetSystemMetrics(SM_CXSCREEN) - display.w;
int yPos = GetSystemMetrics(SM_CYSCREEN) - display.h;
RECT windowRect = {0, 0, display.w, display.h}; // Define Our Window Coordinates
AdjustWindowRectEx (&windowRect, WS_POPUP, 0, winStyle );
// Create the window
if (!(hwnd = CreateWindowEx(
winStyle, // extended style
WINDOW_CLASS_NAME, // class
gl.winTitle.c_str(), // title
( gl.borderless || CmdLine.Option("-borderless") ) ? (WS_POPUPWINDOW | WS_VISIBLE)
: (gl.noFullscreen ? ((CmdLine.Option("-bordered") ? WS_BORDER : 0) | WS_VISIBLE)
: (WS_POPUP | WS_VISIBLE)), // use POPUP for full screen
gl.noFullscreen && !CmdLine.Option("-recenter") ? xPos / 2 : 0,
gl.noFullscreen && !CmdLine.Option("-recenter") ? yPos / 2 : 0, // initial game window x,y
display.w, // initial game width
display.h, // initial game height
HWND_DESKTOP, // handle to parent
NULL, // handle to menu
hinstance, // instance of this application
NULL)
) // extra creation parms
) {
OUTPUT("WinAPI ERROR: Could not open window.\n");
return(0);
}
if (gl.borderless || CmdLine.Option("-borderless") ) {
LONG lStyle = GetWindowLong(hwnd, GWL_STYLE);
lStyle &= ~(WS_CAPTION | WS_THICKFRAME | WS_MINIMIZE | WS_MAXIMIZE | WS_SYSMENU);
SetWindowLong(hwnd, GWL_STYLE, lStyle);
LONG lExStyle = GetWindowLong(hwnd, GWL_EXSTYLE);
lExStyle &= ~(WS_EX_DLGMODALFRAME | WS_EX_CLIENTEDGE | WS_EX_STATICEDGE);
SetWindowLong(hwnd, GWL_EXSTYLE, lExStyle);
SetWindowPos(hwnd, NULL, 0, 0, display.w, display.h, SWP_FRAMECHANGED | SWP_NOMOVE | SWP_NOSIZE | SWP_NOZORDER | SWP_NOOWNERZORDER);
}
// Temporary change to full screen mode
ZeroMemory(&game_screen, sizeof(game_screen)); // clear out size of DEVMODE struct
game_screen.dmSize = sizeof(game_screen);
game_screen.dmPelsWidth = display.w;
game_screen.dmPelsHeight = display.h;
game_screen.dmBitsPerPel = display.bits;
game_screen.dmFields = DM_PELSWIDTH | DM_PELSHEIGHT | DM_BITSPERPEL;
ChangeDisplaySettings(&game_screen, CDS_FULLSCREEN);
// save the game window handle
display.hwnd = game_window = hwnd;
display.hdc = game_dc = GetDC(display.hwnd = game_window); // get the GDI device context
// set up the pixel format desc struct
pfd = {
sizeof(PIXELFORMATDESCRIPTOR), // size of this PFD
1, // version number
PFD_DRAW_TO_WINDOW | // supports window
PFD_SUPPORT_OPENGL | // supports OpenGL
PFD_DOUBLEBUFFER, // support double buff
PFD_TYPE_RGBA, // request RGBA format
(BYTE)display.bits, // select color depth
0, 0, 0, 0, 0, 0, // color bits ignored
0, // no alpha buff
0, // shift bit ignored
0, // no accum buff
0, 0, 0, 0, // accum bits ignored
16, // 16-bit Z-buff (depth buff)
0, // no stencil buff
0, // no aux buff
PFD_MAIN_PLANE, // main drawing layer
0, // reserved
0, 0, 0 // layer masks ignored
};
int pf; // pixel format
if (!gl.arbMultisampleSupported) {
if (!(pf = ChoosePixelFormat(game_dc, &pfd))) // match the pixel format
{
MessageBox(game_window, "OpenGL could not be initialized -- ChoosePixelFormat Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
}
} else {
pf = gl.arbMultisampleFormat;
}
if (!SetPixelFormat(game_dc, pf, &pfd)) // set the pixel format
{
MessageBox(game_window, "OpenGL could not be initialized -- SetPixelFormat Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
}
if (!(game_rc = wglCreateContext(game_dc))) // create the rendering context
{
MessageBox(game_window, "OpenGL could not be initialized -- CreateContext Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
}
if (!(upload_rc = wglCreateContext(game_dc))) // create the rendering context
{
MessageBox(game_window, "Multiple OpenGL contexts could not be initialized -- CreateContext Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
} else { // Share as much as you can between two contexts
if (!wglShareLists(game_rc, upload_rc)) {
// could use GetLastError here
MessageBox(game_window, "wglShareLists -- Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
}
}
if (!wglMakeCurrent(game_dc, display.hglrc = game_rc)) // make it current
{
MessageBox(game_window, "OpenGL could not be initialized -- MakeCurrent Error ; report this to program authors for help!", "OpenGL Error", MB_OK);
return FALSE; // error returned
}
ShowCursor(false);
ShowWindow(game_window, SW_SHOWNORMAL);
SetForegroundWindow(game_window);
In the above code, what I get is a window that has no resize functionality, hides the OS mouse cursor, and can only be exitted with ALT-TAB (or ALT-F4), and when it is exitted appears at the back of the windows Z-order. I always open my window using a parameter that sets display.w to 1920 and display.h to 1080, either in full screen or in Windowed mode. WM_SIZE is then called to adjust it to the client area.
Please note that the following WM_SIZE is called during the WinProc right after the initial time I set display.Resized(w,h):
case WM_SIZE:
{
display.Resized(LOWORD(lparam), HIWORD(lparam));
return (0);
}
break;
This is executed exactly once during app load, and in the first case it looks like the values are: 1918,1078
UPDATE: If I use the result of GetWindowRect() here, or GetClientRect() as shown below, the window mysteriously moves to Center-X,Center-Y of screen! What gives??
// RECT rect;
// if ( GetClientRect(hwnd,&rect) ) {
// display.Resized((int)rect.right,(int)rect.bottom);
// }
//if ( GetWindowRect( hwnd, &rect ) ) {
// display.Resized((int)ADIFF(rect.left,rect.right),(int)ADIFF(rect.top,rect.bottom));
//}
display.Resized(LOWORD(lparam), HIWORD(lparam));
return (0);
What steps do I need to take to make the window stretchable such that the context is resized to the view, and the mouse is properly adjusted based on the screen ratio?
Basically, there are too many edge cases to make sense of all of this. As time has gone on since the 2 years ago that I asked this question, I've had other inconsistencies between full screen and window emerge.
From what I understand there are basically 3 types of windows:
Your normal on-screen moveable/resizable window for windowing GUIs, like this browser window (if you are not on mobile)
One matched to a display's resolution support (including resolutions smaller than its native) -- we call this "Full screen" (or Fullscreen, which isn't even a word)
One that is a normal on-screen window, but lacks a title bar, borders and scroll bars, and appears as large as the screen. Referred to "on the street" as a "Borderless Window"
I want to master all of these but in a way that makes them all accessible and doesn't require special cases. I've basically given up on doing so with WinAPI, but obviously multiple companies do this. Following Microsoft's documentation isn't very helpful, and I've experimented with a lot of different CreateWindow CreateWindowEx -- many of these features are deprecated by the way, or don't work at all.
(Maybe the best question is, WHEN WILL MICROSOFT CLEAN UP THIS CRAP? But I think we all know the answer.) .. any help to get it working would be appreciated.
I'm now working in: C++, Windows API, OpenGL 3.x / 4.x, Windows 10.

OpenGL ES 2.0 on SGX540 OpenGL Offscreen PIXMAP Support

On the DM370 ( TI OMAP 3 ) with the Imagination Technologies PowerVR SGX 530 I was able to use the following code to initialize my EglSurface using CMEM and PIXMAP offscreen surfaces:
// Index to bind the attributes to vertex shaders
#define VERTEX_ARRAY 0
#define TEXCOORD_ARRAY 1
// Bit types
#define SGXPERF_RGB565 0
#define SGXPERF_ARGB8888 2
// SurfaceTypes
#define SGXPERF_SURFACE_TYPE_WINDOW 0
#define SGXPERF_SURFACE_TYPE_PIXMAP_16 1
#define SGXPERF_SURFACE_TYPE_PIXMAP_32 2
typedef struct _NATIVE_PIXMAP_STRUCT
{
long pixelFormat;
long rotation;
long width;
long height;
long stride;
long sizeInBytes;
long pvAddress;
long lAddress;
} NATIVE_PIXMAP_STRUCT;
// Init EGL with offscreen PIXMAP support
void* GLWidget::commonEglInit(int surfaceType, NATIVE_PIXMAP_STRUCT** pNativePixmapPtr) {
int windowWidthTi, windowHeightTi;
EGLint iMajorVersion, iMinorVersion;
EGLint ai32ContextAttribs[] = { EGL_CONTEXT_CLIENT_VERSION, 2, EGL_NONE };
eglDisplay = eglGetDisplay((int)0);
if (!eglInitialize(eglDisplay, &iMajorVersion, &iMinorVersion))
return NULL;
if ( !eglBindAPI(EGL_OPENGL_ES_API) ) {
return NULL;
}
EGLint pi32ConfigAttribs[5];
pi32ConfigAttribs[0] = EGL_SURFACE_TYPE;
pi32ConfigAttribs[1] = EGL_WINDOW_BIT | EGL_PIXMAP_BIT;
pi32ConfigAttribs[2] = EGL_RENDERABLE_TYPE;
pi32ConfigAttribs[3] = EGL_OPENGL_ES2_BIT;
pi32ConfigAttribs[4] = EGL_NONE;
int iConfigs;
if (!eglChooseConfig(eglDisplay, pi32ConfigAttribs, &eglConfig, 1, &iConfigs) || (iConfigs != 1))
{
fprintf(stderr,"Error: eglChooseConfig() failed.\n");
return NULL;
}
commonCreateNativePixmap(SGXPERF_ARGB8888,WIDTH, HEIGHT, pNativePixmapPtr);
eglSurface = eglCreatePixmapSurface(eglDisplay, eglConfig, *pNativePixmapPtr, NULL);
if (!fprintf(stderr,"eglCreateSurface\n"))
return NULL;
eglContext = eglCreateContext(eglDisplay, eglConfig, NULL, ai32ContextAttribs);
if (!fprintf(stderr,"eglCreateContext\n"))
return NULL;
eglMakeCurrent(eglDisplay, eglSurface, eglSurface, eglContext);
if (!fprintf(stderr,"eglMakeCurrent\n"))
return NULL;
EGLBoolean success = eglSwapInterval(eglDisplay, 1);
if ( !success ) {
fprintf(stderr,"eglSwapInterval\n");
sleep(3600);
return NULL;
}
eglQuerySurface(eglDisplay, eglSurface, EGL_WIDTH, &windowWidthTi);
eglQuerySurface(eglDisplay, eglSurface, EGL_HEIGHT, &windowHeightTi);
fprintf(stderr,"Window width=%d, Height=%d\n", windowWidthTi, windowHeightTi);
(void*)(*pNativePixmapPtr)->lAddress;
return (void*)(*pNativePixmapPtr)->lAddress;
}
On the OMAP 5 / Sitara - AM57xx EVM, with the SGX 540 GPU, I've built and deployed the processor SDK with the OpenGL libraries, cmemk.ko, and pvrsrvctl. I can successfully run the PVR OpenGL demos and they show up on the display. I'm trying to run my application on this new EVM and it always fails with:
Error: eglChooseConfig() failed.
Error creating EGL surface!
If I remove the EGL_PIXMAP_BIT in the pi32ConfigAttribs, then it gets further.
Do the AM57xx OpenGL libraries not support PIXMAP surfaces? If they do, how can I get them to work? Thanks!
You should not be using the EGL_PIXMAP_BIT. It requires the EGL to provide surfaces in a format which is directly compatible with the OS's windowing system for off-screen image transfers. Use FBOs for this instead.
Note that pixmaps are not the same thing as pixel buffers or (pbuffers).
It looks like you are using TI's embedded Linux distribution, so pixmaps would have to be compatible with something like Qt, DirectFB or X11. TI has never provided EGL drivers for OMAP that were that well integrated for specific windowing system's off-screen images. EGL_PIXMAP_BIT may have worked in the past with some specific windowing system, but not necessarily the one you are using. This article explains in more detail the differences between various types of off-screen images for OpenGL ES:
Render to Texture with OpenGL ES

Arduino/MPU6050/AdafruitMotorShieldV2: script hangs/freezes when turn on motors

I'm a newby to robotics and electronics in general, so please don't assume I tried anything you might think is obvious.
I'm trying to create a cart which will basically run around by itself (simple AI routines to avoid obstacles, go from pt. A to pt. B around corners, follow lines, etc.). I'm putting together an Adafruit Arduino Uno R3 with the Adafruit Motor Shield v2 and an MPU-6050. I'm using the "breadboard" on the Motor Shield for the circuitry, soldering everything there.
I can get all the pieces working independently with their own scripts: the Motor Shield drives the 4 motors as expected using the Adafruit library; I'm using the "JRowberg" library for the MPU-6050, and started with the example MPU6050_DMP6.ino, which works fine as long as the cart motors are turned off. My only changes in the example script below are motor startup and some simple motor commands.
As long as I leave the switch which powers the motors off, everything seems fine: it outputs to the Serial window continuously with Euler data which, I assume, is correct. However, a few seconds after I turn on the power to the motors (and the wheels start turning), it just hangs/freezes: the output to the Serial window stops (sometimes in mid-line), and the wheels keep turning at the speed of their last change. Sometimes I see "FIFO overflow" errors, but not always. Sometimes I see "nan" for some of the floating point values before it hangs, but not always.
Some things I've tried, all of which changed noting:
* I've swapped out the MPU-6050 board for another from the same manufacturer.
* I've tried moving the MPU-6050 away from the motors using a ribbon cable.
* I've changed the I2C clock using JRowber's advice (a change in a .h file and changing the value of the TWBR variable), but I don't think I've tried all possible values.
* I've changed the speed of the MotorShield in the AFMS.begin() command, although, again, there are probably other values I haven't tried, and I'm not sure how in-sync this and the TWBR value need to be.
And some other things, all to no avail.
Below is an example script which fails for me:
#include "I2Cdev.h"
#include "MPU6050_6Axis_MotionApps20.h"
// is used in I2Cdev.h
#if I2CDEV_IMPLEMENTATION == I2CDEV_ARDUINO_WIRE
#include "Wire.h"
#endif
#include "Adafruit_MotorShield.h"
#include "utility/Adafruit_PWMServoDriver.h"
#define DEBUG 1
MPU6050 mpu;
#define OUTPUT_READABLE_EULER
#define LED_PIN 13
bool blinkState = false;
bool dmpReady = false; // set true if DMP init was successful
uint8_t mpuIntStatus; // holds actual interrupt status byte from MPU
uint8_t devStatus; // return status after each device operation (0 = success, !0 = error)
uint16_t packetSize; // expected DMP packet size (default is 42 bytes)
uint16_t fifoCount; // count of all bytes currently in FIFO
uint8_t fifoBuffer[64]; // FIFO storage buffer
Quaternion q; // [w, x, y, z] quaternion container
VectorInt16 aa; // [x, y, z] accel sensor measurements
VectorInt16 aaReal; // [x, y, z] gravity-free accel sensor measurements
VectorInt16 aaWorld; // [x, y, z] world-frame accel sensor measurements
VectorFloat gravity; // [x, y, z] gravity vector
float euler[3]; // [psi, theta, phi] Euler angle container
float ypr[3]; // [yaw, pitch, roll] yaw/pitch/roll container and gravity vector
uint8_t teapotPacket[14] = { '$', 0x02, 0,0, 0,0, 0,0, 0,0, 0x00, 0x00, '\r', '\n' };
Adafruit_MotorShield AFMS = Adafruit_MotorShield();
#define NUM_MOTORS 4
#define MOTOR_FL 0
#define MOTOR_FR 1
#define MOTOR_RR 2
#define MOTOR_RL 3
Adafruit_DCMotor *myMotors[NUM_MOTORS] = {
AFMS.getMotor(1),
AFMS.getMotor(2),
AFMS.getMotor(3),
AFMS.getMotor(4),
};
#define CHANGE_SPEED_TIME 500
long changeSpeedMillis = 0;
int curSpeed = 30;
volatile bool mpuInterrupt = false; // indicates whether MPU interrupt pin has gone high
void dmpDataReady() {
mpuInterrupt = true;
}
void setup() {
#if I2CDEV_IMPLEMENTATION == I2CDEV_ARDUINO_WIRE
Wire.begin();
TWBR = 24; // 400kHz I2C clock (200kHz if CPU is 8MHz)
#elif I2CDEV_IMPLEMENTATION == I2CDEV_BUILTIN_FASTWIRE
Fastwire::setup(400, true);
#endif
Serial.begin(115200);
while (!Serial); // wait for Leonardo enumeration, others continue immediately
// start the motor shield.
AFMS.begin(); // create with the default frequency 1.6KHz
// AFMS.begin(4000); // OR with a different frequency, say 4KHz
// kill all the motors.
myMotors[MOTOR_FL]->run(BRAKE);
myMotors[MOTOR_FL]->setSpeed(0);
myMotors[MOTOR_FR]->run(BRAKE);
myMotors[MOTOR_FR]->setSpeed(0);
myMotors[MOTOR_RR]->run(BRAKE);
myMotors[MOTOR_RR]->setSpeed(0);
myMotors[MOTOR_RL]->run(BRAKE);
myMotors[MOTOR_RL]->setSpeed(0);
Serial.println("Motor Shield ready!");
Serial.println(F("Initializing I2C devices..."));
mpu.initialize();
// verify connection
Serial.println(F("Testing device connections..."));
Serial.println(mpu.testConnection() ? F("MPU6050 connection successful") : F("MPU6050 connection failed"));
// wait for ready
Serial.println(F("\nSend any character to begin DMP programming and demo: "));
while (Serial.available() && Serial.read()); // empty buffer
while (!Serial.available()); // wait for data
while (Serial.available() && Serial.read()); // empty buffer again
// load and configure the DMP
Serial.println(F("Initializing DMP..."));
devStatus = mpu.dmpInitialize();
// supply your own gyro offsets here, scaled for min sensitivity
mpu.setXGyroOffset(220);
mpu.setYGyroOffset(76);
mpu.setZGyroOffset(-85);
mpu.setZAccelOffset(1788); // 1688 factory default for my test chip
// make sure it worked (returns 0 if so)
if (devStatus == 0) {
// turn on the DMP, now that it's ready
Serial.println(F("Enabling DMP..."));
mpu.setDMPEnabled(true);
// enable Arduino interrupt detection
Serial.println(F("Enabling interrupt detection (Arduino external interrupt 0)..."));
attachInterrupt(0, dmpDataReady, RISING);
mpuIntStatus = mpu.getIntStatus();
// set our DMP Ready flag so the main loop() function knows it's okay to use it
Serial.println(F("DMP ready! Waiting for first interrupt..."));
dmpReady = true;
// get expected DMP packet size for later comparison
packetSize = mpu.dmpGetFIFOPacketSize();
} else {
// ERROR!
// 1 = initial memory load failed
// 2 = DMP configuration updates failed
// (if it's going to break, usually the code will be 1)
Serial.print(F("DMP Initialization failed (code "));
Serial.print(devStatus);
Serial.println(F(")"));
}
// configure LED for output
pinMode(LED_PIN, OUTPUT);
}
void loop() {
// if programming failed, don't try to do anything
if (!dmpReady) return;
// wait for MPU interrupt or extra packet(s) available
while (!mpuInterrupt && fifoCount < packetSize) {
// as per Vulpo's post.
delay(10);
if (millis() > changeSpeedMillis) {
curSpeed += 20;
if (curSpeed > 256) {
curSpeed = 30;
}
Serial.print("changing speed to: ");
Serial.println(curSpeed);
myMotors[MOTOR_FL]->run(FORWARD);
myMotors[MOTOR_FL]->setSpeed(curSpeed);
myMotors[MOTOR_FR]->run(FORWARD);
myMotors[MOTOR_FR]->setSpeed(curSpeed);
myMotors[MOTOR_RR]->run(FORWARD);
myMotors[MOTOR_RR]->setSpeed(curSpeed);
myMotors[MOTOR_RL]->run(FORWARD);
myMotors[MOTOR_RL]->setSpeed(curSpeed);
changeSpeedMillis = millis() + CHANGE_SPEED_TIME;
}
}
// reset interrupt flag and get INT_STATUS byte
mpuInterrupt = false;
mpuIntStatus = mpu.getIntStatus();
// get current FIFO count
fifoCount = mpu.getFIFOCount();
// check for overflow (this should never happen unless our code is too inefficient)
if ((mpuIntStatus & 0x10) || fifoCount == 1024) {
// reset so we can continue cleanly
mpu.resetFIFO();
Serial.println(F("FIFO overflow!"));
// otherwise, check for DMP data ready interrupt (this should happen frequently)
} else if (mpuIntStatus & 0x02) {
// wait for correct available data length, should be a VERY short wait
while (fifoCount < packetSize) fifoCount = mpu.getFIFOCount();
// read a packet from FIFO
mpu.getFIFOBytes(fifoBuffer, packetSize);
// track FIFO count here in case there is > 1 packet available
// (this lets us immediately read more without waiting for an interrupt)
fifoCount -= packetSize;
#ifdef OUTPUT_READABLE_EULER
// display Euler angles in degrees
mpu.dmpGetQuaternion(&q, fifoBuffer);
mpu.dmpGetEuler(euler, &q);
Serial.print("euler\t");
Serial.print(euler[0] * 180/M_PI);
Serial.print("\t");
Serial.print(euler[1] * 180/M_PI);
Serial.print("\t");
Serial.println(euler[2] * 180/M_PI);
#endif
// blink LED to indicate activity
blinkState = !blinkState;
digitalWrite(LED_PIN, blinkState);
}
}
Have you considered that your troubles are caused by interference from the currents flowing into your motors?
If your motors are DC brush, then more interference may be radiated from the brushes back into your various wires.
As a first step, perhaps let only one motor work and see if hangups diminish in frequency (although, to be sure, you need a 'scope onto a few wires carrying logic signals.

Unexpected failure when using XCreateImage with LodePNG

I am trying to add a background image to a X11 window, in Linux. I use the simple LodePNG to decode the PNG image into raw data (RGBA) then I try to set the window background.
What happens is that the window shows up for a while then it unexpectedly closes back. If I comment out the XCreateImage and XPutImage function (including the destructors) the window shows up correctly, so the window creation is not a problem.
My code looks like this:
// Headers here (xlib, lodepng) ...
// Global vars ...
Display *display;
Window window;
int window_width = 640;
int window_height = 480;
// Entry point, initialization, window creation ...
int main(int argc, char* argv[]) {
vector<unsigned char> image; //the raw pixels
unsigned width, height;
// Decode
unsigned error = lodepng::decode(image, width, height, "bg.png");
// If there's no error continue
if(!error)
{
Pixmap pixmap = XCreatePixmap
(
display,
XDefaultRootWindow(display),
width,
height,
DefaultDepth(display, 0)
);
XGCValues gr_values;
GC gr_context = XCreateGC
(
display,
window,
GCBackground,
&gr_values
);
// Here is where it fails !!!
unsigned rowbytes = 0;
XImage *ximage = XCreateImage
(
display,
CopyFromParent,
32,
XYPixmap,
0,
(char*)image.data(),
width,
height,
32,
rowbytes
);
XPutImage(
display,
pixmap,
gr_context,
ximage,
0, 0,
0, 0,
window_width,
window_height
);
XSetWindowBackgroundPixmap(display, window, pixmap);
XFreePixmap(display, pixmap);
XFreeGC(display, gr_context);
XDestroyImage(ximage);
}
}
After I decode the PNG I can see that I get the correct width and height of the image. The size of the raw data (image variable) is always 819200, no matter what image I choose, which is a bit weird and I wonder if LodePNG is not loading the image correctly (yet it gives no error and correct width and height). Other causes of this problem, I don't know. I don't get any error message, the window just closes after being saw a little bit. Maybe some of the arguments of XCreateImage is wrong but I can't figure out which.

Resources