User interaction in Processing - processing

I have a general question (I know I should present specific code with a problem, but in my case the problem is of a more general nature).
In Processing, let's say I make an ellipse:
ellipse(30, 30, 10, 10);
Now, is there a way to get the pixels where this ellipse is on the canvas? The reason would be to have a way of creating user interaction with the mouse (for instance). So when someone clicks the mouse over the ellipse, something happens.
I thought of turning everything into objects and use a constructor to somehow store the position of the shape, but this is easier said than done, particularly for more complex shapes. And that is what I am interested in. It's one thing to calculate the position of an ellipse, but what about more complex shapes? Are there any libraries?

Check out the geomerative library. It has a way to check whether the mouse is inside any SVG shape. I can't remember off the top of my head but it works something like you make a shape:
myShape = RG.loadShape("shape.svg");
and a point:
RPoint p = new RPoint(mouseX, mouseY);
and the boolean function contains() will tell you if the point is inside the shape:
myShape.contains(p);

It's better to use a mathematical formula than pixel-by-pixel checking of the mouse position (it's much faster, and involves less code).
For a perfect circle, you can calculate the Euclidean distance using Pythagoras' theorem. Assume your circle is centred at position (circleX,circleY), and has a radius (not diameter) of circleR. You can check if the mouse is over the circle like this:
if(sq(mouseX-circleX)+sq(mouseY-circleY) <= sq(circleR)) {
// mouse is over circle
} else {
// mouse is not over circle
}
This approach basically imagines a right-angled triangle, where the hypotenuse (the longest side) runs from the centre of the circle to the mouse position. It uses Pythagoras' theorem to calculate the length of that hypotenuse, and if it's less than the circle's radius then the mouse is inside the circle. (It includes a slight optimisation though -- it's comparing squares to avoid doing a square root, as that can be comparatively slow.)

An alternative to my original mathematical answer also occurred to me. If you can afford the memory and processing power of drawing all your UI elements twice then you can get good results by using a secondary buffer.
The principle involves having an off-screen graphics buffer (e.g. using PGraphics). It must be exactly the same size as the main display, and have anti-aliasing disabled. Draw all your interactive UI elements (buttons etc.) to this buffer. However, instead of drawing them the normal way, give each one a unique colour which it uses for fill and stroke (don't add any text or images... just solid colours). For example, one button might be entirely red, and another entirely green. Any other RGB value works, as long as each item has a unique colour. Make sure the background has a unique colour too.
The user never sees that buffer, so don't draw it to the screen (unless you're debugging or something). When you want to detect what item the mouse is over, just lookup the mouse position on that off-screen buffer. Get the pixel colour at that location, and match it to the UI element.
After you've done all that, just go ahead and draw everything to the main display as normal.
It's worth noting that you can cut-down the processing time of this approach a lot if your UI elements never (or rarely) move. You only need to redraw the secondary buffer when something appears/disappears, animates, or changes size/position.

Related

understanding the display of the pixels on the screen

I'm sorry if this is a stupid question, but I want to make sure that I'm right or not.
suppose we have an 8x8 pixel screen and we want to represent a 2x2 square, a pixel can be black - 1 and white - 0. I would imagine this as an 8x8 matrix
[[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,1,1,0,0,0],
[0,0,0,1,1,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0],
[0,0,0,0,0,0,0,0]]
using this matrix, we paint over the pixels and update them (for example) every second. we also have the coordinates of the pixels representing the square : (4,4) (4,5) (5,4) (5,5) and if we want to move the square we add 1 to x part of coordinate.
is it true or not?
Graphics Rendering is a complex mesh of art, mathematics, and hardware, assuming you're asking about how the screen actually works instead of a pet problem on simulating displays.
The buffer you described in the question is the interface which software uses to tell the hardware (video card) what to draw on the screen, and how it is actually done is in the realm of hardware. Hence, the logic for manipulating graphics objects (things you want drawn) is separate from the rendering process itself. Your program tells the buffer which pixels you want to update, and that's all; this can be done as often as you like, regardless of whether the hardware actually manages to flush its buffers onto the screen.
The software would be responsible for sorting out what exactly to draw on the screen; this is usually handled on multiple logical levels. Higher levels would construct a virtual worldspace for your objects and determine their interactions and attributes (position, velocity, collision, etc.), as well as a camera to determine the FOV the screen should display (if your world is 3D). Lower levels would then figure out the actual pixel values to write to the buffer, based on the camera FOV (3D), or just plain pixel coordinates after applying the desired transformations (rotation, shear, resize, etc.) to the associated image (2D).
It should be noted that virtual worldspace coordinates do not necessarily reflect pixel coordinates, even in 2D worlds. I'm not an expert on this subject, frankly, but I suspect it'll be easier if you first determine how far you want the object to move in virtual space first, and then apply the necessary transformations to show the results in a viewing window with customizable dimensions.
In short, you probably don't want to 'add 1 to x' when you want to move something on screen; you move it in a high abstraction layer, and then draw the results. This will save you a lot of trouble, especially if you have a complex scene with all kinds of stuff and a background.
Assuming you want to move a group of pixels to the right, then yes, all you need to do is identify the group of pixels and add 1 to their X coordinate. Of course you need to fill in the vacated spots with zeroes, otherwise that would have been a copy operation.
Keep in mind, my answer is a bit naive in the sense that when you reach the rightmost boundary, you have to wrap.

Direct2D: Check if image is outside visible area before drawing?

Is it a reasonable optimization to omit calls to ID2D1HwndRenderTarget::DrawBitmap() if the image will end up outside the visible area? If I implement the checking logic in the application that will cost some performance, so if the first thing D2D does is doing the same check then I'd rather not do it.
I had a test with my application which renders some UI part using Direct2D (and attaching renderdoc), seems it is a bit random.
I render a mix of Rectangles, Text, Path geometries (beziers) and Rectangle with a bitmap brush (which should be equivalent to your DrawBitmap call).
Then I capture a frame with all those objects visible, and another one panning my UI (using transform) so objects are not visible.
From there could check what is drawn or not:
Text is always culled
Solid color rectangles are not culled
Most of the times path geometries are culled, but sometimes not.
Rectangle with bitmap brush are NEVER culled
So it seems Direct2D is making different decisions depending on the types of elements you plan to draw.
Since rectangles are easily batched and cheap to draw, it seems that they are just drawn regardless.
Bitmap rectangles and text require more work, so it seems they are effectively culled.
Path geometry was looking to depend on how many polygon the geometry is tesselated to (I had a path that was translating to 26 primitives and it was not culled, another one translating to 120 and it got culled).
So you can either trust Direct2D that it will perform that optimization, but I would personally implement a quick rectangle to rectangle check just in case (it's not gonna hurt your performances as its an extremely simple operation).

Need line drawing algorithm for simulating natural pencil

I'm writing a drawing program that uses a pressure sensitive table for input. I'd like to be able to simulate the soft pencil effect that many other art programs have (such as Paint Tool SAI, Art Rage). Technique I'm using at the moment is functional, but is missing the cleanness I see in more professional programs.
My algorithm at the moment works like this:
Create a bitmap representing the head of the brush. This is just a transparent bitmap with a black circle drawn on it. The circle has an inner radius that is solid black and an outer radius. The blackness linearly fades from opaque to transparent as you move from the inner to the outer radius.
Capture input events from my tablet. Each point contains an (x, y) coordinate as well as a pressure value
For every point after the first one, draw a line from the previous point to the current one. This is done by drawing (daubing) the brush bitmap several times between the two points. The step size between each daub is chosen so there is an overlap between subsequent daubs.
This works reasonably well, but the result is a line that is somewhat blobby and jagged.
One thing I need to do is somehow smooth out the input points so that the stroke as a whole is smooth.
The other thing I need to do is figure out how to 'drag' the brush head along this path to make the stroke. If the spacing is too far apart, the stroke looks like a line of circles. If too close together, the stroke builds up on itself and becomes very dark. (I tried to fix this by attenuating the brush by the spacing. This does make things more consistent, but stops the stroke from being fully opaque).
Anyhow, I'd expect that there's a lot of research already done on this, if only I knew where to look. Please let me know if there are any better pencil drawing algorithms out there.
Instead of drawing the new circle over what has already been drawn, using the standard blending functions (so that regions of overlap get a higher opacity), you need to keep the maximum opacity so far.
Only after you have built up the complete stroke (as on a white sheet), you can blend it to the existing line art.
The picture illustrates the difference between blending and keeping the maximum opacity.

three.js - Overlapping layers flickering

When several objects overlap on the same plane, they start to flicker. How do I tell the renderer to put one of the objects in front?
I tried to use .renderDepth, but it only works partly -
see example here: http://liveweave.com/ahTdFQ
Both boxes have the same size and it works as intended. I can change which of the boxes is visible by setting .renderDepth. But if one of the boxes is a bit smaller (say 40,50,50) the contacting layers are flickering and the render depth doesn't work anymore.
How to fix that issue?
When .renderDepth() doesn't work, you have to set the depths yourself.
Moving whole meshes around is indeed not really efficient.
What you are looking for are offsets bound to materials:
material.polygonOffset = true;
material.polygonOffsetFactor = -0.1;
should solve your issue. See update here: http://liveweave.com/syC0L4
Use negative factors to display and positive factors to hide.
Try for starters to reduce the far range on your camera. Try with 1000. Generally speaking, you shouldn't be having overlapping faces in your 3d scene, unless they are treated in a VERY specific way (look up the term 'decal textures'/'decals'). So basically, you have to create depth offsets, and perhaps even pre sort the objects when doing this, which all requires pretty low-level tinkering.
If the far range reduction helps, then you're experiencing a lack of precision (depending on the device). Also look up 'z fighting'
UPDATE
Don't overlap planes.
How do I tell the renderer to put one of the objects in front?
You put one object in front of the other :)
For example if you have a camera at 0,0,0 looking at an object at 0,0,10, if you want another object to be behind the first object put it at 0,0,11 it should work.
UPDATE2
What is z-buffering:
http://en.wikipedia.org/wiki/Z-buffering
http://msdn.microsoft.com/en-us/library/bb976071.aspx
Take note of "floating point in range of 0.0 - 1.0".
What is z-fighting:
http://en.wikipedia.org/wiki/Z-fighting
...have similar values in the z-buffer. It is particularly prevalent with
coplanar polygons, where two faces occupy essentially the same space,
with neither in front. Affected pixels are rendered with fragments
from one polygon or the other arbitrarily, in a manner determined by
the precision of the z-buffer.
"The renderer cannot reposition anything."
I think that this is completely untrue. The renderer can reposition everything, and probably does if it's not shadertoy, or some video filter or something. Every time you move your camera the renderer repositions everything (the camera is actually the only thing that DOES NOT MOVE).
It seems that you are missing some crucial concepts here, i'd start with this:
http://www.opengl-tutorial.org/beginners-tutorials/tutorial-3-matrices/
About the depth offset mentioned:
How this would work, say you want to draw a decal on a surface. You can 'draw' another mesh on this surface - by say, projecting a quad onto it. You want to draw a bullet hole over a concrete wall and end up with two coplanar surfaces - the wall, the bullet hole. You can figure out the depth buffer precision, find the smallest value, and then move the bullet hole mesh by that value towards the camera. The object does not get scaled (you're doing this in NDC which you can visualize as a cube and moving planes back and forth in the smallest possible increment), but does translate in depth direction, ending up in front of the other.
I don't see any flicker. The cube movement in 3D seems to be super-smooth. Can you try in a different computer (may be faster one)? I used Chrome on Macbook Pro.

How to get consistent gradient fill in GDI+ when using a rotated LinearGradientBrush?

I'm using GDI+ in my application, and I need to use a rotated LinearGradientBrush to paint several rects in the exact same way. However, although I'm calling the same code to fill each rect, the results aren't what I expect. Here's the code to create the gradient fill, where rcDraw is the rect containing the area to paint for each rect. These coordinates are in the parent window's coordinates, so they are not identical for the 2 rects.
g_hbrLinear = new LinearGradientBrush( Rect( 0, rcDraw.top, 0, rcDraw.bottom - rcDraw.top ),
clrStart, clrEnd, (REAL) 80, FALSE );
What I see on screen looks like this (http://www.nnanime.com/bugs/LinGradBrush-rotate10.png). You can see that it's as if the fill from the first rect continues into the second one. What I really want is to have the 2 rects look identical. I think I can do that if I paint each rect separately using its own client coordinates, but for the purposes of my app, I need to use the parent window's coordinates.
I guess what I'm asking is, how does GDI+ calculate the "origin" of a fill? Is it always based on 0,0 in the coordinate system you use? Is there a way to shift it? I tried TranslateTransform, but it doesn't seem to shift the fill in a way that I find predictable or understandable.
The rect passed to the linear gradient brush determines the where the left and right colors will sit, and the gradient will be painted within this rectangle.
So, I think you need to create a brush for each rectangle you are painting, where the rectangle you are painting is also passed to the constructor for the linear gradient brush.
My experience with the "transform" of linear gradient brushes matches yours; I haven't been able to understand what it's supposed to do.
You can think of a brush in GDI+ as a function mapping world co-ordinates to a color. What the brush looks like at a given point does not change based on the shape being filled.
It does change with the transform of the Graphics object you're drawing on. So, if you don't want to change the brush, you could temporarily change the transform of the Graphics object so that the rectangle you're drawing has a specific, known size and position in world coordinates. The BeginContainer and EndContainer methods should make this easy.
(There is also the RenderingOrigin property but it only affects hatch brushes, which oddly are unaffected by world transforms.)

Resources