I'm using "threejs-examples-webgl-gpgpu_water" and would like to put an object on the surface following this surface which means sync with the vertex displacement on the point where the object is.
I cannot find out how to get the current height of coordinates x,y in the water surface. I guess the image is calculated in the shader and there is no access to the intermediate value of the deformed water surface. Can someone help?
The height values of the water are stored in a texture that is part of a render target so you could use renderer.readRenderTargetPixels ( renderTarget, x, y, width, height, buffer ) to pull out the height of the water at a particular point.
Unfortunately that texture is a floating point texture and there's no way to directly read a floating point texture in WebGL into JavaScript.
What you'd need to do is render that floating point texture into a non-floating point RGBA 8bit texture/rendertarget while quantizing it into some other representation using some code related to this.
In pseudo code
// at init time
.. make rendertarget with RGBA,UNSIGNED_BYTE texture
// at render time
.. render `gpuCompute.getCurrentRenderTarget( heightmapVariable ).texture;`
.. into your render target using a shader that converts the
.. floating point height values in RGBA like the example above
// read out a pixel from your render target
var heightData = new Uint8Array(4);
var x = ... // compute which pixel you need to read
var y = ... // to get the height you want
var width = 1
var height = 1;
renderer.readRenderTargetPixels ( renderTarget, x, y, width, height, heightData );
.. convert heightData back into a height value.
Related
(More info at end)----->
I am trying to render a small picture-in-picture display over my scene. The PiP is just a smaller texture, but it is intended to reveal secret objects in the scene when it is placed over them.
To do this, I want to render my scene, then render the SAME scene on the smaller texture, but with the exact same positioning as the main scene. The intended result would be something like this:
My problem is... I cannot get the scene on the smaller texture to match up 1:1. I keep trying various kludges, but ultimately I suspect that I need to do something to the projection matrix to pan it over to the location of the frame. I can get it to zoom correctly...just can't get it to pan.
Can anyone suggest what I need to do to my projection matrix to render my scene 1:1 (but panned by x,y) onto a smaller texture?
The data I have:
Resolution of the full-screen framebuffer
Resolution of the smaller texture
XY coordinate where I want to draw the smaller texture as an overlay sprite
The world/view/projection matrices from the original full-screen scene
The viewport from the original full-screen scene
(Edit)
Here is the function I use to produce the 3D camera:
void Make3DCamera(Vector theCameraPos, Vector theLookAt, Vector theUpVector, float theFOV, Point theRez, Matrix& theViewMatrix,Matrix& theProjectionMatrix)
{
Matrix aCombinedViewMatrix;
Matrix aViewMatrix;
aCombinedViewMatrix.Scale(1,1,-1);
theCameraPos.mZ*=-1;
theLookAt.mZ*=-1;
theUpVector.mZ*=-1;
aCombinedViewMatrix.Translate(-theCameraPos);
Vector aLookAtVector=theLookAt-theCameraPos;
Vector aSideVector=theUpVector.Cross(aLookAtVector);
theUpVector=aLookAtVector.Cross(aSideVector);
aLookAtVector.Normalize();
aSideVector.Normalize();
theUpVector.Normalize();
aViewMatrix.mData.m[0][0] = -aSideVector.mX;
aViewMatrix.mData.m[1][0] = -aSideVector.mY;
aViewMatrix.mData.m[2][0] = -aSideVector.mZ;
aViewMatrix.mData.m[3][0] = 0;
aViewMatrix.mData.m[0][1] = -theUpVector.mX;
aViewMatrix.mData.m[1][1] = -theUpVector.mY;
aViewMatrix.mData.m[2][1] = -theUpVector.mZ;
aViewMatrix.mData.m[3][1] = 0;
aViewMatrix.mData.m[0][2] = aLookAtVector.mX;
aViewMatrix.mData.m[1][2] = aLookAtVector.mY;
aViewMatrix.mData.m[2][2] = aLookAtVector.mZ;
aViewMatrix.mData.m[3][2] = 0;
aViewMatrix.mData.m[0][3] = 0;
aViewMatrix.mData.m[1][3] = 0;
aViewMatrix.mData.m[2][3] = 0;
aViewMatrix.mData.m[3][3] = 1;
if (gG.mRenderToSprite) aViewMatrix.Scale(1,-1,1);
aCombinedViewMatrix*=aViewMatrix;
// Projection Matrix
float aAspect = (float) theRez.mX / (float) theRez.mY;
float aNear = gG.mZRange.mData1;
float aFar = gG.mZRange.mData2;
float aWidth = gMath.Cos(theFOV / 2.0f);
float aHeight = gMath.Cos(theFOV / 2.0f);
if (aAspect > 1.0) aWidth /= aAspect;
else aHeight *= aAspect;
float s = gMath.Sin(theFOV / 2.0f);
float d = 1.0f - aNear / aFar;
Matrix aPerspectiveMatrix;
aPerspectiveMatrix.mData.m[0][0] = aWidth;
aPerspectiveMatrix.mData.m[1][0] = 0;
aPerspectiveMatrix.mData.m[2][0] = gG.m3DOffset.mX/theRez.mX/2;
aPerspectiveMatrix.mData.m[3][0] = 0;
aPerspectiveMatrix.mData.m[0][1] = 0;
aPerspectiveMatrix.mData.m[1][1] = aHeight;
aPerspectiveMatrix.mData.m[2][1] = gG.m3DOffset.mY/theRez.mY/2;
aPerspectiveMatrix.mData.m[3][1] = 0;
aPerspectiveMatrix.mData.m[0][2] = 0;
aPerspectiveMatrix.mData.m[1][2] = 0;
aPerspectiveMatrix.mData.m[2][2] = s / d;
aPerspectiveMatrix.mData.m[3][2] = -(s * aNear / d);
aPerspectiveMatrix.mData.m[0][3] = 0;
aPerspectiveMatrix.mData.m[1][3] = 0;
aPerspectiveMatrix.mData.m[2][3] = s;
aPerspectiveMatrix.mData.m[3][3] = 0;
theViewMatrix=aCombinedViewMatrix;
theProjectionMatrix=aPerspectiveMatrix;
}
Edit to add more information:
Just playing and tweaking numbers, I have come to a "close" result. However the "close" result requires a multiplication by some kludge numbers, that I don't understand.
Here's what I'm doing to to perspective matrix to produce my close result:
//Before calling Make3DCamera, adjusting FOV:
aFOV*=smallerTexture.HeightF()/normalRenderSize.HeightF(); // Zoom it
aFOV*=1.02f // <- WTH is this?
//Then, to pan the camera over to the x/y position I want, I do:
Matrix aPM=GetCurrentProjectionMatrix();
float aX=(screenX-normalRenderSize.WidthF()/2.0f)/2.0f;
float aY=(screenY-normalRenderSize.HeightF()/2.0f)/2.0f;
aX*=1.07f; // <- WTH is this?
aY*=1.07f; // <- WTH is this?
aPM.mData.m[2][0]=-aX/normalRenderSize.HeightF();
aPM.mData.m[2][1]=-aY/normalRenderSize.HeightF();
SetCurrentProjectionMatrix(aPM);
When I do this, my new picture is VERY close... but not exactly perfect-- the small render tends to drift away from "center" the further the "magic window" is from the center. Without the kludge number, the drift away from center with the magic window is very pronounced.
The kludge numbers 1.02f for zoom and 1.07 for pan reduce the inaccuracies and drift to a fraction of a pixel, but those numbers must be a ratio from somewhere, right? They work at ANY RESOLUTION, though-- so I have have a 1280x800 screen and a 256,256 magic window texture... if I change the screen to 1024x768, it all still works.
Where the heck are these numbers coming from?
If you don't care about sub-optimal performance (i.e., drawing the whole scene twice) and if you don't need the smaller scene in a texture, an easy way to obtain the overlay with pixel perfect precision is:
Set up main scene (model/view/projection matrices, etc.) and draw it as you are now.
Use glScissor to set the rectangle for the overlay. glScissor takes the screen-space x, y, width, and height and discards anything outside that rectangle. It looks like you have those four data items already, so you should be good to go.
Call glEnable(GL_SCISSOR_TEST) to actually turn on the test.
Set the shader variables (if you're using shaders) for drawing the greyscale scene/hidden objects/etc. You still use the same view and projection matrices that you used for the main scene.
Draw the greyscale scene/hidden objects/etc.
Call glDisable(GL_SCISSOR_TEST) so you won't be scissoring at the start of the next frame.
Draw the red overlay border, if desired.
Now, if you actually need the overlay in its own texture for some reason, this probably won't be adequate...it could be made to work either with framebuffer objects and/or pixel readback, but this would be less efficient.
Most people completely overcomplicate such issues. There is absolutely no magic to applying transformations after applying the projection matrix.
If you have a projection matrix P (and I'm assuming default OpenGL conventions here where P is constructed in a way that the vector is post-multiplied to the matrix, so for an eye space vector v_eye, we get v_clip = P * v_eye), you can simply pre-multiply some other translate and scale transforms to cut out any region of interest.
Assume you have a viewport of size w_view * h_view pixels, and you want to find a projection matrix which renders only a tile w_tile * h_tile pixels , beginning at pixel location (x_tile, y_tile) (again, assuming default GL conventions here, window space origin is bottom left, so y_tile is measured from the bottom). Also note that the _tile coordinates are to be interpreted relative to the viewport, in the typical case, that would start at (0,0) and have the size of your full framebuffer, but this is by no means required nor assumed here.
Since after applying the projection matrix we are in clip space, we need to transform our coordinates from window space pixels to clip space. Note that clip space is a 4D homogeneous space, but we can use any w value we like (except 0) to represent any point (as a point in the 3D space we care about forms a line in the 4D space we work in), so let's just use w=1 for simplicity's sake.
The view volume in clip space is denoted by the [-w,w] range, so in the w=1 hyperplane, it is [-1,1]. Converting our tile into this space yields:
x_clip = 2 * (x_tile / w_view) -1
y_clip = 2 * (y_tile / h_view) -1
w_clip = 2 * (w_tile / w_view) -1
h_clip = 2 * (h_tile / h_view) -1
We now just need to translate the objects such that the center of the tile is moved to the center of the view volume, which by definition is the origin, and scale the w_clip * h_clip sized region to the full [-1,1] extent in each dimension.
That means:
T = translate(-(x_clip + 0.5*w_clip), -(y_clip + 0.5 *h_clip), 0)
S = scale(2.0/w_clip, 2.0/h_clip, 1.0)
We can now create the modified projection matrix P' as P' = S * T * P, and that's all there is. Rendering with P' instead of P will render exactly the region of your tile to whatever viewport you are using, so for it to be pixel-exact with respect to your original viewport, you must now render with a viewport which is also w_tile * h_tile pixels big.
Note that there is also another approach: The viewport is not clamped against the framebuffer you're rendering to. It is actually valid to provide negative values for x and y. If your framebuffer for rendering your tile into is exactly w_tile * h_tile pixels, you simply could set glViewport(-x_tile, -y_tile, x_tile + w_tile, y_tile + h_tile) and render with the unmodified projection matrix P instead.
I am writing a mesh editor where I have manipulators with the help of which I change the vertices of the mesh. The task is to render the manipulators with constant dimensions, which would not change when changing the camera and viewport parameters. The projection matrix is perspective. I will be grateful for ideas how to implement the invariant scale geometry.
If I got it right you want to render some markers (for example vertex drag editation area) with the same visual size for any depth they are rendered to.
There are 2 approaches for this:
scale with depth
compute perpendicular distance to camera view (simple dot product) and scale the marker size so it has the same visual size invariant on the depth.
So if P0 is your camera position and Z is your camera view direction unit vector (usually Z axis). Then for any position P compute the scale like this:
depth = dot(P-P0,Z)
Now the scale depends on wanted visual size0 at some specified depth0. Now using triangle similarity we want:
size/dept = size0/depth0
size = size0*depth/depth0
so render your marker with size or scale depth/depth0. In case of using scaling you need to scale around your target position P otherwise your marker would shift to the sides (so translate, scale, translate back).
compute screen position and use non perspective rendering
so you transform target coordinates the same way as the graphic pipeline does until you got the screen x,y position. Remember it and in pass that will render your markers just use that instead of real position. For this rendering pass either use some constant depth (distance from camera) or use non perspective view matrix.
For more info see Understanding 4x4 homogenous transform matrices
[Edit1] pixel size
you need to use FOVx,FOVy projection angles and view/screen resolution (xs,ys) for that. That means if depth is znear and coordinate is at half of the angle then the projected coordinate will go to edge of screen:
tan(FOVx/2) = (xs/2)*pixelx/znear
tan(FOVy/2) = (ys/2)*pixely/znear
---------------------------------
pixelx = 2*znear*tan(FOVx/2)/xs
pixely = 2*znear*tan(FOVy/2)/ys
Where pixelx,pixely is size (per axis) representing single pixel visually at depth znear. In case booth sizes are the same (so pixel is square) you have all you need. In case they are not equal (pixel is not square) then you need to render markers in screen axis aligned coordinates so approach #2 is more suitable for such case.
So if you chose depth0=znear then you can set size0 as n*pixelx and/or n*pixely to get the visual size of n pixels. Or use any dept0 and rewrite the computation to:
pixelx = 2*depth0*tan(FOVx/2)/xs
pixely = 2*depth0*tan(FOVy/2)/ys
Just to be complete:
size0x = size_in_pixels*(2*depth0*tan(FOVx/2)/xs)
size0y = size_in_pixels*(2*depth0*tan(FOVy/2)/ys)
-------------------------------------------------
sizex = size_in_pixels*(2*depth0*tan(FOVx/2)/xs)*(depth/depth0)
sizey = size_in_pixels*(2*depth0*tan(FOVy/2)/ys)*(depth/depth0)
---------------------------------------------------------------
sizex = size_in_pixels*(2*tan(FOVx/2)/xs)*(depth)
sizey = size_in_pixels*(2*tan(FOVy/2)/ys)*(depth)
---------------------------------------------------------------
sizex = size_in_pixels*2*depth*tan(FOVx/2)/xs
sizey = size_in_pixels*2*depth*tan(FOVy/2)/ys
I have a background pixmap, basically a canvas, which I draw a bunch of
rectangles on and I need to rotate the pixmap and rectangles.
However rotating the background pixmap and the rectangles needs to be done
seperately, that is the rotation of the background pixmap gets handled via an
external library routine and I need to rotate and redraw the rectangles
on top manually.
So far I am actually able to rotate the rectangles by applying a
transformation matrix I got from Wikipedia
to each vertex. What I don't know is how to translate them that each rectangle retains its position relative to the canvas.
Here is a quick drawing for illustration of what I want to achieve:
I need to do this with C and Xlib, but I'm not necessarily looking for code but would appreciate some general hints/algorithms.
To get the translated position for the child object, you need to rotate the relative position vector for the child object, and then add it to the origin:
Pseudocode would be:
public static Vector2 OffsetByRotation(Vector2 childPos, Vector2 parentPos, float angle)
{
var relativeVector = childPos - parentPos;
relativeVector = Rotate(relativeVector, angle);
return parentPos + relativeVector;
}
Note that your example image not only rotates the parent object, but also translates it: your left image is rotated around (0, 300), but this point is then translated to (0, 0).
The requested transformation is
X' = 300 - Y
Y' = X
I'm about to project image into cylindrical panorama. But first I need to get the pixel (or color from pixel) I'm going to draw, then then do some Math in shaders with polar coordinates to get new position of pixel and then finally draw pixel.
Using this way I'll be able to change shape of image from polygon shape to whatever I want.
But I cannot find anything about this method (get pixel first, then do the Math and get new position for pixel).
Is there something like this, please?
OpenGL historically doesn't work that way around; it forward renders — from geometry to pixels — rather than backwards — from pixel to geometry.
The most natural way to achieve what you want to do is to calculate texture coordinates based on geometry, then render as usual. For a cylindrical mapping:
establish a mapping from cylindrical coordinates to texture coordinates;
with your actual geometry, imagine it placed within the cylinder, then from each vertex proceed along the normal until you intersect the cylinder. Use that location to determine the texture coordinate for the original vertex.
The latter is most easily and conveniently done within your geometry shader; it's a simple ray intersection test, with attributes therefore being only vertex location and vertex normal, and texture location being a varying that is calculated purely from the location and normal.
Extemporaneously, something like:
// get intersection as if ray hits the circular region of the cylinder,
// i.e. where |(position + n*normal).xy| = 1
float planarLengthOfPosition = length(position.xy);
float planarLengthOfNormal = length(normal.xy);
float planarDistanceToPerimeter = 1.0 - planarLengthOfNormal;
vec3 circularIntersection = position +
(planarDistanceToPerimeter/planarLengthOfNormal)*normal;
// get intersection as if ray hits the bottom or top of the cylinder,
// i.e. where |(position + n*normal).z| = 1
float linearLengthOfPosition = abs(position.z);
float linearLengthOfNormal = abs(normal.z);
float linearDistanceToEdge = 1.0 - linearLengthOfPosition;
vec3 endIntersection = position +
(linearDistanceToEdge/linearLengthOfNormal)*normal;
// pick whichever of those was lesser
vec3 cylindricalIntersection = mix(circularIntersection,
endIntersection,
step(linearDistanceToEdge,
planarDistanceToPerimeter));
// ... do something to map cylindrical intersection to texture coordinates ...
textureCoordinateVarying =
coordinateFromCylindricalPosition(cylindricalIntersection);
With a common implementation of coordinateFromCylindricalPosition possibly being simply return vec2(atan(cylindricalIntersection.y, cylindricalIntersection.x) / 6.28318530717959, cylindricalIntersection.z * 0.5);.
When laying out a figure in MATLAB, typing axis equal ensures that no matter what the figure dimensions, the axes will always be square:
My current problem is that I want to add a second axes to this plot. Usually, that's no problem; I would just type axes([x1 y1 x2 y2]), and a new square figure would be added with corners at (x1, y1), (x2, y2), which is a fixed location relative to the figure. The problem is, I want this new axes to be located at a fixed location relative to the first axes.
So, my questions are:
Does anyone know how I can position an axes in a figure by specifying the location relative to another axes?
Assuming I can do 1, how can I have this new axes remain in the same place even if I resize the figure?
An axis position property is relative to its parent container. Therefore, one possibility is to create a transparent panel with the same size as the first axis, then inside it create the second axis, and set its location and size as needed. The position specified would be as if it were relative to the first axis.
Now we need to always maintain the panel to be the same size/location as the first axis. Usually this can be done using LINKPROP which links a property of multiple graphic objects (panel and axis) to be the same, namely the 'Position' property.
However, this would fail in your case: when calling axis image, it fixes the data units to be the same in every direction by setting aspect ratio properties like 'PlotBoxAspectRatio' and 'DataAspectRatio'. The sad news is that the 'Position' property will not reflect the change in size, thus breaking the above solution. Here is an example to illustrate the problem: if you query the position property before/after issuing the axis image call, it will be the same:
figure, plot(1:10,1:10)
get(gca,'Position')
pause(1)
axis image
get(gca,'Position')
Fortunately for us, there is a submission on FEX (plotboxpos) that solves this exact issue, and returns the actual position of the plotting region of the axis. Once we have that, it's a matter of syncing the panel position to the axis position. One trick is to create a event listener for when the axis changes size (it appears that the 'TightInset' property changes unlike the 'Position' property, so that could be the trigger in our case).
I wrapped the above in a function AXESRELATIVE for convenience: you call it as you would the builtin AXES function. The only difference is you give it as first argument the handle to the axis you want to relatively-position the newly created axis against. It returns handles to both the new axis and its containing panel.
Here is an example usage:
%# automatic resize only works for normalized units
figure
hParentAx = axes('Units','normalized');
axis(hParentAx, 'image')
%# create a new axis positioned at normalized units with w.r.t the previous axis
%# the axis should maintain its relative position on resizing the figure
[hAx hPan] = axesRelative(hParentAx, ...
'Units','normalized', 'Position',[0.7 0.1 0.1 0.1]);
set(hAx, 'Color','r')
And the function implementation:
function [hAx hPan] = axesRelative(hParentAx, varargin)
%# create panel exactly on top of parent axis
s = warning('off', 'MATLAB:hg:ColorSpec_None');
hPan = uipanel('Parent',get(hParentAx, 'Parent'), ...
'BorderType','none', 'BackgroundColor','none', ...
'Units',get(hParentAx,'Units'), 'Position',plotboxpos(hParentAx));
warning(s)
%# sync panel to always match parent axis position
addlistener(handle(hParentAx), ...
{'TightInset' 'Position' 'PlotBoxAspectRatio' 'DataAspectRatio'}, ...
'PostSet',#(src,ev) set(hPan, 'Position',plotboxpos(hParentAx)) );
%# create new axis under the newly created panel
hAx = axes('Parent',hPan, varargin{:});
end
On a completely different note: before you recent edit, I got the impression that you were trying to produce a scatter plot of images (i.e like a usual scatter plot, but with full images instead of points).
What you suggested (from what I understand) is creating one axis for each image, and setting its position corresponding to the x/y coordinates of the point.
My solution is to use the IMAGE/IMAGESC functions and draw the small images by explicitly setting the 'XData' and 'YData' properties to shift and scale the images appropriately. The beauty of this is it require a single axis, and doesn't suffer from having to deal with resizing issues..
Here is a sample implementation for that:
%# create fan-shaped coordinates
[R,PHI] = meshgrid(linspace(1,2,5), linspace(0,pi/2,10));
X = R.*cos(PHI); Y = R.*sin(PHI);
X = X(:); Y = Y(:);
num = numel(X);
%# images at each point (they don't have to be the same)
img = imread('coins.png');
img = repmat({img}, [num 1]);
%# plot scatter of images
SCALE = 0.2; %# image size along the biggest dimension
figure
for i=1:num
%# compute XData/YData vectors of each image
[h w] = size(img{i});
if h>w
scaleY = SCALE;
scaleX = SCALE * w/h;
else
scaleX = SCALE;
scaleY = SCALE * h/w;
end
xx = linspace(-scaleX/2, scaleX/2, h) + X(i);
yy = linspace(-scaleY/2, scaleY/2, w) + Y(i);
%# note: we are using the low-level syntax of the function
image('XData',xx, 'YData',yy, 'CData',img{i}, 'CDataMapping','scaled')
end
axis image, axis ij
colormap gray, colorbar
set(gca, 'CLimMode','auto')
This is usually the sort of thing you can take care of with a custom 'ResizeFcn' for your figure which will adjust the position and size of the smaller axes with respect the the larger. Here's an example of a resize function that maintains the size of a subaxes so that it is always 15% the size of the larger square axes and located in the bottom right corner:
function resizeFcn(src,event,hAxes,hSubAxes)
figurePosition = get(get(hAxes,'Parent'),'Position');
axesPosition = get(hAxes,'Position').*figurePosition([3 4 3 4]);
width = axesPosition(3);
height = axesPosition(4);
minExtent = min(width,height);
newPosition = [axesPosition(1)+(width-minExtent)/2+0.8*minExtent ...
axesPosition(2)+(height-minExtent)/2+0.05*minExtent ...
0.15*minExtent ...
0.15*minExtent];
set(hSubAxes,'Units','pixels','Position',newPosition);
end
And here's an example of its use:
hFigure = figure('Units','pixels'); %# Use pixel units for figure
hAxes = axes('Units','normalized'); %# Normalized axes units so it auto-resizes
axis(hAxes,'image'); %# Make the axes square
hSubAxes = axes('Units','pixels'); %# Use pixel units for subaxes
set(hFigure,'ResizeFcn',{#resizeFcn,hAxes,hSubAxes}); %# Set resize function