I have read some tutorials about GLSL.
In certain position attribute is a vec4 in some vec3.
I know that the matrix operations need a vec4, but is it worth to send an additional element?
Isn't it better to send vec3 and later cast in the shader vec4(position, 1.0)?
Less data in memory - it will be faster? Or we should pack an extra element to avoid casting?
Any tips what should be better?
layout(location = 0) in vec4 position;
MVP*position;
or
layout(location = 0) in vec3 position;
MVP*vec4(position,1.0);
For vertex attributes, this will not matter. The 4th component is automatically expanded to 1.0 when it is absent.
That is to say, if you pass a 3-dimensional vertex attribute pointer to a 4-dimensional vector, GL will fill-in W with 1.0 for you. I always go with this route, it avoids having to explicitly write vec4 (...) when doing matrix multiplication on the position and it also avoids wasting memory storing the 4th component.
This works for 2D coordinates too, by the way. A 2D coordinate passed to a vec4 attribute becomes vec4 (x, y, 0.0, 1.0). The general rule is this: all missing components are replaced with 0.0 except for W, which is replaced with 1.0.
However, to people who are unaware of the behavior of GLSL in this circumstance, it can be confusing. I suppose this is why most tutorials never touch on this topic.
Related
Having a 16-bit uint texture in my C++ code, I would like to use it for z-testing in an OpenGL ES 3.0 app. How can I achieve this?
To give some context, I am making an AR app where virtual objects can be occluded by real objects. The depth texture of real environment is generated, but I can't figure out how to apply it.
In my app, I first use glTexImage2D to render backdrop image from the camera feed, then I draw some virtual objects. I would like the objects to be transparent based on a depth texture. Ideally, the occlusion testing needs to be not binary, but gradual, so that I can alpha blend the objects with background near the occlusion edges.
I can pass and read the depth texture in the fragment shader, but not sure how to use it for z-testing instead of rendering.
Lets assume you have a depth texture uniform sampler2D u_depthmap and the internal format of the depth texture is a floating point format.
To read the texel from the texture, where the current fragment is on, you have to know the size of the viewport (uniform vec2 u_vieport_size). gl_FragCoord contains the window relative coordinate (x, y, z, 1/w) values for the fragment. So the texture coordinate for the depth map is calcualted by:
vec2 map_uv = gl_FragCoord.xy / u_vieport_size;
The depth from the depth texture u_depthmap is given in range [0.0, 1.0], because of the internal floating point format. The depth of the fragment is contained in the gl_FragCoord.z, in range [0.0, 1.0], too.
That means that the depth of the map and the depth of the fragment can be calculated as follows:
uniform sampler2D u_depthmap;
uniform vec2 u_vieport_size;
void mian()
{
vec2 map_uv = gl_FragCoord.xy / u_vieport_size;
float map_depth = texture(u_depthmap, map_uv).x;
float frag_depth = gl_FragCoord.z;
.....
}
Note, map_depth and frag_depth are both in the range [0.0, 1.0]. If the were generated both with the same projection (especially the same near and far plane), then they are comparable. This means you have to ensure that the shader generates the same depth values as the ones in the depth map, for the same point in the world. If this is not the case, then you have to linearize the depth values and you have to calculate the view space Z-coordinate.
I want to create a simple heat distortion on my texture, but can't seem to figure out the steps required to accomplish this. So far, I've been able to change pixel colors the following way (using pixel shader):
varying vec3 v_Position;
varying vec4 v_Color;
varying vec3 v_Normal;
varying vec2 v_TexCoordinate;
void main()
{
var col = texture2D(u_Texture, v_TexCoordinate);
col.r = 0.5;
gl_FragColor = col;
}
This is where I get lost. How can I modify pixel locations to distort the texture? can I set any other properties, but gl_FragColor? or do I have to create a plane with many vertices and distort the vertex locations? Is it possible to get 'neighbour' pixel color values? Thanks!
How can I modify pixel locations to distort the texture?
By modifying the location from which you sample the texture. That would be the second parameter of texture2D
var col = texture2D(u_Texture, v_TexCoordinate);
^-------------^
Texture distortion goes here
Is it possible to get 'neighbour' pixel color values?
Yes, and that's the proper way to do it. In a fragment shader the location you're writing to is immutable¹, so it has all to be done through the fetch location. Also note that you can sample from the same texture an arbitrary² number of times, which enables you to implement blurring³ effects.
¹: writes to freely determined image locations (scatter writes) are supported by OpenGL-4 class hardware, but scatter writes are extremely inefficient and should be avoided.
²: there's a practical limit of the total runtime of the shader, which may be limited by the OS, and also by the desired frame rate.
³: blurring effects should be implemented using so called separable filters for largely improved performance.
I've looked around and haven't found anything relevant. I'm tyring to create a shader to give a texture smoke effect animation like here:
Not asking for a complete/full solution (although that would be awesome) but any pointers towards where I can get started to achieve this effect. Would i need to have the vertices for the drawing or is this possible if I have the texture only?
Modelling smoke with a fluid simulation isn't simple and can be very slow for a detailed simulation. Using noise to add finer details can be a fair bit faster. If this is the direction you want to head in, this answer has some good links to the little grasshopper. If you have a texture, use it to initialize the smoke density (or spawn particles for that matter) and run the simulation. If you start with vector data, and want the animation to trail along the curve as in your example it gets more complex. Perhaps draw the curve over the top of the smoke simulation, gradually drawing less of it and drawing the erased bits as density into the simulation. Spawning particles along its length and using "noise based particles" as linked above sounds like a good alternative too.
Still, it sounds like you're after something a bit simpler. I've created a short demo on shadertoy, just using perlin noise for animated turbulence on a texture. It doesn't require any intermediate texture storage or state information other than a global time.
https://www.shadertoy.com/view/Mtf3R7
The idea started with trying to create streaks of smoke that blur and grow with time. Start with a curve, sum/average colour along it and then make it longer to make the smoke appear to move. Rather than add points to the curve over time to make it longer, the curve has a fixed number of points and their distance increases with time.
To create a random curve, perlin noise is sampled recursively, providing offsets to each point in turn.
Using mipmapping, the samples towards the end of the curve can cover a larger area and make the smoke appear to blur into nothing, just as your image does. However, since this is a gather operation the end of the smoke curve is actually the start (hence the steps-i below).
//p = uv coord, o = random offset for per-pixel noise variation, t = time
vec3 smoke(vec2 p, vec2 o, float t)
{
const int steps = 10;
vec3 col = vec3(0.0);
for (int i = 1; i < steps; ++i)
{
//step along a random path that grows in size with time
p += perlin(p + o) * t * 0.002;
p.y -= t * 0.003; //drift upwards
//sample colour at each point, using mipmaps for blur
col += texCol(p, float(steps-i) * t * 0.3);
}
return col.xyz / float(steps);
}
As always with these effects, you can spend hours playing with constants getting it to look that tiny bit better. I've used a linearly changing value for the mipmap bias as the second argument to texCol(), which I'm sure could be improved. Also averaging a number of smoke() calls with varying o will give a smoother result.
[EDIT] If you want the smoke to animate along a curve with this method, I'd use a second texture that stores a "time offset" to delay the simulation for certain pixels. Then draw the curve with a gradient along it so the end of the curve will take a little while to start animating. Since it's a gather operation you should draw a much fatter lines into this time offset texture as it's the pixels around them which will gather colour. Unfortunately this will break when parts of the curve are too close or intersect.
In the example pictured it appears as if they have the vertices. Possibly the "drawing" of the flower shape was recorded and then played back continuously. Then the effect hits the vertices based on a time offset from when they were drawn. The effect there appears to mostly be a motion blur.
So to replicate this effect you would need the vertices. See how the top of the flower starts to disappear before the bottom? If you look closely you'll see that actually the blur effect timing follows the path of the flower around counter clockwise. Even on the first frame of your gif you can see that the end of the flower shape is a brighter yellow than the beginning.
The angle of the motion blur also appears to change over time from being more left oriented to being more up oriented.
And the brightness of the segment is also changing over time starting with the yellowish color and ending either black or transparent.
What I can't tell from this is if the effect is additive, meaning that they're applying the effect to the whole frame and then to the results of that effect each frame, or if it's being recreated each frame. If recreated each frame you'd be able to do the effect in reverse and have the image appear.
If you are wanting this effect on a bitmapped texture instead of a line object that's also doable, although the approach would be different.
Let's start with the line object and assume you have the vertices. The way I would approach it is that I would add a percentage of decay as an attribute to the vertex data. Then each frame that you render you'd first update the decay percentage based on the time for that vertex. Stagger them slightly.
Then the shader would draw the line segment using a motion blur shader where the amount of motion blur, the angle of the blur, and the color of the segment are controlled by a varying variable that is assigned by the decay attribute. I haven't tested this shader. Treat it like pseudocode. But I'd approach it this way... Vertex Shader:
uniform mat4 u_modelViewProjectionMatrix;
uniform float maxBlurSizeConstant; // experiment with value and it will be based on the scale of the render
attribute vec3 a_vertexPosition;
attribute vec2 a_vertexTexCoord0;
attribute float a_decay;
varying float v_decay;
varying vec2 v_fragmentTexCoord0;
varying vec2 v_texCoord1;
varying vec2 v_texCoord2;
varying vec2 v_texCoord3;
varying vec2 v_texCoord4;
varying vec2 v_texCoordM1;
varying vec2 v_texCoordM2;
varying vec2 v_texCoordM3;
varying vec2 v_texCoordM4;
void main()
{
gl_Position = u_modelViewProjectionMatrix * vec4(a_vertexPosition,1.0);
v_decay = a_decay;
float angle = 2.8 - a_decay * 0.8; // just an example of angles
vec2 tOffset = vec2(cos(angle),sin(angle)) * maxBlurSizeConstant * a_decay;
v_fragmentTexCoord0 = a_vertexTexCoord0;
v_texCoordM1 = a_vertexTexCoord0 - tOffset;
v_texCoordM2 = a_vertexTexCoord0 - 2.0 * tOffset;
v_texCoordM3 = a_vertexTexCoord0 - 3.0 * tOffset;
v_texCoordM4 = a_vertexTexCoord0 - 4.0 * tOffset;
v_texCoord1 = a_vertexTexCoord0 + tOffset;
v_texCoord2 = a_vertexTexCoord0 + 2.0 * tOffset;
v_texCoord3 = a_vertexTexCoord0 + 3.0 * tOffset;
v_texCoord4 = a_vertexTexCoord0 + 4.0 * tOffset;
}
Fragment Shader:
uniform sampler2D u_textureSampler;
varying float v_decay;
varying vec2 v_fragmentTexCoord0;
varying vec2 v_texCoord1;
varying vec2 v_texCoord2;
varying vec2 v_texCoord3;
varying vec2 v_texCoord4;
varying vec2 v_texCoordM1;
varying vec2 v_texCoordM2;
varying vec2 v_texCoordM3;
varying vec2 v_texCoordM4;
void main()
{
lowp vec4 fragmentColor = texture2D(u_textureSampler, v_fragmentTexCoord0) * 0.18;
fragmentColor += texture2D(u_textureSampler, v_texCoordM1) * 0.15;
fragmentColor += texture2D(u_textureSampler, v_texCoordM2) * 0.12;
fragmentColor += texture2D(u_textureSampler, v_texCoordM3) * 0.09;
fragmentColor += texture2D(u_textureSampler, v_texCoordM4) * 0.05;
fragmentColor += texture2D(u_textureSampler, v_texCoord1) * 0.15;
fragmentColor += texture2D(u_textureSampler, v_texCoord2) * 0.12;
fragmentColor += texture2D(u_textureSampler, v_texCoord3) * 0.09;
fragmentColor += texture2D(u_textureSampler, v_texCoord4) * 0.05;
gl_FragColor = vec4(fragmentColor.rgb, fragmentColor.a * v_decay);
}
Of course the trick is in varying the decay amount per vertex based on a slight offset in time.
If you want to do the same with a sprite you're going to do something very similar except that the difference between the decay per vertex would have to be played with to get right as there are only 4 vertices.
SORRY - EDIT
Sorry... The above shader blurs the incoming texture. It doesn't necessarily blur the color of the line being drawn. This might or might not be what you want to do. But again without knowing more of what you are actually trying to accomplish it's difficult to give you a perfect answer. I get the feeling you'd rather do this on a sprite anyway than a line vertex based object. So no you can't copy and paste this shader in to your code as is. But it shows the concept of how you'd do what you're looking to do. Especially if you're doing it on a texture instead of on a vertex based line.
Also the above shader isn't complete. For example it doesn't expand to allow the blur to get beyond the bounds of the texture. And it gets texture info from outside the area where the sprite is in the sprite sheet. To fix this you'd have to start with a bounding box larger than the sprite and shrink the sprite in the vertex to be the right size. And you'd have to not grab textels from the spite sheet beyond the bounds of the sprite. There are ways of doing this without having to include a bunch of white space around the sprite in the sprite sheet.
Update
On second look it might be particle based. If it is they again have all the vertices but as particle locations. I sort of prefer the idea that it's line segments because I don't see any gaps. So if it is particles there are a lot and they're tightly placed. The particles are still decaying cascading from the top petal around to the last. Even if it's line segments you could treat the vertices as particles to apply the wind and gravity.
As for how the smoke effect works check out this helper app by 71 squared: https://71squared.com/particledesigner
The way it works is that you buy the Mac app to use to design and save your particle. Then you go to their github and get the iOS code. But this particular code creates a particle emitter. Doing a shape out of particles would be different code. But the evolution of the particles is the same.
OpenGL ES suggests that your target platform may not have the computional power to do a real smoke simulation (and if it does, it would consume quite a bit of power, which is undesirable on a device like a phone).
However, your target device will definitively have the power to create a fake texture-space effect which looks good enough to be convincing.
First look at the animation you posted. The flower is blurring and fading, there is a sideway motion to the left ("wind") and an upwards motion of the smoke. Thus, what is primarily needed is ping-ponging between two textures, sampling for each fragment at the fragment's location offset by a vector pointing downwards and right (you only have gather available, not scatter).
No texelFetchOffset or such function in ES 2.0 so you'll have to use plain old texture2D and do the vector add yourself, but that shouldn't be a lot of trouble. Note that since you need to use texture2D anyway you'll not need to worry about gl_FragCoord either. Have the interpolator give you the correct texture coordinate (simply set texcoord of vertices of the quad to 0 on one end, and to 1 on the other end).
To get the blur effect, randomize the offset vector (e.g. by adding another random vector with a much smaller magnitude, so the "overall direction" remains the same), to get the fade effect either multiply alpha with an attenuation factor (such as 0.95) or do the same with the color (which will give you "black" rather than "transparent", but depending on wheter or not you want premultiplied alpha, that may be the correct thing).
Alternatively you could implement the blur and fade effect by generating mipmaps first (gradually fade them to transparent), and using the optional bias value in texture2D, slightly increasing the bias as time progresses. That will be, yet lower quality (possibly with visible box artefacts), but it allows you to preprocess much of the calculation ahead of time and has a much more cache-friendly access pattern.
I'm using this code in a fragment shader to round the edges of a GL point
precision mediump float;
varying vec4 fragColor;
void main() {
gl_FragColor = fragColor;
if(length(gl_PointCoord-vec2(0.5)) > 0.5)
discard;
}
The problem is, the rounding is applied to every type of primitive drawn in the context, including triangle strips. Is there a way of adding an if statement to limit the rounding to only GL_POINTS?
I think you should just use a new shader for other primitives.
Two minor comments:
Did you consider using a small texture (containing a circle) instead of doing a calculation like this? It might be a bit faster but it obviously depends on the details.
Also try to avoid using the discard keyword. It might have a negative impact on performance. You could for example set the alpha value to 0 for those fragments that you currently discard.
So, I'm trying to make sense of some code before I copy and paste it into my application. In openGL I'm seeing some variables typed as in and out. I see no such thing in the following code snippet. From what I understand, the vertex shader "magically" gets the input for the "in" typed variables via the program, which incidentally can have a fragment and vertex shader attached to it(the program). Heres the code:
<script id="shader-vs" type="x-shader/x-vertex">
attribute vec2 aVertexPosition;
attribute vec2 aPlotPosition;
varying vec2 vPosition;
void main(void) {
gl_Position = vec4(aVertexPosition, 1.0, 1.0);
vPosition = aPlotPosition;
}
</script>
So, my question is, by attaching an appropriate program here, aVertexPosition and aPlotPosition will both be properly initialized and furthermore, vPosition could be used somewhere else in my app, namely the fragment shader?
Let me try and explain how the GPU pipeline I/O works:
Each Vertex has a set of attributes associated with it. Given your example code:
attribute vec2 aVertexPosition;
attribute vec2 aPlotPosition;
You are saying that each vertex has a 2D vertex position and plot position. If you added:
attribute vec3 vNormal;
Then every vertex would also have a normal. You could think of these as vertex "properties".
You must tell the GPU where to fetch the values for each of these attributes.
Each vertex attribute is assigned an attribute array index when the shader is compiled. You must enable each attribute array index that your shader requires
enableVertexAttribArray(int attributeIndex);
Once you've enabled it, you want to bind the attribute array to a vertex buffer.
bindBuffer(ARRAY_BUFFER, buffer);
You now describe how to fetch the attribute with this call:
vertexAttribPointer(int attributeIndex, int count, int type, bool normalized, int stride, int offset);
Given your example code:
vertexAttribPointer(0, 2, FLOAT, false, 16, 0); // vertex position
vertexAttribPointer(1, 2, FLOAT, false, 16, 8); // plot position
16 or the stride is the number of bytes between each vertex. Each vertex consists of 4 floats and each float is 4 bytes wide. The offset is where the attribute starts within a vertex. The vertex position is at the 0th byte of the vertex and plot position is at the 8th.
You can think of these as describing how to index into an array. The Nth vertex:
aVertexPosition.x = BUFFER[offset + N * stride + sizeof(FLOAT) * 0];
aVertexPosition.y = BUFFER[offset + N * stride + sizeof(FLOAT) * 1];
Vertex attributes are fetched automatically for you by the GPU and filled in before your vertex shader function is executed. Yes your vertex shader main is called once for every single vertex you draw.
The output of the vertex shader stage are the 'varying' variables. They are 'varying' because they are interpolated across the surface of the primitive (triangle) between vertices. You write the values out for each vertex but when the triangle is rasterized into fragments, each fragment gets the interpolated value of each varying variable. The fragment shader gets run for every fragment (pixel) that is "covered" by the draw call. If you draw a small triangle that covers a 4x4 patch of pixels then the fragment shader is executed 16 times.
Concisely:
Vertex Shader Inputs: Vertex Attributes & Uniform values (not covered)
Vertex Shader Outputs: Varying Values at each vertex
Fragment Shader Inputs: Varying Values for a given fragment (pixel)
Fragment Shader Outputs: Color & Depth values which are stored in the color and depth buffer.
Vertex Shader is run for every vertex in the draw call.
Fragment shader is run for every "covered" or "lit" fragment (pixel) in the draw call.
In other, newer versions of OpenGL, which have more shader stages than just vertex and fragment, in and out are used instead of attribute and varying.
attribute corresponds to in for a vertex shader.
varying corresponds to out for a vertex shader.
varying corresponds to in for a fragment shader.
(I haven't actually used in and out, so this description may be inaccurate. Please feel free to improve my answer by editing. I don't know how uniforms fit in.)