display three.js scene across multiple screens - three.js

is it possible to use threejs to display the same scene in different browser windows? or even in Fullscreen across multiple displays?
i know it is possible to render the same scene from multiple views into one window, so the breaking point is if it is possible to pass the render context to another browser window.
i have not found any examples for this use case and a rough first try fails:
https://codepen.io/tp_up/pen/vYBqLEq?page=1
secondWindow = window.open();
secondWindow.document.body.innerHTML = '<canvas id="scene"></canvas>';
renderer = new THREE.WebGLRenderer({canvas :secondWindow.document.getElementById('scene')});
the reason for the question is that we want to know if a web app that renders to multiple screens is feasible or if we have to make a native application for that use case.
so a yes / no answer with a reason would be enough.

This example in the official list of examples shows exactly that use case. You just need to add some options (like a query parameter) to decide which view to draw and base display off the time. This example also shows that use case assuming the monitors are in a circle instead of a grid. And this example shows that use case as well assuming the monitors are of different sizes.
The first and the last example use PerspectiveCamera.setViewOffset to choose which portion of a larger image to draw.
From the docs:
.setViewOffset ( fullWidth : Float, fullHeight : Float, x : Float, y : Float, width : Float, height : Float ) : null
fullWidth — full width of multiview setup
fullHeight — full height of multiview setup
x — horizontal offset of subcamera
y — vertical offset of subcamera
width — width of subcamera
height — height of subcamera
Sets an offset in a larger frustum. **This is useful for multi-window or multi-monitor/multi-machine setups.
For example, if you have 3x2 monitors and each monitor is 1920x1080 and the monitors are in grid like this:
+---+---+---+
| A | B | C |
+---+---+---+
| D | E | F |
+---+---+---+
then for each monitor you would call it like this:
var w = 1920;
var h = 1080;
var fullWidth = w * 3;
var fullHeight = h * 2;
// A
camera.setViewOffset( fullWidth, fullHeight, w * 0, h * 0, w, h );
// B
camera.setViewOffset( fullWidth, fullHeight, w * 1, h * 0, w, h );
// C
camera.setViewOffset( fullWidth, fullHeight, w * 2, h * 0, w, h );
// D
camera.setViewOffset( fullWidth, fullHeight, w * 0, h * 1, w, h );
// E
camera.setViewOffset( fullWidth, fullHeight, w * 1, h * 1, w, h );
// F
camera.setViewOffset( fullWidth, fullHeight, w * 2, h * 1, w, h );
Note there is no reason monitors have to be the same size or in a grid.
As for different browser windows you'll need to come up with a way for them to communicate with each other. If all the animation is based on the time then you just need to keep the time in sync (or use the system clock as in time = Date.now()) to keep things in sync. otherwise you can pass messages across windows using postMessage.
Here's an example. Open multiple windows and they should stay in sync.

Related

How is this ray casting algorithm flawed?

Matrix operations performed on the GPU can be pretty hard to debug because GPU operations don't really allow for console logs.
I've written one designed for a real time 2D rendering engine based on a very simple form of I guess what could be called ray casting and am having trouble figuring out what's wrong with it (it's outputting [0,0,0,255,0,0,0,255,...] instead of populating colors).
this.thread.x is the index of the current unit (color channel) in the matrix being operated on.
scene is a buffer made up of 6-unit clumps, each value containing, in order:
The type of entity, always 1 for "sprite" in this case.
The sprite ID, corresponding the the index in this.constants.textures containing the buffer for the entity's sprite.
X offset, the left edge of the sprite
Y offset, the top edge of the sprite
width of the sprite
height of the sprite
bufferWidth is the width of the render area multiplied by 4 channels.
this.constants.textures is an array containing buffers of each sprite which the sprite IDs from the scene refer to.
Note: For those curious, this is being done with GPU.js, a JavaScript lib that converts a JS func into GLSL code to be run via WebGL.
function(scene, sceneLength, bufferWidth) {
var channel = this.thread.x % 4;
if (channel === 3) {
return 255;
}
var x = this.thread.x % bufferWidth;
var y = Math.floor(this.thread.x / bufferWidth);
for (let i1 = 0; i1 < sceneLength; i1 += 6) {
var id = scene[i1 + 1];
var x1 = scene[i1 + 2];
var y1 = scene[i1 + 3];
var w1 = scene[i1 + 4];
var h1 = scene[i1 + 5];
var r1 = scene[i1 + 6];
var offsetX1 = x1 - x;
if (offsetX1 > 0 && offsetX1 < w1) {
var offsetY1 = y1 - y;
if (offsetY1 > 0 && offsetY1 < h1) {
var c1 = offsetY1 * w1 * 4 + offsetX1 * 4;
var c1R = c1 - (c1 % 4);
var c1A = c1R + 3;
if (this.constants.textures[id][c1A] != 0) {
return this.constants.textures[id][c1];
}
}
}
}
return 0;
}
Explanation for the concept I'm trying to implement:
With a matrix operation, when you want to draw a sprite if you were to perform a pass on the entire render area, you'd be doing far more work than necessary. If you break the rendering area down into chunks and only update the sections involved in the sprite being drawn, that would be a fairly decent way to do it. It would certainly be good enough for real time game rendering. This would be a multi-pass approach, where sprites are rendered one at a time.
Alternatively, for what seems to me to be the most optimal approach possible, instead of that, we can use a single-pass approach that performs a single matrix operation for the entire rendering area, evaluating for each color channel what should be there based on doing a very basic form of collision detection with each sprite in the scene and the relevant pixel in that sprite.
You're calculating your sprite offsets backwards, the calculations should be:
var offsetX1 = x - x1;
and
var offsetY1 = y - y1;
The offsets should increase as x and y increase (assuming the sprite co-ordinates have the same co-ordinate system as the screen co-ordinates), so you shouldn't be subtracting x and y.

Three JS - Scaling texture to fit a (any size) Plane perfectly

In essence, I want to replicate the behaviour of how the CSS, background-size: cover works.
Looking here you can see the image is being scaled keeping its aspect ratio, but it's not really working correctly, as the image does not fill the Plane, leaving margins either side - https://next.plnkr.co/edit/8650f9Ji6qWffTqE?preview
Code snippet (Lines 170 - 175) -
var geometryAspectRatio = 5/3;
var imageAspectRatio = 3264/2448;
textTile.wrapT = THREE.RepeatWrapping;
textTile.repeat.x = geometryAspectRatio / imageAspectRatio;
textTile.offset.x = 0.5 * ( 1 - textTile.repeat.x );
What I want to happen is for it so scale-up and then reposition its self in the centre (much how cover works).
var repeatX, repeatY;
repeatX = w * this.textureHeight / (h * this.textureWidth);
if (repeatX > 1) {
//fill the width and adjust the height accordingly
repeatX = 1;
repeatY = h * this.textureWidth / (w * this.textureHeight);
mat.map.repeat.set(repeatX, repeatY);
mat.map.offset.y = (repeatY - 1) / 2 * -1;
} else {
//fill the height and adjust the width accordingly
repeatX = w * this.textureHeight / (h * this.textureWidth);
repeatY = 1;
mat.map.repeat.set(repeatX, repeatY);
mat.map.offset.x = (repeatX - 1) / 2 * -1;
}
Updated https://next.plnkr.co/edit/LUk37xLG2yvv6hgg?preview
For anyone confused by this as I was, the missing piece for me is that .repeat.x and .repeat.y properties of any texture can be values less than one, and scales up the image when is under 1 as the inverse of the scale. Think about it, when it's scale 2, in a way it repeats .5 times because you only see half of the image.
So...
Something not supported by textures in THREE.js and common in some libraries, would be
.scaleX = 2; (not supported in THREE.js textures as of v1.30.1)
And the THREE.js texture equivalent would be
texture.repeat.x = .5;
To convert scale to "repeat", simply do the inverse of the scale
var desiredScaleX = 3;
var desiredRepeatX = 1 / desiredScaleX;
The repeat for scale 3 comes out to (1/3) = .3333; In other words a 3x image would be cropped and only show 1/3 of the image, so it repeats .3333 times.
As for scaling to fit to cover, generally choosing the larger scale of the two will do the trick, something like:
var fitScaleX = targetWidth / actualWidth;
var fitScaleY = targetHeight / actualHeight;
var fitCoverScale = Math.max(fitScaleX,fitScaleY);
var repeatX = 1 / fitCoverScale;
var repeatY = 1 / fitCoverScale;

MATLAB coding problem

Hey guys, I got this error message when I tried to trigger the function below. Can anybody help me out? Thanks!
>> changeYuv('tilt.yuv',352,288,1:40,40);
??? Index exceeds matrix dimensions.
Error in ==> changeYuv at 32
j=histogram(imgYuv(:,:,1,k+1));
>> [x,y,z,a]=size(imgYuv)
x =
288
y =
352
z =
3
a =
40
The source code:
function [imgYuv, S]= changeYuv(fileName, width, height, idxFrame, nFrames)
% load RGB movie [0, 255] from YUV 4:2:0 file
fileId = fopen(fileName, 'r');
subSampleMat = [1, 1; 1, 1];
nrFrame = length(idxFrame);
for f = 1 : 1 : nrFrame
% search fileId position
sizeFrame = 1.5 * width * height;
fseek(fileId, (idxFrame(f) - 1) * sizeFrame, 'bof');
% read Y component
buf = fread(fileId, width * height, 'uchar');
imgYuv(:, :, 1,f) = reshape(buf, width, height).';
% read U component
buf = fread(fileId, width / 2 * height / 2, 'uchar');
imgYuv(:, :, 2,f) = kron(reshape(buf, width / 2, height / 2).', subSampleMat); % reshape and upsample
% read V component
buf = fread(fileId, width / 2 * height / 2, 'uchar');
imgYuv(:, :, 3,f) = kron(reshape(buf, width / 2, height / 2).', subSampleMat); % reshape and upsample
%histogram difference of Y component
for k=1:(nFrames-1)
h=histogram(imgYuv(:,:,1,k));
j=histogram(imgYuv(:,:,1,k+1));
X=abs(h-j)/256;
S(k)=sum(X);
end
end
fclose(fileId);
On every iteration of the outer loop, you appear to be growing imgYuv by one in the 4th dimension, starting from empty. But your inner loop always loops from 1 to nFrames-1. Therefore, it would seem to me like you're trying to access beyond the extent of imgYuv.
On an unrelated note, growing an array like this is typically very slow. You're much better off initialising imgYuv before you start, i.e. imgYuv = zeros([height,width,3,nFrames]).

How do I find a dimension of aspect ratio 4:3 which fits within a predetermined size?

The problem here is I have a display window of size x by y, and I need to display an image inside the window without any scrolling, and to maintain the aspect ratio of 4:3. I have the following snippet of code:
// Lock the current height, calculate new width of the canvas and scale the viewport.
// get width of the movie canvas
qreal width = canvas_->width();
// Find the height of the video
qreal height = (width/4.0) * 3;
// find original width and height for video to calculate scaling factor
qreal videoWidth = movieData_->GetWidth();
qreal videoHeight = movieData_->GetHeight();
// calculate scaling factor
qreal scaleFactorWidth = width/videoWidth;
qreal scaleFactorHeight = height/videoHeight;
Of course, by using either the height, or the width as the 'anchor', one way or other the new image will cause scrolling (assuming the original image is larger than the window in the first place). How do I find a dimension of aspect ratio 4:3 which fits within a predetermined size?
Edit
I would need to pass in a scale factor for both x and y to do the scaling
canvas_->scale(scaleFactorWidth, scaleFactorHeight);
Just take the minimum of the both calculated values:
scale = min(scaleFactorWidth, scaleFactorHeight)
or (if you want outer-fit)
scale = max(scaleFactorWidth, scaleFactorHeight)
struct dimensions resize_to_fit_in(struct dimensions a, struct dimensions b) {
double wf, hf, f;
struct dimensions out;
wf = (double) b.w / a.w;
hf = (double) b.h / a.h;
if (wf > hf)
f = hf;
else
f = wf;
out.w = a.w * f;
out.h = a.h * f;
return out;
}
An here is a C version where the returned dimension will be a dimension 'a' fitted in dimension 'b' without loosing aspect ratio.
Find the largest of the two values width, w and height h. Say your maximum width x height is 100 x 80. Note that 100/80 = 1.25.
Case 1: If w/h > 1.25, then divide w by 100 to get the ratio of your original size to the new size. Then multiply h by that ratio.
Case 2: Otherwise, then divide h by 80 to get the ratio of your original size to the new size. Then multiply w by that ratio.
Here's an ActionScript version of what you ask (resize while maintaining aspect ratio)... shouldn't be too hard to port to whatever:
private static function resizeTo(dispObj:DisplayObject, width:Number, height:Number) : void
{
var ar:Number = width / height;
var dispObjAr:Number = dispObj.width/dispObj.height;
if (ar < dispObjAr)
{
dispObj.width = width;
dispObj.height = width / dispObjAr;
}
else
{
dispObj.height = height;
dispObj.width = height * dispObjAr;
}
return;
}
EDIT: In order to maintain 4:3 the source images would need to be 4:3

Programmatically Lighten a Color

Motivation
I'd like to find a way to take an arbitrary color and lighten it a few shades, so that I can programatically create a nice gradient from the one color to a lighter version. The gradient will be used as a background in a UI.
Possibility 1
Obviously I can just split out the RGB values and increase them individually by a certain amount. Is this actually what I want?
Possibility 2
My second thought was to convert the RGB to HSV/HSB/HSL (Hue, Saturation, Value/Brightness/Lightness), increase the brightness a bit, decrease the saturation a bit, and then convert it back to RGB. Will this have the desired effect in general?
As Wedge said, you want to multiply to make things brighter, but that only works until one of the colors becomes saturated (i.e. hits 255 or greater). At that point, you can just clamp the values to 255, but you'll be subtly changing the hue as you get lighter. To keep the hue, you want to maintain the ratio of (middle-lowest)/(highest-lowest).
Here are two functions in Python. The first implements the naive approach which just clamps the RGB values to 255 if they go over. The second redistributes the excess values to keep the hue intact.
def clamp_rgb(r, g, b):
return min(255, int(r)), min(255, int(g)), min(255, int(b))
def redistribute_rgb(r, g, b):
threshold = 255.999
m = max(r, g, b)
if m <= threshold:
return int(r), int(g), int(b)
total = r + g + b
if total >= 3 * threshold:
return int(threshold), int(threshold), int(threshold)
x = (3 * threshold - total) / (3 * m - total)
gray = threshold - x * m
return int(gray + x * r), int(gray + x * g), int(gray + x * b)
I created a gradient starting with the RGB value (224,128,0) and multiplying it by 1.0, 1.1, 1.2, etc. up to 2.0. The upper half is the result using clamp_rgb and the bottom half is the result with redistribute_rgb. I think it's easy to see that redistributing the overflows gives a much better result, without having to leave the RGB color space.
For comparison, here's the same gradient in the HLS and HSV color spaces, as implemented by Python's colorsys module. Only the L component was modified, and clamping was performed on the resulting RGB values. The results are similar, but require color space conversions for every pixel.
I would go for the second option. Generally speaking the RGB space is not really good for doing color manipulation (creating transition from one color to an other, lightening / darkening a color, etc). Below are two sites I've found with a quick search to convert from/to RGB to/from HSL:
from the "Fundamentals of Computer Graphics"
some sourcecode in C# - should be easy to adapt to other programming languages.
In C#:
public static Color Lighten(Color inColor, double inAmount)
{
return Color.FromArgb(
inColor.A,
(int) Math.Min(255, inColor.R + 255 * inAmount),
(int) Math.Min(255, inColor.G + 255 * inAmount),
(int) Math.Min(255, inColor.B + 255 * inAmount) );
}
I've used this all over the place.
ControlPaint class in System.Windows.Forms namespace has static methods Light and Dark:
public static Color Dark(Color baseColor, float percOfDarkDark);
These methods use private implementation of HLSColor. I wish this struct was public and in System.Drawing.
Alternatively, you can use GetHue, GetSaturation, GetBrightness on Color struct to get HSB components. Unfortunately, I didn't find the reverse conversion.
Convert it to RGB and linearly interpolate between the original color and the target color (often white). So, if you want 16 shades between two colors, you do:
for(i = 0; i < 16; i++)
{
colors[i].R = start.R + (i * (end.R - start.R)) / 15;
colors[i].G = start.G + (i * (end.G - start.G)) / 15;
colors[i].B = start.B + (i * (end.B - start.B)) / 15;
}
In order to get a lighter or a darker version of a given color you should modify its brightness. You can do this easily even without converting your color to HSL or HSB color. For example to make a color lighter you can use the following code:
float correctionFactor = 0.5f;
float red = (255 - color.R) * correctionFactor + color.R;
float green = (255 - color.G) * correctionFactor + color.G;
float blue = (255 - color.B) * correctionFactor + color.B;
Color lighterColor = Color.FromArgb(color.A, (int)red, (int)green, (int)blue);
If you need more details, read the full story on my blog.
Converting to HS(LVB), increasing the brightness and then converting back to RGB is the only way to reliably lighten the colour without effecting the hue and saturation values (ie to only lighten the colour without changing it in any other way).
A very similar question, with useful answers, was asked previously:
How do I determine darker or lighter color variant of a given color?
Short answer: multiply the RGB values by a constant if you just need "good enough", translate to HSV if you require accuracy.
I used Andrew's answer and Mark's answer to make this (as of 1/2013 no range input for ff).
function calcLightness(l, r, g, b) {
var tmp_r = r;
var tmp_g = g;
var tmp_b = b;
tmp_r = (255 - r) * l + r;
tmp_g = (255 - g) * l + g;
tmp_b = (255 - b) * l + b;
if (tmp_r > 255 || tmp_g > 255 || tmp_b > 255)
return { r: r, g: g, b: b };
else
return { r:parseInt(tmp_r), g:parseInt(tmp_g), b:parseInt(tmp_b) }
}
I've done this both ways -- you get much better results with Possibility 2.
Any simple algorithm you construct for Possibility 1 will probably work well only for a limited range of starting saturations.
You would want to look into Poss 1 if (1) you can restrict the colors and brightnesses used, and (2) you are performing the calculation a lot in a rendering.
Generating the background for a UI won't need very many shading calculations, so I suggest Poss 2.
-Al.
IF you want to produce a gradient fade-out, I would suggest the following optimization: Rather than doing RGB->HSB->RGB for each individual color you should only calculate the target color. Once you know the target RGB, you can simply calculate the intermediate values in RGB space without having to convert back and forth. Whether you calculate a linear transition of use some sort of curve is up to you.
Method 1: Convert RGB to HSL, adjust HSL, convert back to RGB.
Method 2: Lerp the RGB colour values - http://en.wikipedia.org/wiki/Lerp_(computing)
See my answer to this similar question for a C# implementation of method 2.
Pretend that you alpha blended to white:
oneMinus = 1.0 - amount
r = amount + oneMinus * r
g = amount + oneMinus * g
b = amount + oneMinus * b
where amount is from 0 to 1, with 0 returning the original color and 1 returning white.
You might want to blend with whatever the background color is if you are lightening to display something disabled:
oneMinus = 1.0 - amount
r = amount * dest_r + oneMinus * r
g = amount * dest_g + oneMinus * g
b = amount * dest_b + oneMinus * b
where (dest_r, dest_g, dest_b) is the color being blended to and amount is from 0 to 1, with zero returning (r, g, b) and 1 returning (dest.r, dest.g, dest.b)
I didn't find this question until after it became a related question to my original question.
However, using insight from these great answers. I pieced together a nice two-liner function for this:
Programmatically Lighten or Darken a hex color (or rgb, and blend colors)
Its a version of method 1. But with over saturation taken into account. Like Keith said in his answer above; use Lerp to seemly solve the same problem Mark mentioned, but without redistribution. The results of shadeColor2 should be much closer to doing it the right way with HSL, but without the overhead.
A bit late to the party, but if you use javascript or nodejs, you can use tinycolor library, and manipulate the color the way you want:
tinycolor("red").lighten().desaturate().toHexString() // "#f53d3d"
I would have tried number #1 first, but #2 sounds pretty good. Try doing it yourself and see if you're satisfied with the results, it sounds like it'll take you maybe 10 minutes to whip up a test.
Technically, I don't think either is correct, but I believe you want a variant of option #2. The problem being that taken RGB 990000 and "lightening" it would really just add onto the Red channel (Value, Brightness, Lightness) until you got to FF. After that (solid red), it would be taking down the saturation to go all the way to solid white.
The conversions get annoying, especially since you can't go direct to and from RGB and Lab, but I think you really want to separate the chrominance and luminence values, and just modify the luminence to really achieve what you want.
Here's an example of lightening an RGB colour in Python:
def lighten(hex, amount):
""" Lighten an RGB color by an amount (between 0 and 1),
e.g. lighten('#4290e5', .5) = #C1FFFF
"""
hex = hex.replace('#','')
red = min(255, int(hex[0:2], 16) + 255 * amount)
green = min(255, int(hex[2:4], 16) + 255 * amount)
blue = min(255, int(hex[4:6], 16) + 255 * amount)
return "#%X%X%X" % (int(red), int(green), int(blue))
This is based on Mark Ransom's answer.
Where the clampRGB function tries to maintain the hue, it however miscalculates the scaling to keep the same luminance. This is because the calculation directly uses sRGB values which are not linear.
Here's a Java version that does the same as clampRGB (although with values ranging from 0 to 1) that maintains luminance as well:
private static Color convertToDesiredLuminance(Color input, double desiredLuminance) {
if(desiredLuminance > 1.0) {
return Color.WHITE;
}
if(desiredLuminance < 0.0) {
return Color.BLACK;
}
double ratio = desiredLuminance / luminance(input);
double r = Double.isInfinite(ratio) ? desiredLuminance : toLinear(input.getRed()) * ratio;
double g = Double.isInfinite(ratio) ? desiredLuminance : toLinear(input.getGreen()) * ratio;
double b = Double.isInfinite(ratio) ? desiredLuminance : toLinear(input.getBlue()) * ratio;
if(r > 1.0 || g > 1.0 || b > 1.0) { // anything outside range?
double br = Math.min(r, 1.0); // base values
double bg = Math.min(g, 1.0);
double bb = Math.min(b, 1.0);
double rr = 1.0 - br; // ratios between RGB components to maintain
double rg = 1.0 - bg;
double rb = 1.0 - bb;
double x = (desiredLuminance - luminance(br, bg, bb)) / luminance(rr, rg, rb);
r = 0.0001 * Math.round(10000.0 * (br + rr * x));
g = 0.0001 * Math.round(10000.0 * (bg + rg * x));
b = 0.0001 * Math.round(10000.0 * (bb + rb * x));
}
return Color.color(toGamma(r), toGamma(g), toGamma(b));
}
And supporting functions:
private static double toLinear(double v) { // inverse is #toGamma
return v <= 0.04045 ? v / 12.92 : Math.pow((v + 0.055) / 1.055, 2.4);
}
private static double toGamma(double v) { // inverse is #toLinear
return v <= 0.0031308 ? v * 12.92 : 1.055 * Math.pow(v, 1.0 / 2.4) - 0.055;
}
private static double luminance(Color c) {
return luminance(toLinear(c.getRed()), toLinear(c.getGreen()), toLinear(c.getBlue()));
}
private static double luminance(double r, double g, double b) {
return r * 0.2126 + g * 0.7152 + b * 0.0722;
}

Resources