I have a video-sphere and I want to map a high resolution (5100 x 2550px) video texture onto it.
<video
id="my-video"
preload="auto"
src="./assets/video/video.mp4"
crossorigin="anonymous"
playsinline
webkit-playsinline
></video>
...
<a-videosphere play-on-click id="skyVideo" rotation="-30 0 0" src="#my-video" class="cantap">
</a-videosphere>
When I load the high resolution video, the videosphere is black and I get the following error:
WebGL Error 501
When I downsize the video texture to 2560 x 1280, the videosphere shows the texture as expected with no errors.
Is there a maximum texture resolution in AFrame or three.js that I am not aware of?
The limit isn't established by A-Frame or Three.js, but by WebGL and your graphics card capabilities. Go to https://webglreport.com/ and look at the Textures > Max Texture Size field to see what the device you're using can handle. My laptop maxes out at 8192, my cellphone is lower, but my desktop goes higher. It varies depending on your GPU. My guess is that your machine maxes out at 4096.
You can get this value in Three.js with WebGLRenderer.capabilities.maxTextureSize, as outlined in the docs so you can read this value on different devices and adjust accordingly. Maybe you'd like to point to lower rez videos based on the limits.
Related
I have a training dataset of 640x512 images that I would like to use with a 320x240 camera.
Is it ok to change the aspect ratio and the size of the training images to that of the camera?
Would it be better to upscale the camera frames?
It is better if you keep the aspect ratio of the images because you will be artificially modifying the composition of the objects in the image. What you can do is downscale the image by a factor of 2, so it's 320 x 256, then crop from the center so you have a 320 x 240 image. You can do this by simply removing the first 8 and last 8 columns of the image to get it to 320 x 240. Removing the first 8 and last 8 columns should be safe because it is very unlikely you will see meaningful information within an 8 pixel band on either side of the image.
If you are using a deep learning framework such as Tensorflow or PyTorch, there are pre-processing methods to automatically allow you to crop from the center as well as downscale the image by a factor of 2 for you. You just need to set up a pre-processing pipeline and have these two things in place. You don't have any code established so I can't help you with implementation details, but hopefully what I've said is enough to get you started.
Finally, do not upsample the images. There will be no benefit because you will be using existing information to interpolate to a larger space which is inaccurate. You can scale down, but never scale up. The only situation where this could be useful is if you use superresolution, but that would be for specific cases and it highly depends on what images you use. In general, I do not recommend upscaling. Take your training set and downscale to the resolution of the camera as the images from the camera would be what is used at inference and at that resolution.
I'm detecting object from a live feed of a camera. The backend model used is ssd_mobilenet_v2. If I capture a image and feed it to the model for 10 times; everytime I get the bounding box of same size. But when I feed a live video to the model (without changing anything in the frame), with every frame I'm getting bouding box of different size (variation of 4 to 5 pixels when the image resolution is 640x480). The reason which I think behind is that due to tiny variations in the digital camera sensors, no two frames will be 100% the same — some pixels will most certainly have different intensity values 1. In this link the user have used GaussianBlur to average pixel intensities across an 21 x 21 region. Is this the only way to fix this ? Or there any better way to correct this.
I'm using Raspberry camera to get the video feed.
https://www.pyimagesearch.com/2015/05/25/basic-motion-detection-and-tracking-with-python-and-opencv/
I have a website using three.js with CanvasRenderer. The renderer's size is set depending on the window size. On an iPad, the size is 1024 x 672 pixels. With this size, I get extremely bad performance (~1 FPS). If I reduce the width to 1023 Pixels, I get about 60 FPS.
Is there any specific reasion why the performance should drop significantly when hitting 1024 pixels? I don't have any issues with this resolution in Firefox.
I have heard about the HTML5 canvas element in some browsers losing a massive amount of performance once a certain number of pixels is being rendered. This would appear to be caching issue with the browser itself. Though the question is still unanswered.
Why does drawImage performance differ greatly with larger than 65776 pixel canvas sources
Render the canvas at a lower resolution and use CSS to scale it up to full HD.
I want to display the kinect color frame in wpf with full screen , but when i am trying it ,
I got only very less quality video frames.
How to do this any idea??
The Kinect camera doesn't have great resolutions. Only 640x480 and 1280x960 are supported. Forcing these images to take up the entire screen, especially if you're using a high definition monitor (1920x1080, for example), will cause the image to be stretched, which generally looks awful. It's the same problem you run into if you try to make any image larger; each pixel in the original image has to fill up more pixels in the expanded image, causing the image to look blocky.
Really, the only thing to minimize this is to make sure you're using the Kinect's maximum color stream resolution. You can do that by specifying a ColorImageFormat when you enable the ColorStream. Note that this resolution has a significantly lower number of frames per second than the 640x480 stream (12 FPS vs 30 FPS). However, it should look better in a fullscreen mode than the alternative.
sensor.ColorStream.Enable(ColorImageFormat.RgbResolution1280x960Fps12);
I'm rendering a lot of big images with alpha testing, and I'm hitting the fill rate.
If I change the texture format from RGBA8888 to RGBA4444 will the fill rate improve?
EDIT: Hardware is Iphone 3GS and OpenGL version 1.1
Doing some tests I've found the following:
Loading png as RGBA8888 I get 42fps
Loading png as RGBA4444 I get 44fps
Loading pvrtc2 I get 53 fps (and I had to double the texture size because it was not squared)
It seems that changing from rgba8888 to rgba4444 does not improve framerate. But using pvrtc2 might do.
You don't specify the particular hardware you're asking about, but Apple has this to say about the PowerVR GPUs in iOS devices:
If your application cannot use
compressed textures, consider using a
lower precision pixel format. A
texture in RGB565, RGBA5551, or
RGBA4444 format uses half the memory
of a texture in RGBA8888 format. Use
RGBA8888 only when your application
needs that level of quality.
While this will improve memory usage, I'm not sure that it will have a dramatic effect on fill rate. You might see an improvement from being able to hold more of the texture in cache at once, but I'd listen to Tommy's point about per-pixel operations being the more likely bottleneck here.
Also, when it comes to texture size, you'll get much better image quality at a smaller size by using texture compression (like PVRTC for the PowerVR GPUs) than lowering the precision of the texture pixel format.