I have some questions on choosing the right architectural decision to solve my problem.
I am planning to create an app, which takes the
input from a client's (a browser) webcam,
sending the input to the server (whether frame by frame, or just live stream video)
getting each frame from the server (into a image)
convert the image using some technology (let's say like a tiktok filter)
returning the image back to the client in real time.
Except for the phase 4 which the technology can only be applied on an image,
Everything else can be changed.
I'm targeting 30fps (or at least 20) with 1080p quality.
The language or framework is completely agnostic as I do not have any preference. Right now, I am thinking of using React with Node, but I'm opened to other options as well. (eg. Python maybe. Language doesn't matter)
If anyone have some prior experiences, can you teach me the best way?
I've tried to create the image blob from client and send it to the server using socket.io but it seemed too slow to use when targeted at 30fps on 1080p image.
I'm currently looking at WebRTC with fluent-ffmpeg, but not sure if it's the right way.
Any kind of help will be appreciated.
Related
I am going to start off this question with; Yes. I have to use Visual Basic 6.0 to program this. It's out of my control which programming language is being used, and the one I have been told to use is VB6.
I am trying to create a program that can fetch the data from a RTSP Stream and save a single shot of the video feed to a bmp file. I have been looking on Google, and on Overflow, but I haven't been able to find a solution on how to go about accomplishing this.
One of the things I am worried about is compatibility issues. For example, one of my searches early on led me to EmguCV, but I can't get that working for VB6, and honestly I never really expected to get it working.
So are there any good libraries, or built in features for VB6 that can help me accomplish what I am trying to do? I am kind of just hitting my head against a brick wall here.
Try using Windows Media Control for the heavy lifting (streaming, video decoding, etc).
Just have to figure out how to pause first frame of the stream and capture it.
Ive noticed recently that two of my wireless providers (freedompop which uses sprint, and net10 which uses at&t) seem to be using some sort of content proxy which is resizing my website images and making them look quite poor. As a test i connected to both of these wireless services and loaded a page through their connection, and then again through a vpn. Both times through the cell services themselves an example image downloaded and was only 9kb, however the actual jpeg is 30kb. The image looks poor and grainy after being recompressed. Over vpn, i got the full size image so this seems to be pointing to the cell carrier using some kind of proxy to reduce their bandwidth.
Has anyone else noticed this recently and is there anything we can do to keep our images looking good, or instruct the proxy not to resize?
Did a little more research and did a few different tests. What I found is that the proxies the carriers are using (at least my two) are only capable of downsizing jpegs. I tried png files through both of my services, and neither one got resized. So if you want to preserve the quality of your images, even for people on a cellular connection it seems that using PNG's might be a start, at least for now until the proxies are capable of downsizing PNG's too.
I need to access web camera using Java. This is what I want to do
Access web cam
Now the user can see web cam working because his face is visible on screen
(have heard some libs are there which doesn't show the video output of webcam)
when user click save button, take a snapshot and save it
I have tried number of ways to do this, from a long time.
JMF - Now it is dead
FMJ - Now it is dead too
VLCJ - too much because I am not creating a music/video player and it expect VLC to be installed
Xuggler - too much and hard work
JMyron - didn't work
JavaFX - I thought it could do it, but seems like it can't
I am even satisfied if the library is just ONLY doing the above mentioned, because that's enough for me. But I expect it to be simple too. Really great if it is not using DLLs, because it is not platform independent if it does. Really appreciate if it can DETECT the camera, without manually passing the camera name and other info as have do in VLCJ (because there might be thousands of camera brands, so I can't create a list of thousand elements in it). And, I am creating a desktop application, not web app.
If you know a library like this, please be kind enough to let me know. Other libraries (which might not suit to all of my requirements, but suits to the basic requirement) also welcome. Please help
I think the project you are looking for is: https://github.com/sarxos/webcam-capture (I'm the author)
There is an example working exactly as you've described - after it's run, the window appear where, after you press "Start" button, you can see live image from webcam device and save it to file after you click on "Snapshot" (source code available, please note that FPS counter in the corner can be disabled):
The project is portable (WinXP, Win7, Win8, Linux, Mac, Raspberry Pi) and does not require any additional software to be installed on the PC.
API is really nice and easy to learn. Example how to capture single image and save it to PNG file:
Webcam webcam = Webcam.getDefault();
webcam.open();
ImageIO.write(webcam.getImage(), "PNG", new File("test.png"));
We are creating animations using HTML5 + Javascript. Now, we'd like to convert these animations to video files (MPEG4, or other, doesn't matter) so that browserly challenged people could also see the animations. Animations contain an audio track.
We are looking for solutions where HTML page would be rendered and recorded on the server side. I know there exist tools for rendering web page thumbnails etc. static images already. However, in our use case we should output a video file.
What options are there? Is headless X server + Firefox a way to go? Preferably we'd be running this in a cloud (Amazon EC2).
Also if there something special we'd need to think when playing animation in non-real-time speeds, I'd like to hear - e.g. syncing HTML5 audio with animation.
Just to note that we have solved this problem by
Running Firefox on a headless server. A server with a decent graphics card.
Having a Selenium Python control script to start/stop rendering
A custom rendering loop which will use Firefox's XPCom API to feed <canvas> raw pixels directly in the encoding pipeline
A custom Javascript rendering loop where clock does not come from a real clock, but it slices frames to the renderer on a stable framerate which is not real-time
Quite a complex system, so doesn't fit into one Answer box :(
One thing that might work, depending on your animation, would be to essentially create an animated gif. With enough still images of your animation strung together, you should be able to turn those into a video with the right software. This is a rather brute-force solution, but if you can advance through your animation in a "frame-by-frame" fashion, it might work.
Sounds interesting. I've done a similar thing with the QtWebKit library that requires a headless X server to run. What I was doing was converting web pages to PDF on a large scale. It might make sense to search for some webkit2pdf tools to see an example that you can build off of.
I would take a look at the library and combine it with another recording solution.
In terms of tooling I've had a pretty decent experience with these Ruby tools:
the headless gem supports video capturing with ffmpeg
capybara will allow you to drive e.g. firefox.
I have NO idea about how they deal with audio though
I would like to know the file formats that IP based surveillance cameras produces. I would then need to build or use available codec/ source code to convert to a format Flash player 10 can support. These are formats other than the usual .FLV for my website which is run on JBoss with Flex 3. I would like to support as many file formats as possible.
I do not want to introduce a streaming server (FMS or RED5 Open source) because of various reasons.
Have anyone any idea about this? Any help would be amazing to get because I have not done anything like this before.
Thanks in advance,
Ranjith
There's no reason to expect these to be standard or consistent; they can do anything they want as long as the camera and the base station agree.
You would have to acquire the target surveillance system and read its documentation, or sniff its network traffic and see what it does. If the first brand seems too difficult, move on to another brand.