How to stream video(and if it possible audio too) from webcam using Gstreamer? I already tried to stream video from source, but I can't stream video from webcam on Windows. How I can do this?
Client:
VIDEO_CAPS="application/x-rtp,media=(string)video,clock-rate=(int)90000,encoding-name=(string)H263-1998"
DEST=localhost
VIDEO_DEC="rtph263pdepay ! avdec_h263"
VIDEO_SINK="videoconvert ! autovideosink"
LATENCY=100
gst-launch -v gstrtpbin name=rtpbin latency=$LATENCY \
udpsrc caps=$VIDEO_CAPS port=5000 ! rtpbin.recv_rtp_sink_0 \
rtpbin. ! $VIDEO_DEC ! $VIDEO_SINK \
udpsrc port=5001 ! rtpbin.recv_rtcp_sink_0 \
rtpbin.send_rtcp_src_0 ! udpsink host=$DEST port=5005 sync=false async=false
Server:
DEST=127.0.0.1
VOFFSET=0
AOFFSET=0
VELEM="ksvideosrc is-live=1"
VCAPS="video/x-raw,width=352,height=288,framerate=15/1"
VSOURCE="$VELEM ! $VCAPS"
VENC="avenc_h263p ! rtph263ppay"
VRTPSINK="udpsink port=5000 host=$DEST ts-offset=$VOFFSET name=vrtpsink"
VRTCPSINK="udpsink port=5001 host=$DEST sync=false async=false name=vrtcpsink"
VRTCPSRC="udpsrc port=5005 name=vrtpsrc"
gst-launch gstrtpbin name=rtpbin
$VSOURCE ! $VENC ! rtpbin.send_rtp_sink_2
rtpbin.send_rtp_src_2 ! $VRTPSINK
rtpbin.send_rtcp_src_2 ! $VRTCPSINK
$VRTCPSRC ! rtpbin.recv_rtcp_sink_2
You will have to use GStreamer 1.3.90 or newer and the ksvideosrc element that is available only since that version.
And then you can stream it just like any other input... the details depend on what codecs, container format, streaming protocol and network protocol you want to use. The same goes for audio, that works basically exactly the same as video.
http://cgit.freedesktop.org/gstreamer/gst-plugins-good/tree/tests/examples/rtp
here you can find some examples that use RTP for streaming. Server side and client side examples, audio-only, video-only or both. And also streaming from real audio/video capture sources (for Linux though, but on Windows it works exactly the same... just with the Windows specific elements for that).
Related
I have five webcams I want to live to stream their content to m3u8(HLS stream), so I can use an HTML web player to play that file.
My current setup:
I have five systems each has a webcam connected to it, so I am using RTSP to stream data from the system to AWS.
./ffmpeg -f avfoundation -s 640x480 -r 30 -i "0" -f rtsp rtsp://awsurl.com:10000/cam1
./ffmpeg -f avfoundation -s 640x480 -r 30 -i "0" -f rtsp rtsp://awsurl.com:10000/cam2
....
./ffmpeg -f avfoundation -s 640x480 -r 30 -i "0" -f rtsp rtsp://awsurl.com:10000/cam5
On the cloud, I want to set up a server. I Googled and learned about GStreamer, with which I can set up an RTSP server. The command below has an error. (I can't figure out how to set up one server for multiple webcam streams)
gst-launch-1.0 udpsrc port=10000 ! rtph264depay ! h264parse ! video/x-h264,stream-format=avc ! \
mpegtsmux ! hlssink target-duration=2 location="output_%05d.ts"\
playlist-root=http://localhost:8080/hls/stream/ playlists-max=3
I question how I can set up the RTSP to differentiate between multiple webcam streams using one server (or do I have to create a server for each webcam stream)?
This might not be a canonical answer, as there are no details about the camera streams, the OS and your programming language, but you may try the following:
1. Install prerequisites
You would need gstrtspserver library (and may be gstreamer dev packages as well if you want to try from C++).
Assuming a Linux Ubuntu host, you would use:
sudo apt-get install libgstrtspserver-1.0 libgstreamer1.0-dev
2. Get information about the received streams
You may use various tools for that, with gstreamer you may use:
gst-discoverer-1.0 rtsp://awsurl.com:10000/cam1
For example, if you see:
Topology:
unknown: application/x-rtp
video: H.264 (Constrained Baseline Profile)
Then it is H264 encoded video sent by RTP so RTPH264.
You would get more details adding verbose flag (-v).
If you want your RTSP server to stream with H264 encoding and the incoming stream is also H264, then you would just forward.
If the received stream has a different encoding than what you want to encode, then you would have to decode video and re-encode it.
3. Run the server:
This python script would run a RTSP server, streaming 2 cams with H264 encoding (expanding to 5 should be straight forward).
Assuming here that the first cam is H264 encoded, it is just forwarding. For the second camera, the stream is decoded and re-encoded into H264 video.
In latter case, it is difficult to give a canonical answer, because the decoder and encoder plugins would depend on your platform. Some also use special memory space (NVMM for Nvidia, 3d11 for Windows, ...), in such case you may have to copy to system memory for encoding with x264enc, or better use an other encoder using same memory space as input.
import gi
gi.require_version('Gst','1.0')
gi.require_version('GstVideo','1.0')
gi.require_version('GstRtspServer','1.0')
from gi.repository import GObject, GLib, Gst, GstVideo, GstRtspServer
Gst.init(None)
mainloop = GLib.MainLoop()
server = GstRtspServer.RTSPServer()
mounts = server.get_mount_points()
factory1 = GstRtspServer.RTSPMediaFactory()
factory1.set_launch('( rtspsrc location=rtsp://awsurl.com:10000/cam1 latency=500 ! rtph264depay ! h264parse ! rtph264pay name=pay0 pt=96 )')
mounts.add_factory("/cam1", factory1)
factory2 = GstRtspServer.RTSPMediaFactory()
factory2.set_launch('( uridecodebin uri=rtsp://awsurl.com:10000/cam2 source::latency=500 ! queue ! x264enc key-int-max=15 insert-vui=1 ! h264parse ! rtph264pay name=pay0 pt=96 )')
mounts.add_factory("/cam2", factory2)
server.attach(None)
print ("stream ready at rtsp://127.0.0.1:8554/{cam1,cam2,...}")
mainloop.run()
If you want using C++ instead of python, you would checkout sample test-launch for your gstreamer version (you can get it with gst-launch-1.0 --version) that is similar to this script and adapt.
4. Test
Note that it may take a few seconds to start before displaying.
gst-play-1.0 rtsp://[Your AWS IP]:8554/cam1
gst-play-1.0 rtsp://[Your AWS IP]:8554/cam2
I have no experience with AWS, be sure that no firewall blocks UDP/8554.
rtsp-simple-server might be a good choice for presenting and broadcasting live streams through various format/protocols such as HLS over HTTP.
Even on meager and old configuration, it still has provided me a decent latency.
If you look for reduced latency, you might be surprised with cam2ip. Unfortunatly this isn't HLS, it's actually mjpeg, and thus without sound, but with far better latency.
I have the following pipelines
sender
gst-launch-1.0 videotestsrc ! "video/x-raw,width=1280, height=720,framerate=30/1" ! shmsink socket-path=/tmp/stream sync=true wait-for-connection=false shm-size=100000000
receiver
gst-launch-1.0 shmsrc socket-path=/tmp/stream is-live=1 ! video/x-raw,width=1280,height=720,framerate=30/1,format=BGR ! videoconvert ! queue ! autovideosink
Is there a way i can include the caps data in the shared memory instead of specifying that data on the receiver side?
what i want is something like
gst-launch-1.0 shmsrc socket-path=/tmp/stream is-live=1 ! video/x-raw ! videoconvert ! queue ! autovideosink
This currently shows
ERROR: from element /GstPipeline:pipeline0/GstCapsFilter:capsfilter0: Filter caps do not completely specify the output format
I can modify the sender as much as possible but want to keep the receiver more general to my needs.
The above is possible with encoded video like h264 but not raw video.
I am designing a video streaming application where host (sender) is on Ubuntu and client (reciever) is on Windows (msvc2015 compiler). My main focus is sending and showing the most recent frame as fast as possible on screen. Both computers have QT 5.15.2
I am utilizing GStreamer with JPEG encryption for sending the frames with the below pipeline. (A little bit modified version of the pipeline from QMediaPlayer documentation https://doc.qt.io/qt-5/qmediaplayer.html)
gst-pipeline: appsrc ! video/x-raw, format=BGRx, framerate=0/1 !
videoconvert ! video/xraw, format=I420 ! jpegenc ! rtpjpegpay ! udpsink
host=127.0.0.1 port=5000
I can decode and show this stream with QML MediaPlayer on Linux and the performance is quite good.
MediaPlayer {
id: mediaPlayer
source: "gst-pipeline: udpsrc port=5000 caps = \"application/x-rtp, media=video, clock-rate=90000, encoding-name=JPEG, payload=26\" ! rtpjpegdepay ! jpegdec ! videoconvert ! qtvideosink sync=false"
autoPlay: true
}
However, my target is to show this stream on Windows machine and I am having difficulties with GStreamer backend. Same url results in the error below. I think it indicates QT is not linked to gstreamer backend and passes the given argument to DirectShow.
DirectShowPlayerService::doSetUrlSource: Unresolved error code
0x800c000d (The specified protocol is unknown.)
Stream plays well with gst-launch command given below, it seems only problem is qt is not able to use gstreamer. In this case, I could not find a way to link GStreamer with QT on windows.
.\gst-launch-1.0.exe udpsrc port=5000 caps = "application/x-rtp,
media=video, clock-rate=90000, encoding-name=JPEG, payload=26" !
rtpjpegdepay ! jpegdec ! videoconvert ! autovideosink sync=false
I can also show the video with K-Lite codec using QML MediaPlayer again but the video has a delay around 1-2 seconds which is not desired for my design. The code snippet is given below:
MediaPlayer {
id: mediaPlayer
source: "rtp://127.0.0.1:5000"
autoPlay: true
}
My question is can I integrate gstreamer to QT somehow? If I can not how can I remove the delay that occurs with K-Lite? I would like to obtain the performance I see with gst-launch command.
I successfully streamed my webcam's image with GStreamer using gst-launch this way :
SERVER
./gst-launch-1.0 -v -m autovideosrc ! video/x-raw,format=BGRA ! videoconvert ! queue ! x264enc pass=qual quantizer=20 tune=zerolatency ! rtph264pay ! udpsink host=XXX.XXX.XXX.XXX port=7480
CLIENT
./gst-launch-1.0 udpsrc port=7480 ! "application/x-rtp, payload=127" ! rtph264depay ! decodebin ! glimagesink
Now I try to reproduce the client side in my app using this pipeline (I don't post the code as I made an Objective-C wrapper around my pipeline and elements) :
udpsrc with caps:"application/x-rtp,media=video,payload=127,encoding-name=H264"
rtph264depay
decodebin
glimagesink (for testing) or a custom appsink (in pull-mode) that converts image to CVPixelBufferRef (tested: it works with videotestsrc / uridecodebin / etc.)
It doesn't work, even if the state messages of the pipeline look quite 'normal'. I have messages in the console concerning SecTaskLoadEntitlements failed error=22 but I have them too when working with the command line.
I'm asking myself what's under gst-launch that I'm missing. I couldn't find any example on the web on udpsrc based pipeline.
My questions are :
Does anybody knows what's actually happening when we launch gst-launch or a way to know what's actually happening?
Are there some examples of working pipelines in code with udpsrc?
EDIT
Here is the image of my pipeline. As you can see, GstDecodeBin element doesn't create a src pad, as it's not receiving - or treating - anything (I set a 'timeout' property to 10 seconds on the udpsrc element, that is thrown). Could it be an OSX sandboxing problem?
Now my pipeline looks like this:
udpsrc
queue
h264 depay
decode bin
video converter
caps filter
appsink / glimagesink
Tested with the method in this question, the app does actually receive something on this port.
Found why it wasn't receiving anything: GstUdpSrc element must be in GST_STATE_NULL to be assigned a port to listen to, or it will listen to the default port (5004) silently.
Everything works fine now.
Setting the environment variable GST_DEBUG to udpsrc:5 helped a lot, for information.
I am having trouble to play the audio from the rtsp server, i have no problem for the video playback, but some error occurred while i tried to play audio,
the following is the command used to play video:
C:\gstreamer\1.0\x86_64\bin>gst-launch-1.0 rtspsrc location=rtsp://192.168.2.116/axis-media/media.amp latency=0 !decodebin ! autovideosink
however, when i change the autovideosink to autoaudiosink, which as in follow:
C:\gstreamer\1.0\x86_64\bin>gst-launch-1.0 rtspsrc location=rtsp://192.168.2.116/axis-media/media.amp latency=0 !decodebin ! autoaudiosink
i get the errors below:
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1: Internal data flow error.
Additional debug info:
gstbasesrc.c(2933): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc1:
streaming task paused, reason not-linked (-1)
I am new to both stackoverflow and Gstreamer, any helps from you would be much appreciated
Thanks to thiagoss's reply , I have my first success on playing both video and audio using the following pipeline:
gst-launch-1.0 rtspsrc location=rtsp://192.168.2.116/axis-media/media.amp latency=0 name=src src. ! decodebin ! videoconvert ! autovideosink src. ! decodebin ! audioconvert ! autoaudiosink
IIRC rtspsrc will output one pad for each stream (video and audio might be separate) so you could be linking your video output to an audiosink.
You can run with -v to see the caps on each pad and verify this. Then you can properly link by using pad names in gst-launch-1.0:
Something like:
gst-launch-1.0 rtspsrc location=rtsp://192.168.2.116/axis-media/media.amp latency=0 name=src src.stream_0 !decodebin ! autovideosink
Check the correct stream_%u number to use for each stream to have it linked correctly.
You can also just be missing a videoconvert before the videosink. I'd also test that.