I know, there are apps out there like steam, Xbox that streams the game video onto network. What I want is the same thing but i need to make my own code for the purpose (open source), so that I may amend it for different applications without any copyright issues. I know Winsock programming and can transmit sample videos (mp4 files) but this thing of capturing the display on one system and producing it on some remote system is different and much complicated (i suppose). Plus, the streaming needs to be live with minimum delays. I have access to gigabit ethernet to address slow fps issues but first i need to transmit recieve a sample buffer
If someone please guide me on how to go about it
I think gstreamer is the closest thing to what you want. Look here and here for more info.
Also, look at ffmpeg.
The solution I found is OBS studio. It is open source which allows me to edit the source code as to fulfill my needs. However, I had to implement a nginx-rtmp server on the client end to receive the video. I then used vlc media player to stream the video received by the rtmp server. I have Gtx 960m installed so I implemented harware encoding which allowed me to stream 720p #60 fps over 100 Mbps ethernet cable. The results were drastic when I used software encoder (instead of hardware) before streaming.
We have an iOS app that we are currently rebuilding for Android. The app relies on being able to scrub video with frame accuracy. We have 3D animations that are rendered out as single frames; we build subsets of frames into lots of small (1-2 second) videos; and the app provides the ability to scrub those videos and see each individual frame.
The MP4 videos we initially created work fine on iOS. When we tried to get them working on Android (using the MediaPlayer class), we entered a world of pain! What we need to do is find a video format that will play and allow frame-accurate scrubbing across all Android devices, using MediaPlayer.seekTo(). Initially we are targetting Android 3.0 and above, but we probably want to stretch back to 2.3.3 after our initial release. Here's what I've discovered so far:
(A) Android claims that H264 "baseline profile" should be supported everywhere: (URL). However, within that, there are dozens of other settings that may or may not be supported. Is there a more fine-grained list anywhere? Currently we are converting to H264 within an MP4 container.
(B) I haven't yet seen an Android device that will accurately scrub H264 files without inserting keyframes ("intra frames"). iOS will happily take H264 files without keyframes and provide accurate scrubbing. It seems that, to allow accurate scrubbing, we need to insert a keyframe for every frame of the video (the relevant ffmpeg setting is "-g 1"). This significantly increases the file size.
(C) However, inserting a keyframe for every frame results in a video that will not play at all on the Samsung Galaxy Note 3 (Snapdragon chipset I believe). Reducing the keyframes to every second frame or above seems to work (ffmpeg setting "-g 2").
To summarise:
MediaPlayer.seekTo() seems very dependent on the video format, and varies across devices. Is this the intention? Is there a base level of behaviour that seekTo() is supposed to provide, regardless of format?
What video format that will allow frame-accurate scrubbing (using MediaPlayer.seekTo()) across all Android devices (at least for 3.0 and above?)
The MediaPlayer isn't likely to give you what you want anytime in the near future. The underlying native players are still relatively immature in some respects, not to mention H.264 in particular has lost favor at Google (and other places) because of patent issues.
You may want to look into using ffmpeg itself on Android. See here for some help with that. Using ffmpeg on Android isn't uncommon, so there's plenty of info on how to do it floating around.
Making the video all i-frames seems a terrible waste. I would avoid that at all costs if I were you.
3gp works fine for me. However, the video seems to restart a couple of frames before.
Is there a particular reason why Firefox does not support playback of MP3 files in <audio> elements, only Ogg format?
Is it a licensing issue?
Are there any plans made for a possible future implementation?
Is it possible to develop an addon to support MP3 playback in <audio> elements?
Update October 2012: Wooohooo! Brendan Eich just announced on his blog that work for MP3 and H264 support in Firefox is underway. You can track the work on BugZilla: Support H.264/AAC/MP3 video/audio playback on desktop Firefox
Update February 2013: After much heavy lifting from Firefox developer Chris Pearce, this patch flips the switch to enable MP3, MP4, H.264, and AAC playback by default in HTML5 <audio> and <video> elements when running on Windows 7 and later. We should see some native web MP3 support in the next stable FF release.
Update April 2013: Woohooo! The latest stable Firefox has experimental support for MP3. To turn it on, type about:config in Firefox, find media.windows-media-foundation.enabled and set it to true. Restart Firefox, and you're all set; go to a site with HTML5 audio (e.g. my radio site) and you'll see Firefox is indeed playing the native MP3 and not resorting to a Flash fallback.
Update May 2013: At last! Firefox 21 was released today, and it includes native HTML5 MP3 support on Windows. I just verified it supports native MP3 audio out-of-the-box, provided your operating system supports it. I tested on Windows 8, but I believe this will automatically work on Windows 7 and Vista.
Update December 2013: Firefox 26 was released today, which gives native MP3 audio support for all versions of Windows going back to Windows XP.
The currently-accepted answer by Ian Devlin is obsolete. The new answer is: while Firefox has historically not supported native MP3 playback for licensing reasons, this will change in the future; we'll soon see a Firefox that plays MP3 natively via the HTML5 <audio> tag.
In March 2012, Mozilla did an about-face on this issue, stating publicly they'll support MP3 and H.264 in their native HTML5 implementation, provided the codec is already available on the end user's system.
In the linked article, Mozilla's director of research, Andreas Gal, makes the following public statements:
“We will support decoding any video/audio format that is supported by
existing decoders present on the system, including H.264 and MP3.
There is really no justification to stop our users from using system
decoders already on the device, so we will not filter any formats.
I don’t think this bug significantly changes our position on
open video. We will continue to promote and support open codecs, but
when and where existing codecs are already installed and licensed on
devices we will make use of them in order to provide people with the
best possible experience.”
This is in contrast to their previous position, which didn't attempt MP3 and H.264 playback even if the operating system supported it.
Bottom line: Firefox will eventually support MP3s in its HTML5 <audio> implementation. As of September 2012, I see no information about when this will happen. It appears to be under development for Firefox on Droid; I speculate we'll see support in desktop Firefox soon afterwards. Edit October 2012: Indeed, this speculation was correct: native MP3 and H264 playback in desktop Firefox is now under development.
Licensing issues: HTML5 video and H.264 – what history tells us and why we’re standing with the web and Mozilla defends Firefox's HTML5 support for only Ogg Theora video (despite their titles, they both also talk about MP3 licensing, albeit briefly).
All you can do is fall back to Flash and play them through that.
UPDATE: Native MP3 (and H264) support is now available on desktop Firefox version 20+
I'm using it to follow podcasts, and the occasional mp4 video, too.
If it doesn't work, there's an hidden option to enable:
about:config → media.windows-media-foundation.enabled → true
It also works with HTML5 YouTube
(that should anyway use webm, but might be more fine tuned for h264)
MP3 usually is a Fraunhofer/Thomson patents problem. They sell their licenses on the slightly shady mp3licensing.com domain.
Regarding patents (update 2016/6) (Wikipedia):
The basic MP3 decoding and encoding technology is patent-free in the
European Union, all patents having expired there. In the United
States, the technology will be substantially patent-free on 31
December 2017 (see below). The majority of MP3 patents expired in the
US between 2007 and 2015.
and
[...] patents expire 20 years after the initial filing date, which can be up
to 12 months later for filings in other countries. As a result,
patents required to implement MP3 expired in most countries by
December 2012, 21 years after the publication of ISO CD 11172.
As to patents finally expire in the US in December 2017:
Except for three patents, the US patents administered by Sisvel had
all expired in 2015, however (the exceptions are: U.S. Patent
5,878,080, expires February 2017, U.S. Patent 5,850,456, expires
February 2017 and U.S. Patent 5,960,037, expires 9. April 2017.
as well as
[...] the MP3 technology will be patent-free in the United States on 30
December 2017 when U.S. Patent 5,703,999, held by the
Fraunhofer-Gesellschaft and administered by Technicolor,
expires.
There is software circumventing those patents, like the LAME MP3 encoder, but they do that by distributing only in source code form.
The LAME developers state that, since their code is only released in
source code form, it should only be considered as an educational
description of an MP3 encoder
Then there are binary distributions of LAME, and, as you can easily see from the domain, they originate from Argentina. This can happen because MP3 patents are deemed invalid in many countries where the very concept of software patent was never legislated upon.
(I'd like to have an exhaustive list of countries, but the situation evolves quite rapidly, and I don't even known what side of the soft-patents divide my country stands in. That's not a level of uncertainty Mozilla wants to cope with)
Then again, Mozilla may have found THEIR way around the patent problem.
It's not perfect. (i.e. it leaves linux in a puddle of mud)
Andreas Gal, Mozilla’s director of research wrote:
(but the discussion revolved around B2G, really read the whole article to form an opinion)
“We will support decoding any video/audio format that is supported by
existing decoders present on the system, including H.264 and MP3.
There is really no justification to stop our users from using system
decoders already on the device, so we will not filter any formats,” he
wrote. “I don’t think this bug significantly changes our position on
open video. We will continue to promote and support open codecs, but
when and where existing codecs are already installed and licensed on
devices we will make use of them in order to provide people with the
best possible experience.”
So, from what I see:
On Windows and Mac (using, already licensed by the OS, dlls/dylibs) Mozilla could end up supporting MP3.
On Linux... I'd like to know. Maybe in selected countries, you'll end up installing some unlicensed libs and get away the way Audacity does.
There's a light at the end of a tunnel, but let's just hope it's not a fast approaching train.
Taken from Wikipedia, for MP3:
MPEG-1 or MPEG-2 Audio Layer 3 (or
III), more commonly referred to as
MP3, is a patented digital audio
encoding format using a form of lossy
data compression.
Taken for Ogg:
Ogg is a free, open standard container
format maintained by the Xiph.Org
Foundation. The creators of the Ogg
format state that it is unrestricted
by software patents and is designed to
provide for efficient streaming and
manipulation of high quality digital
multimedia.
Mozilla doesn't want patent issues, so Ogg was chosen as the better candidate.
It is possible to make such an implementation, so that Firefox can play mp3 in <audio> tag, but this won't be done because of issues I mentioned.
Sometimes politics, and other real world issues, dictate what gets implemented and what doesn't.
For Linux to turn on HTML5 streaming for MP3, MP4, H.264, and AAC, you got to set media.gstreamer.enabled to true into about:config!
Gstreamer is present in most, if not all, Linux distributions!
I believe that the Mozilla developers decided against implementing MP3 support, in order to avoid paying for patent licences from a number of organisations (Technicolor/Thomson Consumer Electronics, the Fraunhofer Institute, Alcatel-Lucent, Sisvel and potentially others, from what I can gather).
Decoding of various audio formats, including MP3 files, can now be done in modern browsers using JavaScript. http://audiocogs.org/codecs/mp3/
If a site used this kind of code (or a Firefox add-on did), Firefox would play MP3 files just fine.
As already said it's a patent problem. There are already plenty of open source MP3 decoders and encoders implementations (among them are LAME and FFmpeg), the problem is that to ship binaries somebody has to pay for the patent license. Here's a quote from LAME wikipedia article confirming this:
Like all MP3 encoders, LAME implements some technology covered by patents owned by the Fraunhofer Society and other entities. The developers of LAME do not themselves license the technology described by these patents. Distributing compiled binaries of LAME, its libraries, or programs that derive from LAME in countries that recognize those patents may be patent infringing.
The LAME developers state that, since their code is only released in source code form, it should only be considered as an educational description of an MP3 encoder, and thus does not infringe any patent by itself when released as source code only. At the same time, they advise users to obtain a patent license for any relevant technologies that LAME may implement before including a compiled version of the encoder in a product.
Mozilla obviously cannot afford to pay patents for any copy of Firefox that gets downloaded.
The two possible solutions are:
use external codecs already preinstalled in the system Firefox is installed in. This solution is the one that has been adopted when running in Windows as mentioned in this other answer. In this case, if the Microsoft MP3 codec is used, Microsoft is the one who paid for the patent license, cost which is part of the Windows license;
wait for MP3 patents to expire. The last one will probably expire on April 16, 2017 which is roughly 3 years and a half from now, but time passes steadily! Firefox 1.0 came out on November 9, 2004, which is more than 9 years ago!
That's why using patent free codecs is, from a user perspective standpoint, more desirable!
2017 UPDATE: patents have expired and open source projects are now starting to pick up MP3 support, e.g. Fedora. Let's hope Firefox does the same soon!
It's now 2020 and I got this issue on Firefox 83.0 on Windows 10. At first I thought my issue was with my profile being too old, from the Firefox 2.x era maybe, and that I got some leftovers such as the mentioned in other answers media.windows-media-foundation.enabled. Unfortunately that wasn't the case.
After searching for a bit in about:config for media decoding, playback and MP3 codec configurations I've found the culprit: media.ffvpx.mp3.enabled. It was set to true and that was preventing a podcast from playing, with Firefox complaining about not finding any compatible codecs to play an audio/mp3 file. Just toggled it false and refresh the podcast page and it started working fine.
We currently have a system with live video encoded to an MPEG-TS multicast stream, being received by televisions with STBs. In addition to televisions we'd like to embed the video in our Windows application.
I know that VLC will receive the stream, but would prefer both a solution that I can embed in an existing application without playing window moving games, and one without licensing problem. I realize that likely means that I'm not looking at a free solution, that's fine, within reason.
Anyone know of a good product for this? Either something easy to use, or a plug-in for WMP.
You'll need to develop a simple DirectShow filter that listens on a given port and just passes down every packet it had received.
I don't have a sample handy, but it's really simple, several hundreds lines of code.
Then you just connect this filter to an MPEG2 Demultiplexer capable of decoding transport stream.
NVidia and Elecard come to mind first, though the former one does not connect under debugger.
Then you connect the demultiplexer to the decoder and finally to the renderer.
The demultiplexers and decoders handle the live stream issues well, you just capture the UDP packets and send down to them.
Due to licensing issues, MPEG2 decoders cannot be free (ffmpeg and VLC violate the license), so you'll have to buy the decoder.
Visit http://elecard.com, they have a nice range of MPEG2 products.
Expanding on Quassnoi's answer...
You might check out the Haali Media Splitter to act as a "MPEG2 Demultiplexer." This is a filter that just pulls the compressed video and sound out of the transport stream, so I'm guessing it doesn't have any licensing issues. Most PCs with a DVD player on them already have a licensed DirectShow MPEG2 decoder, so you can probably just use one that's already installed (or purchase a license from a place like elecard if you really want to be safe).
As you are developing your DirectShow application, you might find Monogram GraphStudio to be a helpful tool in designing the filter chains.