I am trying to use a websocket to send audio message, what type of message should I change the audio stream into so that I can use a socket to send?
If I directly use a websocket.send(audio), I would get an error "DOMException", should I change it into binary data? and how?
I am totally new to program, so please help!!!
The Websocket API and supports sending Blob and ArrayBuffer binary types.
If your browser supports Mozilla's audio data API then you can read out the data from an audio tag as an ArrayBuffer and send that. However, this API is unlikely to be standardized.
If your browser supports the Web Audio API (proposed by Google) then you can also extract an ArrayBuffer of the data and send that. The Web Audio API will likely see greater adoption. Here is an intro to Web Audio API. This FAQ has an answer that describes how to use the Web Audio API to read from normal audio/video tags.
Updated:
The Aurora.js library mentioned in this blog post looks like it might be helpful in dealing with audio and binary data.
Related
We want to be able to embed an audio file in a Slack chat via the web API.
So far I managed to files.remote.add but that's just an external link.
Then I tried files.upload and share it to a channel, it does embed and play right inside Slack, but for us it's a suboptimal solution, as users can easily download the file, which is against our requirements.
Is there another way to achieve this?
PS: I've read almost all the documentation, there doesn't seem to be an audio or media block type, although I've seen examples that cite a file block type which is neither documented in the API and it's not defined in Bolt. (at least there are no type definitions for it for typescript)
It's not currently possible to embed a media player (audio or video) via the API. We're constantly tuning our APIs, if you have a specific use case in mind, would you mind writing in to feedback#slack.com and we'll get it logged. Thank you!
I am trying to transcribe the audio in an OpenTok session in real-time. The OpenTok API does not not seem to have that feature. Is there any way I can capture the data in some form and push it to another script/ tool that makes the transcription?
The issue is not with transcribing, the issue is in accessing the live audio stream data and using it in real-time.
You can get access to the video/audio stream (MediaStream) with https://tokbox.com/developer/sdks/js/reference/OT.html#getUserMedia in client SDK.
You can manipulate audio using available API from WebAudio spec.
Publish audio from an audio MediaStreamTrack object. For example, you can use the AudioContext object and the Web Audio API to dynamically generate audio. You can then call createMediaStreamDestination().stream.getAudioTracks()[0]on the AudioContext object to get the audio MediaStreamTrack object to use as the audioSource property of the optionsobject you pass into the OT.initPublisher() method. For a basic example, see the Stereo-Audio sample OpenTok-web-samples repo on GitHub.
This above GitHub example is about injecting your audio stream. However, you can also extract/capture your audio before injecting it. See detail here...
https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API.
What is the Dialogflow webhook format? I want to capture the data in an MVC controller and then respond to the webhook whilst maintaining context. How do I do this in C# with the Dialogflow v2 client library? I haven't found any examples to follow in this language and am unsure how to capture the data.
You can read more about Dialogflow webhook format here. Regarding the Dialoglow v2 client library, it is specifically stating that you don't need to use it and can answer in JSON instead using JObject.
If you want to use Protobuf to lower the overhead and save some traffic you will have to use utilities from idiomatic cloud libraries described in the previous link.
The "Dialogflow V2 client library" is for people who are writing clients that send requests to Dialogflow. Dialogflow calls this a "Detect Intent Request", since you're sending a text string (or audio stream) to Dialogflow and it will determine and process the Intent.
If you're interested in building a Fulfillment webhook that is called when an Intent is matched, then you want to look at the Dialogflow Fulfillment API, which is different. There is no supported library for C# yet, but you can parse and return the JSON. The JSON for V1 and V2 are slightly different (mostly in the names of the fields used, but also occasionally in what the values should be).
You can see the fields for the JSON as well as several examples of the request and response formats. There was also an article on Medium recently posted by a Google Developer Relations member discussing using C# for fulfillment in Dialogflow.
i am using tokbox api integration for conference calls between two people.however ,sometimes based on user choice in my web application,i want to have only audio enabled in the tokbox session.
Is there a way to do so.
i am using api calls to generate a session id and javascript workaround to generate a token,unable to use sdk as GAE doesnt support it.
Inputs appreciated.
You can use JavaScript to disable publishing video, which results in audio only. The code is publisher.publishVideo(false). See the docs for Publisher for more information: https://tokbox.com/developer/sdks/js/reference/Publisher.html#publishVideo
Which language Server SDK is not working on GAE to generate tokens? It's a very bad idea to generate your tokens in JS on the client because your API Secret will be sent in the clear to anyone who visits your page.
If I want to create online phone for small office with web client (through Astrisk/Adhearsion) how can I stream IN/OUT audio right in my browser (like GTalk do). Preferably without Java applets and without Flash.
I need to get voice stream from Astriks call, stream it in browser, receive audio from mic and send it back to Astrisk.
And what shoud I choose for streaming audio IN/OUT as backend? XMPP?
You can't. There isn't a standard way (currently) to do what you are looking to do. You need something to help you out. This is often done in the form of a Flash application.
Google actually uses a browser plugin from Vidyo to make this happen effectively.