I try to save post data to local file. But if i do that, in browser i see
The connection was reset
The connection to the server was reset while the page was
loading. The site could be temporarily unavailable or too busy. Try
again in a few
moments. If you are unable to load any pages, check your
computer's network
connection. If your computer or network is protected by a
firewall or proxy, make sure
that Firefox is permitted to access the Web.
There is my code:
<?xml version="1.0"?>
<?xml-stylesheet href="chrome://global/skin/" type="text/css"?>
<window id="yourwindow" xmlns="http://www.mozilla.org/keymaster/gatekeeper/there.is.only.xul">
<script>
<![CDATA[
var MIKI =
{
observe: function(subject, topic, data)
{
if (topic == "http-on-modify-request")
{
var httpChannel = subject.QueryInterface(Components.interfaces.nsIHttpChannel);
if(httpChannel.requestMethod == "POST"){
var uploadChannel = httpChannel.QueryInterface(Components.interfaces.nsIUploadChannel);
var uploadChannelStream = uploadChannel.uploadStream;
uploadChannelStream.QueryInterface(Components.interfaces.nsISeekableStream);
uploadChannelStream.seek(0,0);
var stream = Components.classes['#mozilla.org/scriptableinputstream;1'].createInstance(Components.interfaces.nsIScriptableInputStream);
stream.init(uploadChannelStream);
var data = stream.read(stream.available());
data = data.split(/\r\n\r\n/).slice(1).join('\r\n\r\n');
var buffer = httpChannel.URI.spec + " " + data+ "\n";
if(data != '') fos.write(buffer, buffer.length);
}
}
}
};
var file = Components.classes["#mozilla.org/file/directory_service;1"].getService(Components.interfaces.nsIProperties).get("Desk", Components.interfaces.nsIFile);
file.append("http-request-log.txt");
var fos = Components.classes["#mozilla.org/network/file-output-stream;1"].createInstance(Components.interfaces.nsIFileOutputStream);
fos.init(file, 0x02 | 0x08 | 0x10, -1, 0);
var observerService = Components.classes["#mozilla.org/observer-service;1"].getService(Components.interfaces.nsIObserverService);
observerService.addObserver(MIKI, "http-on-modify-request", false);
]]>
</script>
</window>
What i do wrong?
Since you are reading the data out of the upload stream to write it to the file, you are probably leaving the stream pointer at the end (so when the request happens it looks like the POST body is empty). Try seeking back to the beginning of the stream once you are done reading out the data.
Related
Problem Description
I am a .NET Core developer and I have recently been asked to transcribe mp3 audio files that are approximately 20 minutes long into text. Thus, the file is about 30.5mb. The issue is that speech is sparse in this file, varying anywhere between 2 minutes between a spoken sentence or 4 minutes of length.
I've written a small service based on the google speech documentation that sends 32kb of streaming data to be processed from the file at a time. All was progressing well until I hit this error that I share below as follows:
I have searched via google-fu, google forums, and other sources and I have not encountered documentation on this error. Suffice it to say, I think this is due to the sparsity of spoken words in my file? I am wondering if there is a programmatical centric workaround?
Code
I have used some code that is a slight modification of the google .net sample for 32kb streaming. You can find it here.
public async void Run()
{
var speech = SpeechClient.Create();
var streamingCall = speech.StreamingRecognize();
// Write the initial request with the config.
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
StreamingConfig = new StreamingRecognitionConfig()
{
Config = new RecognitionConfig()
{
Encoding =
RecognitionConfig.Types.AudioEncoding.Flac,
SampleRateHertz = 22050,
LanguageCode = "en",
},
InterimResults = true,
}
});
// Helper Function: Print responses as they arrive.
Task printResponses = Task.Run(async () =>
{
while (await streamingCall.ResponseStream.MoveNext(
default(CancellationToken)))
{
foreach (var result in streamingCall.ResponseStream.Current.Results)
{
//foreach (var alternative in result.Alternatives)
//{
// Console.WriteLine(alternative.Transcript);
//}
if(result.IsFinal)
{
Console.WriteLine(result.Alternatives.ToString());
}
}
}
});
string filePath = "mono_1.flac";
using (FileStream fileStream = new FileStream(filePath, FileMode.Open))
{
//var buffer = new byte[32 * 1024];
var buffer = new byte[64 * 1024]; //Trying 64kb buffer
int bytesRead;
while ((bytesRead = await fileStream.ReadAsync(
buffer, 0, buffer.Length)) > 0)
{
await streamingCall.WriteAsync(
new StreamingRecognizeRequest()
{
AudioContent = Google.Protobuf.ByteString
.CopyFrom(buffer, 0, bytesRead),
});
await Task.Delay(500);
};
}
await streamingCall.WriteCompleteAsync();
await printResponses;
}//End of Run
Attempts
I've increased the stream to 64kb of streaming data to be processed and then I received the following error as can be seen below:
Which, I believe, means the actual api timed out. Which is decidely a step in the wrong direction. Has anybody encountered a problem such as mine with the Google Speech Api when dealing with a audio file with sparse speech? Is there a method in which I can filter the audio down to only spoken words progamatically and then process that? I'm open to suggestions, but my research and attempts have only lead me to further breaking my code.
There is to way for recognize audio in Google Speech API:
normal recognize
long running recognize
Your sample is uses the normal recognize, which has a limit for 15 minutes.
Try to use the long recognize method:
{
var speech = SpeechClient.Create();
var longOperation = speech.LongRunningRecognize( new RecognitionConfig()
{
Encoding = RecognitionConfig.Types.AudioEncoding.Linear16,
SampleRateHertz = 16000,
LanguageCode = "hu",
}, RecognitionAudio.FromFile( filePath ) );
longOperation = longOperation.PollUntilCompleted();
var response = longOperation.Result;
foreach ( var result in response.Results )
{
foreach ( var alternative in result.Alternatives )
{
Console.WriteLine( alternative.Transcript );
}
}
return 0;
}
I hope it helps for you.
I'm sending the contents of this Flex form (Don't ask why) over to node. There is a post paramteter called "photo" which is a base64 encoded image.
Contents of photo get sent over ok. Problem is when I am trying to decode the content and write them to a file.
var fs = require("fs");
fs.writeFile("arghhhh.jpg", new Buffer(request.body.photo, "base64").toString(), function(err) {});
I've tried toString("binary") as well. But it seems node doesnt decode all of the content. It seems it only decodes jpg header info and leaves the rest.
Can anyone please help me with this?
Thanks
Try removing the .toString() entirely and just write the buffer directly.
this is my full solution which would read any base64 image format, decode it and save it in the proper format in the database:
// Save base64 image to disk
try
{
// Decoding base-64 image
// Source: http://stackoverflow.com/questions/20267939/nodejs-write-base64-image-file
function decodeBase64Image(dataString)
{
var matches = dataString.match(/^data:([A-Za-z-+\/]+);base64,(.+)$/);
var response = {};
if (matches.length !== 3)
{
return new Error('Invalid input string');
}
response.type = matches[1];
response.data = new Buffer(matches[2], 'base64');
return response;
}
// Regular expression for image type:
// This regular image extracts the "jpeg" from "image/jpeg"
var imageTypeRegularExpression = /\/(.*?)$/;
// Generate random string
var crypto = require('crypto');
var seed = crypto.randomBytes(20);
var uniqueSHA1String = crypto
.createHash('sha1')
.update(seed)
.digest('hex');
var base64Data = 'data:image/jpeg;base64,/9j/4AAQSkZJRgABAQEAZABkAAD/4Q3zaHR0cDovL25zLmFkb2JlLmN...';
var imageBuffer = decodeBase64Image(base64Data);
var userUploadedFeedMessagesLocation = '../img/upload/feed/';
var uniqueRandomImageName = 'image-' + uniqueSHA1String;
// This variable is actually an array which has 5 values,
// The [1] value is the real image extension
var imageTypeDetected = imageBuffer
.type
.match(imageTypeRegularExpression);
var userUploadedImagePath = userUploadedFeedMessagesLocation +
uniqueRandomImageName +
'.' +
imageTypeDetected[1];
// Save decoded binary image to disk
try
{
require('fs').writeFile(userUploadedImagePath, imageBuffer.data,
function()
{
console.log('DEBUG - feed:message: Saved to disk image attached by user:', userUploadedImagePath);
});
}
catch(error)
{
console.log('ERROR:', error);
}
}
catch(error)
{
console.log('ERROR:', error);
}
In nodejs 8.11.3 new Buffer(string, encoding) is deprecated, instead of this the new way to do that is Buffer.from(string, encoding) always without .toString().
For more details read the documentation in nodejs docs: Buffer
Remove .toString()
Here you decode the base64 to a buffer, which is fine, but then you convert the buffer into a string. This means that it is a string object whose code points are bytes of the buffer.
I am try to make an upload picture webpage in mobile phones, but it always run out of memory and the browser quit once the picture is too big.
Here are my code:
<input type="file" id="testFile" />
<img src="" style="position:relative;" id="uploadingImg" />
function setImage() {
var fileList = this.files;
var imageType = /image/;
for(var i = 0;i < fileList.length;i++) {
document.getElementById('uploadingImg').file = fileList[i];
var reader = new FileReader();
reader.onload = function(e) {
document.getElementById('uploadingImg').src = e.target.result;
}
reader.readAsDataURL(fileList[i]);
}
}
document.getElementById('testFile').addEventListener('change', setImage, false);
Does anyone knows how to preview one picture by using <img> or <canvas> element?
Please don't use "readAsDataURL", because when it comes to document.getElementById('uploadingImg').src = e.target.result;
It will run out of memory. Because the base64 Url occupy too much memory and it exist in the memory with many three or four copies.
How can I use "readAsArrayBuffer" and use "drawImage" by canvas?
or other method?
All is ok if I can preview one picture in local disk without using "readAsDataUrl".
Thanks!
In most browsers you don't need to use FileReader (and consequently readAsDataUrl) for this. Instead, you can use the File API's createObjectURL method. Here's your setImage() function rewritten:
function setImage() {
var file = this.files[0];
var URL = window.URL || window.webkitURL;
if (URL.createObjectURL && (file.type == "image/jpeg" || file.type == "image/png" || file.type == "image/gif")) {
document.getElementById('uploadingImg').src = URL.createObjectURL(file);
} else {
alert('Unable to show preview');
}
}
Browser support is still a bit patchy, though, so check support tables such as caniuse.com before deciding whether to use it:
http://caniuse.com/#search=createobjectURL
I'm currently working on a proxy server where we in this case have to modify the data (by using regexp) that we push through it.
In most cases it works fine except for websites that use gzip as content-encoding (I think), I've come across a module called compress and tried to push the chunks that I receive through a decompress / gunzip stream but it isn't really turning out as I expected. (see below for code)
figured i'd post some code to support my prob, this is the proxy that gets loaded with mvc (express):
module.exports = {
index: function(request, response){
var iframe_url = "www.nu.nl"; // site with gzip encoding
var http = require('http');
var httpClient = http.createClient(80, iframe_url);
var headers = request.headers;
headers.host = iframe_url;
var remoteRequest = httpClient.request(request.method, request.url, headers);
request.on('data', function(chunk) {
remoteRequest.write(chunk);
});
request.on('end', function() {
remoteRequest.end();
});
remoteRequest.on('response', function (remoteResponse){
var body_regexp = new RegExp("<head>"); // regex to find first head tag
var href_regexp = new RegExp('\<a href="(.*)"', 'g'); // regex to find hrefs
response.writeHead(remoteResponse.statusCode, remoteResponse.headers);
remoteResponse.on('data', function (chunk) {
var body = doDecompress(new compress.GunzipStream(), chunk);
body = body.replace(body_regexp, "<head><base href=\"http://"+ iframe_url +"/\">");
body = body.replace(href_regexp, '<a href="#" onclick="javascript:return false;"');
response.write(body, 'binary');
});
remoteResponse.on('end', function() {
response.end();
});
});
}
};
at the var body part i want to read the body and for example in this case remove all hrefs by replacing them with an #. The problem here of course is when we have an site which is gzip encoded/ compressed it's all jibberish and we can't apply the regexps.
now I've already tired to mess around with the node-compress module:
doDecompress(new compress.GunzipStream(), chunk);
which refers to
function doDecompress(decompressor, input) {
var d1 = input.substr(0, 25);
var d2 = input.substr(25);
sys.puts('Making decompression requests...');
var output = '';
decompressor.setInputEncoding('binary');
decompressor.setEncoding('utf8');
decompressor.addListener('data', function(data) {
output += data;
}).addListener('error', function(err) {
throw err;
}).addListener('end', function() {
sys.puts('Decompressed length: ' + output.length);
sys.puts('Raw data: ' + output);
});
decompressor.write(d1);
decompressor.write(d2);
decompressor.close();
sys.puts('Requests done.');
}
But it fails on it since the chunk input is an object, so i tried supplying it as an chunk.toString() which also fails with invalid input data.
I was wondering if I am at all heading in the right direction?
The decompressor expects binary encoded input. The chunk that your response receives is an instance of Buffer which toString() method does by default give you an UTF-8 encoded string back.
So you have to use chunk.toString('binary') to make it work, this can also be seen in the demo.
PLEASE READ THE UPDATE #2 BELOW IF YOU ARE INTERESTED IN THIS PROBLEM ;)
Say I put this code into the JS of my extension.
var reader = {
onInputStreamReady : function(input) {
var sin = Cc["#mozilla.org/scriptableinputstream;1"]
.createInstance(Ci.nsIScriptableInputStream);
sin.init(input);
sin.available();
var request = '';
while (sin.available()) {
request = request + sin.read(512);
}
console.log('Received: ' + request);
input.asyncWait(reader,0,0,null);
}
}
var listener = {
onSocketAccepted: function(serverSocket, clientSocket) {
console.log("Accepted connection on "+clientSocket.host+":"+clientSocket.port);
input = clientSocket.openInputStream(0, 0, 0).QueryInterface(Ci.nsIAsyncInputStream);
output = clientSocket.openOutputStream(Ci.nsITransport.OPEN_BLOCKING, 0, 0);
input.asyncWait(reader,0,0,null);
}
}
var serverSocket = Cc["#mozilla.org/network/server-socket;1"].
createInstance(Ci.nsIServerSocket);
serverSocket.init(-1, true, 5);
console.log("Opened socket on " + serverSocket.port);
serverSocket.asyncListen(listener);
Then I run Firefox and connect to the socket via telnet
telnet localhost PORT
I send 5 messages and they get printed out, but when I try to send 6th message I get
firefox-bin: Fatal IO error 11 (Resource temporarily unavailable) on X server :0.0.
Even worse, when I try to put this same code into an XPCOM component (because that's where I actually need it), after I try sending a message via telnet I get
Segmentation fault
or sometimes
GLib-ERROR **: /build/buildd/glib2.0-2.24.1/glib/gmem.c:137: failed to allocate 32 bytes
aborting...
Aborted
printed to the terminal from which I launched firefox.
This is really weird stuff.. Can you spot something wrong with the code I've pasted or is smth wrong with my firefox/system or is the nsIServerSocket interface deprecated?
I'm testing with Firefox 3.6.6.
I would really appreciate some answer. Perhaps you could point me to a good example of using Sockets within an XPCOM component. I haven't seen many of those around.
UPDATE
I just realised that it used to work so now I think that my Console
component breaks it. I have no idea how this is related. But if I
don't use this component the sockets are working fine.
Here is the code of my Console component. I will try to figure out
what's wrong and why it interferes and I'll post my findings later.
Likely I'm doing something terribly wrong here to cause Segmentation
faults with my javascript =)
Voodoo..
components/Console.js:
const Cc = Components.classes;
const Ci = Components.interfaces;
const Cr = Components.results;
Console.prototype = (function() {
var win;
var initialized = false;
var ready = false;
var _log = function(m, level, location) {
if (initialized&&ready) {
var prefix = "INFO: ";
switch (level) {
case "empty":
prefix = ""
break;
case "error":
prefix = "ERORR: "
break;
case "warning":
prefix = "WARNING: "
break;
}
win.document.getElementById(location).value =
win.document.getElementById(location).value + prefix + m + "\n";
win.focus();
} else if (initialized&&!ready) {
// Now it is time to create the timer...
var timer = Components.classes["#mozilla.org/timer;1"]
.createInstance(Components.interfaces.nsITimer);
// ... and to initialize it, we want to call
event.notify() ...
// ... one time after exactly ten second.
timer.initWithCallback(
{ notify: function() { log(m); } },
10,
Components.interfaces.nsITimer.TYPE_ONE_SHOT
);
} else {
init();
log(m);
}
}
var log = function(m, level) {
_log(m, level, 'debug');
}
var poly = function(m, level) {
_log(m, "empty", 'polyml');
}
var close = function() {
win.close();
}
var setReady = function() {
ready = true;
}
var init = function() {
initialized = true;
var ww = Components.classes["#mozilla.org/embedcomp/window-
watcher;1"]
.getService(Components.interfaces.nsIWindowWatcher);
win = ww.openWindow(null, "chrome://polymlext/content/
console.xul",
"console", "chrome,centerscreen,
resizable=no", null);
win.onload = setReady;
return win;
}
return {
init: init,
log : log,
poly : poly,
}
}());
// turning Console Class into an XPCOM component
Components.utils.import("resource://gre/modules/XPCOMUtils.jsm");
function Console() {
this.wrappedJSObject = this;
}
prototype2 = {
classDescription: "A special Console for PolyML extension",
classID: Components.ID("{483aecbc-42e7-456e-b5b3-2197ea7e1fb4}"),
contractID: "#ed.ac.uk/poly/console;1",
QueryInterface: XPCOMUtils.generateQI(),
}
//add the required XPCOM glue into the Poly class
for (attr in prototype2) {
Console.prototype[attr] = prototype2[attr];
}
var components = [Console];
function NSGetModule(compMgr, fileSpec) {
return XPCOMUtils.generateModule(components);
}
I'm using this component like this:
console = Cc["#ed.ac.uk/poly/console;1"].getService().wrappedJSObject;
console.log("something");
And this breaks the sockets :-S =)
UPDATE #2
Ok, if anyone is interested in checking this thing out I would really
appreciate it + I think this is likely some kind of bug (Seg fault
from javascript shouldn't happen)
I've made a minimal version of the extension that causes the problem,
you can install it from here:
http://dl.dropbox.com/u/645579/segfault.xpi
The important part is chrome/content/main.js:
http://pastebin.com/zV0e73Na
The way my friend and me can reproduce the error is by launching the
firefox, then a new window should appear saying "Opened socket on
9999". Connect using "telnet localhost 9999" and send a few messages.
After 2-6 messages you get one of the following printed out in the
terminal where firefox was launched:
1 (most common)
Segmentation fault
2 (saw multiple times)
firefox-bin: Fatal IO error 11 (Resource temporarily unavailable) on
X
server :0.0.
3 (saw a couple of times)
GLib-ERROR **: /build/buildd/glib2.0-2.24.1/glib/gmem.c:137: failed
to
allocate 32 bytes
aborting...
Aborted
4 (saw once)
firefox-bin: ../../src/xcb_io.c:249: process_responses: Assertion
`(((long) (dpy->last_request_read) - (long) (dpy->request)) <= 0)'
failed.
Aborted
If you need any more info or could point me to where to post a bug
report :-/ I'll be glad to do that.
I know this is just one of the many bugs... but perhaps you have an
idea of what should I do differently to avoid this? I would like to
use that "console" of mine in such way.
I'll try doing it with buffer/flushing/try/catch as people are suggesting, but I wonder whether try/catch will catch the Seg fault...
This is a thread problem. The callback onInputStreamReady happened to be executed in a different thread and accessing UI / DOM is only allowed from the main thread.
Solution is really simple:
change
input.asyncWait(reader,0,0,null);
to
var tm = Cc["#mozilla.org/thread-manager;1"].getService();
input.asyncWait(reader,0,0,tm.mainThread);