I Googled this but couldn't find an answer but it must be a common problem. This is the same question as Node request (read image stream - pipe back to response), which is unanswered.
How do I send an image file as an Express .send() response? I need to map RESTful urls to images - but how do I send the binary file with the right headers? E.g.,
<img src='/report/378334e22/e33423222' />
Calls...
app.get('/report/:chart_id/:user_id', function (req, res) {
//authenticate user_id, get chart_id obfuscated url
//send image binary with correct headers
});
There is an api in Express.
res.sendFile
app.get('/report/:chart_id/:user_id', function (req, res) {
// res.sendFile(filepath);
});
http://expressjs.com/en/api.html#res.sendFile
a proper solution with streams and error handling is below:
const fs = require('fs')
const stream = require('stream')
app.get('/report/:chart_id/:user_id',(req, res) => {
const r = fs.createReadStream('path to file') // or any other way to get a readable stream
const ps = new stream.PassThrough() // <---- this makes a trick with stream error handling
stream.pipeline(
r,
ps, // <---- this makes a trick with stream error handling
(err) => {
if (err) {
console.log(err) // No such file or any other kind of error
return res.sendStatus(400);
}
})
ps.pipe(res) // <---- this makes a trick with stream error handling
})
with Node older then 10 you will need to use pump instead of pipeline.
Related
I followed the Twilio instructions to setup receive & respond to a text message as in the URL below which uses a Rest Post. It works. Great.
app.post('/sms', (req, res) => {
// Start our TwiML response.
const twiml = new MessagingResponse();
// Add a text message.
const msg = twiml.message('Check out this sweet owl!');
// Add a picture message.
msg.media('https://demo.twilio.com/owl.png');
res.writeHead(200, {'Content-Type': 'text/xml'});
res.end(twiml.toString());
});
I then wanted to convert the REST POST to a graphql POST to be consistent with my code base. I set it up, and my graphql POST responds with the following format which is not xml (which I believe Twilio requires) but json as per graphql. Thus, I can see the response move through the system but Twilio registers an error. If I'm correct, is there a way for Twilio to process the graphql json response or for me to adjust graphql to return xml rather than json (as below)?
My latest graphql attempt wraps the rest post in a graphql query as such.
sms: async () => {
// console.log(request, response);
const { MessagingResponse } = require("twilio").twiml;
const twiml = new MessagingResponse();
twiml.message("The Robots are coming! Head for the hills!");
let test = "";
return axios({
method: "post",
url: "https://2b52-98-38-82-19.ngrok.io/sms",
responseType: 'text/xml'
})
.then(res => test = res.data)
// .then(function (response) {
// console.log('axios response =', response);
// return twiml.toString();
// })
.catch(function (error) {
console.log(error);
});
}
}
returning the following via Apollo Sandbox and/or insomnia.
{
"data": {
"sms": "<?xml version="1.0" encoding="UTF-8"?><Response><Message>The Robots are coming! Head for the hills POST POST!</Message></Response>"
}
}
I'm testing out Cloudwatch logs as a trigger for serverless functions. However when my serverless function is triggered, it's just outputting jibberish, which I figured was some form of encryption. Unfortunately I don't know what that encryption is.
Here's the output:
{"awslogs":{"data":"H4sIAAAAAAAAAGVR...4BmF05wEAAA=="}}
And here's the function's code:
const handler = async (event) => {
console.log('*********RECEIVED EVENT FROM CLOUDWATCH**********')
console.log(JSON.stringify(event));
return {
statusCode: 200,
body: JSON.stringify(
event,
null,
2
)
}
};
export { handler };
The "data" section that you get back is base64 encoded and compressed. To get the information out from the event, you just need to decode the base64 information and unzip the data.
Here's a code snippet that shows basically what needs to be done in order to read the log data.
...
const payload = Buffer.from(event.awslogs.data, 'base64');
zlib.gunzip(payload, (err, res) => {
if (err) {
...
}
const parsed = JSON.parse(res.toString('utf8'));
...
});
I am using NodeJS. One of my function (lets call it funcOne) receives some input which I pass to another function (lets call it funcTwo) which produces some output.
Before I pass the input to funcTwo I need to make an Ajax call to an endpoint passing the input and then I must pass the output produced by the AJAX call to funcTwo. funcTwo should be called only when the AJAX call is successful.
How can I achieve this in NodeJS. I wonder if Q Library can be utilized in this case
Using request
function funcOne(input) {
var request = require('request');
request.post(someUrl, {json: true, body: input}, function(err, res, body) {
if (!err && res.statusCode === 200) {
funcTwo(body, function(err, output) {
console.log(err, output);
});
}
});
}
function funcTwo(input, callback) {
// process input
callback(null, input);
}
Edit: Since request is now deprecated you can find alternatives here
Since request is deprecated. I recommend working with axios.
npm install axios#0.16.2
const axios = require('axios');
axios.get('https://api.nasa.gov/planetary/apod?api_key=DEMO_KEY')
.then(response => {
console.log(response.data.url);
console.log(response.data.explanation);
})
.catch(error => {
console.log(error);
});
Using the standard http library to make requests will require more effort to parse/get data. For someone who was used to making AJAX request purely in Java/JavaScript I found axios to be easy to pick up.
https://www.twilio.com/blog/2017/08/http-requests-in-node-js.html
Here is what I'm trying to do:
Retrieve raw data of an image (jpeg) from a URL given to me by an API
Pass the raw data or buffer to a function that uploads it to another server
NEVER PIPE THE IMAGE TO THE DISK
I've followed every example I can find (that doesn't pipe to disk), but still the content comes out corrupted. I have tried forcing various "accept-encodings" (gzip, deflate) but they basically resolve to the same data, just compressed.
I believe this has something to do with the response encoding rather than how I am asking for the data.
Here's the code so far:
var parsedUrl = require('url').parse(PATH_TO_IMAGE)
var params = {
hostname: parsedUrl.hostname,
path: parsedUrl.path,
}
return http.get(params, function(photo_res) {
var photoData = '';
res.setEncoding('binary');
photo_res.on('data', function(chunk) {
photoData += chunk;
});
photo_res.on('end', function() {
// DO STUFF TO UPLOAD IMAGE
});
photo_res.on('error', function(err) {
console.error('Unable to download photo:', err);
return done(err);
});
});
You have a simple typographic error which may be causing Node to interpret your data stream with the incorrect type. Your error is in this line:
res.setEncoding('binary');
To avoid confusion you should keep the response variable named res, and since your data is in binary format, it might be better to keep it as a buffer.
http.get(options, function(res) {
var photoData = [];
res.setEncoding('binary');
res.on('data', function(chunk) {
photoData.push(chunk);
});
res.on('end', function() {
var photo = Buffer.concat(photoData);
});
res.on('error', function(err) {
console.error('Unable to download photo:', err);
});
});
In the example, I store all chunks of data into an array, then use Buffer.concat() to create a single buffer. It is better this way because you were originally appending your image's data onto a string, which may have cause the corruption.
I'm building a super simple server in node and in my onRequest listener I'm trying to determine if I should serve a static file (off the disk) or some json (probably pulled from mongo) based on the path in request.url.
Currently I'm trying to stat the file first (because I use mtime elsewhere) and if that doesn't fail then I read the contents from disk. Something like this:
fs.stat(request.url.pathname, function(err, stat) {
if (!err) {
fs.readFile(request.url.pathname, function( err, contents) {
//serve file
});
}else {
//either pull data from mongo or serve 404 error
}
});
Other than cacheing the result of fs.stat for the request.url.pathname, is there something that could speed this check up? For example, would it be just as fast to see if fs.readFile errors out instead of the stat? Or using fs.createReadStream instead of fs.readFile? Or could I potentially check for the file using something in child_process.spawn? Basically I just want to make sure I'm not spending any extra time messing w/ fileio when the request should be sent to mongo for data...
Thanks!
var fs = require('fs');
fs.exists(file, function(exists) {
if (exists) {
// serve file
} else {
// mongodb
}
});
I don't think you should be worrying about that, but rather how can you improve the caching mechanism. fs.stat is really ok for file checking, doing that in another child process would probably slow you down rather then help you here.
Connect implemented the staticCache() middleware a few months ago, as described in this blog post: http://tjholowaychuk.com/post/9682643240/connect-1-7-0-fast-static-file-memory-cache-and-more
A Least-Recently-Used (LRU) cache algo is implemented through the
Cache object, simply rotating cache objects as they are hit. This
means that increasingly popular objects maintain their positions while
others get shoved out of the stack and garbage collected.
Other resources:
http://senchalabs.github.com/connect/middleware-staticCache.html
The source code for staticCache
this snippet can help you
fs = require('fs') ;
var path = 'sth' ;
fs.stat(path, function(err, stat) {
if (err) {
if ('ENOENT' == err.code) {
//file did'nt exist so for example send 404 to client
} else {
//it is a server error so for example send 500 to client
}
} else {
//every thing was ok so for example you can read it and send it to client
}
} );
In case you want to serve a file using express, I would recommend to just use the sendFile error Handler of express
const app = require("express")();
const options = {};
options.root = process.cwd();
var sendFiles = function(res, files) {
res.sendFile(files.shift(), options, function(err) {
if (err) {
console.log(err);
console.log(files);
if(files.length === 0) {
res.status(err.status).end();
} else {
sendFiles(res, files)
}
} else {
console.log("Image Sent");
}
});
};
app.get("/getPictures", function(req, res, next) {
const files = [
"file-does-not-exist.jpg",
"file-does-not-exist-also.jpg",
"file-exists.jpg",
"file-does-not-exist.jpg"
];
sendFiles(res, files);
});
app.listen(8080);
If the file is not existent then it will go to the error that sends it self.
I made a github repo here https://github.com/dmastag/ex_fs/blob/master/index.js