I'm using Ember on the frontend and Node/Express on the back end. I've got image uploading directly from my client to S3. My issue now is that the image sizes are huge (970 × 728 pixels (Natural: 3264 × 2448 pixels)) and I want to optimize the size. I've been able to do that using the canvas. But the resulting dataURI is also, very, very large. I could store it on my server, but that seems silly since then I'll have 1 large file sitting in s3 and another sitting in my server for every single image. I feel like the solution should be to upload the dataURI (in blob form) directly to s3 in the first place, but that's where I'm stumped. In order to draw the image in the canvas, I need the src url from s3. So how can I get the dataURI and save it to s3 without already having the original in s3 in the first place? Or should I be saving both versions in s3? Any advice on how to achieve this would be great. Thanks!
Browser support for this type of thing is limited. You could upload the large image to your server, use your server to resize the image, then upload the resized image from your server to s3 (making sure to delete the working copy). This is also good for hiding your AWS creds.
Another option is to use a service like: http://filepicker.io
Related
I'm trying to build a E-commerce site with a admin page where the administrator can upload images of certain products.
I'd like Meteor to upload those images to a folder and then display those images in the product page of that product.
I know that normally the image files that the client will be using should be inside the 'public' folder, but I'd like to know more about what other options I might have.
Also, if I upload a new file to the 'public' folder or if I delete a file in the 'public' folder, the website refreshes itself...and this is good and bad at the same time depending on what effect you are after....
Here are my questions:
What if I create a 'uploads' folder in the server and upload the images to that folder. Would it be possible to display the images inside the 'uploads' folder in the client browser??? How??
Is there a way to use the browser to access the contents of the 'public' folder???
Is there a way to stop the 'reactivity' of the site if changes happen in the 'uploads' folder created?
Is uploading the images to the 'public' folder the best solution available to this problem?
Thank you very much for the help
When dealing with what will likely be a large number of images I like to offload not only the storage but also the processing to a third party.
My go-to app in this situation would be Cloudinary. Here's why:
Storage - I can store the original images outside of my application. A huge benefit to keep images in sync from dev to prod.
CDN - I get the extra benefits of images being quickly loaded from the Cloudinary CDN.
Off-load Processing - All of the processing of images is handled by Cloudinary which doesn't slow down my app as a whole.
Image Manipulation - I can make calls to the original image, calls to just get a thumbnail, and calls to change manipulate, ie :effect => grayscale. So if a 1000x1000px image was uploaded, I can request a 50x50px from Cloudinary that will return the image cropped to that exact size rather than using CSS to shrink a huge image.
Flexibility - I can change the size of images and return that exact size to the app without having to re-upload images. For example, if my product page pulled in thumbs at 40px, I could easily make a call to grab the same image at 50px.
Hope this helps.
http://cloudinary.com/
You can do all of this using the meteor package collectionFS. The package is well documented and you have a variety of options that you can uses for storing the uploaded files. CollectionFS also gives the ability for image manipulation on the upload, such as creating a resized thumbnail.
I realized this question is a bit old.
I had the same problem, one of the solution that works for me is using meteor-upload https://github.com/tomitrescak/meteor-tomi-upload-jquery
Definitely don't store stuff in the public directory - it will slow down starting up the app, and hot code refreshes on image upload could easily cause it to crash once there are a decent number of images in there.
Any of the above solutions with storing images elsewhere would work. One other option is using the peerlibrary:aws-sdk package to upload stuff to S3, which is what I use for several apps and have found to be very clean.
Storing the image as a base64 string in MongoDB is also a method. Useful for posting to APIs and save the worry of having to handle other 3rd Parties.
My hosting has very limited memory space, and I want part of my images(or all of them) be saved at amazon S3 service.
For example, User uploads a picture at site. I take original image, create 5 different thumbnails with it, send original image to S3 with putObject function, and delete it locally. Then user wants to edit his picture, lets say rotate it. I will download original image, rotate it, re-generate 5 thumbnails from it, and put again to S3?
What if i want one of my generated images thumbnails be at Amazon S3, this image is rarely viewed(lets say it is a big portrait, and served only on click at user profile). Do i download image to machine, send it to page, and delete it? Will this approach be fast? Is there better logic for this part?
Struggle for advices.
There is no reason you can't store all your images on S3 - the original and all of the generated thumbnails.
The host receives the uploaded images, generates the required thumbnails and the PUTs them all to S3. Your web application then references all of your images directly from the S3 location - there is no need to download them to your host in order to show them on your website.
Serving your images (and in fact all of your static content - i.e. css files and js files) from S3 will in all likelihood speed up the the page load of your website and if you need a further performance boost, with just a few clicks of the mouse you can use AWS Cloudfront to push your s3 files to geographically dispersed edge locations around the globe to get those items 'closer' to your intended users.
I have a mobile app where users can upload JPGs but I want to limit the size of the image before sending it to my GAE blobstore. I don't want to have to force a user to resize the image they want to upload but rather just do it on the fly. Anyone got a way to do this without distorting the image?
There is no straight forward way to do it and no third party library in gwt yet for this.
Option 1
The best you can try is for GWT wrappers over html5 apis . You can track this on another stackoverflow question Image resizing client-side with javascript before upload to the server
Option 2
Upload and process on server side before pushing to GAE blobstore.
If I am correct blobstores maximum file size is around 2Gb. And you can send around 1Mb of data in one request of GAE blobstore service.
There is an app called AppImage that divides the file into smaller pieces. For more info on that click the below link
Large Image Resizing for Google App Engine
This is just a thought : why do you want to upload it through GAE as you can directly upload it to blobstore unless it greater the 2Gb !!!!
The first bit before the _ is the id of the pin...what are the ZZtfjmGQ used for? I'm assuming the _c is probalby something to do with size.
http://media-cache-lt0.pinterest.com/upload/33284484717557666_HZtfjmFQ_c.jpg
I'm building an image upload service in node.js and was curious what other sites do to store the image.
Final images are served from a CDN, evident by the subdomain in the URL. The first bit, as you pointed out, is the id of the image, the second bit is a UID to get around cache limitations for image versions, and the last bit is image size.
A limitation of CDNs is the inability to process the image after upload. To get around this, my service uploads the files to my Nodejs server where I then serve the image back to the client. I use a jQuery script the user can use to crop the image which sends crop coordinates back to the server and I use ImageMagick to create the various sizes of the the uploaded image. You can obviously eliminate the crop step and just use aspect ratio's to automatically create the needed image sizes. I then upload the final images to the CDN for hosting to end users.
When a user needs to update a photo already in the CDN, the user uploads to nodejs server, images are processed and sized, the UID hash is updated and then uploaded to the CDN. If you want to keep things clean (and cut on CDN cost) you can delete the old "version" at this step as well. However, in my service I give the option to backtrack to an older version if needed.
The rest of the CRUD actions are pretty self explanatory. You can read a list of images available from the CDN using the ID (My service has a user id as well as an image id to allow more robust query operations) and deleting is as simple as identifying the image you want to delete.
I've got an app on Google App Engine that will accept image uploads from users. The problem that I envision is that users will upload these images directly from their cameras, and file sizes are often greater than 1MB, which is the limit for the image API (which would be used to resize the images).
What's the best way to accept the upload of say a 1.5MB image file, and resize it to under 1MB?
While this is not clear in the App Engine documentation, this is possible by using a combination of the Blobstore and the Image Manipulation Service.
You must:
Upload the Image into the Blobstore
Retrieve the Image from the Blobstore
Perform the Image Manipulation with an Image resulting in less than 1mb in size
I've written up a post about this -> http://socialappdev.com/uploading-and-re-sizing-large-images-on-app-engine-11-2010.
Here are two (similar) ways to solve this:
If you want to keep everything controlled yourself, you can put a resize script on a server of yours, which takes the URL to the raw uploaded image (which can be up to 10MB due to HTTP response size limit, but you would have to store it as 1MB chunks in the datastore), downloads it from your application, resizes it, and then POSTs it back to your application. All this interaction would need some kind of authorization of course, so that it can't be abused. Alternatively, POST the image directly to your external server, but then you have to either send the other form data back to your application, or use a separate form for the image upload.
Use an external imaging service. I would recommend Picnik. See their API documentation. As you can see, it lets you make a form that posts the image directly to their servers, then the user can edit the image (and resize), then the image is posted back to your server. With this solution you have to upload the image in a separate form, since Picnik receives all your POST data.
I recommend point 2, because it doesn't require you to go around Google App Engine limitations and since your users are uploading images straight from the camera, they will probably want to do something with them anyways (such as crop.)
That's a conundrum. The "obvious" answer, using google.appengine.api.images.resize, won't work because it's too big. :) So you will have to use third-party software, either on the server (which will be tricky because of App Engine's limitations) or the cilent (e.g. a Java uploader).