I was reading about attacks on sites with the ability to upload and download files. Some attacks were about uploading a jpg which is really a html file and a comment about what if you want users to be allowed to store html and download them (or perhaps view them in the browser w/o using the save as feature).
Is there some type of flag i can use to say do not execute? I will want users to view images or video files other have uploaded. What if i'd like user html to be displayed but i dont want to force users to download them (content-disposition attachment).
Is there a way i can say hey here is some user data. It could be an image so i should allow img src to work. It could be an html so i'd like users to see it but dont allow it to read/write cookies/localstorage/call ajax request/etc?
-edit- Come to think of it. All of my user data is hosted on its own cookieless subdomain for static files. That would get rid of many problems i mention but what else is left to deal with? Also i believe my mime response completely depends on what my web server does (nginx atm) which could simply be look at the file extension.
-edit2- I adjusted my nginx config to add the application/unknown Content-Type. It seems to do exactly what i want. I saw a suggestion to use octet-stream for unknown files but that causes browsers (at least firefox) to try to download it even if its a jpg capable being viewed in browser.
It all depends on the Content-Type in your HTTP Response.
Browsers handle the data returned by the Content-Type in HTTP response.
For example if let say a user uploads a HTML file in a upload field supposedly for photo upload, as long as your web server gives Content-Type as image/jpeg (or image/png et al) the browser should handle it as an image - and in this case an invalid image because the image contains weird HTML stuff inside instead of the usual binary.
In any case, if you are feeling unsecure, you can always peek into the file data during upload validation.
Related
I am working on an xpages app, where the client wants to restrict the size of images that are pasted into the document. I have restricted docs with the upload component, but not with a pasted image.
Has anyone done anything like this?
Thanks
walt
Yes it is possible, I haven't done this but there are a few ways you could go about it.
Excuse me if you already know this, but it is worth pointing out for anyone who comes across this question in the future, RichText as we knew it from Notes/Domino is not used XPages, it is all MIME. When I started Xpages I knew nothing about MIME so after understanding it all I have written a little summary on my blog to help anyone get used to the idea. If you are going to do anything with the InputRichText control and File Uploads etc. it is worth getting familiar with MIMEEntity's.
http://camerongregor.com/2016/04/21/webmail-ui-you-must-learn-about-mime/
There are 2 ways that images can make it into the 'rich text field', either uploaded using the 'Image' button on the editor toolbar, or alternatively in some web browsers, the image can be pasted from the clipboard. The 2 different methods will include the image in 2 different ways
Uploaded
If the image is uploaded, then the image becomes a multipart/related mime entity of some sort of image/jpeg, image/png etc., related to the text/html. Between the time that the image is uploaded, and the time that the Document is saved, the image file will be located on the server's disk, in the xpages persistence folder associated with the Document e.g.
xsppers_directory/application-id/session-id/document-id/someimage.png
Pasted
When pasted the image is not uploaded as a file to the server's persistence folder, but is instead included within the text/html as a data URI image (as Stephan mentioned in the comment above).
So, if you want a true solution you will need to address both avenues.
Handling Pasted images
I have shared a plugin on my blog which blocks pasted images, so this could be used to eliminate the 'paste' avenue altogether, and force users down the upload avenue.
http://camerongregor.com/2016/11/14/preventing-pasting-of-images-in-ckeditor/
Alternatively, if you are happy with having smaller pasted images, you could just 'restrict' it by modifying the plugin so that it checks the 'length' of the data URI (as Stephan suggested above) and allows the paste if the length is below your predetermined limit.
If even more alternatively, you could find some sort of client side javascript library that will resize images and do it there in the client before the paste.
Or another server-side option is you could process the contents of the 'text/html' MIMEEntity when saving, and find any img tag's that used Data URI's and strip it out, resize the image using some server side library such as ImageMagick and replace it.
Handling Uploaded Images
For uploaded images, an intense solution would be to extend the InputRichText control (similar to how the have demonstrated in the Mastering Xpages 2nd edition) and override the processAjaxRequest event so that it resizes the image during the process of uploading of the image and putting it in the persistence folder. You could resize it there and then using something like ImageMagick.
Or a less intense version would be to process the 'pending' attachments during a before save event. If you get the DominoRichTextItem from the DominoDocument, you should be able to iterate through the Attachments, checking their attachment state, there will be a state which means 'pending/about to upload' and for these ones you can process them while they are in the persistence folder, before they are finally attached to the document.
Finally if you don't want the xpages runtime / persistence directory to be involved, you could do it all after save, just using the backend NotesDocument and MimeEntity processing. Extract the Image MimeEntities to disk, process them with ImageMagick or similar, and then re-attach them.
Someone else may have a simpler solution that I haven't thought of but at least this gives you some idea of what may be involved.
I know gmail and hotmail have introduced their own image proxies. I have sent email with simple img tag with src
src="http://www.zong360office.com/assets/images/logo/product-logo.png"
and gmail converts it like this
src="https://ci3.googleusercontent.com/proxy/PCxvUIqBvaJWn_7H8deiQhdJD1QKV_pU5n1JhrPh3xaXMLMzcdSBYCo6-tjs69AYy858H4j5ShIRjw0UrH_VWMtCxC0zIT-Gi4j_dlQv9uRZjzGWFWA=s0-d-e1-ft#http://www.zong360office.com/assets/images/logo/product-logo.png"
Original URL is the direct URL to the image. In the attached image, you can see the request and response headers for this image.
Problem: Please let me know what steps should I take to make this image visible inside gmail.
PS: I am facing the same issue with hotmail as well.
I think I have figured out the GoogleImageProxy issue.
I have answered this in another question as well ( images inside gmail email not being displayed ).
This is something related to CACHING concept. suppose, you have recently deployed your php code on your server but you forgot to upload images. you tested once with your email logic. your system generated an HTML email. When this email will hit the gmail server GoogleImageProxy will try to fetch and store the images from your site to its own proxy server. while fetching the images, GoogleImageProxy found some 404 statuses against your missing images and 403 against some protected images. GoogleImagesProxy has stored these statuses into its own proxy server.
Now tried to open your email, you noticed some 404 statuses against your images. This is something understandable. You immediately realized that you forgot to upload some images, so you uploaded them to your server. and also you have fixed some permissions against protected images.
You are all done now. Now you try to run your php-email script once again. As a result you receive another email in your Gmail or Hotmail inbox. you had fixed all the issues with your images. Now the images must be displayed in your email content. but you are still unable to see the images.
Ah, possibly you forgot to clear your browser's cache. Clear your browser's cache and load the gmail or hotmail page once again. But the result will be still the same. Try to apply dozens of fixes/patches and try to run your php-email script a thousands time. But the result will be still the same. No improvement.
THE REAL PROBLEM
What the hell is going on? Let me explain it to you. Go to your access log and try to find requests from GoogleImageProxy. You'll be surprised to see that there will be only 2 or 3 three requests from GoogleImageProxy depending on the number of different images used in your email. GoogleImageProxy never tried to fetch images Even after you have fixed the issues with your images by uploading missing images and setting permissions for protected images. Why? Clearing your browser's cache has no impact. GoogleImageProxy will never fetch the fresh images even for your newer email because the images are now cached into GoogleImageProxy along with their last status code and not cached in your own browser's.
GoogleImageProxy has set its own expiry date for the images. I think one month. so now the fresh copy of images will be fetch after expiry date. I mean after one month. You can not force GoogleImageProxy to fetch the images. But its important for you to display images in your email. What can be the solution?
THE SOLUTION
Following is the only way to force GoogleImageProxy to fetch your images
Rename your images to something else with png, jpg or gif extensions
only.
Don't use any kind of query string in your image url like "?t=34343"
your image must include png, jpg or gif as an extension.
your image url must be mapped onto your image directly.
If you need to use some proxy url for your protected images then your
response must include the proper header like
"content-type:image/jpeg"
File extension and content-type header must match
Status-code must be 200 instead of 403, 500 etc
IMPORTANT NOTE
Try to repeat the whole process for every run of php-email script. because every time GoogleImageProxy will cache your images and you'll have to repeat the same process for every new try.
Hopefully this will fix the issue for most of the people. In my case this has fixed the issue.
hese are embedded images. You send an image as an attachment and then use it in your img element like: . Gmail transforms the "cid" link to its internal file storage link and that's why it gives element.
This is one of two common ways to have images within email.
The other way is to keep images in your hosting and send img elements with external links: http://yourdomain.com/yourimage.png I strongly recommend to use the second approach especially if you expect to have a lot of images in your email. I had problems with attached images display on Gmail which I couldn't solve.
Make sure your from address is a real valid email and not in the form realname+something#example.com.
I'm using Amazon SES to send emails, and inlining some but not all images as attachments. For me it was the non-attachments that were breaking.
Eventually I realized I was using a 'fake' email for the from field.
My real email is
simon#example.com
To make it easy to filter emails in gmail I used the + syntax to create a 'fake' email for testing purposes.
simon+test#example.com
This works great, except it seems to be changing the behavior of the spam/image filter. My image tags ended up completely stripped of the src attribute.
As soon as I switched back to using the actual email then all the images showed up.
I've got some automatic emails that are sent out upon signup completion for my site.
Until recently, they worked fine. Now Google's new system is rewriting the images and storing them in it's cache (supposedly)
However, Google's new rewriting of my image links are completely breaking them, giving a 500 error and a broken link image.
Lets say my normal image url is:
http://www.mysite.com/images/pic1.jpg
Google is rewriting this to:
https://ci5.googleusercontent.com/proxy/vI79kajdUGm6Wk-fjyicDLjZbCB1w9NfkoZ-zQFOB2OpJ1ILmSvfvHmE56r72us5mIuIXCFiO3V8rgkZOjfhghTH0R07BbcQy5g=s0-d-e1-ft#http://www.mysite.com/images/pic1.jpg
However, there is nothing at that URL.
So, either there is something wrong with the links that are being created by Google or the images are just not being uploaded to the googleusercontent server, but I have no idea how to solve the issue.
Im using PHP, the phpmailer library and a Ubuntu server on Amazon EC2, but Im not sure that is related to the issue.
I think I have figured out the GoogleImageProxy issue.
This is something related to CACHING concept. suppose, you have recently deployed your PHP code on your server but you forgot to upload images. you tested once with your email logic. your system generated an HTML email. When this email will hit the Gmail server GoogleImageProxy will try to fetch and store the images from your site on its own proxy server. while fetching the images, GoogleImageProxy found some 404 statuses against your missing images and 403 against some protected images. GoogleImagesProxy has stored these statuses into its own proxy server.
Now tried to open your email, and you noticed some 404 statuses against your images. This is something understandable. You immediately realized that you forgot to upload some images, so you uploaded them to your server. and also you have fixed some permissions against protected images.
You are all done now. Now you try to run your PHP-email script once again. As a result, you receive another email in your Gmail or Hotmail inbox. you had fixed all the issues with your images. Now the images must be displayed in your email content. but you are still unable to see the images.
Ah, possibly you forgot to clear your browser's cache. Clear your browser's cache and load the Gmail or Hotmail page once again. But the result will be still the same. Try to apply dozens of fixes/patches and try to run your PHP-email script a thousand times. But the result will be still the same. No improvement.
THE REAL PROBLEM
What the hell is going on? Let me explain it to you. Go to your access log and try to find requests from GoogleImageProxy. You'll be surprised to see that there will be only 2 or 3 three requests from GoogleImageProxy depending on the number of different images used in your email. GoogleImageProxy never tried to fetch images Even after you have fixed the issues with your images by uploading missing images and setting permissions for protected images. Why? Clearing your browser's cache has no impact. GoogleImageProxy will never fetch the fresh images even for your newer email because the images are now cached into GoogleImageProxy along with their last status code and not cached in your own browser.
GoogleImageProxy has set its own expiry date for the images. I think one month. so now the fresh copy of images will be fetched after the expiry date. I mean after one month. You can not force GoogleImageProxy to fetch the images. But it is important for you to display images in your email. What can be the solution?
THE SOLUTION
Following is the only way to force GoogleImageProxy to fetch your images
Rename your images to something else with png, jpg, or gif extensions
only.
Don't use any kind of query string in your image URL like ?t=34343
your image must include png, jpg, or gif as an extension.
your image URL must be mapped onto your image directly.
If you need to use some proxy URL for your protected images then your response must include the proper header like
Content-Type: image/jpeg
File extension and content-type header must match
Status-code must be 200 instead of 403, 500, etc
IMPORTANT NOTE
Try to repeat the whole process for every run of PHP-email script. because every time GoogleImageProxy will cache your images and you'll have to repeat the same process for every new try.
Hopefully this will fix the issue for most of the people.
Based on your example, it looks like you are using traditional extensions (.jpg, .png, .gif). Some folks on this thread, describing the same issues you are facing, have stated that using those extensions solves the problem.
Other possible solutions:
Image links broken in Gmail because of google's Image proxy
Doubtful, but maybe a cookie problem
Image URL proxy whitelist setting - this has turned out to be the solution for a few users who are under Google Apps. Via Gmail is not showing image when image url is getting appended with https://ci4.googleusercontent.com/proxy
I was having a similar issue, but it was caused by the length of the URL. Google generates the following URL when caching an image from gmail:
https://ci4.googleusercontent.com/proxy/[hash]#[url])
The hash generated is based on the URL of the image, but the size will vary based on characters used. I ran several tests with different sized URLs, and found the cached image would fail to load consistently (400/Invalid Request) if the hash exceeds 2076 characters in length (close to 2048 bytes + meta? not sure).
Again, the image URL could generate a hash that exceeds this many characters at ~1000 special characters, or 1500+ simple characters. If the hash exceeds 2076 characters in length, the request fails.
I realize this is an old post, but hopefully this helps other devs scouring Google
I know this is an old question but the same thing happened to me. When I checked my access logs this is what I found -
www.example.ca 66.249.85.50 - - [10/Apr/2014:17:57:18 -0400] "GET /newsletters/Apr10_2014/cad/cad2.jpg HTTP/1.1" 403 457 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (via ggpht.com GoogleImageProxy)"
You can see that my server was blocking the GOOGLEIMAGEPROXY giving it a 403 Forbidden reply. I decided to check my .htaccess and sure enough I was blocking the term PROXY. After removing the term, the images appear just fine now on Gmail. Hope that helps.
I just tried ,
after replacing the image (without changing image name)
Open email in new browser , it shows new image
Ctrl+f5 (forces a cache refresh) in the chrome (my default browser) ,
also shows new image
use .png or .jpg
otherwise image will not render
url add auto
https://ci3.googleusercontent.com/proxy/jTpYlM6RUv7Wi8Hxjha4fzExKFy9mjyh133MKKfo3FuV3toLToG6zJcA0IAdIMEW75pY6pkEd2aOSVhWIn0A82q-24YaAd-_k00wIMHwIuUBiy9vEGrMpAW73HaHQmViuESP7A=s0-d-e1-ft#
HTTPS image locations do cache. Several of our production environments have no problems with gmail proxying image locations using a HTTPS uri. I could see gmail ignoring your content if the SSL certificate is invalid in some way.
Check that the content-type returned for the image file by your server is correct.
You can check this using Fiddler.
In my case the size of file was the problem, it was 22 Mb (i know right?), and after we reduced the size everything started working like a charm.
Check file size and if it's too big, compress it.
I know this is an old question but I've met this problem. In my case images are stored at Google Cloud Storage. What is interesting is that link
https://storage.cloud.google.com/{bla_bla}/logo.png
returns 307 (temporary redirect) and Location header containing something like
https://{xxx}-apidata.googleusercontent.com/{bla-bla_bla}/logo.png?{zzz}
Seems like GoogleImageProxy does not process 307 correctly
I have a perfect solution of this problem, which worked for me if you are using PHPMailer then you just have to add another option in PHPMailer for attaching image like this
$mail = new PHPMailer();
$mail->AddEmbeddedImage('../absolutepath/image/image.jpg', 'logoimg', '../absolutepath/image/image.jpg');
Here we have given absolute path of image and give it a name call 'logoimg' or whatever you want.
Now you can add this logoimg to wherever in your HTML Body like this
$mail->Body = "
<h1>Test of PHPMailer html body with image</h1>
<p>This is a test picture: <img src=\"cid:logoimg\" /></p>";
$mail->send();
That's All.
I had this issue when I was sending gifs. I found that the file size matters to Googles Proxy server. I suggest making the files as small as possible and see if that works. You can use your Gmail account and add a photo from a URL to test. If the gif shows up when you are composing your email it will be receivable.
happy coding.
Is it working from Outook/hotmail? It should then we can isolate it as google issue. In your case it is not.
Size of the image can be a problem. Try to reduce it and see
www.mysite.com this site might be accessible from your system. But
is it also accessible from google server?
Try changing extension.. this is the trick: You might have tried several things but it would still fetch from cache(which invalidates your efforts) but when the extension changes, it fetches again and all the work you did before comes into play and if it works you might think it is the 'extension' that did the trick!! (like many of those who speaks about extensions)
In my case of running into this issue, the problem was that accidentally the path to the image in the email template had triple slashes in the URL, e.g. https:///content.example.org/image.png. This was hard to spot, and while it was working in other email clients who could successfully resolve the URL, Google's image proxy wasn't able to handle it and resulted in a 404 for the proxied image address.
It's March 6 and you've probably already figured this out, but thought I'd chime in to help others. I discovered that JPGs don't work in gmail. The PNG format works great. Sorry I can't explain why, but sometimes it's better not to ask why. Use PNG!
I need to create, for a specific project, an image manager that works via Ajax (to get the list of images, display them, ...).
The upload of new images, or image modification, is done via an Ajax script (using the new javascript File API).
The upload works fine, but I encounter a problem in case of image modification : the image displayed by the browser after upload is the cached one and not the uploaded one !!
I know it's a classic cache problem, that can be solved via the 'imagesrc?new Date.getTime()' hack, but I can't use it here.
in fact, this hack doesn't really reload the image, it only create a new instance of the image into the cache, associated to the image url 'imagesrc?new Date.getTime()'.
So, if at any moment, into the image manager, I retry to display the image, without adding the '?new Date.getTime()' to the src, it will display again the old image.
And I either cannot add this hack systematically (because, for example, if the image manager needs to display a lot of very heavy images, it's usefull to get them from the browser cache until they are modified).
I searched a way to solve this problem on internet (really replace the cached image after a javascript upload instead of using the above hack), but I found nothing.
Is there a way to do this, or is it totally impossible ?
Any help or suggestion would be greatly appreciated.
Many thanks in advance
Olivier
Configure your server to send ETag-headers for the images.
An ETag is a hash-value of the file that changes when the file is modified. If an ETag is sent, the browser will add an If-None-Match-header containing the last received ETag of that ressource on its next request and the server will respond with 304: not modified to save traffic if nothing has changed or send the new file if there is one.
I'm currently developing a picture-storage on MVC3 and have a question how to restrict access to images.
If I insert image into HTML lice <img src="/ImagesFolder/image0001.jpg"> it will be shown correctly, but anyone who write full path in browser will get that image too. I don't wand to permit it.
One way is to embed image as base64 string, but it is suitable only for small images, I have large ones.
I've seen recommendations to create image-accessing action, and use something like
<img src="/GetImage?ID=1123">, but at that GetImage page I will still use either direct-path or base64 methods? and in first way full path to imagefile will be translated into parent view and still can be seen in picture properties?
Is there a way to use System.Drawing.Image in <img src=""> or any other way? Do you know any samples?
One way to do this is put all of the images into a path which is not published by the http server. This way there is no direct path users can put into the browser.
Scripts themselves are on the server so they can access this path. Your image-access method should be a script which returns actual data and identify itself as the data it is serving image/jpeg for jpegs for example. This way if somebody accesses GetImage method by some other means than the one you have designed the script can detect it (by referer, or other means) and return nothing.
I think that the "image-accessing action" solution you mention is the easiest one to implement. The GetImage script could for example check for a cookie that you set in your application, so that only people that have first visited your site can receive the images.
Now, if you want to prevent people to display the images by typing the URL in the browser after they visited your site, this is more tricky and in my opinion there isn't an easy solution for that. You might check the Referer HTTP header to see if the request is related to your website (as Referer will be empty when putting the URL into the URL bar, but will contain the URL of the page containing the image when it is included with <img src=...>), but this solution has a drawback, as it will prevent browsers not sending the Referer header (generally for privacy reasons; it is not a very wide-spread configuration but this happens) from seeing your images at all. A second possible drawback is that people will probably be able to see the images that are already cached in your browser by typing their URL, unless you can configure the server to serve them with some headers disallowing caching.