Gmail's new image caching is breaking image links in newsletter - image

I've got some automatic emails that are sent out upon signup completion for my site.
Until recently, they worked fine. Now Google's new system is rewriting the images and storing them in it's cache (supposedly)
However, Google's new rewriting of my image links are completely breaking them, giving a 500 error and a broken link image.
Lets say my normal image url is:
http://www.mysite.com/images/pic1.jpg
Google is rewriting this to:
https://ci5.googleusercontent.com/proxy/vI79kajdUGm6Wk-fjyicDLjZbCB1w9NfkoZ-zQFOB2OpJ1ILmSvfvHmE56r72us5mIuIXCFiO3V8rgkZOjfhghTH0R07BbcQy5g=s0-d-e1-ft#http://www.mysite.com/images/pic1.jpg
However, there is nothing at that URL.
So, either there is something wrong with the links that are being created by Google or the images are just not being uploaded to the googleusercontent server, but I have no idea how to solve the issue.
Im using PHP, the phpmailer library and a Ubuntu server on Amazon EC2, but Im not sure that is related to the issue.

I think I have figured out the GoogleImageProxy issue.
This is something related to CACHING concept. suppose, you have recently deployed your PHP code on your server but you forgot to upload images. you tested once with your email logic. your system generated an HTML email. When this email will hit the Gmail server GoogleImageProxy will try to fetch and store the images from your site on its own proxy server. while fetching the images, GoogleImageProxy found some 404 statuses against your missing images and 403 against some protected images. GoogleImagesProxy has stored these statuses into its own proxy server.
Now tried to open your email, and you noticed some 404 statuses against your images. This is something understandable. You immediately realized that you forgot to upload some images, so you uploaded them to your server. and also you have fixed some permissions against protected images.
You are all done now. Now you try to run your PHP-email script once again. As a result, you receive another email in your Gmail or Hotmail inbox. you had fixed all the issues with your images. Now the images must be displayed in your email content. but you are still unable to see the images.
Ah, possibly you forgot to clear your browser's cache. Clear your browser's cache and load the Gmail or Hotmail page once again. But the result will be still the same. Try to apply dozens of fixes/patches and try to run your PHP-email script a thousand times. But the result will be still the same. No improvement.
THE REAL PROBLEM
What the hell is going on? Let me explain it to you. Go to your access log and try to find requests from GoogleImageProxy. You'll be surprised to see that there will be only 2 or 3 three requests from GoogleImageProxy depending on the number of different images used in your email. GoogleImageProxy never tried to fetch images Even after you have fixed the issues with your images by uploading missing images and setting permissions for protected images. Why? Clearing your browser's cache has no impact. GoogleImageProxy will never fetch the fresh images even for your newer email because the images are now cached into GoogleImageProxy along with their last status code and not cached in your own browser.
GoogleImageProxy has set its own expiry date for the images. I think one month. so now the fresh copy of images will be fetched after the expiry date. I mean after one month. You can not force GoogleImageProxy to fetch the images. But it is important for you to display images in your email. What can be the solution?
THE SOLUTION
Following is the only way to force GoogleImageProxy to fetch your images
Rename your images to something else with png, jpg, or gif extensions
only.
Don't use any kind of query string in your image URL like ?t=34343
your image must include png, jpg, or gif as an extension.
your image URL must be mapped onto your image directly.
If you need to use some proxy URL for your protected images then your response must include the proper header like
Content-Type: image/jpeg
File extension and content-type header must match
Status-code must be 200 instead of 403, 500, etc
IMPORTANT NOTE
Try to repeat the whole process for every run of PHP-email script. because every time GoogleImageProxy will cache your images and you'll have to repeat the same process for every new try.
Hopefully this will fix the issue for most of the people.

Based on your example, it looks like you are using traditional extensions (.jpg, .png, .gif). Some folks on this thread, describing the same issues you are facing, have stated that using those extensions solves the problem.
Other possible solutions:
Image links broken in Gmail because of google's Image proxy
Doubtful, but maybe a cookie problem
Image URL proxy whitelist setting - this has turned out to be the solution for a few users who are under Google Apps. Via Gmail is not showing image when image url is getting appended with https://ci4.googleusercontent.com/proxy

I was having a similar issue, but it was caused by the length of the URL. Google generates the following URL when caching an image from gmail:
https://ci4.googleusercontent.com/proxy/[hash]#[url])
The hash generated is based on the URL of the image, but the size will vary based on characters used. I ran several tests with different sized URLs, and found the cached image would fail to load consistently (400/Invalid Request) if the hash exceeds 2076 characters in length (close to 2048 bytes + meta? not sure).
Again, the image URL could generate a hash that exceeds this many characters at ~1000 special characters, or 1500+ simple characters. If the hash exceeds 2076 characters in length, the request fails.
I realize this is an old post, but hopefully this helps other devs scouring Google

I know this is an old question but the same thing happened to me. When I checked my access logs this is what I found -
www.example.ca 66.249.85.50 - - [10/Apr/2014:17:57:18 -0400] "GET /newsletters/Apr10_2014/cad/cad2.jpg HTTP/1.1" 403 457 "-" "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.0.7) Gecko/2009021910 Firefox/3.0.7 (via ggpht.com GoogleImageProxy)"
You can see that my server was blocking the GOOGLEIMAGEPROXY giving it a 403 Forbidden reply. I decided to check my .htaccess and sure enough I was blocking the term PROXY. After removing the term, the images appear just fine now on Gmail. Hope that helps.

I just tried ,
after replacing the image (without changing image name)
Open email in new browser , it shows new image
Ctrl+f5 (forces a cache refresh) in the chrome (my default browser) ,
also shows new image

use .png or .jpg
otherwise image will not render
url add auto
https://ci3.googleusercontent.com/proxy/jTpYlM6RUv7Wi8Hxjha4fzExKFy9mjyh133MKKfo3FuV3toLToG6zJcA0IAdIMEW75pY6pkEd2aOSVhWIn0A82q-24YaAd-_k00wIMHwIuUBiy9vEGrMpAW73HaHQmViuESP7A=s0-d-e1-ft#

HTTPS image locations do cache. Several of our production environments have no problems with gmail proxying image locations using a HTTPS uri. I could see gmail ignoring your content if the SSL certificate is invalid in some way.

Check that the content-type returned for the image file by your server is correct.
You can check this using Fiddler.

In my case the size of file was the problem, it was 22 Mb (i know right?), and after we reduced the size everything started working like a charm.
Check file size and if it's too big, compress it.

I know this is an old question but I've met this problem. In my case images are stored at Google Cloud Storage. What is interesting is that link
https://storage.cloud.google.com/{bla_bla}/logo.png
returns 307 (temporary redirect) and Location header containing something like
https://{xxx}-apidata.googleusercontent.com/{bla-bla_bla}/logo.png?{zzz}
Seems like GoogleImageProxy does not process 307 correctly

I have a perfect solution of this problem, which worked for me if you are using PHPMailer then you just have to add another option in PHPMailer for attaching image like this
$mail = new PHPMailer();
$mail->AddEmbeddedImage('../absolutepath/image/image.jpg', 'logoimg', '../absolutepath/image/image.jpg');
Here we have given absolute path of image and give it a name call 'logoimg' or whatever you want.
Now you can add this logoimg to wherever in your HTML Body like this
$mail->Body = "
<h1>Test of PHPMailer html body with image</h1>
<p>This is a test picture: <img src=\"cid:logoimg\" /></p>";
$mail->send();
That's All.

I had this issue when I was sending gifs. I found that the file size matters to Googles Proxy server. I suggest making the files as small as possible and see if that works. You can use your Gmail account and add a photo from a URL to test. If the gif shows up when you are composing your email it will be receivable.
happy coding.

Is it working from Outook/hotmail? It should then we can isolate it as google issue. In your case it is not.
Size of the image can be a problem. Try to reduce it and see
www.mysite.com this site might be accessible from your system. But
is it also accessible from google server?
Try changing extension.. this is the trick: You might have tried several things but it would still fetch from cache(which invalidates your efforts) but when the extension changes, it fetches again and all the work you did before comes into play and if it works you might think it is the 'extension' that did the trick!! (like many of those who speaks about extensions)

In my case of running into this issue, the problem was that accidentally the path to the image in the email template had triple slashes in the URL, e.g. https:///content.example.org/image.png. This was hard to spot, and while it was working in other email clients who could successfully resolve the URL, Google's image proxy wasn't able to handle it and resulted in a 404 for the proxied image address.

It's March 6 and you've probably already figured this out, but thought I'd chime in to help others. I discovered that JPGs don't work in gmail. The PNG format works great. Sorry I can't explain why, but sometimes it's better not to ask why. Use PNG!

Related

images inside gmail email not being displayed

I know gmail and hotmail have introduced their own image proxies. I have sent email with simple img tag with src
src="http://www.zong360office.com/assets/images/logo/product-logo.png"
and gmail converts it like this
src="https://ci3.googleusercontent.com/proxy/PCxvUIqBvaJWn_7H8deiQhdJD1QKV_pU5n1JhrPh3xaXMLMzcdSBYCo6-tjs69AYy858H4j5ShIRjw0UrH_VWMtCxC0zIT-Gi4j_dlQv9uRZjzGWFWA=s0-d-e1-ft#http://www.zong360office.com/assets/images/logo/product-logo.png"
Original URL is the direct URL to the image. In the attached image, you can see the request and response headers for this image.
Problem: Please let me know what steps should I take to make this image visible inside gmail.
PS: I am facing the same issue with hotmail as well.
I think I have figured out the GoogleImageProxy issue.
I have answered this in another question as well ( images inside gmail email not being displayed ).
This is something related to CACHING concept. suppose, you have recently deployed your php code on your server but you forgot to upload images. you tested once with your email logic. your system generated an HTML email. When this email will hit the gmail server GoogleImageProxy will try to fetch and store the images from your site to its own proxy server. while fetching the images, GoogleImageProxy found some 404 statuses against your missing images and 403 against some protected images. GoogleImagesProxy has stored these statuses into its own proxy server.
Now tried to open your email, you noticed some 404 statuses against your images. This is something understandable. You immediately realized that you forgot to upload some images, so you uploaded them to your server. and also you have fixed some permissions against protected images.
You are all done now. Now you try to run your php-email script once again. As a result you receive another email in your Gmail or Hotmail inbox. you had fixed all the issues with your images. Now the images must be displayed in your email content. but you are still unable to see the images.
Ah, possibly you forgot to clear your browser's cache. Clear your browser's cache and load the gmail or hotmail page once again. But the result will be still the same. Try to apply dozens of fixes/patches and try to run your php-email script a thousands time. But the result will be still the same. No improvement.
THE REAL PROBLEM
What the hell is going on? Let me explain it to you. Go to your access log and try to find requests from GoogleImageProxy. You'll be surprised to see that there will be only 2 or 3 three requests from GoogleImageProxy depending on the number of different images used in your email. GoogleImageProxy never tried to fetch images Even after you have fixed the issues with your images by uploading missing images and setting permissions for protected images. Why? Clearing your browser's cache has no impact. GoogleImageProxy will never fetch the fresh images even for your newer email because the images are now cached into GoogleImageProxy along with their last status code and not cached in your own browser's.
GoogleImageProxy has set its own expiry date for the images. I think one month. so now the fresh copy of images will be fetch after expiry date. I mean after one month. You can not force GoogleImageProxy to fetch the images. But its important for you to display images in your email. What can be the solution?
THE SOLUTION
Following is the only way to force GoogleImageProxy to fetch your images
Rename your images to something else with png, jpg or gif extensions
only.
Don't use any kind of query string in your image url like "?t=34343"
your image must include png, jpg or gif as an extension.
your image url must be mapped onto your image directly.
If you need to use some proxy url for your protected images then your
response must include the proper header like
"content-type:image/jpeg"
File extension and content-type header must match
Status-code must be 200 instead of 403, 500 etc
IMPORTANT NOTE
Try to repeat the whole process for every run of php-email script. because every time GoogleImageProxy will cache your images and you'll have to repeat the same process for every new try.
Hopefully this will fix the issue for most of the people. In my case this has fixed the issue.
hese are embedded images. You send an image as an attachment and then use it in your img element like: . Gmail transforms the "cid" link to its internal file storage link and that's why it gives element.
This is one of two common ways to have images within email.
The other way is to keep images in your hosting and send img elements with external links: http://yourdomain.com/yourimage.png I strongly recommend to use the second approach especially if you expect to have a lot of images in your email. I had problems with attached images display on Gmail which I couldn't solve.
Make sure your from address is a real valid email and not in the form realname+something#example.com.
I'm using Amazon SES to send emails, and inlining some but not all images as attachments. For me it was the non-attachments that were breaking.
Eventually I realized I was using a 'fake' email for the from field.
My real email is
simon#example.com
To make it easy to filter emails in gmail I used the + syntax to create a 'fake' email for testing purposes.
simon+test#example.com
This works great, except it seems to be changing the behavior of the spam/image filter. My image tags ended up completely stripped of the src attribute.
As soon as I switched back to using the actual email then all the images showed up.

broken images on browser

So, there's this blog I visit often, and it has a lot of pictures.
But the thing is, they are all broken. The image is there, and I found that if I use a proxy server and access the blog, some pictures are there. Some are still broken, so then I have to right click the broken image, copy the image URL, open the URL in a different tab/windows, and once that picture finishes loading, I refresh the blog, and the picture I loaded on a different windows now shows....
Does anyone know why this happens, and how I can get it fixed?? Thanks.
In case you don't understand what I mean, This is the blog that makes this happen
http://wersierre.tumblr.com/
This applies to Firefox, chrome, IE in my case.
This is very likely to be the case that the target server is configured to prevent hotlinking. When accessing the images via the page you posted, the HTTP response code of the images is 403, it indicates the server can be reached and understood the HTTP request, but refuses to take any further action.

Javascript Upload and real image refresh/reload

I need to create, for a specific project, an image manager that works via Ajax (to get the list of images, display them, ...).
The upload of new images, or image modification, is done via an Ajax script (using the new javascript File API).
The upload works fine, but I encounter a problem in case of image modification : the image displayed by the browser after upload is the cached one and not the uploaded one !!
I know it's a classic cache problem, that can be solved via the 'imagesrc?new Date.getTime()' hack, but I can't use it here.
in fact, this hack doesn't really reload the image, it only create a new instance of the image into the cache, associated to the image url 'imagesrc?new Date.getTime()'.
So, if at any moment, into the image manager, I retry to display the image, without adding the '?new Date.getTime()' to the src, it will display again the old image.
And I either cannot add this hack systematically (because, for example, if the image manager needs to display a lot of very heavy images, it's usefull to get them from the browser cache until they are modified).
I searched a way to solve this problem on internet (really replace the cached image after a javascript upload instead of using the above hack), but I found nothing.
Is there a way to do this, or is it totally impossible ?
Any help or suggestion would be greatly appreciated.
Many thanks in advance
Olivier
Configure your server to send ETag-headers for the images.
An ETag is a hash-value of the file that changes when the file is modified. If an ETag is sent, the browser will add an If-None-Match-header containing the last received ETag of that ressource on its next request and the server will respond with 304: not modified to save traffic if nothing has changed or send the new file if there is one.

Don't execute flag in http response?

I was reading about attacks on sites with the ability to upload and download files. Some attacks were about uploading a jpg which is really a html file and a comment about what if you want users to be allowed to store html and download them (or perhaps view them in the browser w/o using the save as feature).
Is there some type of flag i can use to say do not execute? I will want users to view images or video files other have uploaded. What if i'd like user html to be displayed but i dont want to force users to download them (content-disposition attachment).
Is there a way i can say hey here is some user data. It could be an image so i should allow img src to work. It could be an html so i'd like users to see it but dont allow it to read/write cookies/localstorage/call ajax request/etc?
-edit- Come to think of it. All of my user data is hosted on its own cookieless subdomain for static files. That would get rid of many problems i mention but what else is left to deal with? Also i believe my mime response completely depends on what my web server does (nginx atm) which could simply be look at the file extension.
-edit2- I adjusted my nginx config to add the application/unknown Content-Type. It seems to do exactly what i want. I saw a suggestion to use octet-stream for unknown files but that causes browsers (at least firefox) to try to download it even if its a jpg capable being viewed in browser.
It all depends on the Content-Type in your HTTP Response.
Browsers handle the data returned by the Content-Type in HTTP response.
For example if let say a user uploads a HTML file in a upload field supposedly for photo upload, as long as your web server gives Content-Type as image/jpeg (or image/png et al) the browser should handle it as an image - and in this case an invalid image because the image contains weird HTML stuff inside instead of the usual binary.
In any case, if you are feeling unsecure, you can always peek into the file data during upload validation.

choosing an image locally from http url and serving that image without a server round trip

I am a complete novice to Flash (never created anything in flash). I am quite familiar with web applications (J2EE based) and have a reasonable expertise in Javascript.
Here is my requirement.
I want the user to select (via an html form) an image. Normally in the post, this image would be sent to server and may be stored there to be served later. I do not want that. I want to store this image locally and then serve it via HTTP to the user.
So, the flow is:
1. Go to the "select image url":mywebsite.com/selectImage
Browse the image and select the image
This would transfer control locally to some code running on the client (Javascript or flash), which would then store the image locally at some place on the client machine.
Go to the "show image url": mywebsite.com/showImage
This would eventually result in some client code running on the browser that retrieves the image and renders it (without any server round trips.)
I considered the following options:
Use HTML5 local storage. Since I am a complete novice to flash, I looked into this. I found that it is fairly straightforward to store and retrieve images in javascript (only strings are allowed but I am hoping storing base64 encoded strings would work at least for small images). However, how do I serve the image via http url that points to my server without a server round trip? I saw the interesting article at http://hacks.mozilla.org/category/fileapi/ but that would work only in firefox and I need to work on all latest browsers (at least the ones supporting HTML5 local storage)
Use flash SharedObjects. OK, this would have been good - the only thing is I am not sure where to start. Snippets of actionscripts to do this are scattered everywhere but I do not know how to use those scripts in an actual html page:) I do not need to create any movies or anything - just need to store an image and serve it locally. If I go this route, I would also use it to store other "strings" locally. If you suggest this, please give me the exact steps (could be pointers to other web sites) on how to do this. I would like to avoid paying for any flash development environment software ideally:)
Thank you!
You could use a data URI to display the file. Essentially you use the image data (plus a prefix) as the src attribute of an image element. If you already figured out how to read the file into memory as a base64 encoded string, using a data URI would probably be the easiest way to display the image.
OK
I was able to implement the following solution (just in case anyone has any comments or would like to know the answer)
wrote server side code that takes an image and returns its base 64 encoded version.
used the hidden iframe trick to get the base 64 encoded data into an iframe and stored it into the image by dynamically changing the image source to the data uri
For the "hidden iframe trick" - in case you are interested, there is a good article at (see www.openjs.com/articles/ajax/ajax_file_upload/response_data.php)
The only limitation is that IE does not work with images whose base 64 encoded string exceeds 32K - see http://msdn.microsoft.com/en-us/ie/dd578309.aspx. Note that only IE 8 works - IE 7 does not support data uri I believe.

Resources