Azure storage - Images url become binary? - image

I use the Azure storage to upload my image,
but some images will display by binary,
like : http://fungogo.blob.core.windows.net/asdf0/18263359_e0d9199e-b2d3-11e5-b71b-46c19c40c550.jpg
and some images will display by images on the browser
like : https://fungogo.blob.core.windows.net/images/14600328358_a00eaa35c5_o.jpg
I want to display the image on the browser instead of download, how can I fixed it?

I believe the issue is with the Content-Type. In the first block below contains the headers for the failing link. You can see it is listed as image/jpg/jpeg.
HTTP/1.1 200 OK
Content-Length: 137496
Content-Type: image/jpg/jpeg
Content-MD5: zPyz4CSRnPhQtW7PT1w9LQ==
Last-Modified: Wed, 03 Feb 2016 11:06:14 GMT
ETag: 0x8D32C8A086B77F5
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: e25e0d4c-0001-0049-5ff0-5ea774000000
x-ms-version: 2009-09-19
x-ms-lease-status: unlocked
x-ms-blob-type: BlockBlob
Date: Thu, 04 Feb 2016 02:08:52 GMT
The response headers for the working link have a Content-Type as image/jpeg.
HTTP/1.1 200 OK
Content-Length: 1689160
Content-Type: image/jpeg
Content-MD5: iAhgwODEpi7EaTAyUCMY1Q==
Last-Modified: Mon, 01 Feb 2016 08:18:24 GMT
ETag: 0x8D32AE0413D02B6
Server: Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0
x-ms-request-id: 9792c6d9-0001-0045-2cf0-5e4985000000
x-ms-version: 2009-09-19
x-ms-lease-status: unlocked
x-ms-blob-type: BlockBlob
Date: Thu, 04 Feb 2016 02:06:50 GMT
If you want to update the Content-Type of a lot of files at once, you can look at the example in the answer for this SO link at Set Content-type of media files stored on Blob
If you're wondering about the difference between jpg and jpeg, you can look at this SO link JPG vs. JPEG image formats

Related

webhdfs rest api throwing file not found exception

I am trying to open a hdfs file that is present on cdh4 cluster from cdh5 machine using webhdfs from the command line as below:
curl -i -L "http://namenodeIpofCDH4:50070/webhdfs/v1/user/quad/source/JSONML.java?user.name=quad&op=OPEN"
I am getting "File Not Found Exception" even if the file JSONML.java is present in the mentioned path in namenode as well as datanode and its trace is as follows:
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Date: Mon, 22 Feb 2016 13:25:35 GMT
Pragma: no-cache
Date: Mon, 22 Feb 2016 13:25:35 GMT
Pragma: no-cache
Set-Cookie: hadoop.auth="u=quad&p=quad&t=simple&e=1456183535737&s=KdZYcA5iwJeIU2F9ZJfLSaT4qMY=";Path=/
Location: http://n3.quadratics.com:50075/webhdfs/v1/user/quad/source/JSONML.java?op=OPEN&user.name=quad&namenoderpcaddress=n1.quadratics.com:8020&offset=0
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.cloudera.4)
HTTP/1.1 404 Not Found
Cache-Control: no-cache
Expires: Mon, 22 Feb 2016 13:26:28 GMT
Date: Mon, 22 Feb 2016 13:26:28 GMT
Pragma: no-cache
Expires: Mon, 22 Feb 2016 13:26:28 GMT
Date: Mon, 22 Feb 2016 13:26:28 GMT
Pragma: no-cache
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(6.1.26.cloudera.4)
{"RemoteException":{"exception":"FileNotFoundException","javaClassName":"java.io.FileNotFoundException","message":"File does not exist: /user/quad/source/JSONML.java\n\tat org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)\n\tat org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)\n\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)\n\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)\n\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)\n\tat org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)\n\tat org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:559)\n\tat org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)\n\tat org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)\n\tat org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)\n\tat org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)\n\tat org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)\n\tat org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)\n\tat org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)\n\tat java.security.AccessController.doPrivileged(Native Method)\n\tat javax.security.auth.Subject.doAs(Subject.java:415)\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)\n\tat org.apache.hadoop.ipc.Server$Handler.run(Server.java:2038)\n"}}
But I don't get any error and get the status of the above file when I use the below command:
curl -i -L http://namenodeIpofCDH4:50070/webhdfs/v1/user/quad/source/JSONML.java?user.name=quad&op=GETFILESTATUS"
I get the output response as below:
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Thu, 01-Jan-1970 00:00:00 GMT
Date: Mon, 22 Feb 2016 13:38:48 GMT
Pragma: no-cache
Date: Mon, 22 Feb 2016 13:38:48 GMT
Pragma: no-cache
Set-Cookie: hadoop.auth="u=quad&p=quad&t=simple&e=1456184328134&s=sE6esO8J39O+itl+ggNzX4/WzjQ=";Path=/
Content-Type: application/json
Transfer-Encoding: chunked
Server: Jetty(6.1.26.cloudera.4)
{"FileStatus":{"accessTime":1456147448567,"blockSize":134217728,"group":"quad","length":14849,"modificationTime":1456143798039,"owner":"quad","pathSuffix":"","permission":"644","replication":3,"type":"FILE"}}
Any ideas of the reason of why opening a file is failing and fixing that would be greatly appreciated.
I saw a similar error when I had misconfigured my /etc/hosts
The OPEN command above returns a redirect which provides a hostname. The localhost will try and resolve this hostname based on the local DNS setup.
It will then look for the file at the IP address that the hostname resolves to. Not necessarily the one you issued the command to.

WebHDFS OPEN command returns empty results

I created a simple file in HDFS at the path /user/admin/foo.txt
I can see the contents of this file in Hue.
How I issue the command
curl -i http://namenode:50070/webhdfs/v1/user/admin/foo.txt?op=OPEN
I get the response
HTTP/1.1 307 TEMPORARY_REDIRECT
Cache-Control: no-cache
Expires: Tue, 24 Nov 2015 16:20:15 GMT
Date: Tue, 24 Nov 2015 16:20:15 GMT
Pragma: no-cache
Expires: Tue, 24 Nov 2015 16:20:15 GMT
Date: Tue, 24 Nov 2015 16:20:15 GMT
Pragma: no-cache
Location: http://datanode:50075/webhdfs/v1/user/admin/foo.txt?op=OPEN&namenoderpcaddress=nameservice1&offset=0
Content-Type: application/octet-stream
Content-Length: 0
Server: Jetty(6.1.26.cloudera.4)
why is the content-length: 0?? I was hoping that this would list the contents of the file.
Execute:
curl -i http://datanode:50075/webhdfs/v1/user/admin/foo.txt?op=OPEN&namenoderpcaddress=nameservice1&offset=0
As for the explanation - when using WebHDFS to open a file you have to do the following:
You don't know which node the file resides on, so you ask the namenode.
The namenode returns you a datanode containing the file.
You can then open the file itself by talking directly to the datanode.
So this activity is expected. See https://hadoop.apache.org/docs/r1.0.4/webhdfs.html for more information.

NGINX + HHVM + Magento

I've been working on a development box deploying an Nginx, Magento, HHVM model. The site loads nicely and I don't get any real clear errors that I can see. However, a good number of my images are not loading properly. All I get are the image dimensions. The version of HHVM I'm using is:
HipHop VM 3.3.0-dev (rel)
Compiler: heads/master-0-g39d07abf36f7cad883d6be2a384a77c0c8aac040
Repo schema: 248cb6ce2d336c890fb26fd16690bf1b53b46994
Extension API: 20140829
The version of Nginx is:
nginx version: nginx/1.7.4
Some images load, and others don't. We're using timthumb for images, and I do see one error:
\nWarning: Is a directory in /home/user/shared/timthumb/index.php on line 90
If I run a curl from the command-line, I get the following result for an image, which means it's being seen as far as I can tell:
HTTP/1.1 200 OK
Content-Type: image/gif
Connection: keep-alive
Vary: Accept-Encoding
X-Powered-By: HHVM/3.3.0-dev
CF-Cache-Status: HIT
CF-RAY: 16513733df33085c-IAD
Server: cloudflare-nginx
Date: Fri, 05 Sep 2014 08:56:47 GMT
Expires: Sun, 05 Oct 2014 08:56:47 GMT
Cache-Control: public, max-age=2592000
Set-Cookie: __cfduid=d6c188df6595a443200995301f0b707981409907407975; expires=Mon, 23-Dec-2019 23:50:00 GMT; path=/; domain=.placehold.it; HttpOnly
Last-Modified: Fri, 19 Nov 2010 07:00:00 GMT
Cache-Control: max-age=31536000, public, must-revalidate, proxy-revalidate
I have a separate test server with the same content/database running NGINX/PHP-FPM, and the only variable I can see that's different in the curl is that when I run the curl against the dev server running php-fpm, the result is Content-Type: image/jpeg. When I run it against HHVM I get Content-Type: image/gif.
I would think this is related to rewrite rules, but I'm not sure. I also just discovered this error here as well in exception.log:
2014-09-05T12:04:06+00:00 ERR (3): Warning: open(tcp://127.0.0.1:11211?persistent=1&weight=2&timeout=10&retry_interval=10/sess_dd885307c2ba602e5faa7020c9c7c038, O_RDWR) failed: No such file or directory (2) in on line 0
Any advice would be greatly appreciated.

Setting Access-Control-Allow-Origin on Cloudfront

I am having problems serving static assets to Firefox using AWS Cloudfront.
Chrome works perfect, but Firefox is returning a CORS error.
If I execute curl , I get:
HTTP/1.1 200 OK
Content-Type: application/x-font-opentype
Content-Length: 39420
Connection: keep-alive
Date: Mon, 11 Aug 2014 21:53:50 GMT
Cache-Control: public, max-age=31557600
Expires: Sun, 09 Aug 2015 01:28:02 GMT
Last-Modified: Fri, 08 Aug 2014 19:28:05 GMT
ETag: "9df744bdf9372cf4cff87bb3e2d68fc8"
Accept-Ranges: bytes
Server: AmazonS3
Age: 2743
X-Cache: Hit from cloudfront
Via: 1.1 c445b20dfbf3128d810e975e5d84e2cd.cloudfront.net (CloudFront)
X-Amz-Cf-Id: ...
Which I think needs the header:
Access-Control-Allow-Origin: *
Can anyone help me? Why is it a problem on Firefox and not Chrome? How can I solve it?
Have you configured your distribution to support CORS by setting Origin header to be forwarded?
Reference:
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/header-caching.html#header-caching-web-cors

GZIP for CSS and JS dosen't work in Google Chrome, but does in Firefox

I have recently figured out how to enable GZIP (or Deflate) on my WAMP server that I will be using to serve my intranet application.
However, when testing in Google Chrome I see that the PHP file is compressed but Javascript files and CSS are not. The Response header shows that it is not compressed and Google Pagespeed confirms this.
Firefox on the other hand recives all files with compression without a problem.
Here are the headers for the main CSS file as an example:
Google Chrome
Date: Wed, 18 Jul 2012 14:48:43 GMT
Content-Length: 6533
Last-Modified: Wed, 18 Jul 2012 00:42:33 GMT
Server: Apache/2.2.21 (Win64) PHP/5.3.10
ETag: "a00000001509b-1985-4c50ff04b26ef"
Vary: Accept-Encoding
Content-Type: text/css
Accept-Ranges: bytes
200 OK
Firefox
Date: Wed, 18 Jul 2012 14:33:14 GMT
Server: Apache/2.2.21 (Win64) PHP/5.3.10
Last-Modified: Wed, 18 Jul 2012 00:42:33 GMT
Etag: "a00000001509b-1985-4c50ff04b26ef"
Accept-Ranges: bytes
Vary: Accept-Encoding
Content-Encoding: gzip
Content-Length: 1273
Content-Type: text/css
200 OK
Is this a problem with my WAMP setup, code, or is it just Google Chrome?
Thank you.
Google chrome does support JS/CSS gzip.
Request URL:http://d1o7y22ifnbryp.cloudfront.net/static/7793/css/all.min.css
Request Method:GET
Status Code:200 OK
Request Headersview source
Accept:text/css,*/*;q=0.1
Accept-Charset:GBK,utf-8;q=0.7,*;q=0.3
Accept-Encoding:gzip,deflate,sdch
Accept-Language:zh-CN,zh;q=0.8
Connection:keep-alive
Host:d1o7y22ifnbryp.cloudfront.net
User-Agent:Mozilla/5.0 (Windows NT 6.1) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.52 Safari/536.5
Response Headers
Accept-Ranges:bytes
Cache-Control:max-age=604800
Connection:keep-alive
Content-Encoding:gzip
Content-Length:17933
Content-Type:text/css
Date:Wed, 18 Jul 2012 15:43:46 GMT
ETag:"805dc-19c87-4c4a305735340"
Expires:Wed, 25 Jul 2012 15:43:46 GMT
Last-Modified:Thu, 12 Jul 2012 14:45:57 GMT
Server:Apache/2.2.22 (Amazon)
Vary:Accept-Encoding
I think the problem should be in apache configuration.

Resources