I have read about Web Hdfs in all this links
link1 link2 link3
checking with telnet - the port is OPEN
made the needed settings in cloudera ( virtual box) to make webhdfs enable
testing directly on cloudera works:
[cloudera#quickstart ~]$ curl -i -L "http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN"
HTTP/1.1 307 TEMPORARY_REDIRECT
http://quickstart.cloudera:50075/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN&namenoderpcaddress=quickstart.cloudera:8020&offset=0
Content-Length: 0
Server: Jetty(6.1.26.cloudera.4)
HTTP/1.1 200 OK
Cache-Control: no-cache
Expires: Tue, 01 Dec 2015 14:08:07 GMT
Date: Tue, 01 Dec 2015 14:08:07 GMT
Access-Control-Allow-Methods: GET
Access-Control-Allow-Origin: *
Server: Jetty(6.1.26.cloudera.4)
trying directly on cloudera web browser also work for :
http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN
I am able to send PING C:\Users\User>ping 10.0.0.88
Telnet working for port 50070.
But when trying this command outside the cluster it dosnt work (HOST)
"http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN"
more info
LISTSTATUS works from HOST
http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFileop=LISTSTATUS&user.name=hadoop`
returns
{"FileStatuses":{"FileStatus":[
{"accessTime":1448982310759,"blockSize":134217728,"childrenNum":0,"fileId":26625,"group":"cloudera","length":47,"modificationTime":1448978300739,"owner":"cloudera","pathSuffix":"","permission":"644","replication":1,"type":"FILE"}
]}}
but op=OPEN from HOST
"http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN&user.name=hadoop"
returns
This webpage is not available
what could be the problem ?
It seems there is syntax problem in
http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?`op=OPEN&user.name=hadoop`
Try
http://10.0.0.88:50070/webhdfs/v1/user/cloudera/testdir/testFile?op=OPEN
Related
Why does Tomcat version 7.0.88 gives "400 Bad Request" error code if the hostname ends with xyx.com-abc.
For testing purpose let's assume we have the following entry in the hosts file
127.0.0.1 hello.hello.hello-erq
And we try to access this url from curl
curl -v hello.hello.hello-er:8080
We get the following output
* Rebuilt URL to: hello.hello.hello-er:8080/
* Trying 127.0.0.1...
* Connected to hello.hello.hello-er (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: hello.hello.hello-er:8080
> User-Agent: curl/7.49.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Server: Apache-Coyote/1.1
< Transfer-Encoding: chunked
< Date: Thu, 20 Dec 2018 19:53:09 GMT
< Connection: close
<
* Closing connection 0
While using the localhost in the url we get
C:\playground\apache-tomcat-7.0.88-windows-x64\apache-tomcat-7.0.88\bin>curl -v localhost:8080
* Rebuilt URL to: localhost:8080/
* Trying ::1...
* Trying 127.0.0.1...
* Connected to localhost (127.0.0.1) port 8080 (#0)
> GET / HTTP/1.1
> Host: localhost:8080
> User-Agent: curl/7.49.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: Apache-Coyote/1.1
< Content-Type: text/html;charset=ISO-8859-1
< Transfer-Encoding: chunked
< Date: Thu, 20 Dec 2018 20:00:07 GMT
<
<!DOCTYPE html>
All the tomcat configurations are the same for both the tests and the same issue is replicated on a vanilla out of the box tomcat server too.
I tried to replicate the same issue on tomcat-8 but both the url's worked fine there. How can i dig deeper and find out the root cause of this issue in 7.0.88 ?
Is there some additional logging which i can enable to get more on this issue ?
Or the only thing i have left is to pull my hairs and upgrade ?
Tomcat was attempting to enforce the domain name specification by refusing your hostname with a hyphen in the TLD. This was deemed a bug in Tomcat and fixed in 7.0.89 (and versions of Tomcat 8.0.x, 8.5.x and 9.0.x released around the same time).
So it seems that all you need is a small version bump.
I updated ddev to version 1.3.0 and ran ddev config. After that I changed the configuration from nginx-fpm to apache-fpm. After starting ddev and checked the HTTP headers, there is nginx/1.15.3 used. Is there something else to do, to get Apache working?
My config.yaml:
APIVersion: v1.3.0
name: example
type: typo3
docroot: public
php_version: "7.2"
webserver_type: apache-fpm
router_http_port: "8080"
router_https_port: "8443"
xdebug_enabled: true
additional_hostnames: []
additional_fqdns: []
provider: default
hooks:
post-start:
- exec: composer install -d /var/www/html
- exec: ../vendor/bin/typo3cms cache:flush
- exec: ../vendor/bin/typo3cms database:updateschema
- exec: yarn --cwd typo3conf/ext/theme/Resources/Private install
That is such a good question! I know because I already got stumped by it myself when writing tests.
The answer is: Apache runs in the web container but when you use the http://*.ddev.local URL, it goes through ddev-router, which is an nginx reverse proxy, and that's why you see the nginx headers. But rest assured you are using Apache. You can confirm that these ways:
ddev ssh and ps -ef to see what's running
Hit the 127.0.0.1 URL reported by ddev start and ddev describe. That URL goes directly to the web container, for example http://127.0.0.1:33221 - You'll see the apache headers on that one.
Your question is so good - Could you please edit the title to something like "Why do I see nginx headers when ddev is configured to use apache?" - I think other people will find it that way.
$ curl -I http://127.0.0.1:33224
HTTP/1.1 200 OK
Date: Fri, 12 Oct 2018 02:18:26 GMT
Server: Apache/2.4.25 (Debian)
Cache-Control: must-revalidate, no-cache, private
X-Drupal-Dynamic-Cache: HIT
X-UA-Compatible: IE=edge
Content-language: en
X-Content-Type-Options: nosniff
X-Frame-Options: SAMEORIGIN
Expires: Sun, 19 Nov 1978 05:00:00 GMT
Vary:
X-Generator: Drupal 8 (https://www.drupal.org)
X-Drupal-Cache: MISS
Content-Type: text/html; charset=UTF-8
I started experimenting with Traefik on Windows 10 and I need assistance with some very basic steps. Two are my main stumbling blocks: configuring Traefik natively on Windows 10 and http2 support.
1) Native Traefik on Windows
Following the instructions at https://docs.traefik.io/ I successfully managed to install the docker image of Traefik and see the dashboard on localhost:8080
However, when I try to start Traefik natively as "traefik -c traefik.toml" Traefik seems to "hang" no matter what I put in traefik.toml.
Can anybody show me a minimal Windows 10 configuration (.toml) to achieve what I achieve using the Docker version of Traefik?
2) HTTP2 support
Following the test shown at https://docs.traefik.io/#test-it, I execute this on the command line:
curl -v --http2 -H Host:whoami.docker.localhost http://127.0.0.1
but on the output I see:
< HTTP/1.1 200 OK
< Content-Length: 327
< Content-Type: text/plain; charset=utf-8
< Date: Wed, 05 Jul 2017 18:51:00 GMT
Does it mean that the response back is http 1.1 ?
Any help is much appreciated.
I have checked it at: http://www.ipchecking.com/ and they say they are the same. but when I visit each of them they are different
ec2-54-206-38-225.ap-southeast-2.compute.amazonaws.com - 404 error Problem accessing /. Reason: Not Found
54.206.38.225 - returns apache default page
ec2-54-206-38-225.ap-southeast-2.compute.amazonaws.com/jenkins - jenkins launchs
54.206.38.225/jenkins - not found
My understanding was that the host name should resolve to the ip address and thus they shouldboth take me to the same place?
What you are probably seeing is due to named based virtual hosts.
When your browser makes an HTTP request, it includes a header that says what host it is looking for. This allows a server to have more than 1 site hosted on a single IP address and port.
This can also allow a load balancer to redirect your traffic to different machines on its network for handling.
You can find more information at
https://en.wikipedia.org/wiki/Virtual_hosting
http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html
Check this:
# curl -I -s http://ec2-54-206-38-225.ap-southeast-2.compute.amazonaws.com | head -3
HTTP/1.1 404 Not Found
Date: Tue, 05 Jan 2016 06:15:49 GMT
Server: Jetty(winstone-2.9)
# curl -I -s http://54.206.38.225 | head -3
HTTP/1.1 200 OK
Date: Tue, 05 Jan 2016 06:16:00 GMT
Server: Apache/2.4.7 (Ubuntu)
# curl -I -s http://ec2-54-206-38-225.ap-southeast-2.compute.amazonaws.com/jenkins | head -3
HTTP/1.1 302 Found
Date: Tue, 05 Jan 2016 06:16:18 GMT
Server: Jetty(winstone-2.9)
# curl -I -s http://54.206.38.225/jenkins | head -3
HTTP/1.1 404 Not Found
Date: Tue, 05 Jan 2016 06:16:28 GMT
Server: Apache/2.4.7 (Ubuntu)
From above commands (look at the HTTP response code closely),
When FQDN is used, the HTTP request is responded by JeTTy.
When IP Address is used, the HTTP request is responded by Apache.
Jetty is aware of /jenkins path
Apache is not aware of /jenkins path.
So this implies that, You have JeTTy acting as Reverse_Proxy/Load_Balancer. So the connection looks like this:
USER --> JeTTy --> Apache
Now, you need to figure out how JeTTy is configured for redirecting/denying requests. This link might be helpful.
However, Usually, I have seen that an application server is fronted by a web server as Reverse_Proxy/Load_Balancer. So, you might find that your setup looks like below:
USER --> Apache --> JeTTy
If this is the case then figure out how Apache is configured for redirecting/denying requests.
I have a problem with HttpFS service of hadoop.When i try to curl some resource:
curl -i http://192.168.4.180:14000/webhdfs/v1/user/hadoop/?op=LISTSTATUS
the response i get is:
HTTP/1.1 404 Not Found
Server: Apache-Coyote/1.1
Content-Length: 0
Date: Mon, 17 Aug 2015 08:57:47 GMT
But when i try to do the same with webhdfs it works:
curl -i http://192.168.4.180:50070/webhdfs/v1/user/hadoop/?op=LISTSTATUS
HTTP/1.1 200 OK
and so on ....
Httpfs service is running on port 14000, i have checked it via nmap.
Any suggestions or ideas what might be the problem?
I meet the same problem. And solved by this link
In brief, you need to add the symbol link:
# Point to the 'webapps' in current.
cd /etc/hadoop-httpfs/tomcat-deployment
ln -s /usr/hdp/current/hadoop-httpfs/webapps webapps