WebHdfsFileSystem local ip vs network ip hadoop - hadoop

Question

have a requirement to read HDFS from a outside of the hdfs cluster. I stumbled upon WebHdfsFileSystem and even though I got the idea but I could not make it work with the network address. For example, the code below works fine as long as I use 127.0.0.1 or localhost. But the moment I use the network ip address 192.168.. , I get "Retrying connect to server" messages followed by ConnectException.
import java.io.IOException;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.hdfs.web.WebHdfsFileSystem;
public class ReadHDFSFile {
public static void main(String[] args) {
Path p = new Path("hdfs://127.0.0.1:9000/user/hduser");
WebHdfsFileSystem web = new WebHdfsFileSystem();
try {
Configuration conf = new Configuration();
conf.set("fs.default.name","hdfs://127.0.0.1:9000/");
web.setConf(conf);
Configuration conf1 = web.getConf();
FileSystem fs = FileSystem.get(web.getConf());
System.out.println(fs.exists(p));
} catch (IOException e) {
e.printStackTrace();
}
}
}
I am not sure what am I missing here.

Answer 1

I have a version of this working on Hadoop 2.4. I had to change two things relative to using the regular Hadoop FileSystem API:
the protocol changes from hdfs:// to webhdfs://
the port changed to the http-port (which on our Hortonworks cluster is 50070), not the default hdfs port (which might also be called the RPC port?), which is 8020 on our system
Example code that works for me:
Configuration conf = new Configuration();
String conxUrl = String.format("webhdfs://%s:%s", NAMENODE_IP_ADDR, WEBHDFS_PORT);
conf.set("fs.defaultFS", conxUrl);
FileSystem fs = WebHdfsFileSystem.get(conf);
Path path = new Path("/path/to/my/file");
System.out.println(fs.exists(path));