I am having HDFS in Ubuntu Environment, Is that possible to connect Ubuntu HDFS using C# application (Windows OS).
All the systems are connected via LAN.
I want to read simple CSV file from HDFS.
I want to know whether it is possible or not.
If you are using Hortonworks Azure HDInsight you can directly use C# to access the HDFS. In your case you are trying to read from windows OS. Please try using webhdfs. But it need configuration. Please check the below url for details.
URL: http://hadoop.apache.org/docs/r2.4.1/hadoop-hdfs-httpfs/
Related
My application is hosted on Amazon Elastic Compute Cloud by the developer. I need to retrieve the source code for my web application. I am a new user so I need to know how can i download the source codes in my local host.
You need to log into the instance using SSH. If you're familiar with SSH then you can SCP from your local machine.
Of you're not familiar, you can use Systems Manager and transfer the data to S3 then download from there:
https://aws.amazon.com/premiumsupport/knowledge-center/systems-manager-ssh-vpc-resources/
I can access my Hadoop File System from Web UI <IP address>:50070 . I want to access the same from a machine present outside of cluster. I tried to do it but it says webpage not found error. How can I access my HDFS from windows machine through a URL?
I have Hadoop v2.6 installed in my one PC in Ubuntu OS 14.04. I have added lots of unstructured data using Hadoop -put command into HDFS.
Can someone tell me how to download this data from another PC which is not in Hadoop Cluster using the Web User Interface provided by Hadoop??
I can access the data from other PC by typing in the address bar of browser (the IP address of HDFS server):Port Number
Like this: 192.168.x.x:50070
The problem is, I am not able to download the data as it gives the error "Webpage Not Available". I also tried other browsers, but still no luck.
Port 50070 is the default name node port. You should try port 14000 which is the default HttpFS port. If it still doesn't work try using the example from the manual:
http://192.168.x.x:14000?user.name=babu&op=homedir
Related: How connect to Hortonworks sandbox Hbase using Java Client API
I currently make a proof of concept using the Hortonworks Sandbox in a VM. However, I fail at properly configuring the client (outside the VM, but on the same computer). I looked for documentation as to how a client needs to be configured, but didn't find one.
I need client configuration for accessing HBase and MapReduce, but most appreciated would be a documentation that lists configuration for clients to all parts of the sandbox.
It is actually even more stupid than I would have expected. It seems that not all necessary ports are forwarded by default, it is necessary to add them all in the VM configuration.
Can any one please tell me, how can I upload a txt file from my local machine to another virtual machine based on IP Address in hdfs.
Regards,
Baskar.V
You might find webHDFS REST API useful. I have tried it to write content from my local FS to the HDFS and it works fine. But being REST based it should work fine from local FS of a remote machine as well, provided both machines are connected.