I am working on Big data and want to fetch data/files from a file system present in remote machine not from any databases i.e to copy files from remote linux / windows machine to hdfs. Please help me.
You can copy file from your local file to a remote file system and vice versa by using
scp commands
scp [Options] [[User#]From_Host:]Source_File [[User#]To_Host:][Destination_File]
Try to install Hue. It's a very useful tool for this.
Follow this link for more details : http://gethue.com/
Enter into the File Browser in Hue, and click on "Add files" button then you can upload your files into HDFS
Related
enter image description here
I have windows with linux subsystem and I am trying to run druid. I am getting a message CANNOT CREATE FIFO. What should I do to avoid it?
I faced the same issue myself. Was trying to run through WSL Ubuntu. It seems that FIFO file can't be created over mounted drive i.e /mnt/c/.
Workaround for this, you'd have to copy the entire Druid installation folder over to any internal folder eg. /usr/share/ and launch from there.
So my class Project3 prints a load of data to the console of PuTTY when I run it, and I've used "java Project3 > variance.data" to create the file variance.data in the pwd.
I think this has worked because when i use "ls" variance.data is there.
However I want to save this file onto my computer, how do I do this?
You can use PuTTy Secure Copy to download the file.
Open CMD on your Windows machine and use the following command:
pscp <user>#<Remote Server IP>:/path/to/file .
You will then be prompted for the password of the user you are trying to connect as.
You can use FileZilla to connect to the server with your credential (the one you use with PuTTY) and then download the file you need only double-clicking on it. In the image you'll fine it in the 5th rectangle. At the same way, you can easily upload a file double-clicking on in the 4th rectangle.
How to download file(s) from remote server directory to local machine in PuTTY ?
I got the command for inserting file to remote directory from local machine. But it is not working for me though there is no error message.
pscp c:\documents\foo.txt fred#example.com:/tmp/foo
(Question is probably more suited to Superuser)
You have your parameters in the wrong order. Please refer to the documentation:
https://the.earth.li/~sgtatham/putty/0.70/htmldoc/Chapter5.html#pscp
To download, you need:
pscp [options] [user#]host:source target
What you have there is the opposite, it's for doing an upload.
I have a jenkins Job in which I build my application on Unix box by making SSH connection to it and run the build script on Unix box. My Jenkins is on windows I want to generate PMD reports on windows so, i want to copy pmd.xml file from Unix box to Windows machine.
Right now my windows Jenkins workspace is empty because I do SVN checkout, Build and generation of binaries on Unix box. for this purpose I am using Jenkins SSH plugin.
Is there any way to copy Unix contents back to Windows Jenkins Workspace?
I have tried Jenkins SCP plugin but confused with source and SCP site. Please suggest.
Thanks in Advance.
You can use plink.exe and pscp.exe to interface from Windows to Linux. These should be part of your PuTTY distribution.
However, if your slave is already connecting to the Linux machine, you can use the same connection to bring the file back to slave's workspace. Then use Copy to Slave plugin (nevermind the name, it does both directions) to move the file from slave to master workspace
I Added Linux machine as slave to windows master and then used Copy-to-Slave plugin to copy generated reports from linux to windows workspace its working fine.
I do a lot of work with files hosted on an FTP server. Currently to edit a file on the server I have to open the server in Cyberduck, navigate with the mouse to the folder I want and then click "Edit," which opens a temporary file. Anyway, editing files on the server would be way easier if I could use the terminal to navigate through the file directory and edit files. Is there a way to create a symbolic link in my home directory to an FTP server?
edit: I'm on a Mac
If your using linux you can mount an ftp site with curlftpfs. Check out this tutorial: Ubuntu Geeks Curlftpfs
If you are on windows you can use NetDrive:
NetDrive.Net
With either of these you can create links/shortcuts to your hearts content.