As we all know, "WC -l" command is used to get number of lines of a file,but i couldn't able to use the same command over ftp server.
Please help me out with this.
You can't run arbitrary shell commands over FTP, you can only run those commands the FTP server accepts.
I'm not aware of any common FTP commands to get the line count of a file. You may want to use SSH instead if possible. Otherwise, you would have to download the file to get the line count.
We would like to transfer a file everyday from a Shared location to remote location.
I was informed to use following script:
ASCII
QUOTE SITE RECFM=FB LRECL=750 BLKSIZE=23250 FILE=SEQ CYL PRI=50 SEC=20
PUT filename ‘filename(+1)'
BYE
Also, was recommended to use MOVEit Freely 5.5.0.0 – Secure FTP client.
Is the script above has to be written in Mainframe or in .dat file ?
Using text editor, put the ftp commands, one command per line.
ASCII
QUOTE SITE RECFM=FB LRECL=750 BLKSIZE=23250 FILE=SEQ CYL PRI=50 SEC=20
PUT filename ‘filename(+1)'
BYE
Let's pretend the script filename you made is send.txt, you can then run the command ftps like:
ftps -s:send.txt hostname
I am trying to move all files under a folder to SFTP folder using shell script for my batch job. But every time it runs only few files are moved. Not all the files.
/usr/local/bin/expect -c "spawn sftp -o Identityfile="/export/home/user/.ssh/example.ppk" $SFTP_USERID#$SFTP_SERVER
expect \"password: \"
send \"$PASSWORD\n\"
expect \"sftp> \"
send \"cd $DESTDIR\r\"
expect \"sftp> \"
send \"lcd $LOCALDIR\r\"
expect \"sftp> \"
send \"put *.* \r \"
expect \"sftp> \"
send \"quit\r\"
expect \"sftp> \"" >> $BATCH_DIR/logs/batch"$todaydatetime".log
This script runs every time succesfully but only few files are moved to SFTP destination folder. In Logs i am always seeing only 19 Files are uploaded from local folder to SFTP Folder(every time same files).
I understand why every time same files but i am not able to figure out why only few files.
Is there any limit on time that SFTP command will be active?
Kindly also help me how can i change the command to take only new files. "rsynch" is not working.
Hi kenster, mput didn't work for me. Files that are transferred are files under my local folder started with numbers. In my local folder there are 236 files of which 19 files that are starting with numbers are getting transferred even if there are spaces in file name or file extension is may be pdf or xls or what ever but always same 19 files are transferred MEANING not a single file that starts with alphabet is transferred. I tried the same steps manually to check whether file names/ permissions are causing some issue but manually is working fine : ( all files are transferred.
Sorry Guys my mistake.
I just added below lines after lcd step. Now only 6 files are transferred.
expect \"sftp> \"
send \"lls -ltr\r\"
Issue looks like something else not with commands or file names.
I echo date '+%Y%m%d%H%M%S' before and end of all steps. the programs executed only 12 secs everytime.THis should be some with my environment.I am working in restricted environment. Thanks Guys. But still can help me how to pick only new files(moving old files to backup folder is not accepted by my boss).
Change the line:
send \"put *.* \r \"
to
send \"mput * \r \"
as PUT *.* is an ugly Windows-ism.
You should also consider putting double quotes around $DESTDIR and $LOCALDIR in case they contain spaces.
You could try using sshfs and then use "regular" commands like cp, which should give you more options and which should behave like a regular filesystem.
sshfs $SFTP_USERID#$SFTP_SERVER: /temporary/mount/path
cp -R $LOCALDIR/* /temporary/mount/path/$DESTDIR
fusermount -u /temporary/mount/path/
I have written a batch script that logs into my ftp server, then navigates to a directory. I am having trouble with the mget command, I want it to download every .dat file in the directory, but it simply returns this error:
Cannot access file '/home/minecraft/multicraft/servers/server267/world/players/*.dat':No such file or directory.
200 Type set to: ANSI
Cannot find list of remote files
Here is my script (ran from cmd)
open 66.71.244.202
USER
PASSWORD
cd /world
cd players
mget *.dat
That is by design. The most recent update to the FTP specification (RFC 3659) explicitly forbids it (see section 2.2.2):
For the commands defined in this specification, all pathnames are to be treated literally. That is, for a pathname given as a parameter to a command, the file whose name is identical to the pathname given is implied. No characters from the pathname may be treated as special or "magic", thus no pattern matching (other than for exact equality) between the pathname given and the files present in the NVFS of the server-FTP is permitted.
Clients that desire some form of pattern matching functionality must obtain a listing of the relevant directory, or directories, and implement their own file name selection procedures.
When you execute your script file with ftp, you have to turn off the globbing which will allow the use of wildcards in the script. For example:
ftp -n -i -s:scriptfile.txt
should work but
ftp -n -i -g -s:scriptfile.txt
will not.
I know this is old, but it might help someone. I had the same issue with wildcards on MGET from Windows FTP, but it was not consistent in that it worked talking to some remote systems, but not to all of them.
My script was doing this:
cd /folder/folder
mget ./-400TA/folder/*_XYZ
In the folder structure I have a set of different folders that begin with hyphens, and for whatever reason the script CD's down to just above there, and uses the relative path in the MGET. I had the same issue that some of you reported, that if I connected interactively and typed the commands one by one, it worked. But in batch, it didn't.
I followed the suggestions in this and other posts, but no joy. I don't have access to the remote systems at the moment to look at them to figure out why some worked and some didn't.
However, what I did find was this. Changing my script as follows:
cd /folder/folder/-400TA/folder
mget *_XYZ
did the trick. Simple. There's some strange interaction going on somewhere possibly with folder protections or something, but it just shows that trying out different things may get you there in the end.
I would make sure glob is on, when turned off the file name in the put and get commands is taken literally and wildcards will not be looked at.
More info:
glob:Toggle filename expansion for mdelete, mget and mput. If globbing
is turned off with glob, the file name arguments are taken literally
and not expanded. Globbing for mput is done as in csh. For mdelete and
mget, each remote file name is expanded separately on the remote
machine and the lists are not merged. Expansion of a directory name is
likely to be different from expansion of the name of an ordinary file:
the exact result depends on the foreign operating system and ftp
server, and can be previewed by doing ‘mls remote-files -’ Note: mget
and mput are not meant to transfer entire directory subtrees of files.
That can be done by transferring a tar archive of the subtree (in
binary mode).
Once you are inside your ftp try to check the glob and set it on if it is off. The default behaviour is on, from the command line when connecting to ftp with the option -g you can turn off the file name globbing.
It could very well also be a firewall issue where it is not permitting or forwarding the servers inbound connection. Happened to me.
Hi I want to LOOP through files on FTP and copy one by one. Every thing is fine with FTP connection and accessing folders.
My question is How can loop through all files on FTP. It looks like there is no "For" type of functionality available to access FTP files because each line is considered as complete command.
open MyServerName 21
MyUserName
MyPassword
lcd E:\LocalDirectory
cd /FTPDirectory/upload
I WANT TO LOOP THROGH ALL FILES AND COPY ONE BY ONE TO LOCAL DIRECTORY
disconnect
bye
Why i want to loop through all files in FTP is, I want to copy only those files which are not locked and available for copy.
Use a different FTP client: wget.
With the -m option (for --mirror), use the following in a script
cd mylocaldirectory
wget -m ftp://username:password#hostname/theremotedirectory