I have written a batch script that logs into my ftp server, then navigates to a directory. I am having trouble with the mget command, I want it to download every .dat file in the directory, but it simply returns this error:
Cannot access file '/home/minecraft/multicraft/servers/server267/world/players/*.dat':No such file or directory.
200 Type set to: ANSI
Cannot find list of remote files
Here is my script (ran from cmd)
open 66.71.244.202
USER
PASSWORD
cd /world
cd players
mget *.dat
That is by design. The most recent update to the FTP specification (RFC 3659) explicitly forbids it (see section 2.2.2):
For the commands defined in this specification, all pathnames are to be treated literally. That is, for a pathname given as a parameter to a command, the file whose name is identical to the pathname given is implied. No characters from the pathname may be treated as special or "magic", thus no pattern matching (other than for exact equality) between the pathname given and the files present in the NVFS of the server-FTP is permitted.
Clients that desire some form of pattern matching functionality must obtain a listing of the relevant directory, or directories, and implement their own file name selection procedures.
When you execute your script file with ftp, you have to turn off the globbing which will allow the use of wildcards in the script. For example:
ftp -n -i -s:scriptfile.txt
should work but
ftp -n -i -g -s:scriptfile.txt
will not.
I know this is old, but it might help someone. I had the same issue with wildcards on MGET from Windows FTP, but it was not consistent in that it worked talking to some remote systems, but not to all of them.
My script was doing this:
cd /folder/folder
mget ./-400TA/folder/*_XYZ
In the folder structure I have a set of different folders that begin with hyphens, and for whatever reason the script CD's down to just above there, and uses the relative path in the MGET. I had the same issue that some of you reported, that if I connected interactively and typed the commands one by one, it worked. But in batch, it didn't.
I followed the suggestions in this and other posts, but no joy. I don't have access to the remote systems at the moment to look at them to figure out why some worked and some didn't.
However, what I did find was this. Changing my script as follows:
cd /folder/folder/-400TA/folder
mget *_XYZ
did the trick. Simple. There's some strange interaction going on somewhere possibly with folder protections or something, but it just shows that trying out different things may get you there in the end.
I would make sure glob is on, when turned off the file name in the put and get commands is taken literally and wildcards will not be looked at.
More info:
glob:Toggle filename expansion for mdelete, mget and mput. If globbing
is turned off with glob, the file name arguments are taken literally
and not expanded. Globbing for mput is done as in csh. For mdelete and
mget, each remote file name is expanded separately on the remote
machine and the lists are not merged. Expansion of a directory name is
likely to be different from expansion of the name of an ordinary file:
the exact result depends on the foreign operating system and ftp
server, and can be previewed by doing ‘mls remote-files -’ Note: mget
and mput are not meant to transfer entire directory subtrees of files.
That can be done by transferring a tar archive of the subtree (in
binary mode).
Once you are inside your ftp try to check the glob and set it on if it is off. The default behaviour is on, from the command line when connecting to ftp with the option -g you can turn off the file name globbing.
It could very well also be a firewall issue where it is not permitting or forwarding the servers inbound connection. Happened to me.
Related
I'm currently practicing basic Shell Commands in WSL, Windows Subsystem for Linux (I do not have a linux system but I want to get familiar with commands).
I start a bash session on the command prompt window and navigate to my desktop using cd . In desktop I noticed that after using ls -lF some files with the prefix ~$ appear, such as: '~$executable.x'* or '~$file.txt'
These files are not currently present under the desktop directory, but I was able to remember that they were at one point (varying from a week to months ago).
When I do the same process in powershell windows (not using linux commands) I noticed that files displayed match the desktop and no extra files are listed.
I was wondering if anyone could explain what ~$ means in this context? my intuition is telling me they are backed up files that are somehow hidden in the desktop. After googling, all I could find is that ~ reefers to the home. I also understand that $ is the default prompt symbol for the bash shell when it is waiting for me to type something, but I'm still confused on why it would show up as a prefix for the name of a file.
Hope I made my question clear.
I'm currently reading "Linux® Command Line and Shell Scripting BIBLE" by Blum and Bresnahan but I could not find an answer there, this is my last resource after many googling attempts. Any other source for more information on the topic would be helpful.
On Windows, files that start with ~ are used for hidden files. More specifically,, the prefix ~$ are often used as backups for programs, should they crash before writing updates to a file (e.g. Microsoft Word, etc.)
From Wikipedia:
The tilde symbol is used to prefix hidden temporary files that are created when a document is opened in Windows. For example, when you open a Word document called “Document1.doc,” a file called “~$cument1.doc” is created in the same directory. This file contains information about which user has the file open, to prevent multiple users from attempting to change a document at the same time.
See: Why does Word make temporary files?
Relevant superuser question: https://superuser.com/questions/405257/what-type-of-file-is-file
I'm building an production site update script using expect and lftp.
I'm trying for sometime to do this simple thing: send a file overwriting if it exists. But I just can't find a command that allow me to overwrite destination on put. put -c just continues if it's the same file. But if it's different it does nothing.
Do I really have to check if the file exists and delete it in order to put the file in the server ? isn't there a direct command ?
If that's the only option, then there's another thing I couldn't find: a command to see if the file exists. My only option is to treat ls's output ?
I also accept recommendations if there's another ftp client that allows me to do these things in an easier manner.
LFTP offers a parameter for put which is a lowercase e.
This allows you to remove the file before uploading it, as opposed to an overwrite option (which LFTP doesn't appear to inherently support). Things like prompt could be available, but if we're strictly discussing LFTP, I would recommend using that option, ala:
put -e file.txt
From the LFTP man pages.
There are multiple folders with subfolders and image files on the FTP server. The -R is disabled. I need to dump the recursive directory listing with the path name in a text file. The logic I have till now is that, traverse in each folder, check the folder name if it consists of '.' to verify it as a file or a folder, if its a folder, go in and check for subfolders or files and list them. Since I cannot go with the -R, I have to go with a function to perform traverse each folder.
#!/bin/sh
ftp_host='1.1.1.1'
userName='uName'
ftp -in <<EOF
open $ftp_host
user $userName
recurList() {
path=`pwd`
level=()
for entry in `ls`
do
`cwd`
close
bye
EOF
I am stuck with the argument for the for loop!
Sorry to see you didn't get any replies yet. I think the reason may be that Bash isn't a good way to solve this problem, since it requires interacting with the FTP client, i.e. sending commands and reading responses. Bash is no good at that sort of thing. So there is no easy answer other than "don't use Bash".
I suggest you look at two other tools.
Firstly, you may be able to get the information you want using http://curlftpfs.sourceforge.net/. If you mount the FTP server using curlftpfs, then you can use the find command to dump the directory structure. This is the easiest option... if it works!
Alternatively, you could write a program using Python with the ftplib module: https://docs.python.org/2/library/ftplib.html. The module allows you to interact with the FTP server through API calls.
Is it possible to have two wildcards? If not, is there another way of going about this problem?
I am trying to recursively get a file type from an ftp server using Bash. But, what I am having trouble with is that I am trying to remove files from multiple directories. Many of these directory names will have matching strings. The client will look for all directories with the matching string and get a certain file type from each. Each directory can have many files with the same extension.
What I have tried to do is use wget recursively.
wget -r 'ftp://anonymous:#$HOST/$PATH/$DIRSTRING*/*.$FILEEXT
This gives me an error message saying the $PATH/$DIRSTRING*/ file or directory could not be found.
I know wget supports globbing. But, is it possible to have two wildcards? If not, is there another way of going about this problem?
Best Regards
wget is not really suited for this kind of ftp usage...but lftp is very good at mirroring ftp site data, it even supports globbing! :)
for your example:
lftp -e "mirror -I '$DIRSTRING*/*$FILEEXT' /$RPATH mirrorSite" ftp://anonymous#$HOST
see man lftp / mirror command
Is it possible to use a wildcard in scp
I am trying to achieve:
loop
{
substitue_host (scp path/file.jar user#host:path1/foo*/path2/jar/)
}
I keep on getting "scp: ambiguous target"
Actually I am calling an api with source and dest that uses scp underneath and loops over diff hosts to put files
Thanks!
In general, yes, it is certainly possible to use a wildcard in scp.
But, in your scp command, the second argument is the target, the first argument is the source. You certainly cannot copy a source into multiple targets.
If you were trying to copy multiple jars, for example, then the following would certainly work:
scp path/*.jar user#host:path2/jar/
"ambigious target" in this case is specifically complaining that the wildcard you're using results in multiple possible target directories on the #host system.
--- EDIT:
If you want to copy to multiple directories on a remote system and have to determine them dynamically, a script like the following should work:
dir_list=$(ssh user#host ls -d '/path1/foo*/path2/jar/')
for dir in $dir_list; do
scp path/file.jar user#host:$dir
done
The dir_list variable will hold the results of the execution of the ls on the remote system. The -d is so that you get the directory names, not their contents. The single quotes are to ensure that wildcard expansion waits to execute on the remote system, not on the local system.
And then you'll loop through each dir to do the remote copy into that directory.
(All this is ksh syntax, btw.)