I am using an FTP server which, while listing the files using e.g. ls command, returns the last modification date in following format:
05/06/12
Is there a way to know what date format the remote server is using?
There is no definition about date format in the listings, it is not even defined that the listings should include the date at all. So you can only guess if you need to parse the listing. For a reliable and defined way to get the modification time of a file use the MDTM command. Unfortunately you need to send this for each file which could make everything slower.
Related
I need to monitor a log file that rotates every day in the same location. The format of the file is: filename.log.YY-MM-DD
To config rsyslog, I use wildcard to map filename.log.*, but I don't want to review old logs, just the actual day.
I tried to use date command in File parameter but its not recognized. Also with a variable is not recognized.
input(type="imfile"
File="/var/log/filename.log.*"
Tag="logstore""
Severity="info"
Facility="local4")
I expected just log the last filename.log.today, not all filename.log.*
Oracle's sqlldr defaults to a .dat extension. That I want to override. I don't like to rename the file. When googled get to know few answers to use . like data='fileName.' which is not working. Share your ideas, please.
Error message is fileName.dat is not found.
Sqlloder has default extension for all input files data,log,control...
data= .dat
log= .log
control = .ctl
bad =.bad
PARFILE = .par
But you have to pass filename without apostrophe and dot
sqlloder pass/user#db control=control data=data
sqloader will add extension. control.ctl data.dat
Nevertheless i do not understand why you do not want to specify extension?
You can't, at least in Unix/Linux environments. In Windows you can use the trailing period trick, specifying either INFILE 'filename.' in the control file or DATA=filename. on the command line. WIndows file name handling allows that; you can for instance do DIR filename. at a command prompt and it will list the file with no extension (as will DIR filename). But you can't do that with *nix, from a shell prompt or anywhere else.
You said you don't want to copy or rename the file. Temporarily renaming it might be the simplest solution, but as you may have a reason not to do that even briefly you could instead create a hard or soft link to the file which does have an extension, and use that link as the target instead. You could wrap that in a shell script that takes the file name argument:
# set variable from correct positional parameter; if you pass in the control
# file name or other options, this might not be $1 so adjust as needed
# if the tmeproary file won't be int he same directory, need to be full path
filename=$1
# optionally check file exists, is readable, etc. but overkill for demo
# can also check temporary file does not already exist - stop or remove
# create soft link somewhere it won't impact any other processes
ln -s ${filename} /tmp/${filename##*/}.dat
# run SQL*Loader with soft link as target
sqlldr user/password#db control=file.ctl data=/tmp/${filename##*/}.dat
# clean up
rm -f /tmp/${filename##*/}.dat
You can then call that as:
./scriptfile.sh /path/to/filename
If you can create the link in the same directory then you only need to pass the file, but if it's somewhere else - which may be necessary depending on why renaming isn't an option, and desirable either way - then you need to pass the full path of the data file so the link works. (If the temporary file will be int he same filesystem you could use a hard link, and you wouldn't have to pass the full path then either, but it's still cleaner to do so).
As you haven't shown your current command line options you may have to adjust that to take into account anything else you currently specify there rather than in the control file, particularly which positional argument is actually the data file path.
I have the same issue. I get a monthly download of reference data used in medical application and the 485 downloaded files don't have file extensions (#2gb). Unless I can load without file extensions I have to copy the files with .dat and load from there.
How would I use wget or curl to download the newest file in a directory?
This seems really easy, however the filename won't always be predictable, and as new data comes in it'll be replaced with a random filename.
Specifically, the directory I wish to download data from has the following naming structure, where the last string of characters is a randomly generated timestamp:
MRMS_RotationTrackML1440min_00.50_20160530-175837.grib2.gz
MRMS_RotationTrackML1440min_00.50_20160530-182639.grib2.gz
MRMS_RotationTrackML1440min_00.50_20160530-185637.grib2.gz
The randomly generated timestamp is in the format of: {hour}{minute}{second}
The directory in question is here: http://mrms.ncep.noaa.gov/data/2D/RotationTrackML1440min/
Could it have to be something with something in the headers, where you'd use curl to sift through the last-modified timestamp?
Any help would be appreciated here, thanks in advance.
You can just run following command periodically:
wget -r -nc --level=1 http://mrms.ncep.noaa.gov/data/2D/RotationTrackML1440min/
It will download recursively whatever is new in the directory after last run.
I want to make a stat via svn,and the input and output like this:
input:svn url,statDate,endDate
output:
person1 20Lines
persion2 30Lines
eg: my.sh http://svn.xxx.com/trunck/xxx/xxx/test.php 2014-12-01 2014-12-10
I use the shell script,and my idea like follows:
1.svn co the whole project
2.get the changed files within a specific date based on the giving url
3.use the command 'svn blame [file url]' get the changed infos by person
but now I don't know how to get the changed files in a directory within a specific date
please give me any ideas,thanks!
You don't want to use svn status, as this only shows if a file has changed in the working copy, whereas you are looking for commit info. You can, however, use
svn log -vq -r {2014-12-01}:{2014-12-10} http://svn.xxx.com/trunck/xxx/xxx/test.php
for your purposes. The -v adds information on the changed paths, the -q suppresses the commit message and only shows the changed paths.
You can then parse this output for the changed paths, user, and revision, and then use svn diff to find out for a given path how many lines were changed by that user in the specified revision.
I wrote a script that sync several servers at once. One of my problem is that I cannot get LFTP to preserve initial file modification time. So basically, it always upload all the files while only some changed.
Do you know how to force LFTP to preserve file modification time when downloading or uploading?
Thanks for any help.
Korchkidu
On the following page
http://www.bouthors.fr/wiki/doku.php?id=en:linux:synchro_lftp
the authors state:
When uploading, it is not possible to set the date/time on the files uploaded, that's why –>ignore-time is needed.
so if you use the flag combination --only-newer and --ignore-time you can achieve decent backup properties, in such a way that all files that differ in size are replaced. Of course it doesn't help if you really need to rely on time-synchronization but if it is just to perform a regular backup of data, it'll do the job.
try this
from here http://lftp.yar.ru/lftp-man.html:
ftp:use-site-utime (boolean)
when true, lftp sends 5-argument `SITE UTIME' command to set file modification time on
uploaded files. Default is true.
According to man lftp, I found this flag:
ftp:use-mdtm-overloaded (boolean)
when true, lftp uses two argument MDTM command to set file modification time on uploaded files. Default is false.
Thus, I set it to:
set ftp:use-mdtm-overloaded true
and it worked as expected.