There are some files in /var/log as below:
cron
cron-20200322
maillog-20200329
tallylog
bootscan.log
bootscan.log-20200115.gz
bootscan.log-20200116.gz
I want to backup files:cron;cron-20200322;bootscan.log;bootscan.log-20200115.gz;bootscan.log-20200116.gz to a single file /mnt/log_backup.tar
Then,I witer bash shell script as below:
#!/bin/bash
backup_dir='/mnt'
tar -cpf $backup_dir/log_backup.tar -C /var/log cron*
tar -cpfr $backup_dir/log_backup.tar -C /var/log boot*
Sadly, I got error as below:
tar: cron*: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
tar: Removing leading `/' from member names
tar: boot*: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
What's the problem in my shell script?
The globbing of * is evaluated before changing the folder to /var/log.
Check How to make tar globbing work with the 'change directory' option
find . /var/log -maxdepth 1 -type f -name "boot*" -print0 | tar -cpf $backup_dir/log_backup.tar --null -T -
find . /var/log -maxdepth 1 -type f -name "cron*" -print0 | tar -rpf $backup_dir/log_backup.tar --null -T -
I think it's the correct answer,maybe not the best one.
Pls contribute you 2 cents if any better solution,thks!
Related
I get error printouts when I use methods of (1) xargs find | xargs mv and (2) xargs find -exec mv to move files, but the moving of the files works as expected. I get no error printouts when I use (3) an intermediate variable, which is what I want.
Finding the inode numbers, which is what I'm doing with stat and sed, shouldn't be related to the problem.
I have condensed my problem into the following. Basically I want to get rid of the error printouts in both of Methods 1 and 2 (no piping to /dev/null please).
Note I'm on macOS, using BSD stat, and I'm also using zsh.
And I also know about find -print0 | xargs -0, I just wanted to condense the problem so that it's not as long. This is as condensed as I want to get.
Setup
Make test subdirectories (which are later moved by inode number).
mkdir -p "./testdir/dir"{1..4}
Method 1 - xargs find | xargs mv
Works to move the subdirectories, but with error printouts.
stat -s ./testdir/* | sed -E -n 's/.*st_ino=([0-9]+).*/\1/p' | xargs -I abc find ./testdir/* -maxdepth 1 -inum abc | xargs -I {} mv {} ~/.Trash
Error printouts:
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir2: No such file or directory
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir2: No such file or directory
find: ./testdir/dir3: No such file or directory
Method 2 - xargs find -exec mv
Works to move the subdirectories, but with error printouts.
stat -s ./testdir/* | sed -E -n 's/.*st_ino=([0-9]+).*/\1/p' | xargs -I abc find ./testdir/* -maxdepth 1 -inum abc -exec mv {} ~/.Trash \;
Error printouts:
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir2: No such file or directory
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir2: No such file or directory
find: ./testdir/dir3: No such file or directory
find: ./testdir/dir1: No such file or directory
find: ./testdir/dir2: No such file or directory
find: ./testdir/dir3: No such file or directory
find: ./testdir/dir4: No such file or directory
Method 3 - intermediate variable
Works to move the subdirectories, but this time without error printouts, which is what I want.
dirs_to_move=$(stat -s ./testdir/* | sed -E -n 's/.*st_ino=([0-9]+).*/\1/p' | xargs -I abc find ./testdir/* -maxdepth 1 -inum abc)
printf '%s\n' "$dirs_to_move" | xargs -I {} mv {} ~/.Trash
Question
How do I eliminate the error printouts from both of Methods 1 and 2? What is causing the printouts?
Short answer: find is being told to search all 4 subdirectories, even after some of them have been deleted.
Detailed eplanation: The root problem is that the wildcard in the xargs -I abc find ./testdir/* -maxdepth 1 ... part gets expanded by the shell before any of the commands get run. So that part of the script becomes:
... | xargs -I abc find ./testdir/dir1 ./testdir/dir2 ./testdir/dir3 ./testdir/dir4 -maxdepth 1 ...
What happens then is the stat | sed part sends the first inode number to xargs, that runs find, find searches all 4 directories for the matching item, and either deletes it directly or sends its path to the next xargs for deletion.
Next, the stat | sed part sends the second inode number to xargs, that runs find, find searches all 4 directories... oops, hey, one of them's missing! So it prints an error message about ./testdir/dir1 not existing, searches the other three, and (one way or another) deletes the next one.
Next comes the third inode number, and this time neither ./testdir/dir1 nor ./testdir/dir2 exists, so you get two error messages. Etc etc etc.
(There's also an additional problem with the second one, where immediately after running the mv command, find then tries to search its contents, and oops it's gone. That's why you get more error messages that way. I think you might've wanted -maxdepth 0 to keep it from trying to do that.)
Solution: I'm not sure what the larger context is, but my immediate reaction is that this is an overcomplex mess and as much as possible should be removed. But without knowing what can be changed without breaking the big picture, the minimal fix I see is to just have find search the entire testdir directory, rather than using a wildcard to list specific (wrong) subdirectories:
... | xargs -I abc find ./testdir -maxdepth 1 ...
(And note that in this form, -maxdepth 1 is actually correct.)
I have tried this command:
cd ~/Desktop
sudo find /tmp -name *file.doc* -exec 'mv "{}" ./file.docx' '{}' \;
but error:
find: mv "a b cfile.docx" ./file.docx: No such file or directory'
But the mv command looks just as it should (first arg - quoted file found in /tmp dir, second arg - new name for that file in current directory - that is mv SOURCE DEST is correct). If I do it in the /tmp dir - it would work, so why the error?
Find is trying to execute the file 'mv "a b cfile.docx" ./file.docx' but there's no such file.
This is what is happening:
$ 'mv "a b cfile.docx" ./file.docx'
-bash: mv "a b cfile.docx" ./file.docx: No such file or directory
bash also is not able to find the file we've asked it to execute.
The intention is to execute mv. Try this fixed version.
I've removed the single quote surrounding the arg to -exec
also the {} in the end is not needed. we just need the file substituted in one place
sudo find /tmp -name *file.doc* -exec mv "{}" ./file.docx \;
When the following command runs from crontab tar errors occur when a file with spaces in the filename are encountered.
/tar -C / -zcvf /root/archive-of-files-with-spaces-in-filename.tar.gz `find /var/www/html -mmin -1450 -print | sed 's|^/||'`
Changing the switch to -print0 or removing the -print switch altogether does not solve the problem:
/tar -C / -zcvf /root/archive-of-files-with-spaces-in-filename.tar.gz `find /var/www/html -mmin -1450 -print0 | sed 's|^/||'`
OR
/tar -C / -zcvf /root/archive-of-files-with-spaces-in-filename.tar.gz `find /var/www/html -mmin -1450 | sed 's|^/||'`
The same errors are thrown by tar.
How can the command be re-written to allow filenames with spaces to be served to tar without being broken up into directory names that do not exist?
Sample of an error produced with filename:
"2016-12-15 Name File Repair exportICS.php"
tar: var/www/html-calendar.northpawfamilycloud.xyz/pages/2016-12-15:
Cannot stat: No such file or directory
tar: Name: Cannot stat: No such file or directory
tar: File: Cannot stat: No such file or directory
tar: Repair: Cannot stat: No such file or directory
tar: exportICS.php: Cannot stat: No such file or directory
NOTE | sed 's|^/||'` strips leading slash off finds so tar error "removing leading slash does not occur.
Since the only filtering you're doing is to remove leading slashes from names under /var/www/html, you can use tar alone — avoiding a myriad problems with find:
tar -C / -czvf /root/archive-of-files-with-spaces-in-filename.tar.gz var/www/html
Given a directory, tar backs up the contents of the directory. If you have sub-directories under /var/html, your archive was probably twice as big as necessary. This one won't be bigger than necessary.
Apparently, the find command includes options. In that case, you have to do things the other way:
(cd /; find var/www/html -type f -mmin -1450 -print0) |
tar -C / --null -T - --czvf /root/archive-of-files-with-spaces-in-filename.tar.gz
The cd /; find var/html avoids leading slashes from find. The -print0 uses null terminators for the file names. The --null tells tar to expect file names to be null-terminated; the -T - says read the list of file names from standard input.
You can put all that on a single line if you wish. It isn't the way I'd do it. I would create a script that would be run by the cron (crontab) system that would do the job, especially since the output file name would inevitably be time-stamped if I was running it, and it would probably identify which site was being backed up, etc. The site to be backed up would be a parameter to the script, supplied by the crontab entry. That's my preference; there are plenty of people who write long lines into their crontab files.
Also, GNU tar (noisily) omits the leading slash from archive names, so that there are never absolute names inside a tar file generated by GNU tar:
tar: Removing leading `/' from member names
That means you could avoid the sub-shell and simply use find /var/www/html … to generate the names, living with the warning message. Since you have verbose output anyway, that's not likely to be much of a problem.
Archive with spaces:
Escape spaces with backslashes or quote the filename. Example:
tar czvf /my/path/filename\ with\ spaces.tgz files...
If the files to be archived have spaces in their names:
Use find ... -print0 to separate filenames with NULL tar ... --null to only consider NULL terminated files. From GNU tar documentation:
find /path ... -print0 | tar ... --null -T -
I would like to rename the following directory:
From 1/2/3/2/2 to 1/2_re/3/2_re/2_re.
Each directory has other contents too -for example file2stay.sh- which should stay untouched.
I tried the command:
find ./ -exec bash -c 'mv 2 2_re' \; but after it successfully renames the first directory the following error message appears:
mv: cannot stat ‘2’: No such file or directory
You need to tell find to process the content of a folder before the folder itself using -depth:
find . -name "2" -type d -depth -execdir mv 2 2_re \;
-execdir executes the mv in the folder where the ./2 was found.
You can use this find with sort -r in a for loop using process substitution:
while read -r f; do
mv "$f" "${f}_re"
done < <(find . -name '2' | sort -r)
Basically I need to run a Unix script to find all folders in the directory /fss/fin, if it exists; then I have tar it and move to another directory /fs/fi.
This is my command so far:
find /fss/fin -type d -name "essbase" -print
Here I have directly mentioned the folder name essbase. But instead, I would like to find all the folders in the /fss/fin and use them all.
How do I find all folders in the /fss/fin directory & tar them to move them to /fs/fi?
Clarification 1:
Yes I need to find only all folders in the directory /fss/fin directory using a Unix shell script and tar them to another directory /fs/fi.
Clarification 2:
I want to make it clear with the requirement. The Shell Script should contain:
Find all the folders in the directory /fss/fin
Tar the folders
Move the folders in another directory /fs/fi which is located on the server s11003232sz.net
On user requests it should untar the Folders and move them back to the orignal directory /fss/fin
here is an example I am working with that may lead you in the correct direction
BackUpDIR="/srv/backup/"
SrvDir="/srv/www/"
DateStamp=$(date +"%Y%m%d");
for Dir in $(find $SrvDir* -maxdepth 0 -type d );
do
FolderName=$(basename $Dir);
tar zcf "$BackUpDIR$DateStamp.$FolderName.tar.gz" -P $Dir
done
Since tar does directories automatically, you really don't need to do very much. Assuming GNU tar:
tar -C /fss/fin -cf - essbase |
tar -C /fs/fi -xf -
The '-C' option changes directory before operating. The first tar writes to standard output (the lone '-') everything found in the essbase directory. The output of that tar is piped to the second tar, which reads its standard input (the lone '-'; fun isn't it!).
Assuming GNU find, you can also do:
(cd /fss/fin; tar -cf - $(find . -maxdepth 1 -type d | sed '/^\.$/d')) |
tar -xf - -C /fs/fi
This changes directory to the source directory; it runs 'find' with a maximum depth of 1 to find the directories and removes the current directory from the list with 'sed'; the first 'tar' then writes the output to the second one, which is the same as before (except I switched the order of the arguments to emphasize the parallelism between the two invocations).
If your top-level directories (those actually in /fss/fin) have spaces in the names, then there is more work to do again - I'm assuming none of the directories to be backed up start with a '.':
(cd /fss/fin; find * -maxdepth 0 -type d -print 0 | xargs -0 tar -cf -) |
tar -xf - -C /fs/fi
This weeds out the non-directories from the list generated by '*', and writes them with NUL '\0' (zero bytes) marking the end of each name (instead of a newline). The output is written to 'xargs', which is configured to expect the NUL-terminated names, and it runs 'tar' with the correct directory names. The output of this ensemble is sent to the second tar, as before.
If you have directory names starting with a '.' to collect, then add '.[a-z]*' or another suitable pattern after the '*'; it is crucial that what you use does not list '.' or '..'. If you have names starting with dashes in the directory, then you need to use './*' and './.[a-z]*'.
If you've got still more perverse requirements, enunciate them clearly in an amendment to the question.
find /fss/fin -d 1 -type d -name "*" -print
The above command gives you the list of 1st level subdirectories of the /fss/fin.
Then you can do anything with this. E.g. tar them to your output directory as in the command below
tar -czf /fss/fi/outfile.tar.gz `find /fss/fin -d 1 -type d -name "*" -print`
Original directory structure will be recreated after untar-ing.
Here is a bash example (change /fss/fin, /fs/fi with your paths):
dirs=($(find /fss/fin -type d))
for dir in "${dirs[#]}"; do
tar zcf "$dir.tgz" "$dir" -P -C /fs/fi && mv -v "$dir" /fs/fi/
done
which finds all the folders, tar them separately, and if successful - move them into different folder.
This should do it:
#!/bin/sh
list=`find . -type d`
for i in $list
do
if [ ! "$i" == "." ]; then
tar -czf ${i}.tar.gz ${i}
fi
done
mv *.tar.gz ~/tardir