I am trying to tar some files into a destination, where FILES path and DESTINATION were entered as arguments ($1 and $2): script FILE DEST
I want to tar the files into the destination and change the name (ex: add the word "update" and the date at the end of the filename) at the end of the new tar file
So I have: date=date dest=$2 files="$1/*" #considers all the files within the directory given, as files is a . directory containing one or more files tar $dest/"update"$date $files
however, this does not work, and I get the error: tar: Old option 'u' requires an argument (as in the u in "update" I assume)
It is a little unclear from your description, but it appears you are trying to make a script that will take a directory name you call FILE as the first argument and then a name for your archive as the second argument you call DEST to which you want to append "_update_$(date +%F).tar.xz" (or choose the compression type you wish). You then want to create a TAR archive similar to:
tar -cJf "${2}_update_$(date +%F).tar.xz" "$1"/*
A script to do that is relatively straight forward, and we will add the -J option to use xz compression for the archive. Your FILE and DEST names are confusing, so let's use dname for the directory containing the files and tarname for the archive name. The after appending "_update_$(date +%F).tar.xz" to your tarname input you could do:
#!/bin/bash
[ -z "$1" ] || [ -z "$2" ] && { ## validate 2 arguments given
printf "error: insufficient input\nusage: %s file dest\n" "${0##*/}" >&2
exit 1
}
dirname="$1" ## name of directory to archive
tarname="${2}_update_$(date +%F).tar.xz" ## archive name
tar -cJf "$tarname" "$dirname"/* ## create compressed archive
Example Directory to Archive
$ ls -al tmpd/
total 4
drwxr-xr-x 2 david david 180 Sep 26 11:15 .
drwxrwxrwt 20 root root 420 Sep 26 11:16 ..
-rw-r--r-- 1 david david 0 Sep 26 11:15 a
-rw-r--r-- 1 david david 0 Sep 26 11:15 b
-rw-r--r-- 1 david david 0 Sep 26 11:15 c
-rw-r--r-- 1 david david 0 Sep 26 11:15 d
-rw-r--r-- 1 david david 0 Sep 26 11:15 e
-rw-r--r-- 1 david david 0 Sep 26 11:15 f
-rw-r--r-- 1 david david 111 Sep 22 22:08 file
Example Use/Output
$ bash quicktar.sh tmpd myfiles
The archive is created:
$ ls -al myfiles_update_2019-09-26.tar.xz
-rw-r--r-- 1 david david 460 Sep 26 11:16 myfiles_update_2019-09-26.tar.xz
Finally check the content of the compressed archive:
$ tar -tJf myfiles_update_2019-09-26.tar.xz
tmpd/a
tmpd/b
tmpd/c
tmpd/d
tmpd/e
tmpd/f
tmpd/file
Look things over and let me know if that is what you intended. You can drop compression or change the type just by changing the -J option in the command above.
Related
I have directory of files called:
foo--ext1
foo--ext2
foo--ext3
...
I would like to rename them to:
foo-bar-ext1
foo-bar-ext2
foo-bar-ext3
....
How can I do this renaming in bash?
I have attempted to understand mywiki.wooledge.org/BashFAQ/030 but I can't work what should go in place of ${f%.foo}.bar"; in the first example.
So I have started with:
for f in foo--*; do echo mv -- "$f"
but what do I put next?
There are several ways to approach this.
I recommend bookmarking this page and referencing it often.
I would use this:
$: for f in foo--*; do mv "$f" "${f//foo--/foo-bar-}"; done
This uses string substitution in the current filename to construct the new name, replacing foo-- with foo-bar-.
Note the // in the replacement. This will replace every occurrence of foo-- with foo-bar- in each filename.
$: ls -l foo-*
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext1
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext2
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext3
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:33 foo-bar-foo-bar-etx4
Remove one of the leading slashes to make it only handle the first occurrence -
$: for f in foo--*; do mv "$f" "${f/foo--/foo-bar-}"; done
$: ls -l foo-*
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext1
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext2
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext3
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-foo--ext4
Another simple method avoiding the loop is to use rename (from the util-linux package). There all that is needed is:
rename "foo-" "foo-bar" foo--*
You can check what will be done before actually doing the rename with the -n (no-act) and -v (verbose) options. For your example files, that would be:
$ rename -nv "foo-" "foo-bar" foo--*
`foo--ext1' -> `foo-bar-ext1'
`foo--ext2' -> `foo-bar-ext2'
`foo--ext3' -> `foo-bar-ext3'
There are two versions of rename that you will find provided in Linux distributions, the rename above from the util-linux package and perl-rename, which some Linux distros use instead which will also install as rename. Both are capable of handling the rename, but the options will be different. Check which you have with rename --version before use.
I have this small script:
#!/bin/bash
for file in "$(ls | grep -v $0)";do
cat $file > "${file}-test"
done
On this directory:
total 40
-rwxr-xr-x 1 root root 783 Dec 11 09:19 appendToLog.sh
-rwxr-xr-x 1 root root 3995 Dec 11 13:22 con2dd.py
-rwxr-xr-x 1 root root 362 Dec 11 13:26 dd.py
-rw-r--r-- 1 root root 566 Dec 11 13:26 dd.pyc
-rw-r--r-- 1 root root 18558 Dec 25 11:24 moshe.log
-rw------- 1 root root 0 Dec 11 09:20 nohup.out
-rwxr-xr-x 1 root root 88 Dec 25 11:28 task.sh
-rwxr-xr-x 1 root root 560 Dec 11 10:33 test.py
Nevermind that I can achieve that with cp, I want to understand why this exactly is producing this file:
-rw-r--r-- 1 root root 24912 Dec 25 11:28 appendToLog.sh?con2dd.py?dd.py?dd.pyc?moshe.log?nohup.out?task.sh?test.py-test
And nothing else.
The problem is parsing output of ls is just wrong (see Why you shouldn't parse the output of ls(1), filenames in unix can have almost any special characters including whitespace, newlines, commas, pipe symbols. Its because you've quoted the output of ls in one construct, you have a single list of all the files concatenated as one string in the value of "${file}-test" which is quite not what you wanted to do.
Also notice how ls sometimes garbles your filename data (in our case, it turned the \n character in between the words into a ? question mark (could indicate a character that cannot be displayed).
Just use the glob expansion in bash to list the files and do actions on them.
for f in *; do
[[ -e $f ]] || continue
...
done
That said, You could probably have some non-printable characters on end of lines (eg. CRLF imported rom Windows)
Run cat -A scriptname it'll show you all characters in your script. Then, you can convert to unix-like format running dos2unix scriptname.
I have a folder called error which contains some 1000 files like below:
-rw-r--r-- 1 orartkp6 dba 1298 Apr 19 09:23 BEN_INV_5_0900091010993_20160419092353.xml.err
-rw-r--r-- 1 orartkp6 dba 1289 Apr 19 09:23 BEN_INV_5_0900091010994_20160419092353.xml.err
-rw-r--r-- 1 orartkp6 dba 1286 Apr 19 09:23 BEN_INV_5_0900091010995_20160419092353.xml.err
-rw-r--r-- 1 orartkp6 dba 1292 Apr 19 09:23 BEN_INV_5_0900091010996_20160419092353.xml.err
-rw-r--r-- 1 orartkp6 dba 1300 Apr 19 09:23 BEN_INV_5_0900091010997_20160419092353.xml.err
Now I have a .txt file which has say only 500 of the above file names.
SO now I have to write a script to read every line from the .txt file and search inside the error folder and move only those 500 files into a new folder (backup_folder).
Can you help me how to do this?
the logic behind this would be,
read only $2 column that is (i suppose) the names of the files,
and in a while loop output each one in a variable and then use mv.
awk '{print $2; var=$2; mv $var dir_to_move}' 500lines.txt
hope i helped.
I used the below while loop to solve this problem.
while read file
do
cp -p "$file" ./backup_folder
done < 500lines.txt
I have a directory which contains a lot of files with .sum extension.
I used the below script to list the contents of all the .sum files to a temp.log file. While the first .sum file gets written to temp.log the awk utility seems to give error for the remaining .sum files. Please help, what i am missing in here.
cd $HOME/aphp/result/${test}
for filename in *.sum
do
tempdir=$filename
awk '/Failed/' "${filename}" > temp.log
awk '/Error/' "${filename}" >> temp.log
if [ -s temp.log ]
then
mkdir -p ${scanresult}/${tempdir}
mv temp.log ${scanresult}/${tempdir}/temp.log
cd ${scanresult}/${tempdir}
mv temp.log ${tempdir}_failed.txt
else
echo Skipping ${tempdir} scanning as it is executed 100 percent with no fail or error.
rm temp.log
fi
done
Errors:
awk: fatal: cannot open file `dss154.sum' for reading (No such file or directory)
awk: fatal: cannot open file `dss235.sum' for reading (No such file or directory)
awk: fatal: cannot open file `dss287.sum' for reading (No such file or directory)
ls -l *.sum
-rwxrwxrwx 1 smruti smruti 1844 Mar 25 16:23 dss103.sum
-rwxrwxrwx 1 smruti smruti 2353 Mar 25 16:40 dss154.sum
-rwxrwxrwx 1 smruti smruti 1023 Mar 25 16:43 dss235.sum
-rwxrwxrwx 1 smruti smruti 908 Mar 25 16:45 dss287.sum
-rwxrwxrwx 1 smruti smruti 867 Mar 25 16:45 dss288.sum
-rwxrwxrwx 1 smruti smruti 1064 Mar 25 16:47 dss350.sum
You are getting into this problem due to this line:
cd ${scanresult}/${tempdir}
Which is changing your current working directory to something else. After which rest of the files cannot be read after 1st file. It is not really clear why are you changing directory inside the loop.
You can use this line to go back to original path:
cd -
However most of your code after awk command looks suspicious and redundant.
What does the command cp $1/. $2 do? I know cp is used for copying from source(stored in variable $1) to destination(stored in variable $2). I am just confused with the /. used along with the variable. Can someone please help me understand this?
The command:
$ cp -R $1/. $2
copies contents of directory pointed by $1 to the directory $2.
Without -R switch this command would fail both when $1 is a file or directory.
In general, . points to the current directory. You can see that by comparing inode's shown by ls:
$ mkdir test
$ ls -ali
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 .
771046 drwxrwxrwt 21 root wheel 714 23 mar 12:30 ..
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 test
$ cd test
$ ls -ali
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 .
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 ..
Note that inode 9525312 points to test when viewed from the parent directory, and points to . when viewed from inside the test directory.