I'm trying to figure out why is the following tar command not working -
I've tried the following 2 versions & both don't work -
Version 1
tar -c --use-compress-program=pigz -f /home/jhonst/data_lake/1m/UX.tar -C '/home/jhonst/data_lake/1m/*.UX.csv'
The error I see is
tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information.
Version 2
tar -c --use-compress-program=pigz -f /home/jhonst/data_lake/1m/UX.tar -C '/home/jhonst/data_lake/1m/*.UX.csv' .
The error I see is
tar:
/home/jhonst/data_lake/1m/*.UX.csv: Cannot open: No such file or directory
tar: Error is not recoverable: exiting now
Please could someone guide me on what I am doing wrong
When you do:
tar -c --use-compress-program=pigz -f /home/jhonst/data_lake/1m/UX.tar /home/jhonst/data_lake/1m/*.UX.csv
You'll get:
tar -t -f /home/jhonst/data_lake/1m/UX.tar
home/jhonst/data_lake/1m/file1.UX.csv
home/jhonst/data_lake/1m/file2.UX.csv
...
Which is not the best. There are two possibilities for getting rid of the "path" inside the tar archive:
Go inside the directory with cd (in a subshell or with pushd/popd if you want to return to the original directory after the tar command):
cd /home/jhonst/data_lake/1m && tar -c --use-compress-program=pigz -f UX.tar *.csv
# returning to the same place after the tar:
(cd /home/jhonst/data_lake/1m && tar -c --use-compress-program=pigz -f UX.tar *.csv)
# or:
pushd /home/jhonst/data_lake/1m && {
tar -c --use-compress-program=pigz -f UX.tar *.csv
popd
}
Use the -C option of GNU tar, which is not that easy to handle:
dirpath=/home/jhonst/data_lake/1m
files=("$dirpath"/*.csv)
tar -c --use-compress-program=pigz -f "$dirpath"/UX.tar -C "$dirpath" "${files[#]#$dirpath/}"
Related
tar: Cowardly refusing to create an empty archive
Try 'tar --help' or 'tar --usage' for more information.
And these are my Commands:
tar -cz -newer $letztes_full_backup -f $zielpfad/inc_backup_home_$datum.tar.gz/home
tar -czf $zielpfad/full_backup_$datum.tar.gz/home
tar -czf $zielpfad/full_backup_$datum.tar.gz/home
ssh user#server "sudo tar -czf - -C /path/To/Directory *.properties" | tar xzf -
It fails with following error -
tar: *.properties: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
If I ssh to server and then on server, instead of tar -czf - use tar -czf abc.tgz and remove the piped command at the end, it works correctly i.e. compress all the .properties file from the directory?
I have tried using --wildcards parameter as well as -P and the complete path ending with *.properties, but they didn't work either.
The shell at the far end expands *.properties in user's home directory, before it execs tar. I'm guessing there's no matching files in ~user.
What you probably want is something more like (assuming user can access the directory)
ssh user#server 'cd /path/To/Directory && sudo tar -czf - *.properties' \
| tar xzf -
Or, use tar's --wildcards option, and stop the remote shell expanding the argument:
ssh user#server 'sudo tar -czf - -C /path/To/Directory --wildcards \*.properties'
So I have a script that takes the zip files in the 2 folders and unzips them into new folders which all works and runs fine
(Folder Names Changed)
But I want to first Know if it is possible to after the files have been unziped and placed in there right folders to then delete the old zip files so all I am left with is the folders and all the unziped files.
And second if it is how would I go about doing that I am new to writing scripts
thanks for any help you can provide
cd ~/Downloads/
mkdir -p -v "./prior/(Folder_Name_1)"
mkdir -p -v "./prior/(Folder_Name_2)"
mkdir -p -v "./prior/(Folder_Name_3)"
mkdir -p -v "./current/(Folder_Name_1)"
mkdir -p -v "./current/(Folder_Name_1)"
mkdir -p -v "./current/(Folder_Name_1)"
tar -xzvf "./prior/Name1.reports.tar.gz" -C "./prior/Folder_Name_1"
tar -xzvf "./prior/Name2.reports.tar.gz" -C "./prior/Folder_Name_2"
tar -xzvf "./prior/Name3.reports.tar.gz" -C "./prior/Folder_Name_3"
tar -xzvf "./current/Name1.reports.tar.gz" -C "./prior/Folder_Name_1"
tar -xzvf "./current/Name2.reports.tar.gz" -C "./prior/Folder_Name_2"
tar -xzvf "./current/Name3.reports.tar.gz" -C "./prior/Folder_Name_3"
I think you want something like this
rm "./prior/Folder_1"
rm "./prior/Folder_2"
rm "./prior/Folder_3"
rm "./current/Folder_1"
rm "./current/Folder_2"
rm "./current/Folder_3"
Question:
I want to untar a tarfile which has many tar files within itself and remove the files in all the tar files and I want all of these processes to run in parallel in Unix bash scripting.
Conditions:
The script should return an error if any untar/remove process has any error.
It should only return success after all N (untar and remove) processes complete successfully.
Proposed solution:
mkdir a
tar -C a -xvf b.tar
cd a
for i in *
do
rm -r $i &
done
If you have GNU Parallel http://www.gnu.org/software/parallel/ installed you can do this:
tar xvf foo.tgz | perl -ne 'print $l;$l=$_;END{print $l}' | parallel rm
It is useful if you do not have space to extract the full tar.gz file, but you need to process files as you unpack them:
tar xvf foo.tgz | perl -ne 'print $l;$l=$_;END{print $l}' | parallel do_stuff {}\; rm {}
You can install GNU Parallel simply by:
wget http://git.savannah.gnu.org/cgit/parallel.git/plain/src/parallel
chmod 755 parallel
cp parallel sem
Watch the intro videos for GNU Parallel to learn more:
https://www.youtube.com/playlist?list=PL284C9FF2488BC6D1
mkdir a
tar -C a -xvf b.tar
cd a
success=$(for i in *
do
rm -r $i || echo failed & # if a job fails false will be echoed
done
wait)
# if any of the jobs failed, success will be set to a value other than ""
[[ -z "$success" ]] && exit 0 || exit 1
The answer tar xvf a.tar | tac | xargs -P 4 rm -rv is inspired from Burton Samograd's comment about xargs -P
$ mkdir -p a/b/c/d
mkdir: created directory `a'
mkdir: created directory `a/b'
mkdir: created directory `a/b/c'
mkdir: created directory `a/b/c/d'
$ touch a/1 a/2 a/3 a/b/4 a/b/5
$ tar cf a.tar a
$ rm -rfv a
removed directory: `a/b/c/d'
removed directory: `a/b/c'
removed `a/b/4'
removed `a/b/5'
removed directory: `a/b'
removed `a/3'
removed `a/1'
removed `a/2'
removed directory: `a'
$ tar xvf a.tar | tac | xargs -P 4 rm -rv
removed `a/2'
removed `a/1'
removed `a/3'
removed `a/b/5'
removed `a/b/4'
removed directory: `a/b/c/d'
removed directory: `a/b/c'
removed directory: `a/b'
removed directory: `a'
I would like to untar an archive e.g. "tar123.tar.gz" to directory /myunzip/tar123/" using a shell command.
tar -xf tar123.tar.gz will decompress the files but in the same directory as where I'm working in.
If the filename would be "tar233.tar.gz" I want it to be decompressed to /myunzip/tar233.tar.gz" so destination directory would be based on the filename.
Does anyone know if the tar command can do this?
tar -xzvf filename.tar.gz -C destination_directory
With Bash and GNU tar:
file=tar123.tar.gz
dir=/myunzip/${file%.tar.gz}
mkdir -p $dir
tar -C $dir -xzf $file
You can change directory before extracting with the -C flag, but the directory has to exist already. (If you create file-specific directories, I strongly recommend against calling them foo.tar.gz - the extension implies that it's an archive file but it's actually a directory. That will lead to confusion.)
Try
file=tar123.tar.gz
dir=/myunzip/$(basename $file .tar.gz) # matter of taste and custom here
[ -d "$dir" ] && { echo "$dir already exists" >&2; exit 1; }
mkdir "$dir" && ( gzip -d "$file | ( cd "$dir" && tar xf - ) )
If you're using GNU tar you can also give it an option -C "$dir" which will cause it to change to the directory before extracting. But the code above should work even with a Bronze Age tar.
Disclaimer: none of the code above has been tested.