I would like to untar an archive e.g. "tar123.tar.gz" to directory /myunzip/tar123/" using a shell command.
tar -xf tar123.tar.gz will decompress the files but in the same directory as where I'm working in.
If the filename would be "tar233.tar.gz" I want it to be decompressed to /myunzip/tar233.tar.gz" so destination directory would be based on the filename.
Does anyone know if the tar command can do this?
tar -xzvf filename.tar.gz -C destination_directory
With Bash and GNU tar:
file=tar123.tar.gz
dir=/myunzip/${file%.tar.gz}
mkdir -p $dir
tar -C $dir -xzf $file
You can change directory before extracting with the -C flag, but the directory has to exist already. (If you create file-specific directories, I strongly recommend against calling them foo.tar.gz - the extension implies that it's an archive file but it's actually a directory. That will lead to confusion.)
Try
file=tar123.tar.gz
dir=/myunzip/$(basename $file .tar.gz) # matter of taste and custom here
[ -d "$dir" ] && { echo "$dir already exists" >&2; exit 1; }
mkdir "$dir" && ( gzip -d "$file | ( cd "$dir" && tar xf - ) )
If you're using GNU tar you can also give it an option -C "$dir" which will cause it to change to the directory before extracting. But the code above should work even with a Bronze Age tar.
Disclaimer: none of the code above has been tested.
Related
I have used a few entries here to guide me through my following bash script:
LOG_FILES=$(find ~/testing.domain.net -name "domain_*log" -printf ' %P ')
NEW_LOG_FILES=$(echo $LOG_FILES | sed -e 's/\r//g')
echo ${NEW_LOG_FILES}
NOW=$(date +"%m-%d-%Y")
echo ${NOW}
tar czf ${NOW}.tar.gz ${NEW_LOG_FILES}
RC=$? # Check whether an error occured
if [[ "$RC" == "0" ]]; then
mv ${NOW}.tar.gz archivedlogs/.
rm ${LOG_FILES}
fi
The objective of the script is to find any log files in the current and sub directories and tar zipped them all before moving tar file to an archivedlogs subdirectory and then deleting log files.
When I execute the script I'm getting:
domain_info_log subdir1/domain_error_log subdir2/domain_error_log domain_error_log
12-27-2020
tar: \r\r: Cannot stat: No such file or directory
tar: Exiting with failure status due to previous errors
./archiveTestLogsDaily.sh: line 12: syntax error: unexpected end of file
I can't seem to get rid of the carriage returns.
I have also tried replacing:
NEW_LOG_FILES=$(echo $LOG_FILES | sed -e 's/\r//g')
with:
NEW_LOG_FILES=${LOG_FILES//$'\r'/}
But same outcome.
Any ideas? My bash script knowledge is not the best but I'm guessing the find part comes back with carriage returns?
Also before anyone else suggests it, I don't have access to logrotate as this is a shared server.
$ uname -a
Linux 3.10.0-962.3.2.lve1.5.26.4.el7.x86_64 #1 SMP Wed Sep 11 10:29:54 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
I think it would be more safe to use a directory before making the tar archive in order to avoid a "tarbomb".
I mean something like:
now="$(date +"%m-%d-%Y")"
archiv_dir="archiv_dir-${now}"
mkdir -p "${archiv_dir}"
find ~/testing.domain.net -type f -a -iname "domain_*log" -exec mv {} "${archiv_dir}" \;
tar cvzf "${now}.tar.gz" "${archiv_dir}" && mv "${now}.tar.gz" "archivedlogs/" && rm -rf "${archiv_dir}"
P.S: make sure to double quote when you're using a variable value in Bash (or even using other POSIX shells), except inside [[.
EDIT to answer to that:
One more thing #Idriss, some of those subdirectories have the same file names, when mv tries to copy them it ignores them because they are already there... is there a way of making sure the parent directory is somehow appended to the name?
With a little bit of bash and awk, you could try something like:
#!/bin/bash
now="$(date +"%m-%d-%Y")"
archiv_dir="archiv_dir-${now}"
ROOT=~/testing.domain.net
find "${ROOT}" -type f -a -iname "domain_*log"|while read; do
subdir="$(echo $REPLY|awk -F '/' '{print $(NF-1)}')"
filename="$(basename "${REPLY}")"
[[ $subdir && $subdir != $(basename $ROOT) ]] && filename="${subdir}_${filename}"
echo mv "${REPLY}" "${archiv_dir}/${filename}"
done
Here is the output:
mv /Users/ineumann/testing.domain.net/subdir3/domain_error_log archiv_dir-12-28-2020/subdir3_domain_error_log
mv /Users/ineumann/testing.domain.net/subdir2/domain_error_log archiv_dir-12-28-2020/subdir2_domain_error_log
mv /Users/ineumann/testing.domain.net/subdir1/domain_error_log archiv_dir-12-28-2020/subdir1_domain_error_log
mv /Users/ineumann/testing.domain.net/domain_info_log archiv_dir-12-28-2020/domain_info_log
N.B: just remove the echo before the mv to perform the moves and I let you merge this example with your previous script that create directories, create the tar archive, etc.
I'm a beginner in writing bash scripts for automating tasks, and I'm trying to untar all the tar files in one directory (there are way too many to do it by hand) for a bunch of source code files. They're all of the type *.tar.gz, *.tar.xz, or *.tar.bz2.
This is for a Linux from Scratch LFS installation I'm doing (I'm a first timer), and I'm not sure how else to automate this task other than using a bash script. The code for my little script to do this is down below.
#!/bin/bash
for afile in 'ls -1'; do
if [ 'afile | grep \"\.tar\.gz\"' ];
then
tar -xzf afile
elif [ 'afile | grep \"\.tar\.xz\"' ]
then
tar -xJf afile
elif [ 'afile | grep \"\.tar\.xz\"' ]
then
tar -xjf afile
else
echo "Something is wrong with the program"
fi
done;
I expected it to untar everything in the directory and create separate directories, but instead it exited with this error:
tar (child): afile: Cannot open: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
Apparently it thinks afile is the actual file, but I don't know how to change afile to be each file that is going through my for construct. How would I write a script for this, especially since there are different types of files?
To get your script to work with minimal changes, use $afile whenever you want the variable's value. The dollar sign makes a variable reference; otherwise you just get the literal string 'afile'. Also get rid of the square brackets and instead echo the variable to grep.
for afile in `ls -1`; do
if echo "$afile" | grep '\.tar\.gz'
then
tar -xzf "$afile"
elif echo $afile | grep '\.tar\.xz'
then
tar -xJf "$afile"
elif echo "$afile" | grep '\.tar\.bz2'
then
tar -xjf "$afile"
else
echo "Something is wrong with the program"
fi
done
Since you're a bash beginner, let's look at various other ways you could write the script. I'd make a couple of improvements. For one, you shouldn't loop over ls. You can get the same thing by looping over *. Second, grep is a heavyweight tool. You can do some simple string comparisons with built-in shell constructs like [[ and ==.
for afile in *; do
if [[ "$afile" == *.tar.gz ]]; then
tar -xzf "$afile"
elif [[ "$afile" == *.tar.xz ]]; then
tar -xJf "$afile"
elif [[ "$afile" == *.tar.bz2 ]]; then
tar -xjf "$afile"
else
echo "Something is wrong with the program"
fi
done
Actually, this would be even nicer with a case statement. Let's try that. Also let's echo the error message to stderr with >&2. That's always a good idea.
for afile in *; do
case "$afile" in
*.tar.gz) tar -xzf "$afile";;
*.tar.xz) tar -xJf "$afile";;
*.tar.bz2) tar -xjf "$afile";;
*) echo "Something is wrong with the program" >&2
esac
done
We could even get rid of the error message if we just list the three types of files we want to loop over. Then there's no way to hit the else case.
for afile in *.tar.{gz,xz,bz2}; do
case "$afile" in
*.tar.gz) tar -xzf "$afile";;
*.tar.xz) tar -xJf "$afile";;
*.tar.bz2) tar -xjf "$afile";;
esac
done
Or a completely different way to do it: use find to find all the files and its -exec action to call a command for each file it finds. Here {} is a placeholder for the files it finds.
find . -name '*.tar.gz' -exec tar -xzf {} \;
find . -name '*.tar.xz' -exec tar -xJf {} \;
find . -name '*.tar.bz2' -exec tar -xjf {} \;
/tmp/-> ls ab*
/tmp/-> ls: ab*: No such file or directory
/tmp/-> tar -cvf ab.tar abc*
tar: abc*: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
/tmp/->
/tmp/-> ls ab*
ab.tar
/tmp/-> tar -tvf ab.tar
/tmp/->
As can be seen there are no files matching pattern abc*, however output file named ab.tar got created with no content. Is there a switch/option than can be passed to tar command so that no output file is created when there are no input file?
I’m fond of using a for-as-if construct for such cases:
for x in abc*; do
# exit the loop if no file matching abc* exists
test -e "$x" || break
# by now we know at least one exists (first loop iteration)
tar -cvf ab.tar abc*
# and since we now did the deed already… exit the “loop”
break
done
The body of the “loop” is run through exactly once, but the shell does the globbing for us. (I normally use continue in the place of the first break, but that’s probably not needed.)
Alternatively, you can use the shell to expand the glob into $*…
set -- abc*
test -e "$1" && tar -cvf ab.tar abc*
If your script runs under set -e, use if test …; then tar …; fi instead, otherwise it will abort when no file exists.
All these variants work in plain sh as well.
There is a way to get the shell to do it:
#!/bin/sh
# safetar -- execute tar safely
sh -O failglob -c 'tar cvf ab.tar abc*'
Is there a switch/option than can be passed to tar command so that no output file is created when there are no input file?
Gnu tar does not have such an option.
Here are two alternatives. You need to study them and figure out what would work for you, as they're a bit of a hack.
You could do something like:
Tar, test, remove when empty
tar -cvf ab.tar abc* ||
tar tf ab.tar | read ||
rm ab.tar
Explanation:
If tar -cvf ... fails, get the contents with tar tf ....
If the read fails, the archive was empty, and it's save to remove it.
Or you could try:
Test, then tar
ls abc* | read && tar -cvf ab.tar abc*
This would not create the empty tar file in the first place.
I have this tiny code here
for i in *.tar.gz;
do tar xzvf $i;
done && find . -name "*.tar.gz" -exec rm {} \;
Now, when I have multiple tars, it will first untar all of them and then delete the tar files.
How can I change the code to untar a file, to delete it and then move to the next tar file?
Thanks in advance
for file in *.tar.gz; do tar xzvf "${file}" && rm "${file}"; done
Don't forget to quote your variables to account for funky filenames with whitespace.
Simply change the order of actions:
for i in *.tar.gz; do
tar xzvf "$i" && rm -r "$i"
done
The requirement is to extract all the *.properties files from multiple tars and put them into a zip.
I tried this:
find . -iwholename "*/ext*/*.tar.gz"|xargs -n 1 tar --wildcards '*.properties' -xvzf | zip -# tar-properties.zip
This is creating a zip with the .properties files in all the tars.
But the issue is the tars are structured as in each tar contains a properties folder which contains the files. The above command is creating a zip with a single properties folder which contains all the files .
Is there a way to put these in the zip with a folder structure like {name of the tar}/properties/*.properties ?
You could use this script. My solution uses --transform as well. Please check first if your tar command supports it with tar --help 2>&1 | grep -Fe --transform.
#!/bin/bash
[ -n "$BASH_VERSION" ] || {
echo "You need bash to run this script." >&2
exit 1
}
TEMPDIR=/tmp/properties-files
OUTPUTFILE=$PWD/tar-properties.zip ## Must be an absolute path.
IFS=
if [[ ! -d $TEMPDIR ]]; then
mkdir -p "$TEMPDIR" || {
echo "Unable to create temporary directory $TEMPDIR." >&2
exit 1
}
fi
NAMES=()
while read -r FILE; do
NAMEOFTAR=${FILE##*/} ## Remove dir part.
NAMEOFTAR=${NAMEOFTAR%.tar.gz} to remove extension ## Remove .tar.gz.
echo "Extracting $FILE."
tar --wildcards '*.properties' -xvzf "$FILE" -C "$TEMPDIR" --transform "s#.*/#${NAMEOFTAR//#/\\#}/properties/#" || {
echo "An error occurred extracting to $TEMPDIR." >&2
exit 1
}
NAMES+=("$NAMEOFTAR")
done < <(exec find . -type f -iwholename '*/ext*/*.tar.gz')
(
cd "$TEMPDIR" >/dev/null || {
echo "Unable to change directory to $TEMPDIR."
exit 1
}
zip -a "$OUTPUTFILE" "${NAMES[#]}"
)
Save it to a script then run it on the directory where those files are to be searched with
bash /path/to/script.sh`
You can probably do the trick with tar option --transform, --xform. This option permits to manipulate path thanks to a sed expression.
find . -iwholename "*/ext*/*.tar.gz"|xargs -n 1 tar --wildcards '*.properties' -xvzf --xform 's#.*/#name_of_the_tar/properties/#' | zip -# tar-properties.zip