I want to produce a.txt from a.tar.gz in the directory. I tried writing the following goal in Makefile:
a.txt: a.tar.gz
tar xvf a.tar.gz
This works, but after running it once a.tar.gz is uncompressed and no longer exists. And subsequent make fails. How can I write a goal that checks if a.txt does not exist, and if so run tar, but otherwise don't do anything?
wildcard will only expand to something if it can find at least one file:
a.txt: $(wildcard a.tar.gz)
tar xvmf $<
Adding the m flag will avoid unnecessary remakes.
The rule will fail if neither file exists however, but without knowing what exactly you're trying to do it's hard to provide a comprehensive solution.
The 'test' or ' [ ' and ' ] ' can do this.
a.txt:
if [ -e a.tar.gz ]; then \
tar xvf a.tar.gz \
fi
Related
I'm trying to compress list of files from a text file (include.txt):
/var/dir/1/some_file_number*.txt
/var/dir/1/some_file_number*.log
and exclude from a list (exclude.txt):
/var/dir/1/some_file_number_exclude*.*
with command
tar -cf archive.tar --files-from="/to/my/dir/include.txt" --exclude-from="/to/my/dir/exclude.txt"
It seems to work for the --exclude-from, but doesn't for --files-from, I gets error saying that:
tar: /var/dir/1/some_file_number*.txt: Warning: Cannot stat: No such file or directory
tar: /var/dir/1/some_file_number*.log: Warning: Cannot stat: No such file or directory
Seems like tar couldn't expand the asterisk in --files-from, any idea how to fix this?
You can create a function that expands glob patterns from a file, and execute it inside process substitutions.
function expand_glob_file {
local patterns __
readarray -t patterns < "$1" || return
for __ in "${patterns[#]}"; do
compgen -G "$__"
done
}
tar -cf archive.tar --files-from=<(expand_glob_file /to/my/dir/include.txt) --exclude-from=<(expand_glob_file /to/my/dir/exclude.txt)
It seems like tar doesn't support pattern matching when creating an archive. From this documentation:
There are no inclusion members in create mode (‘--create’ and ‘--append’), since in this mode the names obtained from the command line refer to files, not archive members.
For other operations such as --extract, you could use the --wildcards flag but apparently creating an archive is an exception.
One idea is you could first expand the wildcards in your include file using something like
xargs -I # bash -c "printf '%s\n' #" < /to/my/dir/include.txt > /tmp/include.txt
mv /tmp/include.txt /to/my/dir/include.txt
and then run your tar command from before. Note: this doesn't handle spaces in file paths well (and possibly many other things since it relies on shell expansion), but konsolebox's answer seems to support that.
I wanted to write a bash script that will unpack .tar.gz archives and for each result file it will set an additional attribute with the name of the original archive. Just to know what the origin is of the unpacked file.
I tried to store the inside files in an array and then for-loop them.
for archive in "$1"*.tar.gz; do
if [ -f "${archive}" ]
then
readarray -t fileNames < <(tar tzf "$archive")
for file in "${fileNames}"; do
echo "${file}"
tar xvzf "${archive}" -C "$1" --no-wildcards "${file}" &&
attr -s package -V "${archive}" "${file}"
done
fi
done
The result is that only one file is extracted and no extra attribute is set.
#! /bin/bash
for archive in "$1"*.tar.gz; do
if [ -f "${archive}" ] ; then
# Unpack the archive into subfolder $1
tar xvf "$archive" -C "$1"
# Assign attributes
tar tf "$archive" | (cd "$1" && xargs -t -L1 attr -s package -V "$archive" )
fi
done
Notes:
Script is unpacking each archive with a single 'tar'. This is more efficient than unpacing one file at a time. It also avoid issues with unpacking folders, which will lead to unnecessary repeated work.
Script is using 'attr'. Will be better to use 'setfattr', if supported on target file system to set attributes on multiple files with a few calls (using xargs, with multiple files per command)
It is not clear what is the structure of the output folder. From the question, it looks as if all archives will be placed into the same folder "$1". The following solution assume that this is the intended behavior, and that each archive will have distinct file names. If each archive is to be placed into different sub folder, it will be easier/more efficient to implement.
I am trying to use a nested for loop for searching and copying some files inside the recipe of one of the targets inside a makefile:
DIR = $(DIR_A) $(DIR_B)
install:
for dirs in $(DIR); do \
for file in $(shell find $(dirs) -type f -and -not -path "*/.svn*" | sed -e "s|$(dirs)||"); do \
folder=$${file%/*}; \
$(INSTALL) -d $(DEST_DIR)$$folder/log; \
$(INSTALL) $(dirs)/$$file $(DEST_DIR)$$folder/log; \
done \
done
However $(dirs) variable always evaluates to empty inside the second for loop and the current working directory gets passed to "find" instead of first directory path from $(DIR).
Can someone please suggest if I am missing something ?
Thanks.
The $(dirs) in the find command is being expanded by make to the make variable dirs which is unset and thus the empty string. To reference the shell variable, you need to escape the $:
for file in $$(find $${dirs} -type f -and \
-not -path "*/.svn*" | sed -e "s|$${dirs}||"); do
But don't do this. It is much cleaner to explicitly list the files you intend to install. If there are many, it is fine to write a script to generate the Makefile. What you are doing is a fragile mess.
You have made several errors, and you will find it almost impossible to solve them as l;ong as you insist on trying to solve them all at once.
Let's try this in stages. Suppose your DIR_A and DIR_B are north and south. On the command line, if you try this:
for dirs in north south; do echo $dirs; done
you will get the correct output:
north
south
If you try it as a makefile rule:
install:
for dirs in $(DIR); do echo $$dirs; done
again, it works correctly.
If you try your makefile recipe:
install:
for dirs in $(DIR); do \ for file in $(shell ls $$dirs); do \ echo "file is $$file"; \ done \ done
it fails, because Make expands the $(shell ...) command before passing the entire for command to the shell, when dirs has not yet been assigned a value. One way to construct a sensible shell command is to use backticks:
for dirs in north south; do for file in `ls $dirs`; do echo "file is $file"; done done
This works on the command line. A makefile rule built around it:
install:
for dirs in $(DIR); do for file in `ls $$dirs`; do echo "file is $$file"; done done
also works.
That should be enough to allow you to rewrite your makefile.
/tmp/-> ls ab*
/tmp/-> ls: ab*: No such file or directory
/tmp/-> tar -cvf ab.tar abc*
tar: abc*: Cannot stat: No such file or directory
tar: Error exit delayed from previous errors
/tmp/->
/tmp/-> ls ab*
ab.tar
/tmp/-> tar -tvf ab.tar
/tmp/->
As can be seen there are no files matching pattern abc*, however output file named ab.tar got created with no content. Is there a switch/option than can be passed to tar command so that no output file is created when there are no input file?
I’m fond of using a for-as-if construct for such cases:
for x in abc*; do
# exit the loop if no file matching abc* exists
test -e "$x" || break
# by now we know at least one exists (first loop iteration)
tar -cvf ab.tar abc*
# and since we now did the deed already… exit the “loop”
break
done
The body of the “loop” is run through exactly once, but the shell does the globbing for us. (I normally use continue in the place of the first break, but that’s probably not needed.)
Alternatively, you can use the shell to expand the glob into $*…
set -- abc*
test -e "$1" && tar -cvf ab.tar abc*
If your script runs under set -e, use if test …; then tar …; fi instead, otherwise it will abort when no file exists.
All these variants work in plain sh as well.
There is a way to get the shell to do it:
#!/bin/sh
# safetar -- execute tar safely
sh -O failglob -c 'tar cvf ab.tar abc*'
Is there a switch/option than can be passed to tar command so that no output file is created when there are no input file?
Gnu tar does not have such an option.
Here are two alternatives. You need to study them and figure out what would work for you, as they're a bit of a hack.
You could do something like:
Tar, test, remove when empty
tar -cvf ab.tar abc* ||
tar tf ab.tar | read ||
rm ab.tar
Explanation:
If tar -cvf ... fails, get the contents with tar tf ....
If the read fails, the archive was empty, and it's save to remove it.
Or you could try:
Test, then tar
ls abc* | read && tar -cvf ab.tar abc*
This would not create the empty tar file in the first place.
Often after unzipping a file I end up with a directory containing nothing but another directory (e.g., mkdir foo; cd foo; tar xzf ~/bar.tgz may produce nothing but a bar directory in foo). I wanted to write a script to collapse that down to a single directory, but if there are dot files in the nested directory it complicates things a bit.
Here's a naive implementation:
mv -i $1/* $1/.* .
rmdir $1
The only problem here is that it'll also try to move . and .. and ask overwrite ./.? (y/n [n]). I can get around this by checking each file in turn:
IFS=$'\n'
for file in $1/* $1/.*; do
if [ "$file" != "$1/." ] && [ "$file" != "$1/.." ]; then
mv -i $file .
fi
done
rmdir $1
But this seems like an inelegant workaround. I tried a cleaner method using find:
for file in $(find $1); do
mv -i $file .
done
rmdir $1
But find $1 will also give $1 as a result, which gives an error of mv: bar and ./bar are identical.
While the second method seems to work, is there a better way to achieve this?
Turn on the dotglob shell option, which allows the your pattern to match files beginning with ..
shopt -s dotglob
mv -i "$1"/* .
rmdir "$1"
First, consider that many tar implementations provide a --strip-components option that allows you to strip off that first path. Not sure if there is a first path?
tar -tf yourball.tar | awk -F/ '!s[$1]++{print$1}'
will show you all the first-level contents. If there is only that one directory, then
tar --strip-components=1 -tf yourball.tar
will extract the contents of that directory in tar into the current directory.
So that's how you can avoid the problem altogether. But it's also a solution to your immediate problem. Having extracted the files already, so you have
foo/bar/stuff
foo/bar/.otherstuff
you can do
tar -cf- foo | tar --strip-components=2 -C final_destination -xf-
The --strip-components feature is not part of the POSIX specification for tar, but it is on both the common GNU and OSX/BSD implementations.