I have to extract all archives in my current folder. What I'm doing only extracts all that are named correctly but the archives can have any messed up extension on them.
file.tar.gvz
or file.tar.potatogz
or file.sss.faz
gunzip *.tar.gz >/dev/null 2>/dev/null
echo $?
echo "gzip"
tar -xvf *.tar >/dev/null 2>/dev/null
echo $?
echo "tar"
unzip '*.zip' >/dev/null 2>/dev/null
echo $?
echo "zip"
Edit:
The file command seems like the right way to go but I can't get it to work in a nested loop. when outside the directory loop it works fine.
for D in *; do
if [ -d "${D}" ]; then
cd $D
bool=false
for files in *
do
type=$( file -b $files | awk '{printf $1;}' )
echo "next is type"
echo $type
done
cd..
done
The File command solved this.
My problems with file were solved with quotation marks.
type=$( file -b "$j" | awk '{printf $1;}' )
When gunzip reads from stdin, it doesn't know that the filename was wrong:
cat archive_with_wrong_name | gunzip -c
Or even easier...
gunzip -c <archive_with_wrong_name
Or even easier...
gzcat archive_with_wrong_name
Looks like you are real close. How about something like this?
for D in *; do
if [ -d "${D}" ]; then
cd $D
for file in *; do
type="$( file -b "$file" | awk '{print $1}' )"
case "$type" in
'gzip' ) gunzip $file; continue ;;
'zip' ) unzip $file; continue ;;
'tar' ) tar xzf $file; continue ;;
'*' ) continue ;;
esac
done
cd ..
fi
done
Related
I'm trying to perform certain operation on each file in a directory but there is a problem with order it's going through. It should do one file at the time. The long line (unzipping, grepping, zipping) works fine on a single file without a script, so there is a problem with a loop. Any ideas?
Script should grep through through each zipped file and look for word1 or word2. If at least one of them exist then:
unzip file
grep word1 and word2 and save it to file_done
remove unzipped file
zip file_done to /donefiles/ with original name
remove file_done from original directory
#!/bin/bash
for file in *.gz; do
counter=$(zgrep -c 'word1\|word2' $file)
if [[ $counter -gt 0 ]]; then
echo $counter
for file in *.gz; do
filenoext=${file::-3}
filedone=${filenoext}_done
echo $file
echo $filenoext
echo $filedone
gunzip $file | grep 'word1\|word2' $filenoext > $filedone | rm -f $filenoext | gzip -f -c $filedone > /donefiles/$file | rm -f $filedone
done
else
echo "nothing to do here"
fi
done
The code snipped you've provided has a few problems, e.g. unneeded nested for cycle and erroneous pipeline
(the whole line gunzip $file | grep 'word1\|word2' $filenoext > $filedone | rm -f $filenoext | gzip...).
Note also your code will work correctly only if *.gz files don't have spaces (or special characters) in names.
Also zgrep -c 'word1\|word2' will also match strings like line_starts_withword1_orword2_.
Here is the working version of the script:
#!/bin/bash
for file in *.gz; do
counter=$(zgrep -c -E 'word1|word2' $file) # now counter is the number of word1/word2 occurences in $file
if [[ $counter -gt 0 ]]; then
name=$(basename $file .gz)
zcat $file | grep -E 'word1|word2' > ${name}_done
gzip -f -c ${name}_done > /donefiles/$file
rm -f ${name}_done
else
echo 'nothing to do here'
fi
done
What we can improve here is:
since we unzipping the file anyway to check for word1|word2 presence, we may do this to temp file and avoid double-unzipping
we don't need to count how many word1 or word2 is inside the file, we may just check for their presence
${name}_done can be a temp file cleaned up automatically
we can use while cycle to handle file names with spaces
#!/bin/bash
tmp=`mktemp /tmp/gzip_demo.XXXXXX` # create temp file for us
trap "rm -f \"$tmp\"" EXIT INT TERM QUIT HUP # clean $tmp upon exit or termination
find . -maxdepth 1 -mindepth 1 -type f -name '*.gz' | while read f; do
# quotes around $f are now required in case of spaces in it
s=$(basename "$f") # short name w/o dir
gunzip -f -c "$f" | grep -P '\b(word1|word2)\b' > "$tmp"
[ -s "$tmp" ] && gzip -f -c "$tmp" > "/donefiles/$s" # create archive if anything is found
done
It looks like you have an inner loop inside the outer one :
#!/bin/bash
for file in *.gz; do
counter=$(zgrep -c 'word1\|word2' $file)
if [[ $counter -gt 0 ]]; then
echo $counter
for file in *.gz; do #<<< HERE
filenoext=${file::-3}
filedone=${filenoext}_done
echo $file
echo $filenoext
echo $filedone
gunzip $file | grep 'word1\|word2' $filenoext > $filedone | rm -f $filenoext | gzip -f -c $filedone > /donefiles/$file | rm -f $filedone
done
else
echo "nothing to do here"
fi
done
The inner loop goes through all the files in the directory if one of them contains file1 or file2. You probably want this :
#!/bin/bash
for file in *.gz; do
counter=$(zgrep -c 'word1\|word2' $file)
if [[ $counter -gt 0 ]]; then
echo $counter
filenoext=${file::-3}
filedone=${filenoext}_done
echo $file
echo $filenoext
echo $filedone
gunzip $file | grep 'word1\|word2' $filenoext > $filedone | rm -f $filenoext | gzip -f -c $filedone > /donefiles/$file | rm -f $filedone
else
echo "nothing to do here"
fi
done
I need to write a bash script that finds and lists symlinks from one directory (lets say some "Directory1") but only the ones pointing to files in certain another directory (lets say "Directory2"). I can`t use "find".
I have tried something like this but it's apparently wrong:
if [[ -d $1 ]]&&[[ -d $2 ]]
then
current_dir='pwd'
cd $1
do
for plik in *
if[[-L $file] && ["$(readlink -- "$file")" = "$2"] ]
then
#ls -la | grep ^l
echo "$(basename "$file")"
fi
done
fi
How about a simple ls with grep :
ls -l Directory1/ | grep "\->" | grep "Directory2"
I have found a solution:
for file1 in $_cat1/* do
if [ -L $file1]; then
dir1="$(dirname `readlink -f $file1`)"
dir2="$(dirname `readlink -f $_cat2`)"
if [dir1 == dir2]
echo $dir1
fi
fi
done
I have no idea I have spent a lot of hours dealing with this problem. I need to write script. Script should loop recursively through subdirectories in current directory. It should check files count in each directory. If file count is greater than 10 it should write all names of these file in file named "BigList" otherwise it should write in file "ShortList". This should look like
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
---<directory name>
<filename>
<filename>
<filename>
<filename>
....
My script only works if subdirecotries don't include subdirectories in turn.
I am confused about this. Because it doesn't work as I expect. It will take less than 5 minutes to write this on any programming language for my.
Please help to solve this problem , because I have no idea how to do this.
Here is my script
#!/bin/bash
parent_dir=""
if [ -d "$1" ]; then
path=$1;
else
path=$(pwd)
fi
parent_dir=$path
loop_folder_recurse() {
local files_list=""
local cnt=0
for i in "$1"/*;do
if [ -d "$i" ];then
echo "dir: $i"
parent_dir=$i
echo before recursion
loop_folder_recurse "$i"
echo after recursion
if [ $cnt -ge 10 ]; then
echo -e "---"$parent_dir >> BigList
echo -e $file_list >> BigList
else
echo -e "---"$parent_dir >> ShortList
echo -e $file_list >> ShortList
fi
elif [ -f "$i" ]; then
echo file $i
if [ $cur_fol != $main_pwd ]; then
file_list+=$i'\n'
cnt=$((cnt + 1))
fi
fi
done
}
echo "Base path: $path"
loop_folder_recurse $path
I believe that this does what you want:
find . -type d -exec env d={} bash -c 'out=Shortlist; [ $(ls "$d" | wc -l) -ge 10 ] && out=Biglist; { echo "--$d"; ls "$d"; echo; } >>"$out"' ';'
If we don't want either to count subdirectories to the cut-off or to list them in the output, then use this version:
find . -type d -exec env d={} bash -c 'out=Shortlist; [ $(ls -p "$d" | grep -v "/$" | wc -l) -ge 10 ] && out=Biglist; { echo "--$d"; ls -p "$d"; echo; } | grep -v "/$" >>"$out"' ';'
I got this
for file in *; do
any command
done
What I want to do is add an exception to the "for file in *; do".
Any ideas?
If you wanted to skip files with a particular extension, for example, ".pl", you could do:
for file in *
do
[ "${file##*.}" != "pl" ] && echo $file
done
One way to do what I think you are asking is to loop through and check a file name with a if statement (or just grep -v the ls cmd):
for file in `ls`; do
if [ "$file" == "something" ]; then
# do something
else
# something else
fi
done
For example, you have files
$ ls -1 ./
./M2U0001.MPG
./M2U0180.MPG
./text
Exception file is M2U0180.MPG
$> filename="M2U0180.MPG"
And
$> for file in $(ls -1 ./* | grep --invert-match "${filename}" ); do echo $file; done
./M2U0001.MPG
./text
Weird solution done :)
use bash's extended globbing
shopt -s extglob
for f in !(excluded-file); do echo "$f"; done
A simple solution :
The following code snippet will print all the files in the pwd , that have a .py extension
for i in * ;do
if [[ $i == *.py ]];then
echo "$i"
fi
done
I have about 100 directories all in the same parent directory that adhere to the naming convention [sitename].com. I want to rename them all [sitename].subdomain.com.
Here's what I tried:
for FILE in `ls | sed 's/.com//' | xargs`;mv $FILE.com $FILE.subdomain.com;
But it fails miserably. Any ideas?
Use rename(1).
rename .com .subdomain.com *.com
And if you have a perl rename instead of the normal one, this works:
rename s/\\.com$/.subdomain.com/ *.com
Using bash:
for i in *
do
mv $i ${i%%.com}.subdomain.com
done
The ${i%%.com} construct returns the value of i without the '.com' suffix.
What about:
ls |
grep -Fv '.subdomain.com' |
while read FILE; do
f=`basename "$FILE" .com`
mv $f.com $f.subdomain.com
done
See: http://blog.ivandemarino.me/2010/09/30/Rename-Subdirectories-in-a-Tree-the-Bash-way
#!/bin/bash
# Simple Bash script to recursively rename Subdirectories in a Tree.
# Author: Ivan De Marino <ivan.demarino#betfair.com>
#
# Usage:
# rename_subdirs.sh <starting directory> <new dir name> <old dir name>
usage () {
echo "Simple Bash script to recursively rename Subdirectories in a Tree."
echo "Author: Ivan De Marino <ivan.demarino#betfair.com>"
echo
echo "Usage:"
echo " rename_subdirs.sh <starting directory> <old dir name> <new dir name>"
exit 1
}
[ "$#" -eq 3 ] || usage
recursive()
{
cd "$1"
for dir in *
do
if [ -d "$dir" ]; then
echo "Directory found: '$dir'"
( recursive "$dir" "$2" "$3" )
if [ "$dir" == "$2" ]; then
echo "Renaming '$2' in '$3'"
mv "$2" "$3"
fi;
fi;
done
}
recursive "$1" "$2" "$3"
find . -name '*.com' -type d -maxdepth 1 \
| while read site; do
mv "${site}" "${site%.com}.subdomain.com"
done
Try this:
for FILE in `ls -d *.com`; do
FNAME=`echo $FILE | sed 's/\.com//'`;
`mv $FILE $FNAME.subdomain.com`;
done