How could I read the content of a parent folder and if sub-folders are found, then make a tar.gz files of those subfolders found. However, I know that subfolders will have the following filename format: name-1.2.3 What I want is to create a tar.gz file that looks like: name-1.2.3-20100928.tar.gz Any help will be appreciated.
#!/bin/sh
DATE=`/bin/date +%y%m%e`
cd /path/to/your/folder
for folder in *; do
if [ -d $folder ]; then
tar -cvzf $folder-$DATE.tar.gz $folder
fi
done
Better immunize your scripts against spaces in names:
#!/bin/bash
# Yes nested quoting is allowed with $(...) syntax, no matter whether syntax coloration does
DATE="$(/bin/date "+%y%m%e")"
cd /path/to/your/folder
for folder in * ; do
if test -d "$folder" ; then
tar cvzf "$folder-$DATE.tar.gz" "$folder"
fi
done
Related
UPDATE
Thank you everyone for sharing up your suggestions. I was able to make the work properly by modifying it as follows.
#!/bin/bash
ROOTPATH="/Volumes/NVME-RAID/ASSET-Processing/CORRUPT-SCAN/SCAN " # manually define root path to your folder that has subfolders
for subdir in *; do
cd ${ROOTPATH}${subdir}
mkdir 00errors
for path in *.{MOV,mov,MP4,mp4}; do
ffmpeg -i "${path}" -f null -; echo$?
RC=$?
if [ "${RC}" -ne "0" ]; then
# Do something to handle the error.
mv ${path} ./00errors
fi
done
cd ../
done
The only issue i have now is that it does not appear to be traversing the subfolders. As I understand it it should create a "00errors" folder in within EACH sub folder and move the errors files within that sub folder.
trying to sift through 14TB of recovered video...
I'm trying to figure out how to properly convert this script so it will run on a bash/MacOS. I'm coming to a wall with the "ROOTPATH" call because MacOS doesn't use /mnt
#!/bin/bash
ROOTPATH="/mnt/f/00test/" # manually define root path to your folder that has subfolders
for subdir in *; do
cd ${ROOTPATH}${subdir}
mkdir 00errors
for path in *.mp4; do
ffmpeg error -i "${path}"
RC=$?
if [ "${RC}" -ne "0" ]; then
# Do something to handle the error.
mv ${path} ./00errors
fi
done
cd ../
done
REF Quickly check the integrity of video files inside a directory with ffmpeg
If your test dir was in your home directory /Users/username aka ~:
ROOTPATH="~/00test" # manually define ROOTPATH to your folder that has subfolders
for subdir in *; do
cd ${ROOTPATH}/${subdir}
I follow the convention to not use trailing slashes in paths; here I have moved the slash to the cd command.
My script should check a directory called /etc/scriptbuilder/default/contents which will hold some directories, if a directory in the contents directory matches one in the project directory the files in the contents directory should be copied to the project directory (so, if /etc/scriptbuilder/default/contents/docs and /project/docs both exist, the contents of /etc/scriptbuilder/default/contents/docs should be copied).
I'm having trouble making this work. It has to find a match of the same directory name and copy the contents over from the contents directory to the project directory if they have the same directory name. This is what I have so far:
#! /bin/bash
if [ -d "$/etc/scriptbuilder/default/contents ]; then
if [[ "/etc/scriptbuilder/default/contents" =~ name ]]; then
cp -a #I'm not sure how to copy and check for the name on the project
directory
fi
fi
Something like that finds the the directories you are looking for and prints them.
source="/etc/scriptbuilder/default/contents"
destination="/home/something/project"
# loop the directories of the source folder
cd "$source"
for name in */ ; do
# create the source and the possible destination directory paths
s="$source/$name"
d="$destination/$name"
# check if the directory exists in the project
if [ -d "$d" ]; then
echo "TODO: Perfrom copy of from $s to $d"
fi
done
If you need help with copy command also post a comment.
With loop it could be like this:
Dir1=path_to_contents/contents
Dir2=path_to_project/project
for i in $(find "$Dir1/" "$Dir2/" -printf '%P\n' | sort | uniq -d)
do
cp -r $Dir1/$i $Dir2
done
I have a folder structure like this:
A big parent folder named Photos. This folder contains 900+ subfolders named a_000, a_001, a_002 etc.
Each of those subfolders contain more subfolders, named dir_001, dir_002 etc. And each of those subfolders contain lots of pictures (with unique names).
I want to move all these pictures contained in the subdirectories of a_xxx inside a_xxx. (where xxx could be 001, 002 etc)
After looking in similar questions around, this is the closest solution I came up with:
for file in *; do
if [ -d $file ]; then
cd $file; mv * ./; cd ..;
fi
done
Another solution I got is doing a bash script:
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
for i in $subs; do
mv $dir1/$i/*/* $dir1/$i/
done
Still, I'm missing something, can you help?
(Then it would be nice to discard the empty dir_yyy, but not much of a problem at the moment)
You could try the following bash script :
#!/bin/bash
#needed in case we have empty folders
shopt -s nullglob
#we must write the full path here (no ~ character)
target="/path/to/photos"
#we use a glob to list the folders. parsing the output of ls is baaaaaaaddd !!!!
#for every folder in our photo folder ...
for dir in "$target"/*/
do
#we list the subdirectories ...
for sub in "$dir"/*/
do
#and we move the content of the subdirectories to the parent
mv "$sub"/* "$dir"
#if you want to remove subdirectories once the copy is done, uncoment the next line
#rm -r "$sub"
done
done
Here is why you don't parse ls in bash
Make sure the directory where the files exist is correct (and complete) in the following script and try it:
#!/bin/bash
BigParentDir=Photos
for subdir in "$BigParentDir"/*/; do # Select the a_001, a_002 subdirs
for ssdir in "$subdir"/*/; do # Select dir_001, … sub-subdirs
for f in "$ssdir"/*; do # Select the files to move
if [[ -f $f ]]; do # if indeed are files
echo \
mv "$ssdir"/* "$subdir"/ # Move the files.
fi
done
done
done
No file will be moved, just printed. If you are sure the script does what you want, comment the echo line and run it "for real".
You can try this
#!/bin/bash
dir1="/path/to/photos/"
subs= `ls $dir1`
cp /dev/null /tmp/newscript.sh
for i in $subs; do
find $dir1/$i -type f -exec echo mv \'\{\}\' $dir1/$i \; >> /tmp/newscript.sh
done
then open /tmp/newscript.sh with an editor or less and see if looks like what you are trying to do.
if it does then execute it with sh -x /tmp/newscript.sh
I have a folder that after an rsync will have a zip in it. I want to unzip it to its own folder(if the zip is L155.zip, to unzip its content to L155 folder). The problem is that I dont know it's name beforehand(although i know it will be "letter-number-number-number"), so I have to unzip an uknown file to its unknown folder and this to be done automatically.
The command “unzip *”(or unzip *.zip) works in terminal, but not in a script.
These are the commands that have worked through terminal one by one, but dont work in a script.
#!/bin/bash
unzip * #also tried .zip and /path/to/file/* when script is on different folder
i=$(ls | head -1)
y=${i:0:4}
mkdir $y
unzip * -d $y
First I unzip the file, then I read the name of the first extracted file through ls and save it in a variable.I take the first 4 chars and make a directory with it and then again unzip the files to that specific folder.
The whole procedure after first unzip is done, is because the files inside .zip, all start with a name that the zip already has, so if L155.ZIP is the zip, the files inside with be L155***.txt.
The zip file is at /path/to/file/NAME.zip.
When I run the script I get errors like the following:
unzip: cannot find or open /path/to/file/*.ZIP
unzip: cannot find or open /path/to/file//*.ZIP.zip
unzip: cannot find or open /path/to/file//*.ZIP.ZIP. No zipfiles found.
mkdir: cannot create directory 'data': File exists data
unzip: cannot find or open data, data.zip or data.ZIP.
Original answer
Supposing that foo.zip contains a folder foo, you could simply run
#!/bin/bash
unzip \*.zip \*
And then run it as bash auto-unzip.sh.
If you want to have these files extracted into a different folder, then I would modify the above as
#!/bin/bash
cp *.zip /home/user
cd /home/user
unzip \*.zip \*
rm *.zip
This, of course, you would run from the folder where all the zip files are stored.
Another answer
Another "simple" fix is to get dtrx (also available in the Ubuntu repos, possibly for other distros). This will extract each of your *.zip files into its own folder. So if you want the data in a different folder, I'd follow the second example and change it thusly:
#!/bin/bash
cp *.zip /home/user
cd /home/user
dtrx *.zip
rm *.zip
I would try the following.
for i in *.[Zz][Ii][Pp]; do
DIRECTORY=$(basename "$i" .zip)
DIRECTORY=$(basename "$DIRECTORY" .ZIP)
unzip "$i" -d "$DIRECTORY"
done
As noted, the basename program removes the indicated suffix .zip from the filename provided.
I have edited it to be case-insensitive. Both .zip and .ZIP will be recognized.
for zfile in $(find . -maxdepth 1 -type f -name "*.zip")
do
fn=$(echo ${zfile:2:4}) # this will give you the filename without .zip extension
mkdir -p "$fn"
unzip "$zfile" -d "$fn"
done
If the folder has only file file with the extension .zip, you can extract the name without an extension with the basename tool:
BASE=$(basename *.zip .zip)
This will produce an error message if there is more than one file matching *.zip.
Just to be clear about the issue here, the assumption is that the zip file does not contain a folder structure. If it did, there would be no problem; you could simply extract it into the subfolders with unzip. The following is only needed if your zipfile contains loose files, and you want to extract them into a subfolder.
With that caveat, the following should work:
#!/bin/bash
DIR=${1:-.}
BASE=$(basename "$DIR/"*.zip .zip 2>/dev/null) ||
{ echo More than one zipfile >> /dev/stderr; exit 1; }
if [[ $BASE = "*" ]]; then
echo No zipfile found >> /dev/stderr
exit 1
fi
mkdir -p "$DIR/$BASE" ||
{ echo Could not create $DIR/$BASE >> /dev/stderr; exit 1; }
unzip "$DIR/$BASE.zip" -d "$DIR/$BASE"
Put it in a file (anywhere), call it something like unzipper.sh, and chmod a+x it. Then you can call it like this:
/path/to/unzipper.sh /path/to/data_directory
simple one liner I use all the time
$ for file in `ls *.zip`; do unzip $file -d `echo $file | cut -d . -f 1`; done
I'm writing a shell script as follows:
for file in `ls`
do
mkdir "$file"_folder
cp $file "$file"_folder
done
What I want to do is to make a folder for each file in the current directory with its name and then underscore folder as the name and then copy that file into it. My problem is that the file names contain spaces in them. How do I escape them?
There are many resources explaining how to do this for variables but none of them can be applied to this situation where I use a for loop to get the names.
Don't use ls there, use shell globbing. (In general, do not parse the output of ls.)
for file in *
do
# only consider files, not directories
if [ -f "$file" ] ; then
new_dir="$file"_folder
# create the directory
if [ ! -d "$new_dir" ] ; then
mkdir "$new_dir"
if [ $? -ne 0 ] ; then
# handle directory creation eror
fi
fi
# possibly check for the copied file existence here
# and deal with that appropriately (i.e. skip/error/copy anyway)
cp "$file" "$new_dir"
fi
done
How about
find . -type f -exec mkdir {}_folder \; -exec cp {} {}_folder \;
It finds all regular files in the current directory, creates the folder (first -exec), and copies the file into the new folder (second -exec).
You do not parse
ls for exactly this reason
for file in *
do
mkdir "${file}_folder"
cp "$file" "${file}_folder"
done