I have a folder called folder1 that has a bunch of pictures in it, and I would like to save each picture in an individual folder. The folder should have the same name as the file, for example if the picture is called "st123" I would like to create a folder called "st123" and move the picture into the folder. I have tried the following, but I get the error cannot move to a subdirectory of itself. Is there another way to solve this?
Otherwise, would it be possible to save the image "st123" in a folder called "123" (so the last 3 characters)?
#!/bin/bash
Parent="/home/me/myfiles/folder1"
for file in $Parent; do
dir="${file%%.*}"
mkdir -p "$dir"
mv "$file" "$dir"
done
This solution might be useful to you, though if you'd like to iterate over the images recursively I suggest you to use find. If that's indeed the case, let me know and I'll edit this answer with relevant code.
for image in /home/me/myfiles/folder1/*; do
if [[ -f $image ]]; then
newf="${image%%?([ab]).*}"
mkdir -p "$newf"
mv -- "$image" "$newf"
fi
done
Test ( extglob is enabled ):
$ [/home/rany/] touch test/st123a.tif test/st123b.tif test/st456.jpg test/st456b.jpg test/st789.tif
$ [/home/rany/] for image in test/*; do newf="${image%%?([ab]).*}"; mkdir -p "$newf"; mv -- "$image" "$newf";done
$ [/home/rany/] tree test
test
├── st123
│ ├── st123a.tif
│ └── st123b.tif
├── st456
│ ├── st456b.jpg
│ └── st456.jpg
└── st789
└── st789.tif
3 directories, 5 files
EDIT:
According to OP request I added the following changes:
Trim file-name's suffix a or b, so that, for example, file names st123a.ext and st123b.ext will go to the same directory st123. Important: this feature requires extglob enabled ( i.e. shopt -s extglob ).
You almost have it working! Change part of line 3 so the for loop iterates over what's inside the directory instead of the directory itself:
#!/bin/bash
Parent="/home/me/myfiles/folder1"
for file in $Parent/*; do
dir="${file%%.*}"
mkdir -p "$dir"
mv "$file" "$dir"
done
https://www.gnu.org/software/bash/manual/html_node/Command-Substitution.html#Command-Substitution
Related
Create Folder for Each File in Recursive Directory, Placing File in Folder
On MacOS, so far I have...
for file in $(ls -R); do
if [[ -f "$file" ]]; then mkdir "${file%.*}"; mv "$file" "${file%.*}"; fi;
done
This operates correctly on the top level of the nested folder, but does nothing with lower levels.
To isolate the error, I tried this instead, operating on rtf files . .
for i in $(ls -R);do
if [ $i = '*.rtf' ];then
echo "I do something with the file $i"
fi
done
This hangs, so I simplified to . .
for i in $(ls -R); do echo "file is $i" done
That hangs also, so I tried . .
for i in $(ls -R); do echo hello
That hangs also.
ls -R works to provide a recursive list of all files.
Suggestions appreciated !!
First of all don't use ls in scripts. It is meant to show output interactively. Although the newish GNU ls version has some features/options for shell parsing, not sure about on a Mac though.
Now using find with the sh shell.
find . -type f -name '*.rtf' -execdir sh -c '
for f; do mkdir -p -- "${f%.*}" && mv -v -- "$f" "${f%.*}"; done' _ {} +
For whatever reason -execdir is not available, one can use -exec
find . -type f -name '*.rtf' -exec sh -c '
for f; do mkdir -p -- "${f%.*}" && mv -v -- "$f" "${f%.*}" ; done' _ {} +
See Understanding-the-exec-option-of-find
Given this file structure:
$ tree .
.
├── 123
│ └── test_2.rtf
├── bar
│ ├── 456
│ │ └── test_1.rtf
│ └── 789
└── foo
There are two common ways to find all the .rtf files in that tree. The first (and most common) is to use find:
while IFS= read -r path; do
echo "$path"
done < <(find . -type f -name "*.rtf")
Prints:
./bar/456/test_1.rtf
./123/test_2.rtf
The second common way is to use a recursive glob. This is not a POSIX way and is only found in more recent shells such as Bash, zsh, etc:
shopt -s globstar # In Bash, this enables recursive globs
# In zsh, this is not required
for path in **/*.rtf; do
echo "$path"
done
# same output
Now that you have the loop to find the files, you can modify the files found.
The first issue you will run across is that you cannot have two files with the same name in a single directory; a directory is just a type of file. So you will need to proceed this way:
Find all the files with their paths;
Create a tmp name and create a sub-directory with that temp name;
Move the found file into the temp directory;
Rename the temp directory to the found file name.
Here is a Bash (or zsh that is default on MacOS) script to do that:
shopt -s globstar # remove for zsh
for p in **/*.rtf; do
[ -f "$p" ] || continue # if not a file, loop onward
tmp=$(uuidgen) # easy temp name -- not POSIX however
fn="${p##*/}" # strip the file name from the path
path_to="${p%/*}" # get the path without the file name
mkdir "${path_to}${tmp}" # use temp name for the directory
mv "$p" "${path_to}$tmp" # move the file to that directory
mv "${path_to}$tmp" "$p" # rename the directory to the path
done
And the result:
.
├── 123
│ └── test_2.rtf
│ └── test_2.rtf
├── bar
│ ├── 456
│ │ └── test_1.rtf
│ │ └── test_1.rtf
│ └── 789
└── foo
I am trying to take a directory filled with an unknown number of files and put each file into a subdirectory based on the name of the file. A file's name could have multiple subdirectories in it all separated by the underscore character so it needs to be able to recursively extract subdirectory names.
Example Files:
2020_Documents_Bills_Water Bill.pdf
2020_Documents_Taxes_W2.pdf
2020_Documents_Receipts_Store Name_Groceries.pdf
2020_Pictures_Family Trip_California_Disney Land_Family Pic.jpg
So the 2020_Documents_Bills_Water Bill.pdf file would end up as 2020/Documents/Bills/Water Bill.pdf.
I would like to limit the tools used to bash, sed, grep, mkdir, and mv if possible.
I had some thoughts on how I believe the script should flow, but I don't know how to make it recursively get subdirectories without a lot of yucky if statements. I was thinking this code could probably get the first subdir and put it in an array and then remove that text and the underscore that follows it from the name of the file and then iterate again until it runs out of underscores.
#!/bin/bash
# cd to directory where files are located
cd /directory/with/files
# iterate over files in directory
for file in *; do
subDirs=() # empty array for subdirs
filePath="" # empty string to build filepath
# ------------------------------------------------------------
# code to extract subdir names and add to subDirs array
# ------------------------------------------------------------
# build filepath using string of all subdirs
for i in ${!subDirs[#]}; do
filepath="${filePath}/${subDirs[$i]}"
done
# set filename to text after last underscore
filename=${file##*_}
# make filepath based on subdirs
mkdir -p "${filepath}"
# move file into filepath without subdirs in name
mv ${file} "${filepath}/${filename}"
done
You can do it simpler because mkdir -p path/to/yours works just with one invocation. You do not have to recursively create subdirectories one by one.
Would you please try:
cd /directory/with/files # cd to directory where files are located
for file in *; do
[[ -f $file ]] || continue # skip non-file entries (just in case)
dir=${file%_*}
base=${file##*_}
dir=${dir//_/\/} # replace "_"s with "/"s
mkdir -p "$dir"
mv -- "$file" "$dir/$base"
done
[Strict Version]
The script below performs the validation of the filenames (with a help of jhnc).
for file in *; do
[[ -f $file ]] || continue # skip non-file entries (just in case)
dir=${file%_*}
base=${file##*_}
dir=${dir//_//} # replace "_"s with "/"s
# validate filenames
case "/$dir/" in
*/../* | */./* | //*) # $dir contains extra dot(s)
echo "skipping invalid filename: $file"
continue
;;
esac
if [[ -z $base ]]; then # the filename ends with "_"
echo "skipping invalid filename: $file"
continue
fi
mkdir -p "$dir"
mv -- "$file" "$dir/$base"
done
Result:
/directory/
└── with
└── files
└── 2020
├── Documents
│ ├── Bills
│ │ └── Water Bill.pdf
│ ├── Receipts
│ │ └── Store Name
│ │ └── Groceries.pdf
│ └── Taxes
│ └── W2.pdf
└── Pictures
└── Family Trip
└── California
└── Disney Land
└── Family Pic.jpg
Just one note to add cd /directory/with/files
Without the exit . Assuming /directory/with/files does not exists.
#!/bin/bash
# cd to directory where files are located
cd /directory/with/files
printf '%s\n' 'rm this' 'rm that' 'mv this' 'mv that'
the output is
myscript: line 4: cd: /directory/with/files: No such file or directory
rm this
rm that
mv this
mv that
all the code after the cd is/was still executed!
With the exit and assuming the /directory/with/files does not exists.
#!/bin/bash
# cd to directory where files are located
cd /directory/with/files || exit
printf '%s\n' 'rm this' 'rm that' 'mv this' 'mv that'
The output.
myscript: line 4: cd: /directory/with/files: No such file or directory
The script did exit and it did not execute the rest of the code.
I have some data for several stations that are separated for the station they are in, and the day they were recorded, so for station 1, for example, I have multiple folders called 2019.001,2019.002 etc. and inside these folders I have the files (all with the same name) ending with HHZ. What I have done is getting these files from each of the stations and putting them on another folder while renaming them to have the name of the folder above and maintaining the name of the station, afterwards I created the folders corresponding to their names. My actual question is how to move the files that correspond to the same day, e.g. 2019.001.station1 and 2019.001.station2 to the folder 2019.001.
dir0=`pwd`
mkdir -p data || exit 1
for pathname in $dir0/stam/*/*HHZ; do
cp "$pathname" "data/$( basename "$( dirname "$pathname" )" )STAMHHZ"
done
for pathname in $dir0/macu/*/*HHZ; do
cp "$pathname" "data/$( basename "$( dirname "$pathname" )" )MACUHHZ"
done
cd $dir0/data
mkdir 2019.0{10..31}
mkdir 2019.00{1..9}
If there is also another way of executing the part of the code where I take the files so I can generalize for several stations that would be nice, since I am only working with two stations right now but in the future I'll work with more.
Here is the tree to where the data is
macu
├── 2019.001
│ └── MACUHHZ
├── 2019.002
│ └── MACUHHZ
├── 2019.003
And
stam
├── 2019.001
│ └── STAMHHZ
├── 2019.002
│ └── STAMHHZ
├── 2019.003
│ └── STAMHHZ
So ideally the final situation would be:
data
├── 2019.001
│ ├── 2019.001MACUHHZ
│ └── 2019.001STAMHHZ
And so on
The script below creates the wanted file structure. The top level directories from which you want to copy data (in your example macu and stam) should be added to the top_dir variable. You can also change it to use a wildcard, or read them from a file, etc.
The basic idea is simple: For each top level directory, for each data directory, create the corresponding directory in data, and for each file, copy the file.
pushd and popd are used as a simple hack to make the * wildcards do what we want. $dir0 contains the root folder of the operation, so we always know where data is.
set -e is used to exit immediately if there is an error.
#!/bin/bash
set -e
top_dirs=( macu stam )
dir0="$(pwd)"
mkdir -p data
for dir in "${top_dirs[#]}" ; do
pushd "$dir" >/dev/null
for datadir in * ; do
mkdir -p "$dir0/data/$datadir"
pushd "$datadir" >/dev/null
for file in *HHZ ; do
cp "$file" "$dir0/data/$datadir/$datadir$file"
done
popd >/dev/null
done
popd >/dev/null
done
I am looking for a way to archive all files of certain file types in one zip file per subfolder.
My folder structure is as follows:
/path/to
└── TopLevel
├── SubLevel1
│ ├── SubSubLevel1
│ ├── SubSubLevel2
│ └── SubSubLevel3
├── SubLevel2
│ ├── SubSubLevel1
│ ├── SubSubLevel2
│ └── SubSubLevel3
├── SubLevel3
│ ├── SubSubLevel1
│ └── SubSubLevel2
└── SubLevel4
In each folder or subfolder or sub-subfolder, there are files of the file type *.abc, *.xyz and also *.001 through *.999 and all these files I want to compress into one zip file per folder, i.e. all files of the specified types in folder "SubSubLevel1" of "SubLevel1" of "TopLevel" should be packaged into one file named "SubSubLevel1_data.zip" inside the "SubSubLevel1" folder. All other files in these folders, which do not match the search criteria as described above, should be kept unzipped in the same directory.
I have found some ideas here or here, but both approaches are based on a different way of archiving the files and I have so far not found a way to adopt them to my needs since I am not very experienced with shell scripting. I have also tried to get a solution with AppleScript, but there I face the problem how to get all files in the folder with the number as an extension (*.001 through *.999). With RegEx, I would do something like ".abc|.xyz.\d\d\d" which would cover my search for certain file types, but I am also not sure now how to implement the result of a grep in AppleScript.
I guess someone out there must have an idea how to address my archiving issue. Thanks in advance for your suggestions.
After some playing around I came up with the following solution:
#!/bin/bash
shopt -s nullglob
find -E "$PWD" -type d -maxdepth 1 -regex ".*201[0-5][0-1][0-9].*" -print0 | while IFS="" read -r -d "" thisFolder ; do
echo "The current folder is: $thisFolder"
to_archive=( "$thisFolder"/*.[Aa][Bb][Cc] "$thisFolder"/*.[Xx][Yy][Zz] "$thisFolder"/*.[0-9][0-9][0-9] )
if [ ${#to_archive[#]} != 0 ]
then
7z a -mx=9 -uz1 -x!.DS_Store "$thisFolder"/"${thisFolder##*/}"_data.7z "${to_archive[#]}" && rm "${to_archive[#]}"
fi
find "$thisFolder" -type d -mindepth 1 -maxdepth 1 -print0 | while IFS="" read -r -d "" thisSubFolder ; do
echo "The current subfolder is: $thisSubFolder"
to_archive=( "$thisSubFolder"/*.[Aa][Bb][Cc] "$thisSubFolder"/*.[Xx][Yy][Zz] "$thisSubFolder"/*.[0-9][0-9][0-9] )
if [ ${#to_archive[#]} != 0 ]
then
7z a -mx=9 -uz1 -x!.DS_Store "$thisSubFolder"/"${thisSubFolder##*/}"_data.7z "${to_archive[#]}" && rm "${to_archive[#]}"
fi
done
done
My script has two nested for loops to iterate through subfolders and sub-subfolders. With "find" I look for a regex pattern in order to only backup folders from 2010-2015 . All files matching the specified extensions inside the folders are compressed in one target archive per folder.
I am trying to mass rename folders and their respective files by use of the command "mv" but its not going very well, I would like the folders to be renamed into:
1/
2/
3/
etc.
The files in each folder should in this process also be renamed while keeping their file extension.
1.png
2.gif
3.jpg
etc
Very thankful for your help
EDIT: Better approach with a function rather than recursive calls to the same script.
Here's hoping I got all corner cases. This descends into directories recursively to handle deeply nested directory trees.
Caveat: Because the script takes care to not overwrite existing files, gaps may appear in the numbering in some corner cases -- if there is a file 0.txt in a directory and the first file that is handled in that directory is a .txt file, it will be moved to 1.txt. This also happens if the first file that is handled is 0.txt, so running the script twice will change the numbering, and running it again will change it back.
So here's the code:
#!/bin/bash
handle_directory() {
local counter=0
for i in *; do
# if the file is a directory (but not a symlink),
# handle it before moving
if [ -d "$i" ] && ! [ -h "$i" ]; then
cd "$i"
handle_directory
cd ..
fi
# extract suffix
suffix="${i##*.}"
if [ "$suffix" != "$i" ]; then
extension=".$suffix"
else
# If there is no filename extension, the counter
# is the whole filename. Without this, we'd get
# 0.Makefile and suchlike.
extension=""
fi
# find a filename that isn't already taken
# this may lead to gaps in the numbering.
while dest="$counter$extension" && [ -e "$dest" ]; do
let ++counter
done
echo mv "$i" "$dest"
let ++counter
done
}
# if a parameter was given, go there to handle it.
# otherwise handle the local directory.
if ! [ -z "$1" ] && ! cd "$1"; then
echo "Could not chdir to directory $1"
exit -1
fi
handle_directory
The general idea is a depth-first search of the directory tree in question. Like any tree, the directory tree is best handled recursively, and the function essentially boils down to: Walk through all things in this directory, if they are a directory, descend and handle it, then find an appropriate file name and rename the thing whether it is a directory or not.
Things used:
local counter=0 # declares a function-local variable counter and initializes it
# to 0. crucially, that it is local means that every invocation
# of handle_directory has its own counter.
[ -d "$i" ] # tests if "$i" is a directory
[ -h "$i" ] # tests if "$i" is a symlink
! [ ... ] # negates the test. ! [ -h "$i" ] is true if "$i" is NOT a symlink
"${i##*.}" # a bashism that cuts off the longest prefix that matches the pattern
[ -e "$dest" ] # tests if "$dest" exists
$1 # the first parameter with which the script is called
[ -z "$1" ] # tests if "$1" is an empty string
! cd "$1" # descends into the directory "$1". true if that failed.
Manpages to read to further understanding:
man test
man bash # that's the big one.
Is it a single level directory? Or do you have subdirectories?
Something like this?
Before:
.
├── folderE
│ ├── conf.properties
│ ├── omg.avi
│ └── test-diff.diff
├── folder_a
│ ├── first.png
│ ├── main.jpg
│ └── second.gif
├── folder_d
│ ├── another.zip
│ └── one.mpeg
└── with space
└── file with spaces too.mov
Command:
countDir=1
for dir in *; do
cd "$dir";
countFile=1
for file in *; do
mv "$file" $countFile.${file#*.}
((countFile++))
done
cd ..
mv "$dir" $countDir
((countDir++))
done
Or, the same in one line:
countDir=1; for dir in *; do cd "$dir"; countFile=1; for file in *; do mv "$file" $countFile.${file#*.}; ((countFile++)); done; cd ..; mv "$dir" $countDir; ((countDir++)); done
After:
.
├── 1
│ ├── 1.properties
│ ├── 2.avi
│ └── 3.diff
├── 2
│ ├── 1.png
│ ├── 2.jpg
│ └── 3.gif
├── 3
│ ├── 1.zip
│ └── 2.mpeg
└── 4
└── 1.mov
Important: keep in mind that this is just a quick and dirty solution and there's no checking for files/directoris already named "1" "2" etc.