I would like to move some files from one folder to another. Say I have:
├── bar
│ ├── a_folder
│ │ └── apicture.png
│ └── another_folder
│ └── myfile.txt
└── foo
And I would like to have every txt files moved from bar to foo:
├── bar
│ ├── a_folder
│ └── apicture.png
│ └── another_folder
└── foo
└── another_folder
└── myfile.txt
I tried using find command using:
find bar -type f -exec mv {} $temp/{} \;
This should work but only if folders are already existing in foo folder. Thus I am looking for a way to create these folders in foo, and it seems that mv does not do that. How can I achieve that?
cp -R bar/. foo/
find bar -type f -delete
Yes that involves a copy and isn't a one-liner, but unless we're talking huge amounts of data and a recurring job here, it's a good stop-gap. (Note the bar/. instead of the usual bar/*, as the latter would miss dot-files.)
I doubt there is a one-liner way to move the files but keep the directories anyway.
rsync has this capability built in.
cd bar && rsync --relative [--no-implied-dirs] . ../foo/
Use --no-implied-dirs if there are symlinks in the tree you want to keep as symlinks.
If you only want to move files where the destination directory already exists, use the --no-dirs flag.
rsync is not in the POSIX toolset, but it is common on UNIX-like machines including OS X and most Linuxen.
Related
I'm hoping someone can help me, I'm looking for help with a bash script to move files by file extension from multiple directories, one directory deeper within their respective directories.
For example I have a 'Projects' directory, that has multiple directories within in it 'Project 001, Project 002, Project 003' and so on. I have '.JPG' files in those folders and want to put them in a 'JPG' folder within their respective project folder - how would I do this with a bash script as there are too many to do by hand?
To make matters more complicated some of those project folders already have 'JPG' folders in them, and some of those have some of the files already in them as duplicates, so I think I'd need to add an overwrite confirmation to it too based on filename.
I know a little bash, but this is a little over my head, so any help would be greatly appreciated.
Thanks,
/ Hami
This is using a CLI on Ubuntu Server 18.04 LTS, on a disk with thousands of directories with human filenames - spaces, parenthesis, and varied unicode character such as Japanese, and European. The names of the directories are varied, and have no particular formula.
Ideally, I'd like to go from this:
Projects
Project 001
Image 001.jpg
Project 002
Image 002.jpg
Image 003.jpg
Project 003
...to this:
Projects
Project 001
JPG
Image 001.jpg
Project 002
JPG
Image 002.jpg
Image 003.jpg
Project 003
You can do what you need with find that searchers for all *.jpg files and a simple helper script called by the -exec option to find to create the jpg directory and move all .jpg files into the new directory.
The helper script will simply get the absolute filename utilitzing readlink -f and then using quick parameter expansion to trim the last /... component from the absolute filename to obtain the full path. Then it is simply a matter of creating the jpg directory at the end of the path and moving the file to the new directory.
Your helper script (I called it helper.sh) could be:
#!/bin/sh
test -z "$1" && exit ## validate 1 argument given or exit
full=$(readlink -f "$1") ## get full filename
dir="${full%/*}" ## get full path
test "${full##*.}" = 'jpg' || exit ## test extension is jpg or exit
test -z "$dir" && dir="/" ## check if file was in / (root)
test -d "$dir/jpg" || mkdir -p "$dir/jpg" ## check/create jpg dir at end of path
mv "$full" "$dir/jpg" ## move file into new jpg dir
(note: after creating the helper script, make sure you make it executable with chmod +x helper.sh)
Original Projects Directory Tree
$ tree Projects/
Projects/
├── Project_001
│ └── Image_001.jpg
├── Project_002
│ ├── Image_002.jpg
│ └── Image_003.jpg
└── Project_003
Your find command operating on the Projects directory calling the helper script for each file would be:
$ find Projects/ -type f -name "*jpg" -exec ./helper.sh '{}' \;
Resulting Projects Directory Tree
$ tree Projects/
Projects/
├── Project_001
│ └── jpg
│ └── Image_001.jpg
├── Project_002
│ └── jpg
│ ├── Image_002.jpg
│ └── Image_003.jpg
└── Project_003
Let me know if you have further questions.
Preserving Files Already In jpg Directory
Per-your additional comment, in order to preserve .jpg files already within a jpg directory under your Projects, all you need to do is add one additional check. If the last component of the path is already jpg, just exit the helper, e.g.
test "${dir##*/}" = 'jpg' && exit ## if already in jpg dir, exit
Shown in context in helper.sh:
test -z "$1" && exit ## validate 1 argument given or exit
full=$(readlink -f "$1") ## get full filename
dir="${full%/*}" ## get full path (trim last /*)
test "${dir##*/}" = 'jpg' && exit ## if already in jpg dir, exit
test "${full##*.}" = 'jpg' || exit ## test extension is jpg or exit
...
Original Projects Directory Tree (w/existing jpg)
$ tree Projects/
Projects/
├── Project_001
│ ├── Image_001.jpg
│ └── jpg
│ └── Image_000.jpg
├── Project_002
│ ├── Image_002.jpg
│ ├── Image_003.jpg
│ └── jpg
│ ├── Image_000.jpg
│ └── Image_001.jpg
└── Project_003
Resulting Projects Directory Tree
$ tree Projects/
Projects/
├── Project_001
│ └── jpg
│ ├── Image_000.jpg
│ └── Image_001.jpg
├── Project_002
│ └── jpg
│ ├── Image_000.jpg
│ ├── Image_001.jpg
│ ├── Image_002.jpg
│ └── Image_003.jpg
└── Project_003
Maybe one of you guys has something like this at hand already? I tried to use robocopy on windows but to no avail. I also tried to write a bash script in linux with find etc... but gave up on that one also ^^ Google search brought no solution also unfortunately. I need this for my private photo library.
Solution could be linux or windows based, both are fine. Any ideas?
I would like to get rid of hundreds of 'intermediary folders'.
I define an 'intermediary folder' as a folder that contains nothing else than exactly one sub-folder. Example
folder 1
file in folder 1
folder 2 <-- 'intermediary folder: contains exactly one sub-folder, nothing else'
folder 3
file in folder 3
What I would like to end up with is:
folder 1
file in folder 1
folder 3
file in folder 3
I do not need the script to be recursive (removing several layers of intermediary folders at once), I'll just run it several times.
Even cooler would be if the script could rename folder 3 in the above example to 'folder 2 - folder 3', but I can live without this feature I guess.
I guess one of you linux experts has a one liner handy for that? ^^
Thank you very much!
Take a look at this code:
#!/usr/bin/env bash
shopt -s nullglob
while IFS= read -rd '' dir; do
f=("$dir"/*)
if ((${#f[#]}==1)) && [[ -d $f ]]; then
mv -t "${dir%/*}" "$f" || continue
rm -r "$dir"
fi
done < <(find folder1 -depth -mindepth 1 -type d -print0)
Explanation:
shopt -s nullglob: allows filename patterns which match no files to expand to a null string
find ... -depth: makes find traverse the file system in a depth-first order
find ... -mindepth 1: processes all directories except the starting-point
find ... -type d: finds only directories
find ... -print0: prints the directories separated by a null character \0 (to correctly handle possible newlines in filenames)
while IFS= read ...: loops over all the directories (the output of find)
f=("$dir"/*): creates an array with all files in the currently processed directory
((${#f[#]}==1)) && [[ -d $f ]]: true if there is only one file and it is a directory
mv -t "${dir%/*}" "$f": moves the only subdirectory one directory above
mv ... || continue: mv can fail if the subdirectory already exists in the directory above. || continue ignores such subdirectory
rm -r "$dir": removes the processed directory
Test run:
$ tree folder1
folder1
├── file1
├── folder2
│ └── folder3
│ └── file3
├── folder4
│ ├── file4a
│ ├── file4b
│ └── file4c
└── folder5
└── folder6
├── file6
└── folder7
└── folder8
└── folder9
├── dir9
└── file9
$ ./script
$ tree folder1
folder1
├── file1
├── folder3
│ └── file3
├── folder4
│ ├── file4a
│ ├── file4b
│ └── file4c
└── folder6
├── file6
└── folder9
├── dir9
└── file9
I'm looking to make a crontab that will search through a directory and all subdirectories and find all files with extension *.mkv then move them to a different directory and create an empty file with the same name and extension in place of the original file.
So it would look like this:
find *.mkv in subdirectories of /home/user/directoryA/~
move *.mkv to /home/user/directoryB/
create empty *.mkv with same filename as the original in place of file in /home/user/directoryA/~
What would be the best way to accomplish this?
The process isn't too difficult if you recognize that when forming your new directory names, your old base directory will simply be a substring within the new directory name. Bash provides a parameter expansion with substring replacement that is tailor made for this process.
Essentially, you find each file below your source directory with the *.mkv extension, you use parameter expansion with substring replacement to form the new full-filename containing your destination directory, (e.g. nffn="${ffn/$srcdir/$destdir}", where ffn is short for full-filename and nffn short for new full-filename)
With your new full-filename formed containing the updated path, it is just a matter of making sure the destination directory exists before moving the file. mkdir -p is perfect here as it will create the full path, and will not complain if the directory already exists. You simply use a parameter expansin with substring removal to isolate the new directory from the new full-filename to pass to mkdir -p, and finally, you check that mkdir -p succeeds or you handle the error, e.g.
## create new directory, handle error if create fails
mkdir -p "${nffn%/*}" || {
echo "error: creating '${nffn%/*}'" >&2
exit 1
}
Putting all the pieces together, you can do what you are attempting with a short script similar to the following.
#!/bin/bash
## source and destination directories, file pattern
# (note: to change destdir, two arguments required
# to change patrn, three arguments required)
srcdir="${1:-/home/david/dev/src-c/tmp/debug/AAA}"
destdir="${2:-/home/david/dev/src-c/tmp/debug/BBB}"
patrn="${3:-*.mkv}"
while read -r ffn; do ## loop over each full-filename
nffn="${ffn/$srcdir/$destdir}" ## form new full-filename
## create new directory, handle error if create fails
mkdir -p "${nffn%/*}" || {
echo "error: creating '${nffn%/*}'" >&2
exit 1
}
mv "$ffn" "$nffn" ## move full-filename to new full-filename
touch "$ffn" ## touch full-filename for zero original
done < <(find "$srcdir" -name "$patrn")
(note: you can pass the directories and file pattern as positional parameters, but note, if you pass more than 1, you must pass each required parameter (or you could implement getotp))
Initial Directories AAA & BBB
$ tree AAA
AAA
├── a.mkv
├── b.mkv
├── dir1
│ ├── a.mkv
│ └── b.mkv
├── dir2
│ ├── a.mkv
│ └── b.mkv
├── dir3
│ ├── a.mkv
│ └── b.mkv
└── dira
├── a.mkv
└── b.mkv
$ tree BBB
BBB [error opening dir]
Final Directories AAA & BBB
$ bash mvemptydir.sh
$ tree AAA
AAA
├── a.mkv
├── b.mkv
├── dir1
│ ├── a.mkv
│ └── b.mkv
├── dir2
│ ├── a.mkv
│ └── b.mkv
├── dir3
│ ├── a.mkv
│ └── b.mkv
└── dira
├── a.mkv
└── b.mkv
$ tree BBB
BBB
├── a.mkv
├── b.mkv
├── dir1
│ ├── a.mkv
│ └── b.mkv
├── dir2
│ ├── a.mkv
│ └── b.mkv
├── dir3
│ ├── a.mkv
│ └── b.mkv
└── dira
├── a.mkv
└── b.mkv
Look things over and let me know if you have further questions.
you can write a script like this :
#!/bin/bash
cd /[ADDRESS]
find . -name *.mkv > /tmp/find_result.txt
mv `cut -f1 /tmp/find_result.txt` /backup/
touch `cut -f1 /tmp/find_result.txt`
1- go to your directory that you want to find this files
2- find all .mkv files and send the result to a file like /tmp/find_result.txt in this example
3- move all files (that save in file "/tmp/find_result.txt") to your desired directory (like "/backup" in this example)
4- finaly create empty file with same name (that save in file "/tmp/find_result.txt")
you can add this script to crontab.
You could use a loop to do this for each file matching your criteria!
for f in `find . -name *.mkv`; do
mv $f /home/user/directoryB/
touch $f
done;
If you wanted to get fancy you could put this into a script and accept directoryA/B as arguments:
for f in `find $1 -name *.mkv`; do mv $f $2; touch $f; done;
and run as ./script.sh /home/user/directoryA/~ /home/user/directoryB/
The directory structure is as follows.
├── input
│ ├── config.ac
│ ├── dir1
│ │ ├── image2.png
│ │ └── image3.jpg
│ ├── dir2
│ └── image1.png
├── main.sh
└── output
Essentially, I am trying to run the following ./main.sh input output which I want to produce the following:
├── input
│ ├── config.ac
│ ├── dir1
│ │ ├── image2.png
│ │ └── image3.jpg
│ ├── dir2
│ └── image1.png
├── main.sh
└── output
├── dir1
│ └── image2.png
├── dir2
└── image1.png
After trying several thing such as find -exec, I went through it step by step. I'm trying to copy the internal directory structure of input into output while at the same time only copying .png files and nothing else.
Here is what I have tried:
for DIR in $1/; do
mkdir $2/$DIR
cp $1/$DIR*.png $1/$DIR $2/$DIR
done
Here is my logic, the for loop will go through every directory structure in the source directory($1), it will then make the exact same directory in the destination directory($2). Now, it looks for any .png files in the current directory it is in and copies it to the exact same corresponding directory that was just created in the destination.
Note, I do plan on doing some conversions to the files later on in the for loop
This doesn't seem to work at all. I get the following errors:
cp: cannot stat 'input/input/*.png': No such file or directory
cp: cannot stat 'input/input/': No such file or directory
Change to the directory you are copying from
Then
tar cf - .| ( cd /other directory; tar xf -)
You'll be probably happy with rsync:
rsync -a --include '*/' --include '*.png' --exclude '*' input/ output
This should copy all directories AND png files. If you want to see what would be copied, add -nv options.
(See the notes on rsync versions here.)
My favorite for copying directory trees (using your input/output structure):
cd input
find . | cpio -pdm ../output
Adjust the 'find' options as required.
Much research has turned almost similar questions yet nothing close enough to give me an idea of how to accomplish part my task. I'll try to keep this clear and short, while explaining the situation and desired result. My structure would be as follows:
-mobile
--Docs
--Downloads
--SomeFile
----this.is.crazy_0.0.1-1_named-silly.txt
----dont.touch.me.pdf
----leave.me.alone.png
----this.is.crazy_0.0.1-2_named-silly.txt
----this.is.crazy_0.0.1-3_named-silly.txt <---- file to keep
--SomeFileA
----this.is.crazy_0.0.1-1_also-silly.txt
----this.is.crazy_0.0.1-2_also-silly.txt
----dont.touch.me.either.pdf
----leave.me.alone.too.png
----this.is.crazy_0.0.1-3_also-silly.txt
----this.is.crazy_0.0.1-11_also-silly.txt <----file to keep
The first part of my script to find the .txt files ignores every directory that is constant in this working directory and prints them to a list (which is a completely ugly hack and most likely a hinder to the way most would accomplish this task) "SomeFileB and SomeFileC" could come along with the same file structure and I'd like to catch them in this script as well.
The idea is to keep the newest .txt file in each directory according to its time stamp which obviously isn't in the filename. The files to keep will continue to change of course. To clarify the question again, how to go about keeping the newest .txt file in each variable directory with variable crazy name, according to timestamp which isn't in the filename? Hopefully I've been clear enough for help. This script should be in bash.
I'm not with the current code right now, as i said its ugly but heres a snippet of what I have find /path/to/working/directory -maxdepth 0 -not -path "*Docs*" -not -path "*Downloads* -name "*.txt" >list
Assuming the question was understood correctly, the task could be expressed as:
Recursively remove all files *.txt except the newest in each respective directory
#!/bin/bash
# Find all directories from top of tree
find a -type d | while read -r dir; do
# skip $dir if doesn't contain any files *.txt
ls "$dir"/*.txt &>/dev/null || continue
# list *.txt by timestamp, skipping the newest file
ls -t "$dir"/*.txt | awk 'NR>1' | while read -r file; do
rm "$file"
done
done
Assuming this directory tree, where a.txt is always the newest:
$ tree -t a
a
├── otherdir
├── b
│ ├── d e
│ │ ├── a.txt
│ │ ├── b.txt
│ │ ├── c.txt
│ │ ├── bar.txt
│ │ └── foo.pdf
│ ├── c
│ │ ├── a.txt
│ │ ├── b.txt
│ │ └── c.txt
│ ├── a.txt
│ ├── b.txt
│ ├── c.txt
│ └── foo.pdf
├── a.txt
├── b.txt
└── c.txt
This is the result after running the script:
$ tree -t a
a
├── b
│ ├── c
│ │ └── a.txt
│ ├── d e
│ │ ├── a.txt
│ │ └── foo.pdf
│ ├── a.txt
│ └── foo.pdf
├── otherdir
└── a.txt
Change rm "$file" to echo rm "$file" to check what would be removed before running "for real"