On my server, I have x number of sub-domains
Their folder names are like this:
example1.mydomain.com , example2.mydomain.com, examplex.mydomain.com
and they all exist in one location.
I am trying to write a simple bash script to copy all folders and files in folder SOURCE for example to all of those sub-domain folders (and replace existing).
In other words, I want to copy my files from source to any folder with the name *.mydomain.com
I tried rsync but couldn't do the *.mydomain part
I suggest:
for i in *.mydomain.com; do rsync -aSv "SOURCE/" "$i"; done
The trailing / after SOURCE is important.
You can use find command to search all files and then use this output to copy them, e.g. assuming you are searching in /home and copying to /target
find /home -name "*.mydomain.com" -exec cp -r {} /target/ \;
But one problem in above solution I see is it might find files / folders with these names and copy them none the less (not sure if it will maintain the same folder hierarchy), perhaps IF you are only looking for folder then try below instead,
find /home -name "*.mydomain.com" -type d -exec cp -r {} /target/ \;
Related
I want to copy all files with specific extensions recursively in bash.
****editing****
I've written the full script. I have list of names in a csv file, I'm iterating through each name in that list, then creating a directory with that same name somewhere else, then I'm searching in my source directory for the directory with that name, inside it there are few files with endings of xlsx,tsv,html,gz and I'm trying to copy all of them into the newly created directory.
sample_list_filepath=/home/lists/papers
destination_path=/home/ds/samples
source_directories_path=/home/papers_final/new
cat $sample_list_filepath/sample_list.csv | while read line
do
echo $line
cd $source_directories_path/$line
cp -r *.{tsv,xlsx,html,gz} $source_directories_path/$line $destination_path
done
This works, but it copies all the files there, with no discrimination for specific extension.
What is the problem?
An easy way to solve your problem is to use find and regex :
find src/ -regex '.*\.\(tsv\|xlsx\|gz\|html\)$' -exec cp {} dest/ \;
find look recursively in the directory you specify (in my example it's src/), allows you to filter with -regex and to apply a command for matching results with -exec
For the regex part :
.*\.
will take the name of the file and the dot before extension,
\(tsv\|xlsx\|gz\|html\)$
verify the extension with those you want.
The exec block is what you do with files you got from regex
-exec cp {} dest/ \;
In this case, you copy what you got ({} meaning) to the destination directory.
I've got a public folder that includes .html and non .html files in its root and subfolders. I need to move all non .html files to a subfolder of the public folder, e.g. to public/assets, preserving their full path.
So far I got something like this:
mkdir -p public/assets
find public -type f -not -name "*.html" -print0 | xargs -0I{} mv {} public/assets/
It doesn't work correctly, because it doesn't move files with full path, e.g. if there's a file public/foo/bar.js, it will be moved to public/assets/bar.js and not into public/assets/foo/bar.js.
I also tried another version:
mkdir -p public/assets
find public -type f -not -name "*.html" | sed 's#\(public\)\(.*\)#\1\2 public/assets\2#' | xargs -I% mv %
but it complains about incorrect syntax of mv command. I pass only one parameter to it, but after running sed it consists of 2 paths separated with one space, so I hoped it will treat it as 2 parameters.
It would be also great, if there was a way to remove empty folders after moving all files, but it's not necessary.
I suggest using rsync for this. Also, I would use a folder in a different location as destination. That folder can then be copied back to your public folder.
mkdir /tmp/assets
rsync -r --include="*/" --exclude="*.html" --include="*" --prune-empty-dirs public/* /tmp/assets
mv /tmp/assets public/assets
I have a folder named accdb under multiple directories all under one parent directory dist. I want to copy the contents of accdb for all directories while preserving the code structure
I succeeded in making the recursive folder structure with:
cd ~/dist; find . -name "accdb" -type d -exec mkdir -p -- ~/acc_trial/{} \;
But i am failing to copy the contents of accdb. This command just makes the structure until directory accdb.
I tried
find . -name "accdb" -type d -exec mkdir -p -- ~/acc_trial/{} \ && cp -r {} ~/acc_trial/{} \;
I get an error:
find: missing argument to `-exec'
I don't know if this is possible using only a find expression, I'm pretty sure it is not. Besides you must consider that if you have one subfolder named accdb inside one accdb folder you'll probably get an error, that's why in the script that I've made I decided to use rsync:
#!/bin/bash
DEST='/home/corronx/provisional/destination_dir'
#Clean destination directory, PLEASE BE CAREFUL IT MUST BE A REMOVABLE DIRECTORY
rm -rf $DEST/*
FIND='test'
LOOK_PATH='/home/corronx/provisional'
FILES=($(find . -type d -name $FIND))
for ((i=0; i<${#FILES[#]};i++))
do
#Remove first character .
FILES[$i]=${FILES[$i]:1:${#FILES[$i]}}
#Create directories in destination path
mkdir -p $DEST${FILES[$i]}
rsync -aHz --delete ${FILES[$i]:1:${#FILES[$i]}}/ $DEST${FILES[$i]}
echo $i
done
Explanation
First of all I'd recommend using full paths in your script because an rm -rf expression inside an script is pretty dangerous (If you want comment that line and delete destination folder before running script).
DEST= Destination path.
FIND= Subfolder name that your are looking for.
LOOK_PATH= Path where you want to execute find
I create an array called FILES that contain all folders that returns find expression, after that I just create destination directories and run rsync to copy files, I've used rsync because I think it is better in case there is any subdirectory with the same name.
PLEASE BE CAREFUL WITH rm -rf expression, if DEST is not set you'll delete everything in your machine
I have lots of projects archived under a directory tree, some of which have a .git folder in them.
What I'd like to do is recursively copy those files and directories to a new destination, keeping the current structure - EXCEPT for those directories containing a .git folder, in which case the script should run a command (let's say "echo", I'll change it later) followed by the folder name, without creating or copying it.
Any help would be much appreciated.
Edit: I'll try to explain myself better: I need to copy every single file and directory, except for those containing .git, which should be skipped and their path should be passed to another command. In this example, path a/b/c/d and its subfolders should be skipped entirely and a/b/c/d should be displayed using echo (just for brevity, I'll replace it with a different command later):
a
a/b
a/b/c
a/b/c/d/.git
a/b/c/d/e
a/b/c/d/f/g
a/b/c/e
a/b/d
a/c
b
b/c
...
IIUC, the following find one-liner will do the job:
find . -type d -mindepth 1 -maxdepth 1 -exec sh -c "test -e '{}/.git' && echo not copy '{}' || cp -r -- '{}' /tmp/copy-here " \;
I tried the search, but couldn't find the answer to my specific problem.
When I use,
find /recovered_files "*.jpg" -type f -exec cp {} /out \;
to copy all .jpg files from directories within the /recovered_files directory, the /out directory gets filled with every single file (jpg, txt, xml etc etc) from within the source directories.
Can anyone please explain wherein my stupidity lies, pleeeeeease???
Many thanks, Mark.
What you're doing at the moment is equivalent to calling cp /dir/dir/dir/file.jpg /out for each file, which will copy the file into /out. Thus, all of the files are being put into the same directory.
rsync allows filters to select only certain files to be copied. Change from and to to the appropriate directories in the following:
rsync -r from/* to --include=*.jpg --filter='-! */' --prune-empty-dirs
Credit to this post for this solution.
Edit: changed to rsync solution. Original as follows:
find from -name "*.jpg" -type f -exec mkdir -p to/$(dirname {}) \; -exec cp --parents {} to \;
You should replace from and to with the appropriate locations, and this form won't quite work if from begins with /. Just cd to / first if you need to. Also, you'll end up with all the files inside to underneath the entire directory structure of from, but you can just move them back out again.