In bash I need to do this:
take all files in a directory
copy them into an existing directory
How do I do this? I tried cp -r t1 t2 (both t1 and t2 are existing directories, t1 has files in it) but it created a directory called t1 inside t2, I don't want that, I need the files in t1 to go directly inside t2. How do I do this?
What you want is:
cp -R t1/. t2/
The dot at the end tells it to copy the contents of the current directory, not the directory itself. This method also includes hidden files and folders.
cp dir1/* dir2
Or if you have directories inside dir1 that you'd want to copy as well
cp -r dir1/* dir2
If you want to copy something from one directory into the current directory, do this:
cp dir1/* .
This assumes you're not trying to copy hidden files.
Assuming t1 is the folder with files in it, and t2 is the empty directory. What you want is something like this:
sudo cp -R t1/* t2/
Bear in mind, for the first example, t1 and t2 have to be the full paths, or relative paths (based on where you are). If you want, you can navigate to the empty folder (t2) and do this:
sudo cp -R t1/* ./
Or you can navigate to the folder with files (t1) and do this:
sudo cp -R ./* t2/
Note: The * sign (or wildcard) stands for all files and folders. The -R flag means recursively (everything inside everything).
cp -R t1/ t2
The trailing slash on the source directory changes the semantics slightly, so it copies the contents but not the directory itself. It also avoids the problems with globbing and invisible files that Bertrand's answer has (copying t1/* misses invisible files, copying `t1/* t1/.*' copies t1/. and t1/.., which you don't want).
For inside some directory, this will be use full as it copy all contents from "folder1" to new directory "folder2" inside some directory.
$(pwd) will get path for current directory.
Notice the dot (.) after folder1 to get all contents inside folder1
cp -r $(pwd)/folder1/. $(pwd)/folder2
Nov, 2021 Update:
This code with Flag "-R" copies perfectly all the contents of "folder1" to existing "folder2":
cp -R folder1/. folder2
Flag "-R" copies symbolic links as well but Flag "-r" skips symbolic links so Flag "-R" is better than Flag "-r".
The latest GNU Grep 3.7:
-R, --dereference-recursive
For each directory operand, read and process all files in that directory,
recursively, following all symbolic links.
-r, --recursive
For each directory operand, read and process all files in that directory,
recursively. Follow symbolic links on the command line, but skip symlinks
that are encountered recursively. Note that if no file operand is given,
grep searches the working directory. This is the same as the
‘--directories=recurse’ option.
Depending on some details you might need to do something like this:
r=$(pwd)
case "$TARG" in
/*) p=$r;;
*) p="";;
esac
cd "$SRC" && cp -r . "$p/$TARG"
cd "$r"
... this basically changes to the SRC directory and copies it to the target, then returns back to whence ever you started.
The extra fussing is to handle relative or absolute targets.
(This doesn't rely on subtle semantics of the cp command itself ... about how it handles source specifications with or without a trailing / ... since I'm not sure those are stable, portable, and reliable beyond just GNU cp and I don't know if they'll continue to be so in the future).
the correct option should be -T. used with -r to copy recursively.
$ cp -r -T t1 t2
Related
I have a bash script I'm trying to write
I have 2 base directories:
./tmp/serve/
./src/
I want to go through all the directories in ./tmp and copy the *.html files into the same folder path in ./src
i.e
if I have a html file in ./tmp/serve/app/components/help/ help.html -->
copy to ./src/app/components/help/ And recursively do this for all subdirectories in ./tmp/
NOTE: the folder structures should exist so just need to copy them only. If it doesn't then hopefully it could create the folder for me (not what I want) but with GIT I can track these folders to manually handle those loose html files.
I got as far as
echo $(find . -name "*.html")\n
But not sure how to actually extract the file path with pwd and do what I need to, maybe it's not a one liner and needs to be done with some vars.
something like
for i in `echo $(find /tmp/ -name "*.html")\n
do
cp -r $i /src/app/components/help/
done
going so far to create the directories would take some more time for me.
I'll try to do it on my own and see if I come up with something
but for argument sake if you do run pwd and get a response the pseudo code for that:
pwd
get response
if that directory does not exist in src create that directory
copy all the original directories contents into the new folder at /src/$newfolder
(possibly running two for loops, one to check the directory tree, and then one to go through each original directory, copying all the html files)
You process substitution to loop the output from your find command and create the destination directory(ies) and then copy the file(s):
#!/bin/bash
# accept first parameters to script as src_dir and dest values or
# simply use default values if no parameter(s) passed
src_dir=${1:-/tmp/serve}
dest=${2-src}
while read -r orig_path ; do
# To replace the first occurrence of a pattern with a given string,
# use ${parameter/pattern/string}
dest_path="${orig_path/tmp\/serve/${dest}}"
# Use dirname to remove the filename from the destination path
# and create the destination directory.
dest_dir=$(dirname "${dest_path}")
mkdir -p "${dest_dir}"
cp "${orig_path}" "${dest_path}"
done < <(find "${src_dir}" -name '*.html')
This script copy .html files from src directory to des directory (create the subdirectory if they do not exist)
Find the files, then remove the src directory name and copy them into the destination directory.
#!/bin/bash
for i in `echo $(find src/ -name "*.html")`
do
file=$(echo $i | sed 's/src\///g')
cp -r --parents $i des
done
Not sure if you must use bash constructs or not, but here is a GNU tar solution (if you use GNU tar), which IMHO is the best way to handle this situation because all the metadata for the files (permissions, etc.) are preserved:
find ./tmp/serve -name '*.html' -type f -print0 | tar --null -T - -c | tar -x -v -C ./src --strip-components=3
This finds all the .html files (-type f) in the ./tmp/serve directory and prints them nul-terminated (-print0), then sends these filenames via stdin to tar as nul-terminated literals (--null) for inclusion (-T -), creating (-c) an archive which is then sent to another tar instance which extracts (-x) the archive printing its contents along the way (optional: -v), changing directory to the destination (-C ./src) before commencing and stripping (--strip-components=3) the ./tmp/serve/ prefix from the files. (You could also cd ./tmp/serve beforehand, using find . instead, and change -C to ../../src.)
Using Ubuntu 18.04. Say we have a file called debug.log. You can create a copy called debug_BACKUP.log with either of these commands:
cp debug.log debug_BACKUP.log
cp debug{,_BACKUP}.log
Alternatively, substitute cp with mv to rename the file.
Now suppose we have debug1.log and debug2.log. We would like to create copies called debug1_BACKUP.log and debug2_BACKUP.log. Is there a single command to achieve this?
When I tried either of the following:
cp debug*.log debug*_BACKUP.log
cp debug*{,_BACKUP}.log
the error is cp: target 'debug*_BACKUP.log' is not a directory.
Brace expansions are an instruction for the shell about how to rewrite your command before glob expansion takes place. They aren't passed to the command itself -- cp has no idea if a brace expansion was used. For that matter, cp doesn't even have any idea if a wildcard is used; when you run cp *.txt dir/, the shell generates an array of C strings corresponding to something like cp foo.txt bar.txt baz.txt dir/ before running it.
This means that if you want to rewrite content after wildcard expansion takes place, you need to do it by hand.
for f in debug*.log; do
[[ $f = *_BACKUP.log ]] && continue # skip things that are already backup files
cp "$f" "${f%.log}_BACKUP.log"
done
There are few excellent bulk rename programs, including Perl based file-rename. You can achieve your bulk copy in 3 steps:
Copy the files to tmp sub folder
Perform bulk rename, moving the files back into the current folder
Remove the tmp folder
I have a whole bunch of files in source folder need to be updated to destination folder. the layout in source folder is flat, no sub directory such as:
src\a.h
\a.c
\b.h
\b.c
\c.h
\c.c
The destination folder contains multiple layers deep sub directory such as:
dst\App\a.h
\App\a.c
\USBD\CDC\b.h
\USBD\CDC\b.c
\USBH\CORE\FUNCTION\c.h
\USBH\CORE\FUNCTION\c.h
I need to search all the files in the "dst" directory recursively, if any of them match any of the file in the "src" directory, then rename the one in "dst" as XX.sav (XX is the original name) and copy the one from "dst" to "src".
thanks
Assuming that you actually want to copy from src to dst, that src and dst are siblings, and that the command is run from the common parent directory, try:
find dst -type f -exec sh -c 'test -f src/${0##*/} &&
{ mv $0 $0.sav && cp src/${0##*/} $0; }' {} \;
If the final two assumptions are not correct, try (untested):
find /path/to/dst -type f -exec sh -c 'test -f $1/${0##*/} &&
{ mv $0 $0.sav && cp $1/${0##*/} $0; }' {} /path/to/src \;
You should be careful: if there are any existing .sav files in dst they may be overwritten, and if there are any .sav files in src you may wind up with .sav.sav files in dst that are duplicates of the related .sav file that this command is itself creating. Use mv -i to address these issues if they are relevant. If the number of existing files is high, mv -i may not be an appropriate solution.
There are several steps to finding a solution to your problem.
First of all, you need to iterate over all files in src folder to make sure that you know what to look for in dst. This can be done easily in bash like this:
for filename in src/*
do
echo "$filename" # simple sanity check
done
Now that you know what you're looking for, you can use command find to try to find the files in dst. Since this command enables recursive look-ups itself, you don't need to do anything special. However, you'll probably want to look for a name of a file, not the whole path also containing directory structure. Something like this should work.
for filename in src/*
do
pattern=$(basename $filename) # remove folder from filename
for another_filename in $(find dst -name "$pattern")
do
echo "$filename -> $another_filename" # another sanity check
done
done
Now that you have a file from src matched with a file from dst, you can simply use cp and mv to achieve what you need.
This is nowhere near the shortest possible solution, but it should be easy to understand and improve once you figure out the commands you need.
I'm trying to make a simple script that copies all of my $HOME into another folder in $HOME called Backup/. This includes all hidden files and folders, and excludes Backup/ itself. What I have right now for the copying part is the following:
shopt -s dotglob
for file in $HOME/*
do
cp -r $file $HOME/Backup/
done
Bash tells me that it cannot copy Backup/ into itself. However, when I check the contents of $HOME/Backup/ I see that $HOME/Backup/Backup/ exists.
The copy of Backup/ in itself is useless. How can I get bash to copy over all the folders except Backup/. I tried using extglob and using cp -r $HOME/!(Backup)/ but it didn't copy over the hidden files that I need.
try rsync. you can exclude file/directories .
this is a good reference
http://www.maclife.com/article/columns/terminal_101_using_rsync_locally
Hugo,
A script like this is good, but you could try this:
cp -r * Backup/;
cp -r .* Backup/;
Another tool used with backups is tar. This compresses your backup to save disk space.
Also note, the * does not cover . hidden files.
I agree that using rsync would be a better solution, but there is an easy way to skip a directory in bash:
for file in "$HOME/"*
do
[[ $file = $HOME/Backup ]] && continue
cp -r "$file" "$HOME/Backup/"
done
This doesn't answer your question directly (the other answers already did that), but try cp -ua when you want to use cp to make a backup. This recurses directories, copies rather than follows links, preserves permissions and only copies a file if it is newer than the copy at the destination.
I am trying to create a bash script for syncing music from my desktop to a mobile device. The desktop is the source.
Is there a way to make rsync recursively sync files but ignore the directory structure? If a file was deleted from the desktop, I want it to be deleted on the device as well.
The directory structure on my desktop is something like this.
Artist1/
Artist1/art1_track1.mp3
Artist1/art1_track2.mp3
Artist1/art1_track3.mp3
Artist2/
Artist2/art2_track1.mp3
Artist2/art2_track2.mp3
Artist2/art2_track3.mp3
...
The directory structure that I want on the device is:
Music/
art1_track1.mp3
art1_track2.mp3
art1_track3.mp3
art2_track1.mp3
art2_track2.mp3
art2_track3.mp3
...
Simply:
rsync -a --delete --include=*.mp3 --exclude=* \
pathToSongs/Theme*/Artist*/. destuser#desthost:Music/.
would do the job if you're path hierarchy has a fixed number of level.
WARNING: if two song file do have exactly same name, while on same destination directory, your backup will miss one of them!
If else, and for answering strictly to your ask ignoring the directory structure you could use bash's shopt -s globstar feature:
shopt -s globstar
rsync -a --delete --include=*.mp3 --exclude=* \
pathToSongsRoot/**/. destuser#desthost:Music/.
At all, there is no need to fork to find command.
Recursively sync all files while ignoring the directory structure
For answering strictly to question, there must no be limited to an extension:
shopt -s globstar
rsync -d --delete sourceRoot/**/. destuser#desthost:destRoot/.
With this, directories will be copied too, but without content. All files and directories would be stored on same level at destRoot/.
WARNING: If some different files with same name exists in defferents directories, they would simply be overwrited on destination, durring rsync, for finaly storing randomly only one.
May be this is a recent option, but I see the option --no-relative mentioned in the documentation for --files-from and it worked great.
find SourceDir -name \*.mp3 | rsync -av --files-from - --no-relative . DestinationDir/
The answer to your question: No, rsync cannot do this alone. But with some help of other tools, we can get there... After a few tries I came up with this:
rsync -d --delete $(find . -type d|while read d ; do echo $d/ ; done) /targetDirectory && rmdir /targetDirectory/* 2>&-
The difficulty is this: To enable deletion of files at the target position, you need to:
specify directories as sources for rsync (it doesn't delete if the source is a list of files).
give it the complete list of sources at once (rsync within a loop will give you the contents of the last directory only at the target).
end the directory names with a slash (otherwise it creates the directories at the target directory)
So the command substitution (the stuff enclosed with the $( )) does this: It finds all directories and adds a slash (/) at the end of the directory names. Now rsync sees a list of source directories, all terminated with a slash and so copies their contents to the target directory. The option -d tells it, not to copy recursively.
The second trick is the rmdir /targetDirectory/* which removes the empty directories which rsync created (although we didn't ask it to do that).
I tested that here, and deletion of files removed in the source tree worked just fine.
If you can make a list of files, you've already solved the problem.
Try:
find /path/to/src/ -name \*.mp3 > list.txt
rsync -avi --no-relative --progress --files-from=list.txt / user#server:/path/to/dest
If you run the script again for new files, it will only copy the missing files.
If you don't like the list, then try a single sentence (but it's another logic)
find /path/to/src/ -name \*.mp3 -type f \
-exec rsync -avi --progress {} user#server:/path/to/dest/ \;
In this case, you will ask for each file, each time, since by the type of sentence, you cannot build the file list previously.