I am writing a short script. One functionality is synchronizing two folders. Now I have two variables with directories to two different folder: DIRECTORY_1 and DIRECTORY_2. In both folders are files and other folders. I need to synchronize these folder to have all files in both folders. For example:
In DIRECTORY_1 I have file1, file2, file3 and folder1
In DIRECTORY_2 I have file4, file5, file6 and folder2
I need comment after which I will have in both directories files1-6 and folders1-2.
I was trying rsync command but it doesn't work properly.
$ mkdir dir1
$ mkdir dir2
$ touch dir1/file1 dir1/file2 dir1/file3
$ mkdir dir1/folder1
$ touch dir2/file4 dir2/file5 dir2/file6
$ mkdir dir2/folder2
$ tree
.
|-- dir1
| |-- file1
| |-- file2
| |-- file3
| `-- folder1
`-- dir2
|-- file4
|-- file5
|-- file6
`-- folder2
$ rsync -a dir1/ dir2
$ tree
.
|-- dir1
| |-- file1
| |-- file2
| |-- file3
| `-- folder1
`-- dir2
|-- file1
|-- file2
|-- file3
|-- file4
|-- file5
|-- file6
|-- folder1
`-- folder2
I guess rsync -d dir2/ dir1 would be next?
Use unison, it's a real folder synchroniser (such as Dropbox or Mega did).
https://www.cis.upenn.edu/~bcpierce/unison/
Mac installation with brew:
brew install unison
I used this command:
unison -auto /path/folder1 /path/folder2
Moreover, and most important to me, if 2 files have the same name, it replaces with the most recent version.
Likely not the most efficient, but this would do the trick:
comm <(ls DIR1) <(ls DIR2) -23 | while read f; do cp -r DIR2/$f DIR1; done
comm <(ls DIR1) <(ls DIR2) -13 | while read f; do cp -r DIR1/$f DIR2; done
Related
Let's assume I have following directory tree:
.
|-- foo
`-- foodir
|-- bardir
| |-- bar
| `-- foo
|-- foo -> bardir/foo
`-- foodir
|-- bar
`-- foo
3 directories, 6 files
How can I rename all foo into buz, including symlinks? like:
.
|-- buz
`-- buzdir
|-- bardir
| |-- bar
| `-- buz
|-- buz -> bardir/buz
`-- buzdir
|-- bar
`-- buz
3 directories, 6 files
I thought it is relatively easy at the first glance, but it turns out that was unexpectedly tough.
Firstly, I tried to mv around all files using git ls-files:
$ for file in $(git ls-files '*foo*'); do mv "$file" "${file//foo/buz}"; done
This gave me a bunch of errors said that I have to create new directories before doing so:
mv: cannot move 'foodir/bardir/bar' to 'buzdir/bardir/bar': No such file or directory
mv: cannot move 'foodir/bardir/foo' to 'buzdir/bardir/buz': No such file or directory
mv: cannot move 'foodir/foo' to 'buzdir/buz': No such file or directory
mv: cannot move 'foodir/foodir/bar' to 'buzdir/buzdir/bar': No such file or directory
mv: cannot move 'foodir/foodir/foo' to 'buzdir/buzdir/buz': No such file or directory
I didn't want to care about cleaning up empty directories after copy, so I tried find -exec expecting it can handle file renaming while finding files based on its names.
$ find . -path .git -prune -o -name '*foo*' -exec bash -c 'mv "$0" "${0//foo/buz}"' "{}" \;
But find seems still tried renaming files from renamed path.
find: ./foodir: No such file or directory
My final solution is to find the first file/directory for every single mv commands.
#!/bin/bash
# Rename file paths recursively
while :; do
path=$(find . -path .git -prune -o -name '*foo*' -print -quit)
if [ -z "$path" ]; then
break
fi
if ! mv "$path" "${path/foo/buz}"; then
break
fi
done
# Change symlink targets as well
find . -path .git -prune -o -type l -exec bash -c '
target=$(readlink "$0")
if [ "$target" != "${target//foo/buz}" ]; then
ln -sfn "${target//foo/buz}"
fi
' "{}" \;
This, kinda lame, but works as I expected. So my questions are:
Can I assume find always output directories before its sub directories/files?
Is there any chance to avoid using find multiple times?
Thank you.
First of all, this is the code;
#!/bin/bash
i=1;
while read line;
do
cp "$i.eps" "$line.eps"
while read line2;
do
mv "$line.eps" "$line2"
sed -i '1d' cate.txt
done < cate.txt
sed -i '1d' cate.txt
((i++))
done < Names.txt
cp "$i.eps" "$line.eps"
Let me explain; I have 2 files totally. One of them is named "Names.txt" which contains the name of files. And the other file is "cate.txt" which contains the name of directories. Also I have .eps files which has name like; 1.eps, 2.eps, etc...
So, what I would like to do is to read the first line in "Names.txt" and than change the first file's name with the first line, and than read first line in "cate.txt" and move the first file under the directory that I read in "cate.txt"
PS 1: I used sed command there because I was always reading the first line in "cate.txt". So, I thought after I read the first line, I can delete it and than read the first line again. But the code was not successful to do that.
PS 2: In this code I can read "Names.txt" and rename .eps files. But when I start reading the "cate.txt" the script doesn't work properly.
Thank you!
Assuming that names.txt and cate.txt have the same number of rows, you can join them together and use that output instead:
#!/bin/bash
i=1
while read filename dirname; do
mkdir -p $dirname
cp $i.file $dirname/$filename
((i++))
done < <(paste names.txt cate.txt)
Example before running:
$ tree
.
|-- 1.file
|-- 2.file
|-- 3.file
|-- cate.txt
`-- dirs.txt
$ cat names.txt
first_file
second_file
third_file
$ cat cate.txt
first_dir
second_dir
third_dir
And after:
$ tree
.
|-- 1.file
|-- 2.file
|-- 3.file
|-- cate.txt
|-- dirs.txt
|-- first_dir
| `-- first_file
|-- second_dir
| `-- second_file
`-- third_dir
`-- third_file
I am getting this error
nameAndMove: line 8: syntax error near unexpected token <'
nameAndMove: line 8:done < <(paste iconNames.rtf iconCate.rtf)'
I googled it and people say that process substitution is not allowed in my terminal. I use mac terminal.
Thank you
I'd like to use find command with excluding a directory.
I have this:
GLUE="$GLUE -not -iwholename */dir3/*"
And I want to use a GLUE variable in a find command (find $GLUE [...]). Unfortunately, instead of -not -iwholename */dir3/* in GLUE I get -not -iwholename dir3/file1 dir3/file2 dir3/file3, i.e., */dir3/* turns into names of files which meet this condition. And, of course, find doesn't work because of it. How to stop that?
find ./ -iname ! -iname <dirname of file name to be not part of search results>
find ./ -iname <file name> ! -path "./dirt o be Excluded/*"
I hope one will help
Let me answer this question first:
How can I use find to search recursively while excluding a directory from the search? My find command supports the -prune action and the common extensions (e.g., provided by GNU find).
You're very lucky that your find supports the -prune action. Congratulations!
In this case, only add:
\! \( -name 'dir_to_exclude' -prune \)
This will be false if the current name processed by find is dir_to_exclude, and at the same time prune (i.e., cut that branch) from the search tree1.
Let's try it (in a scratch directory):
$ mkdir {a,b,c,d}/{,1,2,3}
$ touch {a,b,c,d}/{,1,2,3}/{file1,file2}
$ tree
.
|-- a
| |-- 1
| | |-- file1
| | `-- file2
| |-- 2
| | |-- file1
| | `-- file2
| |-- file1
| `-- file2
|-- b
| |-- 1
| | |-- file1
| | `-- file2
| |-- 2
| | |-- file1
| | `-- file2
| |-- file1
| `-- file2
`-- c
|-- 1
| |-- file1
| `-- file2
|-- 2
| |-- file1
| `-- file2
|-- file1
`-- file2
9 directories, 18 files
$ find \! \( -name a -prune \)
.
./b
./b/file2
./b/1
./b/1/file2
./b/1/file1
./b/file1
./b/2
./b/2/file2
./b/2/file1
./c
./c/file2
./c/1
./c/1/file2
./c/1/file1
./c/file1
./c/2
./c/2/file2
./c/2/file1
Looks good!
Now, you shouldn't put your arguments to find (or, more generally your commands and arguments) in a variable, but in an array! If you don't you'll very soon run into problems with arguments containing spaces or, like in your OP, wildcards.
Put your glue stuff in an array: for example, to find all the files that have a 1 in their name and are empty2:
glue=( '-name' '*1*' '-type' 'f' '-empty' )
Then an array of the exclude this directory:
exclude_dir=( '!' '(' '-name' 'a' '-prune' ')' )
Then find all files that have a 1 in their name and are empty, while excluding directory a (in the same scratch directory as before):
$ find "${exclude_dir[#]}" "${glue[#]}"
./b/1/file1
./b/file1
./b/2/file1
./c/1/file1
./c/file1
./c/2/file1
Looks really good! (and observe the quotes!).
1
If you really want to be sure that you're only excluding the directory with name dir_to_exclude, in case you have a file called dir_to_exclude, you can specify it thus:
\! \( -name 'dir_to_exclude' -type d -prune \)
2
I'm using a lot of quotes here. Just a good habit that actually saves me for the wildcard part *1* and for the parentheses too!
I am trying to extract a file from tar using the following command:
tar xzfv mysql-connector-java-5.1.29.tar.gz mysql-connector-java-5.1.29-bin.jar
However tar command is unable to find mysql-connector-java-5.1.29-bin.jar even though it seems to be present. The folder structure which I get when extracting the tar file is:
The jar is present at http://cdn.mysql.com/Downloads/Connector-J/mysql-connector-java-5.1.29.tar.gz.
|-- mysql-connector-java-5.1.29
| |-- CHANGES
| |-- COPYING
| |-- README
| |-- README.txt
| |-- build.xml
| |-- docs
| | |-- README.txt
| | |-- connector-j.html
| | `-- connector-j.pdf
| |-- mysql-connector-java-5.1.29-bin.jar
| `-- src
| |-- com
| | `-- mysql
| | `-- jdbc
I have not included the complete structure for brevity. But it can be seen that mysql-connector-java-5.1.29-bin.jar is present. I am unable to figure out the issue here.
First, try to list the files in the tar.
tar tvf mysql-connector-java-5.1.29.tar.gz
Then you can find that your file is located inside a directory in the tar
mysql-connector-java-5.1.29/mysql-connector-java-5.1.29-bin.jar
So to extract that file, you should use
tar zxvf mysql-connector-java-5.1.29.tar.gz mysql-connector-java-5.1.29/mysql-connector-java-5.1.29-bin.jar
I need a bash script like
headers ~/headers-folder ~/output-folder
so it recursively finds all .h files in ~/headers-folder and put them all in ~/output-folder with the folder hierarchy maintained?
Thanks!
find /path/to/find -name "*.h" -type f | xargs -I {} cp --parents {} /path/to/destination
Check this out.
rsync is great for that too:
rsync --include '*.h' --filter 'hide,! */' -avm headers-folder/ output-folder/
This will copy all the *.h files, and create only the necessary directories.
Example:
mkdir -p headers-folder/{subdir,empty}
touch headers-folder/foo.h
touch headers-folder/subdir/foo.h
tree headers-folder
# headers-folder/
# |-- empty
# |-- foo.h
# `-- subdir
# `-- foo.h
rsync --include '*.h' --filter 'hide,! */' -avm headers-folder/ output-folder/
tree output-folder
# output-folder/
# |-- foo.h
# `-- subdir
# `-- foo.h