I have the following folder structure:
.
`-- top_level/
|-- sub-01_ses-01/
| `-- filtered_data.tar.gz*
|-- sub-01_ses-02/
| `-- filtered_data.tar.gz*
|-- sub-02_ses-01/
| `-- filtered_data.tar.gz*
|-- sub-02_ses-02/
| `-- filtered_data.tar.gz*
I wanted to create symbolic links to these files preserving the parent structure (since they all have the same filenames).
Here's what I tried:
find -name "filtered_data.tar.gz" \
-exec cp -s --parents --no-clobber -t /home/data/filtered {} \;
Now, I notice that cp does create the parent structure, but the symbolic links fail and I get the following notice:
cp: '/home/data/filtered/./sub-01_ses-01/filtered_data.tar.gz': can make relative symbolic links only in current directory
I'd like to understand why this is hapenning, and what the cp warning is trying to tell me. Also, any pointers on how to fix the issue would be greatly appreciated.
Found the solution here: symlink-copying a directory hierarchy
The path of the file to cp must be absolute, not ./something. So, this should work for you:
find $(pwd) -name "filtered_data.tar.gz" \
-exec cp -s --parents --no-clobber -t /home/data/filtered {} \;
Per your comment about what you're really trying to do, here's a Python script that does it. You should be able to tweak it.
#!/usr/bin/env python3
import os
target_filename = 'filtered_data.tar.gz'
top_src_dir = '.'
top_dest_dir = 'dest'
# Walk the source directory recursively looking for
# target_filename
for parent, dirs, files in os.walk(top_src_dir):
# debugging
# print(parent, dirs, files)
# Skip this directory if target_filename not found
if target_filename not in files:
continue
# Strip off all path parts except the immediate parent
local_parent = os.path.split(parent)[-1]
# Compute the full, relative path to the symlink
dest_file = os.path.join(top_dest_dir, local_parent, target_filename)
# debugging
# print('{} {}'.format(dest_file, os.path.exists(dest_file)))
# Nothing to do if it already exists
if os.path.exists(dest_file):
print('{} already exists'.format(dest_file))
continue
# Make sure the destination path exists
dest_dir = os.path.dirname(dest_file)
os.makedirs(dest_dir, exist_ok=True)
# Translate the relative path to target_filename
# to be relative based on the new destination dir
src_file = os.path.join(parent, target_filename)
src_file = os.path.relpath(src_file, start=dest_dir)
os.symlink(src_file, dest_file)
print('{} --> {}'.format(dest_file, src_file))
Related
Let's assume I have following directory tree:
.
|-- foo
`-- foodir
|-- bardir
| |-- bar
| `-- foo
|-- foo -> bardir/foo
`-- foodir
|-- bar
`-- foo
3 directories, 6 files
How can I rename all foo into buz, including symlinks? like:
.
|-- buz
`-- buzdir
|-- bardir
| |-- bar
| `-- buz
|-- buz -> bardir/buz
`-- buzdir
|-- bar
`-- buz
3 directories, 6 files
I thought it is relatively easy at the first glance, but it turns out that was unexpectedly tough.
Firstly, I tried to mv around all files using git ls-files:
$ for file in $(git ls-files '*foo*'); do mv "$file" "${file//foo/buz}"; done
This gave me a bunch of errors said that I have to create new directories before doing so:
mv: cannot move 'foodir/bardir/bar' to 'buzdir/bardir/bar': No such file or directory
mv: cannot move 'foodir/bardir/foo' to 'buzdir/bardir/buz': No such file or directory
mv: cannot move 'foodir/foo' to 'buzdir/buz': No such file or directory
mv: cannot move 'foodir/foodir/bar' to 'buzdir/buzdir/bar': No such file or directory
mv: cannot move 'foodir/foodir/foo' to 'buzdir/buzdir/buz': No such file or directory
I didn't want to care about cleaning up empty directories after copy, so I tried find -exec expecting it can handle file renaming while finding files based on its names.
$ find . -path .git -prune -o -name '*foo*' -exec bash -c 'mv "$0" "${0//foo/buz}"' "{}" \;
But find seems still tried renaming files from renamed path.
find: ./foodir: No such file or directory
My final solution is to find the first file/directory for every single mv commands.
#!/bin/bash
# Rename file paths recursively
while :; do
path=$(find . -path .git -prune -o -name '*foo*' -print -quit)
if [ -z "$path" ]; then
break
fi
if ! mv "$path" "${path/foo/buz}"; then
break
fi
done
# Change symlink targets as well
find . -path .git -prune -o -type l -exec bash -c '
target=$(readlink "$0")
if [ "$target" != "${target//foo/buz}" ]; then
ln -sfn "${target//foo/buz}"
fi
' "{}" \;
This, kinda lame, but works as I expected. So my questions are:
Can I assume find always output directories before its sub directories/files?
Is there any chance to avoid using find multiple times?
Thank you.
I have a directory full of files. The tree looks something like this:
|-- test1a
| |-- test1b
| |-- foo.txt
| |-- bar.txt
|-- test2a
| |-- test2b
Where the directory names match the regular expression test[1-9][ab].
Using find in bash, I'm trying to create blank files in test2b with the same filenames and extensions as those in test1b.
So far, I've tried the following:
find test1a/test1b -type f -exec touch test2a/test2b {} \;
This, however, does not work. I don't have much experience with bash, so I'm not sure where to go from here. Where am I going wrong?
I was able to solve this problem using the following:
$ cd test2a/test2b
$ find ../../test1a/test1b -type f -exec sh -c 'touch $(basename {})' \;
I believe the problem was resulting from {} giving the full path rather than the filename. It was then trying to create a file that already existed, so it left it alone and did nothing.
Here is a second approach:
find test1a/test1b -type f -execdir echo touch test2a/test2b/{} \; > adhoc.sh
sh adhoc.sh
I have a long list of folders that are siblings to each other, they all start with "0" and are numerically named (001, 002, 003...) but names are not only numerical and are not correlative (for example I have 0010_foo, 0032_bar, 0150_baz, etc).
I need to create a new folder (js) inside each of the folders on my list. I'd like to do it recursively using the command line.
I've tried:
$ cd path/to/my/root/folder
$ find 0* -type d -exec mkdir js {} \;
But I get an error for each attempt: "mkdir: js: file exists". No need to say there's no directory named js inside my folders but they are files with .js extension.
Where is the error in my command and how can I fix it? Thanks!
(Why your find command doesn't work is already explained in bishop's (now deleted) answer — I'm only giving an alternative to find).
You can replace find by a shell for loop as so:
for i in 0*/; do mkdir "$i"js; done
mkdir js {} tries to create two directories; you want mkdir {}/js.
To prevent find from repeatedly finding your new directory, ignore any directory named js.
find 0* ! -path '*/js' -type d -exec mkdir {}/js \;
I'm not 100% sure of your directory structure after your edit, but give this a whirl:
cd /path/to/my/root/folder
find . -maxdepth 1 ! -path . -type d -exec mkdir -p {}/js \;
Seems to work ok:
$ cd /path/to/my/root/folder
$ tree
.
├── 001
│ └── js
└── 002
$ find . -maxdepth 1 ! -path . -type d -exec mkdir -p {}/js \;
.
├── 001
│ └── js
└── 002
└── js
What this find does: In the current directory (.), it finds sub-directories (-type d) -- except the current directory itself (! -path .) and any sub-sub-directories (-maxdepth 1). In those found directories, it creates the desired sub-directory (-exec ...). The mkdir -p part creates the directory and silences any errors about parents not existing. find replaces the {} part with the actual directory it found.
I have a directory structure
Dir_1
Dir_2
Dir_3
Source
. The directory Source contains the files File_1.txt and File_2.txt.
I want to copy all the files from the directory Source to all the remaining directories, in this case Dir_1, Dir_2 and Dir_3.
For this, I used
for i in $(ls -d */ | grep -v 'Source'); do echo $i | xargs -n 1 cp ./Source/*; done
. I, however, keep getting the message
cp: target ‘5’ is not a directory
It seems cp has problems with the directory names which have spaces in them. How do I resolve this (keeping the spaces in the directory names, obviously)?
Using find you could do something like this:
find . -mindepth 1 -maxdepth 1 -type d ! -name Source -exec cp Source/*.txt {} \;
This command searches the current directory for all subdirectories one level deep, excluding Source and then copies the text files into each.
Hope this helps :)
I need a bash script like
headers ~/headers-folder ~/output-folder
so it recursively finds all .h files in ~/headers-folder and put them all in ~/output-folder with the folder hierarchy maintained?
Thanks!
find /path/to/find -name "*.h" -type f | xargs -I {} cp --parents {} /path/to/destination
Check this out.
rsync is great for that too:
rsync --include '*.h' --filter 'hide,! */' -avm headers-folder/ output-folder/
This will copy all the *.h files, and create only the necessary directories.
Example:
mkdir -p headers-folder/{subdir,empty}
touch headers-folder/foo.h
touch headers-folder/subdir/foo.h
tree headers-folder
# headers-folder/
# |-- empty
# |-- foo.h
# `-- subdir
# `-- foo.h
rsync --include '*.h' --filter 'hide,! */' -avm headers-folder/ output-folder/
tree output-folder
# output-folder/
# |-- foo.h
# `-- subdir
# `-- foo.h