I have a long list of folders that are siblings to each other, they all start with "0" and are numerically named (001, 002, 003...) but names are not only numerical and are not correlative (for example I have 0010_foo, 0032_bar, 0150_baz, etc).
I need to create a new folder (js) inside each of the folders on my list. I'd like to do it recursively using the command line.
I've tried:
$ cd path/to/my/root/folder
$ find 0* -type d -exec mkdir js {} \;
But I get an error for each attempt: "mkdir: js: file exists". No need to say there's no directory named js inside my folders but they are files with .js extension.
Where is the error in my command and how can I fix it? Thanks!
(Why your find command doesn't work is already explained in bishop's (now deleted) answer — I'm only giving an alternative to find).
You can replace find by a shell for loop as so:
for i in 0*/; do mkdir "$i"js; done
mkdir js {} tries to create two directories; you want mkdir {}/js.
To prevent find from repeatedly finding your new directory, ignore any directory named js.
find 0* ! -path '*/js' -type d -exec mkdir {}/js \;
I'm not 100% sure of your directory structure after your edit, but give this a whirl:
cd /path/to/my/root/folder
find . -maxdepth 1 ! -path . -type d -exec mkdir -p {}/js \;
Seems to work ok:
$ cd /path/to/my/root/folder
$ tree
.
├── 001
│ └── js
└── 002
$ find . -maxdepth 1 ! -path . -type d -exec mkdir -p {}/js \;
.
├── 001
│ └── js
└── 002
└── js
What this find does: In the current directory (.), it finds sub-directories (-type d) -- except the current directory itself (! -path .) and any sub-sub-directories (-maxdepth 1). In those found directories, it creates the desired sub-directory (-exec ...). The mkdir -p part creates the directory and silences any errors about parents not existing. find replaces the {} part with the actual directory it found.
Related
I want to tar a directory that looks like this:
dir
└── workspace
└── node_modules
└── subfolder
└── workspace
└── node_modules
└── other_folder
I want to exclude all folders named node_modules and exclude the top level folder called workspace, but no sub folders called workspace.
So what I want to end up with is this:
dir
└── subfolder
└── workspace
└── other_folder
I'm running this command: tar -czf ./output.tar.gz --exclude=node_modules --exclude=./workspace dir/.
But it's removing all folders called workspace and node_modules, so I instead end up with this:
dir
└── subfolder
└── other_folder
How do I remove only the specific workspace folder that I want, and not all folders with the same name?
For the required case, possible to use tar excludes:
--exclude dir/./folder -- apply to folder directly under dir
--exclude folder -- will exclude folder anywhere in the tree
Should be possible to use:
tar -czf ./output.tar.gz --exclude=node_modules --exclude=dir/./workspace dir/.
Of course possible to use --files-from, and to generate the list using another tool. This is usually preferred when the list could be large number of files, vs using xargs.
find dir/. -type f ... | tar cvz ./output.tar.gz -T-
find has many, many, many options for including, excluding paths, files, directories, generally filtering options however you want to.
For your case I think it would be:
# exclude all folders named node_modules
# exclude the top level folder called workspace
# but no sub folders called workspace
find dir -type f \
-not -regex '.*/node_modules/.*' -a \
-not -regex 'dir/workspace/.*' \
-exec tar -czf ./output.tar.gz {} +
You may prefer instead of -exec for example find ... -print0 | xargs -0 tar -czf ./output.tar.gz. I think the best would be find ... -print0 | tar -czf ./output.tar.gz --null -T - as it will not fail if there are too many files, ie. too many arguments to pass to tar, I think.
I recreated dir directory with:
while read l; do
mkdir -p "$(dirname "$l")"
touch "$l"
done <<EOF
dir/workspace/1.txt
dir/node_modules/2.txt
dir/subfolder/workspace/3.txt
dir/subfolder/node_modules/4.txt
dir/subfolder/other_folder/5.txt
EOF
then tested on repl and the tar -tf ./output.tar.gz prints:
dir/subfolder/workspace/3.txt
dir/subfolder/other_folder/5.txt
Been trying to piece together a couple previous posts for this task.
The directory tree looks like this:
TEST
|ABC_12345678
3_XYZ
|ABC_23456789
3_XYZ
etc
Each folder within the parent folder named "TEST" always starts with ABC_\d{8} -the 8 digits are always different. Within the folder ABC_\d{8} is always a folder entitled 3_XYZ that always has a file named "MD2_Phd.txt". The goal is to rename each "MD2_PhD.txt" file with the specific 8 digit ID found in the ABC folder name i.e. "\d{8}_PhD.txt"
After several iterations on various bits of code from different posts this is the best I can come up with,
cd /home/etc/Desktop/etc/TEST
find -type d -name 'ABC_(\d{8})' |
find $d -name "*_PhD.txt" -execdir rename 's/MD2$/$d/' "{}" \;
done
find + bash solution:
find -type f -regextype posix-egrep -regex ".*/TEST/ABC_[0-9]{8}/3_XYZ/MD2_Phd\.txt" \
-exec bash -c 'abc="${0%/*/*}"; fp="${0%/*}/";
mv "$0" "$fp${abc##*_}_PhD.txt" ' {} \;
Viewing results:
$ tree TEST/ABC_*
TEST/ABC_12345678
└── 3_XYZ
└── 12345678_PhD.txt
TEST/ABC_1234ss5678
└── 3_XYZ
└── MD2_Phd.txt
TEST/ABC_23456789
└── 3_XYZ
└── 23456789_PhD.txt
You are piping find output to another find. That won't work.
Use a loop instead:
dir_re='^.+_([[:digit:]]{8})/'
for file in *_????????/3_XYZ/MD2_PhD.txt; do
[[ -f $file ]] || continue
if [[ $file =~ $dir_re ]]; then
dir_num="${BASH_REMATCH[1]}"
new_name="${file%MD2_PhD.txt/$dir_num.txt}" # replace the MD2_PhD at the end
echo mv "$file" "$new_name" # remove echo from here once tested
fi
done
I have the following structure:
/home/
├── DIR1/
│ └── file_ab.csv
├── DIR2/
│ └── file_cd.csv
└── DIR3/
└── file3_ef.csv
Where file_**.csv contains rows of floats, different floats for each DIR.
I want to grab the contents of all of the file_**.csv files and concatenate them.
I found this answer here:
find /home -type f -name '*.csv' -exec cat {} \; > pl_parameters
But I get an empty file called 'pl_parameters'. Why is the file empty? How can I fix this?
find /home/DIR* -name 'file*csv' |xargs cat > output.csv
find /home/DIR* -name '*csv' gives you the files absolute paths.
xargs cat will iterate the files and cat print the files content
With Bash 4.0+, you can use globstar and use a more straight forward command:
shopt -s globstar
cd /home
cat **/*.csv > pl_parameters
**/ expands to the entire directory tree underneath the current directory.
Your command:
find /home -type f -name '*.csv' -exec cat {} \; > pl_parameters
looks good to me - not sure why you got a zero by output file.
For a number of files I want to get the parent directory and append its name to the filename. For example, in the following path:
A/B/C/file.zip
I want to rename file.zip to file_C.zip.
Here is my code. I have to find directory which does not contain subdirectory and zip files in it, and I want to rename it to refer to the parent directory.
find ${WORKDIR} -daystart -mtime +3 -type d -links 2 -exec bash -c 'zip -rm "${1%}".zip "$1"' _ {} \;
Here is a pure Bash solution:
find "$WORKDIR" -type f -name '*.zip' | while read file
do
basename=$(basename "$file")
dirname=$(dirname "$file")
suffix=$(basename "$dirname")
if [[ "$basename" != *"_${suffix}.zip" ]]; then
mv -v "$file" "${dirname}/${basename%.zip}_${suffix}.zip"
fi
done
The script processes all *.zip files found in $WORKDIR with a loop. In the loop it checks whether $file already has a suffix equal to the parent directory name. If it hasn't such suffix, the script renames the file appending "_{parent_directory_name}" to the filename just before the extension.
Sample Tree
A
├── B
│ ├── abc.zip.zip
│ └── C
│ └── file_C.zip
└── one.zip
Sample Output
‘./t/A/one.zip’ -> ‘./t/A/one_A.zip’
‘./t/A/B/abc.zip.zip’ -> ‘./t/A/B/abc.zip_B.zip’
A
├── B
│ ├── abc.zip_B.zip
│ └── C
│ └── file_C.zip
└── one_A.zip
where WORKDIR=./t.
Note, I deliberately simplified the find command, as it is not important for the algorithm. You can adjust the options according to your needs.
The best tool for this job is the rename utility that comes with Perl. (Beware that util-linux also contains a utility named rename. It is not what you want. If you have that on your system, investigate how to get rid of it and replace it with the one that comes with Perl.)
With this utility, it's as simple as
find $WORKDIR -name '*.zip' -exec \
rename 's:/([^/]+)/(.+?)\.zip$:/${2}_${1}.zip:' '{}' +
You can stick arbitrary Perl code in that first argument, which makes it even more powerful than it looks from this example.
Note that your find command appears to do something unrelated, involving the creation of .zip files.
I have a script to move files of type .txt to a particular folder .It looks for the files in work folder and move it to completed folder.
I would like to make the script generic i.e to enhance the script so that the scripts works not for just one particular folder but other similar folders as well.
Example: If there is a .txt file in folder /tmp/swan/test/work and also in folder /tmp/swan/test11/work, the files should move to /tmp/swan/test/done and /tmp/swan/test11/done respectively.
EDIT:Also, if there is a .txt file in a sub folder like /tmp/swan/test11/work/APX that should also move to /tmp/swan/test11/done
Below is the current script.
#!/bin/bash
MY_DIR=/tmp/swan
cd $MY_DIR
find . -path "*work*" -iname "*.txt" -type f -execdir mv '{}' /tmp/swan/test/done \;
With -execdir, the mv command is executed in whatever directory the file is found in. Since you just want to move the file to a "sibling" directory, each command can use the same relative path ../done.
find . -path "*work*" -iname "*.txt" -type f -execdir mv '{}' ../done \;
One way to do it:
Background:
$ tree
.
├── a
│ └── work
└── b
└── work
Renaming:
find . -type f -name work -exec \
sh -c 'echo mv "$1" "$(dirname "$1")"/done' -- {} \;
Output:
mv ./a/work ./a/done
mv ./b/work ./b/done
You can remove the echo if it does what you want it to.
What about:
find . -path '*work/*.txt' -exec sh -c 'd=$(dirname $(dirname $1))/done; mkdir -p $d; mv $1 $d' _ {} \;
(also creates the target directory if it does not exist already).