Try to copy files recursively from directory (d1) to directory (d2) exclude some files and folders using lists:
filepaths to exclude stored as files.txt
and directories list to exlude stored as dirs.txt.
Already read some SOF articles like
Copy folder recursively, excluding some folders
BASH copy all files except one
tried rsync
rsync -avr --exclude=*/exclude_filename_* d1/ d2/
but wildcards don't suit either due to a large number of different file names.
Maybe I have to use WHILE loop with rsync..? Still looking for solution.
Does anybody know how to do this?
This is not the best way to do it, but if rsync is not working, you can use find to composite a list and then remove files from that list.
find . -name "*" | grep -v "<filename you dont want>" | xargs -i cp --parents {} /output/dir/name
The only thing to keep in mind here is that your current directory must be the base directory of the files you want to copy, in order to preserve the parent structure.
note:
Add another | grep -v "<filename you dont want>" to remove a second file. Alternatively, you can use wildcard matching in one grep grep -v "<file1>\|<file2>"
Related
I have a directory named XYZ which has directories ABC, DEF, GHI inside it. I want to search for a pattern 'writeText' in all *.c in all directories (i.e XYZ, XYZ/ABC, XYZ/DEF and XYZ/GHI)
What grep command can I use?
Also if I want to search only in XYZ, XYZ/ABC, XYZ/GHI and not XYZ/DEF, what grep command can I use?
Thank you!
grep -R --include="*.c" --exclude-dir={DEF} writeFile /path/to/XYZ
-R means recursive, so it will go into subdirectories of the directory you're grepping through
--include="*.c" means "look for files ending in .c"
--exclude-dir={DEF} means "exclude directories named DEF. If you want to exclude multiple directories, do this: --exclude-dir={DEF,GBA,XYZ}
writeFile is the pattern you're grepping for
/path/to/XYZ is the path to the directory you want to grep through.
Note that these flags apply to GNU grep, might be different if you're using BSD/SysV/AIX grep. If you're using Linux/GNU grep utils you should be fine.
You can use the following command to answer at least the first part of your question.
find . -name *.c | xargs grep "writeText"
How do I use sed (or any other command line utility) to search recursively for all .r and .R files, and remove all lines in all files with a certain pattern?
I've seen this command
sed -i '/"pattern"/d' folder/*
But this works on all the files in folder/, which might miss some stuff, and harm other files I don't want to touch. My directory has a tree-like pattern, and so I need to search everything recursively.
Probably something like this:
find folder -name "*.[rR]" -exec sed -i '/pattern/d' {} \;
I am stuck with one issue, and being a starter in shell script its getting confusing to achieve it.
I need to pull the file name while unzipping a file (which contains different file with different paths) in shell.
Example:
/java/server/test/Class1.java
/java/server/Xml1.xml
Output
I should get below as output in some local variables:
/java/server/test/
/java/server/
Note: I am using unzip utility for this.
If you'e got the full filename including path in $Filename then
${Filename##*/}
will give the path and
${Filename%/*}
will give the filename.
You can done this using combination of sed.
unzip a.zip | sed 's/.*: \(.*\)\/.*$/\1/'
It will give you the directory names without file names. If you need a uniq value, then use uniq command with that.
unzip a.zip | sed 's/.*: \(.*\)\/.*$/\1/' | uniq
Just unzip the file normally
unzip myfolder.zip
and then recurse through the folder and get the filepaths:
find $(pwd)/myfolder -type d
This last command gets (find) all folders (-type d) in extracted directory (myfolder) in the current directory (pwd).
Then you can redirect its output to a file
find $(pwd)/myfolder -type d > dirnames
or store it in a variable:
DIRNAMES=$(find $(pwd)/myfolder -type d)
I would appreciate any help, relatively new here
I have the following directory structure
Main_dir
|-Barcode_subdirname_01\(many further subfolders)\filename.pdf
|-Barcode_subdirname_02\(many further subfolders)\filename.csv
There are 1000s of files within many subfolders
The first level sub directories have the barcode associated to all files within. eg 123456_dirname
I want to copy all files within all subfoders to the main_dir and
rename the files subdirname_barcode_filename.extension (based only on the first subdirectory name and barcode)
I've been attempting to write a bash script to do this from the main_dir but have hit the limit of my coding ability (i'm open to any other way that'll work).
firstly identifying the first level sub folders
find -maxdepth 1 -type d |
then cut out the first 2 parts deliminated by the underscores
cut -d\_ -f1 > barcode
then find the files within the subfolders, rename and move
find -type f -print0 |
while IFS= read -r filenames; do
newname="${barcode/sudirname/filename\/}"
mv "filename" "main_dir"/"newname"
done
I can't get it to work and may be headed in the wrong direction.
You can use rename with sed like substitute conventions, for example
$ rename 's~([^_]+)_([^_]+)_.*/([^/.]+\..*)~$1_$2_$3~' barcode_subdir_01/a/b/c/file2.csv
will rename file to
barcode_subdir_file2.csv
I used ~ instead of the more common / separator to make it more clear.
You can test the script with -n option to show the renamed files without actually doing the action.
I would like to paste a number of files in a unique file named, for example, "output.txt".
Nevertheless I would like to assign the name of the folder to the name of the output file so that it will be: "output_<name of the Folder>.txt".
I have thousands of folders so that the "paste" command will run in a for loop I'm able to write. Can anyone help me please?
The question is incredibly unclear. I'm going to interpret it to mean that you have a large number of directories that all contain a file named 'output.txt', and you want to move those files to a single directory with the original path embedded in the name. Assuming that the root of the directory tree containing all the files is /path/to/source and you want to move them to /path/to/destination:
find /path/to/source -name output.txt -exec sh -c 'd=$(dirname "$1" |
tr / _); cp "$1" "/path/to/destination/output_$d.txt"' sh {} \;
Relative paths will work fine as well as absolute paths.
I too am unclear about what you want, but mktemp(1) has TEMPLATES which might help.