Currently I am trying this:
diff -r /develop /us-prod
which shows all the differences between the two, but all I really care about here is a file named schema.json, which is guaranteed to be there in all directories, but this file can be different.
I want to diff these two directories, but only if the file name is schema.json.
I see that you can do -x to exclude files, but it is difficult to say which other files could be in there.
There are some guaranteed files to be there, but some are not. Is there more an "inclusion" than an exclude?
You can try this :
find /develop -type f -name schema.json -exec bash -c\
'diff "$1" "/us-prod${1#/develop}"' _ {} \;
Assuming the both directories have just one schema.json file for each directory
including their subdirectories, would you please try:
diff $(find /develop -type f -name schema.json) $(find /us-prod -type f -name schema.json)
Related
I want to delete all files and sub directories created by using the specified files. I am currently using the command to delete files and directories
find . ! -name file.txt -type d -exec rm -r {} + #this is for sub directories
find . ! -name file.txt -type f -exec rm -f {} + #this is for files
It deletes all the files and sub directories when I run these command twice but I want to delete all files and directories created leaving one file at once sort. Any help is highly appreciable.
Regards
Jitendra
GNU find can directly delete files and directories:
find ! -name file.txt -delete
It will give error messages, because it cannot delete the directories up to file.txt, but everything else will still be deleted.
If you want to keep the files file1.txt, file2.txt, and file3.txt, chain the conditions like this:
find ! \( -name file1.txt -o -name file2.txt -o -name file3.txt \) -delete
There is a script on a server that I need to run over all the files in a folder. To run this script over one file I use this shell script:
for input in /home/arashsa/duo-bokmaal/Bokmaal/DUO_BM_28042.txt ; do
name=$(basename "$input")
/corpora/bokm/tools/The-Oslo-Bergen-Tagger/./tag-lbk.sh "$input" > "/home/arashsa/duo-bokmaal-obt/$name"
done
I'm terrible at writing shell scripts, and have not managed to found out how to iterate over files. What I want it is to make the script iterate over all files in a given folder that end with .txt and not those that end with _metadata.txt. So I'm thinking I would give it the folder path as argument, make it iterate over all the files in that folder, and run script on files ending with .txt and not _metadata.txt
Use find and the exec option.
$ find /path/to/dir -exec <command here> \;
Each file or directory can be obtained by using {}.
Example usage: $ find . -exec echo {} \;, this will echo each file name recursively or directory name in the current directory. You can use some other options to further specify the desired files and directories you wish to handle. I will briefly explain some of them. Note that the echo is redundant because the output of find will automatically print but I'll leave it there to illustrate the working of exec. This being said, following commands yield the same result: $ find . -exec echo {} \; and $ find .
maxdepth and mindepth
Specifying the maxdepth and mindepth allows you to go as deep down the directory structure as you like. Maxdepth determines how many times find will enter a directory and mindepth determines how many times a directory should be entered before selecting a file or dir.
Example usages:
(1) listing only elements from this dir, including . (= current dir).
(2) listing only elements from current dir excluding .
(3) listing elements from root dir and all dirs in this dir
(1)$ find . -maxdepth 1 -exec echo {} \;
(2)$ find . -mindepth 1 -maxdepth 1 -exec echo {} \;
# or, alternatively
(2)$ find . ! -path . -maxdepth 1 -exec echo {} \;
(3)$ find / -maxdepth 2 -exec echo {} \;
type
Specifying a type option allows you to filter files or directories only, example usage:
(1) list all files in this dir
(2) call shell script funtion func on every directory in the root dir.
(1)$ find . -maxdepth 1 -type f -exec echo {} \;
(2)$ find / -maxdepth 1 -type d -exec func {} \;
name & regex
The name option allows you to search for specific filenames, you can also look for files and dirs using a regex format.
Example usage: find all movies in a certain directory
$ find /path/to/dir -maxdepth 1 -regextype sed -regex ".*\.\(avi\|mp4\|mkv\)"
size
Another filter is the file size, any file or dir greater than this value will be returned. Example usage:
(1) find all empty files in current dir.
(2) find all non empty files in current dir.
(1)$ find . -maxdepth 1 -type f -size 0
(2)$ find . -maxdepth 1 -type f ! -size 0
Further examples
Move all files of this dir to a directory tmp present in .
$ find . -type f -maxdepth 1 -exec mv {} tmp \;
Convert all mkv files to mp4 files in a dir /path/to/dir and child directories
$ find /path/to/dir -maxdepth 2 -regextype sed -regex ".*\.mkv" -exec ffmpeg -i {} -o {}.mp4 \;
Convert all your jpeg files to png (don't do this, it will take very long to both find them and convert them).
$ find ~ -maxdepth 420 -regextype sed -regex '.*\.jpeg' -exec mogrify -format png {} \;
Note
The find command is a strong tool and it can prove to be fruitful to pipe the output to xargs. It's important to note that this method is superior to the following construction:
for file in $(ls)
do
some commands
done,
as the latter will handle files and directories containing spaces the wrong way.
In bash:
shopt -s extglob
for input in /dir/goes/here/*!(_metadata).txt
do
...
done
I have a folder with tens of thousands of different file types. Id like to copy them all to a new folder (Copy1) but also rename them all to $RANDOM but keep the extension intact. I realize I can write a line specifying which extension to find and how to name it, but there is got to be a way to do it dynamically, because there are at least 100 file types and may be more in the future.
I have the following so far:
find ./ -name '*.*' -type f -exec bash -c 'cp "$1" "${1/\/123_//_$RANDOM}"' -- {} \;
but that puts the random number after the extension, and also it puts the all in the same folder. I cant figure out how to do the following 2 things:
1 - Keep all paths intact, but in a new root folder (Copy1)
2 - How to have name be $RANDOM.extension, instead of .extension.$RANDOM
PS - by $RANDOM i mean actual randomly generated number. I am interested in keeping folder structure, so we are dealing with a few hundred files at most per directory, but all directories/files need to be renamed to $RANDOM. Another way to look at what I need to do. Copy all contents or Folder1 with all subdirectories and files to Folder2 (where Fodler2 is a $RANDOM name), then rename all folders and files to random names but keep all extensions.
EDIT: Ok i figured out how to rename and keep extension. But I have a problem where its dumping all of the files into the root directory where script is run from. How do I keep them in their respective folders? Command Im using is:
find ./ -name '*.*' -type f -exec bash -c 'mv "$1" $RANDOM.${1##*.}' -- {} \;
Thanks!
Ok i figured out how to rename and keep extension. But I have a
problem where its dumping all of the files into the root directory
where script is run from. How do I keep them in their respective
folders? Command Im using is:
find ./ -name '*.*' -type f -exec bash -c 'mv "$1" $RANDOM.${1##*.}' -- {} \;
Change your command to:
PATH=/bin:/usr/bin find . -name '*.*' -type f -execdir bash -c 'mv "$1" $RANDOM.${1##*.}' -- {} \;
Or alternatively using uuids instead of random numbers:
PATH=/bin:/usr/bin find . -name '*.*' -type f -execdir bash -c 'mv "$1" $(uuidgen).${1##*.}' -- {} \;
Here's what I came up with :
i=1
random="whatever"
find . -name "*.*" -type f | while read f
do
newbase=${f/*./$random$i.} //added counter to filename
cp $f /Path/Name/"$newbase"
((i++))
done
I had to add a counter to random (i), otherwise, if the extensions are similar, your files would overwrite themselves when copied.
In your new folder, your files should look like this :
whatever1.txt
whatever2.txt
etc etc
I hope this is what you were looking for.
Here is the command that worked for me.
find . -name '*.pdf' -type f -exec bash -c 'echo "{}" && cp "$1" ./$RANDOM.${1##*.}' -- {} \;
find /path/to/files -type f -not -name "M*'.jpg" -exec mv "{}" "{}".mxg \;
I fear I made two mistakes.
Files are stored in a directory structure. Goal is to keep the filenames and change the file extension from .jpg to .mxg. But only for files that have 'M' as the first character of there filename.
The above line has this result:
all files have .mxg added. So the .jpg isn't and all files are changed.
This should do it:
find /path/to/files -type f -name 'M*.jpg' -exec bash -c 'echo mv "$1" "${1/jpg/mxg}"' -- {} \;
A somewhat cleaner solution is if you have the rename command. However, there are different implementations out there, so read your man page first to check you have the same as mine. The version I have in Debian is Larry Wall's implementation in perl. You can recognize this by the example rename 's/\.bak$//' *.bak near the top, or the AUTHOR section near the bottom. With this implementation you can rename your files like this:
find /path/to/files -type f -name 'M*.jpg' -exec rename 's/jpg$/mxg/' {} \;
I have some folder hierarchy, in some of the folders there are images, I need a shell script which can list all images and copy them into one specified folder, where listing them is not important, I just want to copy all images into a folder?
I know I can
ls -R *.png
but how do I copy them all to one folder?
Thanks!
Update: As glenn jackman has pointed out, this would be slightly more efficient to use over the answer I provided:
file . -type f -name \*.png | xargs cp -t destination
For the explanation, see glenn's comments that follow this answer.
One way is to use find:
find . -type f -name "*.png" -exec cp {} ~/path/to/your/destination/folder \;
Explanation:
find is used to find files / directories
. start finding from the current working directory (alternatively, you can specify a path)
-type f: only consider files (as opposed to directories)
-name "*.png": only consider those with png extension
-exec: for each such result found, do something (see below)
cp {} ~/path/to/your/destination/folder \;: this is the do something part: copy each such result found (substituted into the {}) to the destination specified.
To copy multiple file patterns in single go we can use -regex instead -name
find . -type f -regex '.*\(jpg\|jpeg\|png\|gif\|mp4\|avi\|svg\|mp3\|vob\)' -exec cp {} /path/to/your/destination/folder \;