I am trying to synchronise two directories, omitting certain files.
find . -type f -print0 |rsync -vupt -0 --files-from=- . /tmp/test
This copies every file of course, but works as expected
The problem arises when I try to limit the files copied. So if I want to copy all but the files ending ".part" :
find . -type f -\! -iname "*.part" -print0 |rsync -vupt -0 --files-from=- . /tmp/test
But this fails to copy anything at all.
If I remove the pipe I can see that find is outputting a stream that looks just like the kind of output from the first command but minus the file names I don't want to copy.
(I've tries -not and -name alternatives too without luck)
What am I doing wrong?
Thanks for any help.
The find negation is not a command, but an operator. You can use ! -iname "*.part" to negate that single expression.
Related
I need to delete all files with a pattern name: 2020*.js
Inside a specific directory: server/db/migrations/
And then show what it have been deleted: `| xargs``
I'm trying this:
find . -name 'server/db/migrations/2020*.js' #-delete | xargs
But nothing is deleted, and shows nothing.
What I'm doing wrong?
The immediate problem is that -name only looks at the last component of the file name (so 2020xxx.js) and cannot match anything with a slash in it. You can use the -path predicate but the correct solution is to simply delete these files directly:
rm -v server/db/migrations/2020*.js
The find command is useful when you need to traverse subdirectories.
Also, piping the output from find to xargs does not do anything useful; if find prints the names by itself, xargs does not add any value, and if it doesn't, well, xargs can't do anything with an empty input.
If indeed you want to traverse subdirectories, try
find server/db/migrations/ -type f -name '2020*.js' -print -delete
If your shell supports ** you could equally use
rm -v server/db/migrations/**/2020*.js
which however has a robustness problem if there can be very many matching files (you get "command line too long"). In that scenario, probably fall back to find after all.
You're looking for something like this:
find server/db/migrations -type f -name '2020*.js' -delete -print
You have try this:
find . -name 'server/db/migrations/2020*.js' | xargs rm
I converted/compressed several MP4 files from several folders using VLC.
The names of the converted/compressed files end with -converted, for example. 2. bubble sort-converted.mp4.
It's really cumbersome to go into each folder and delete all the original files and leave the converted files.
Using some zsh/bash command I'd like to recursively delete all the original files and leave the converted files.
For example I'll delete 3 - sorting/2. bubble sort.mp4 and will leave 3 - sorting/2. bubble sort-converted.mp4.
TLDR;
In easy words, delete all the files with .mp4 extension, where filesnames don't end with -converted using some zsh/bash command.
Also If there is some way to rename the converted file to the original name after deleting the original files, that will be a plus.
Thank you!
find can be used with a logical expression to match the desired files and delete them.
In your case the following can be used to verify whether it matches the files you want to delete. It finds all files that don't have converted in their names but do end in .mp4.
find . -type f -not \( -name '*converted*' \) -a -name "*.mp4"
Once you are satsified with the file list result then add -delete to do the actual delete.
find . -type f -not \( -name '*converted*' \) -a -name "*.mp4" -delete
Give this a try:
find . -name '*.mp4' | grep -v 'converted' | xargs rm -f
The zsh pure solution:
rm -f ^(*.mp4-converted)(.)
^ ................. negates
*-converted ....... pattern
(.) ............... regular files
Using gnu parallel (in case of many files)
parallel --no-notice rm -rf ::: ^(*converted)(.)
This will work even if your file names contain ', " or space:
find . -name '*.mp4' |
grep -v 'converted' |
parallel -X rm -f
I have this find command:
find . -type f -not -path '**/.git/**' -not -path '**/node_modules/**' | xargs sed -i '' s/typescript-library-skeleton/xxx/g;
for some reason it's giving me these warnings/errors:
find: ./.git/objects/3c: No such file or directory
find: ./.git/objects/3f: No such file or directory
find: ./.git/objects/41: No such file or directory
I even tried using:
-not -path '**/.git/objects/**'
and got the same thing. Anybody know why the find is searching in the .git directory? Seems weird.
why is the find searching in the .git directory?
GNU find is clever and supports several optimizations over a naive implementation:
It can flip the order of -size +512b -name '*.txt' and check the name first, because querying the size will require a second syscall.
It can count the hard links of a directory to determine the number of subdirectories, and when it's seen all it no longers needs to check them for -type d or for recursing.
It can even rewrite (-B -or -C) -and -A so that if the checks are equally costly and free of side effects, the -A will be evaluated first, hoping to reject the file after 1 test instead of 2.
However, it is not yet clever enough to realize that -not -path '*/.git/*' means that if you find a directory .git then you don't even need to recurse into it because all files inside will fail to match.
Instead, it dutifully recurses, finds each file and matches it against the pattern as if it was a black box.
To explicitly tell it to skip a directory entirely, you can instead use -prune. See How to exclude a directory in find . command
Both more efficient and more correct would be to avoid the default -print action, change -not -path ... to -prune, and ensure that xargs is only used with NUL-delimited input:
find . -name .git -prune -o \
-name node_modules -prune -o \
-type f -print0 | xargs -0 sed -i '' s/typescript-library-skeleton/xxx/g '{}' +
Note the following points:
We use -prune to tell find to not even recurse down the undesired directories, rather than -not -path ... to tell it to discard names in those directories after they were found.
We put the -prunes before the -type f, so we're able to match directories for pruning.
We have an explicit action, not depending on the default -print. This is important because the default -print effectively has a set of parenthesis: find ... behaves like find '(' ... ')' -print, not like find ... -print, no if explicit action is given.
We use xargs only with the -0 argument enabling NUL-delimited input, and the -print0 action on the find side to generate a NUL-delimited list of names. NUL is the only character which cannot be present in an arbitrary file path (yes, newlines can be present) -- and thus the only character which is safe to use to separate paths. (If the -0 extension to xargs and the -print0 extension to find are not guaranteed to be available, use -exec sed -i '' ... {} + instead).
I've got the following line in my bash script:
for i in $(find ./TTDD* -type f)
do
It works when there's files in the directory, but when it's empty I get the following:
find ... No such file or directory
How can I suppress that exact error message, as I'm logging output and doesn't care about that specific error message.
The problem is that globs that don't have any matches expand to themselves, and since there's no file named TTDD*, you get this error.
You can rewrite it in different ways. The most straight forward is:
find . -path './TTDD*' -type f
This will show the same files.
If there are other directories in the current dir, it will waste some time going through their files even if they'll never match. If required, you can short-circuit such directories with a less readable find . -path . -o -not -path './TTDD*' -prune -o -type f -print.
NB: iterating over these files with a for loops will break for files with spaces and various other special characters. You can combine this with anubhava's answer to safely read all filenames while also not suppressing all of find's other potentially useful error messages.
while IFS= read -rd '' f; do
printf "Processing [%s]\n" "$f"
done < <(find . -path './TTDD*' -type f -print0 2>/dev/null)
How about you use "-exec"?
This way you can redirect all results of your find command to the input of some other command, e.g., let's say that I want to find all the files within a given folder that are owned by the root user and I want to change the ownership of these files, here is the code:
find ${DataFS}/Data -user root -exec chown someuser:someuser {} \;
The approach you are using is similar to
find ${DataFS}/Data -user root -print0 | xargs -0 chown someuser:someuser
Which is not ideal because it will fail in case it can't find any files (i.e., if it prints an empty string), i.e., chown: missing operand after `someuser:someuser'
OK, so simple enough.. I want to recursively search a directory for files with a specific extension - and then perform an action on those files.
# pwdENTER
/dir
# ls -R | grep .txt | xargs -I {} open {} ENTER
The file /dir/reallyinsubfolder.txt does not exist. ⬅ fails (bad)
Not output, but succeeds.. /dir/fileinthisfolder.txt ⬅ opens silently (good)
This does find ALL the files I am interested in… but only OPEN's those which happen to be "1-level" deep. In this case, the attempt to open /dir/reallyinsubfolder.txt fails, as reallyinsubfolder.txt is actually /dir/sub/reallyinsubfolder.txt.
I understand that grep is simply returning the matched filename… which then chokes (in this case), the open command, as it fails to reach down to the correct sub-directory to execute the file..
How do I get grep to return the full path of a match?
How about using the find command -
find /path/to/dir -type f -iname "*.txt" -exec action to perform {} \;
find . -name *.txt -exec open {};
(Decorate with backslashes of your needing)
I believe you're asking the wrong question; parsing ls(1) output in this fashion is far more trouble than it is worth.
What would work far more reliably:
find /dir -name '*.txt' -print0 | xargs -0 open
or
find /dir -name '*.txt' -exec open {} \;
find(1) does not mangle names nearly as much as ls(1) and makes executing programs on matched files far more reliable.