I am trying to encrypt a folder on our Synology Nas but have found roughly 250 files with filenames longer than 143 characters. Is there any command I can use to remove all characters from the end of the file names so it is under 143 characters in length.
The command i used to find the files
find . -type f -name '???????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????????*'
I was hoping to be able to navigate to the 8-9 or so directories that hold these chunks of files and be able to run a line of code that found the files with names longer than n characters and drop the extra characters to get it under 143.
I'm not familiar with Synology, but if you have a rename command which accepts Perl regex substitutions, and you are fine with assuming that no two files in the same directory have the same 143-character prefix (or losing one of them in this case is acceptable), I guess something like
find . -type f -regex '.*/[^/]\{143\}[^/]+' -exec rename 's%([^/]{143})[^/]+$%$1%' {} +
If you don't have this version of the nonstandard rename command, the simplest solution might be to pipe find's output to Perl and then pipe that to sh:
find . -type f -regex '.*/[^/]\{143\}[^/]+' |
perl -pe 's%(.*/)([^/]{143})([^/]+)$%mv "$1$2$3" "$1$2"' |
sh
If you don't have access to Perl, the same script could be refactored into a sed command, though the regex will be slightly different because they speak different dialects.
find . -type f -regex '.*/[^/]\{143\}[^/]+' |
sed 's%\(.*/\)\([^/]\{143\}\)\([^/]\+\)$%mv "\1\2\3" "\1\2"' |
sh
This has some naïve assumptions about your file names - if they could contain newlines or double quotes, you need something sturdier (see https://mywiki.wooledge.org/BashFAQ/020). In rough terms, maybe try
find . -type f -regex '.*/[^/]\{143\}[^/]+' -exec bash -c 'for f; do
g=${f##*/}
mv -- "$f" "${f%/*}/${g:0:143}"
done' _ {} +
Related
I have arraylist of files and I am trying to use rm with xargs to remove files like:
dups=["test.csv","man.csv","teams.csv"]
How can I pass the complete dups array to find and delete these files?
I want to make changes below to make it work
find ${dups[#]} -type f -print0 | xargs -0 rm
Your find command is wrong.
# XXX buggy: read below
find foo bar baz -type f -print0
means look in the paths foo, bar, and baz, and print any actual files within those. (If one of the paths is a directory, it will find all files within that directory. If one of the paths is a file in the current directory, it will certainly find it, but then what do you need find for?)
If these are files in the current directory, simply
rm -- "${dups[#]}"
(notice also how to properly quote the array expansion).
If you want to look in all subdirectories for files with these names, you will need something like
find . -type f \( -name "test.csv" -o -name "man.csv" -o -name "teams.csv" \) -delete
or perhaps
find . -type f -regextype egrep -regex '.*/(test\.csv|man\.csv|teams\.csv)' -delete
though the -regex features are somewhat platform-dependent (try find -E instead of find -regextype egrep on *BSD/MacOS to enable ERE regex support).
Notice also how find has a built-in predicate -delete so you don't need the external utility rm at all. (Though if you wanted to run a different utility, find -exec utility {} + is still more efficient than xargs. Some really old find implementations didn't have the + syntax for -exec but you seem to be on Linux where it is widely supported.)
Building this command line from an array is not entirely trivial; I have proposed a duplicate which has a solution to a similar problem. But of course, if you are building the command from Java, it should be easy to figure out how to do this on the Java side instead of passing in an array to Bash; and then, you don't need Bash at all (you can pass this to find directly, or at least use sh instead of bash because the command doesn't require any Bash features).
I'm not a Java person, but from Python this would look like
import subprocess
command = ["find", ".", "-type", "f"]
prefix = "("
for filename in dups:
command.extend([prefix, "-name", filename])
prefix = "-o"
command.extend([")", "-delete"])
subprocess.run(command, check=True, encoding="utf-8")
Notice how the backslashes and quotes are not necessary when there is no shell involved.
I want to rename all files in a folder and its subfolders.
I need to change the string HEX20 to the string HEX8.
Some filenames have other numbers, so I cannot simply change the 20 to an 8.
An example of the full path is:
\\FRDS01006\z188018\FEM\Linear\HEX20\3HEX20\3HEX20.bof
I would like to do the same replacement for the folder names.
How about this:
find . -name "*HEX20*" -exec rename HEX20 HEX8 '{}' +
This will search recursively through the current directory and any subdirectories to match HEX20. (The flag -type f is omitted because the asker wants to change the names of directories in addition to files.) It will then build a long rename command and ultimately call it. This type of construction may be simpler than building a series of commands with sed and then executing them one-by-one.
Try this:
find . -type f -name "*HEX20*" | sed 's/\(.*\)HEX20\(.*\)/mv \0 \1HEX8\2/' | sh
This way you find for regular files having HEX20 in their names:
find . -type f -name "*HEX20*"
then change the last occurrence of HEX20 whith HEX8 and compile the mv command:
find . -type f -name "*HEX20*" | sed 's/\(.*\)HEX20\(.*\)/mv \0 \1HEX8\2/'
finally you execute the compiled commands with sh:
find . -type f -name "*HEX20*" | sed 's/\(.*\)HEX20\(.*\)/mv \0 \1HEX8\2/' | sh
I'm reviewing for an exam and one of the questions is asking me to write a single command that will delete the files in a given directory that are at least 6 characters long.
Example:
person#ubuntumachine:~$ ls
abc.txt, abcdef.txt, 123456.txt, helloworld.txt, rawr.txt
The command would delete the files "abcdef.txt", "12346.txt" and "helloworld.txt".
I'm aware the at the * would be used at some point but I'm not sure what to use to indicate 6 characters long...
Thank you <3
Since the question can have 2 interpretations, both answers are given:
1. To delete files with 6 or more characters in the FILE NAME:
rm ??????*
Explanation:
??????: The ?'s are to be entered literally. Each ? matches any single character. So here it means "match any 6 characters"
*: The * wildcard matches zero or more characters
Therefore it removes any file with 6 or more characters.
Alternatively:
find -type f -name "??????*" -delete
Explanation:
find: invoke the find command
-type f: find only files.
-name "??????*": match any file with at least 6 characters, same idea as above.
-delete: delete any such files found.
2. To delete files with 6 or more characters in its CONTENTS:
find -type f -size +5c -delete
Explanation:
find: invoke the find command
-type f: find only files (not directories etc)
-size +5c: find only files greater than 5 characters long. Note: recall that EOF (end of file) counts as a character in this case. If you'd like to exclude EOF from your counter, change it from 5 to 6.
-delete: delete any such files found
Something like this should work:
$ ls|while read filename; do test ${#filename} -gt 6 && echo rm "$filename"; done
The trick is to use the ${#foo} construct to get the length of the filename.
Once you're satisfied with the output, immediately run the following after the previous command:
$ !! | sh
This repeats the last command (which shows the rm command to delete the files) and pipe it to sh to really execute it.
This will perform the requested logic on the current directory and all subdirectories.
find . -type f -regextype posix-egrep -regex ".*/[^/]{5}[^/]+$" -exec rm -vf {} \;
find .
searches the local directory (change the .
to search elsewhere)
-type f
considers files only
-regextype posix-egrep
use egrep regex syntax (this is what I know)
-regex ".*/[^/]{5}[^/]+$"
find will match all paths matching this regex
the regex deconstructs as follows:
.*/ effectively ignores the path until the filename
[^/]{5} finds 5 characters that are not slashes
[^/]+$ requires at least one more character (thus: 6 or more) that is not a slash to appear before the end of line ($)
-exec rm -vf {} \;
find will replace the {} with each file its search query matches (in this case, files with paths that match our regex). Thus, this achieves the deletion. -vf added to print the results so you know what's happened.
-exec is picky about syntax - the \; is necessary to avoid find: missing argument to '-exec' encountered if a simple ; is used in its place.
You can test this by using -print instead of -exec rm -vf {} \; or simply removing the -exec rm -vf {} \; (-print is find's default behavior)
I am trying to remove all empty files that are older than 2 days. Also I am ignoring hidden files, starting with dot. I am doing it with this code:
find /u01/ -type f -size 0 -print -mtime +2 | grep -v "/\\." | xargs rm
It works fine until there are spaces in the name of the file. How could I make my code ignore them?
OS is Solaris.
Option 1
Install GNU find and GNU xargs in an appropriate location (not /usr/bin) and use:
find /u01/ -type f -size 0 -mtime +2 -name '[!.]*' -print0 | xargs -0 rm
(Note that I removed (what I think is) a stray -print from your find options. The options shown removes empty files modified more than 2 days ago where the name does not start with a ., which is the condition that your original grep seemed to deal with.)
Option 2
The problem is primarily that xargs is defined to split its input at spaces. An alternative is to write your own xargs surrogate that behaves sensibly with spaces in names; I've done that. You then only run into problems if the file names contain newlines — which the file system allows. Using a NUL ('\0') terminator is guaranteed safe; it is the only character that can't appear in a path name (which is why GNU chose to use it with -print0 etc).
Option 3
A final better option is perhaps:
find /u01/ -type f -size 0 -mtime +2 -name '[!.]*' -exec rm {} \;
This avoids using xargs at all and handles all file names (path names) correctly — at the cost of executing rm once for each file found. That's not too painful if you're only dealing with a few files on each run.
POSIX 2008 introduces the notation + in place of the \; and then behaves rather like xargs, collecting as many arguments as will conveniently fit in the space it allocates for the command line before running the command:
find /u01/ -type f -size 0 -mtime +2 -name '[!.]*' -exec rm {} +
The versions of Solaris I've worked on do not support that notation, but I know I work on antique versions of Solaris. GNU find does support the + marker and therefore renders the -print0 and xargs -0 workaround unnecessary.
I'm working in Linux & bash (or Cygwin & bash).
I have a huge--huge--directory structure, and I have to find a few needles in the haystack.
Specifically, I'm looking for these files (20 or so):
foo.c
bar.h
...
quux.txt
I know that they are in a subdirectory somewhere under ..
I know I can find any one of them with
find . -name foo.c -print. This command takes a few minutes to execute.
How can I print the names of these files with their full directory name? I don't want to execute 20 separate finds--it will take too long.
Can I give find the list of files from stdin? From a file? Is there a different command that does what I want?
Do I have to first assemble a command line for find with -o using a loop or something?
If your directory structure is huge but not changing frequently, it is good to run
cd /to/root/of/the/files
find . -type f -print > ../LIST_OF_FILES.txt #and sometimes handy the next one too
find . -type d -print > ../LIST_OF_DIRS.txt
after it you can really FAST find anything (with grep, sed, etc..) and update the file-lists only when the tree is changed. (it is a simplified replacement if you don't have locate)
So,
grep '/foo.c$' LIST_OF_FILES.txt #list all foo.c in the tree..
When want find a list of files, you can try the following:
fgrep -f wanted_file_list.txt < LIST_OF_FILES.txt
or directly with the find command
find . type f -print | fgrep -f wanted_file_list.txt
the -f for fgrep mean - read patterns from the file, so you can easily grepping input for multiple patterns...
You shouldn't need to run find twenty times.
You can construct a single command with a multiple of filename specifiers:
find . \( -name 'file1' -o -name 'file2' -o -name 'file3' \) -exec echo {} \;
Is the locate(1) command an acceptable answer? Nightly it builds an index, and you can query the index quite quickly:
$ time locate id_rsa
/home/sarnold/.ssh/id_rsa
/home/sarnold/.ssh/id_rsa.pub
real 0m0.779s
user 0m0.760s
sys 0m0.010s
I gave up executing a similar find command in my home directory at 36 seconds. :)
If nightly doesn't work, you could run the updatedb(8) program by hand once before running locate(1) queries. /etc/updatedb.conf (updatedb.conf(5)) lets you select specific directories or filesystem types to include or exclude.
Yes, assemble your command line.
Here's a way to process a list of files from stdin and assemble your (FreeBSD) find command to use extended regular expression matching (n1|n2|n3).
For GNU find you may have to use one of the following options to enable extended regular expression matching:
-regextype posix-egrep
-regextype posix-extended
echo '
foo\\.c
bar\\.h
quux\\.txt
' | xargs bash -c '
IFS="|";
find -E "$PWD" -type f -regex "^.*/($*)$" -print
echo find -E "$PWD" -type f -regex "^.*/($*)$" -print
' arg0
# note: "$*" uses the first character of the IFS variable as array item delimiter
(
IFS='|'
set -- 1 2 3 4 5
echo "$*" # 1|2|3|4|5
)