Hello and thanks for reading!
I have list like that:
user#laptop:~/lol$ find ./ \( -iname 'latest*' \) -type f -printf "%p\n"
/leagueoflegends/images/e/e6/CaitlynSquare.png/revision/latest?cb=20150402215548
/leagueoflegends/images/e/ef/WillumpSquare.png/revision/latest?cb=20150402222523
/leagueoflegends/images/d/d8/MorganaSquare.png/revision/latest?cb=20150402220702
/leagueoflegends/images/d/d1/UdyrSquare.png/revision/latest?cb=20150402221631
(its longer) and I want to copy each image ("latest?cb=20150402221631" are png images) to another folder and rename it for the name of 2 level up directory, like this:
latest?cb=20150402221631 --> to --> UdyrSquare.png
And if its possible, remove Square.
latest?cb=20150402221631 --> to --> Udyr.png
With bash:
Append this:
| while read -r line; do echo mv -v "$line" "${line%Square*}.png"; done
If everything looks okay, remove echo.
If you have the Perl rename command (which maybe prename or perl-rename on your distribution), you can use regular expressions to rename files. It can read filenames from input:
find . -iname 'latest*' -type f | rename -n ':Square.png.*cb=\d+:.png:'
Or, more specifically:
find . -iname 'latest*' -type f | rename -n 's:Square.png/[^/]*/latest\?cb=\d+:.png:'
For example:
$ cat foo.txt| perl-rename -n 's:Square.png/[^/]*/latest\?cb=\d+:.png:'
/leagueoflegends/images/e/e6/CaitlynSquare.png/revision/latest?cb=20150402215548 -> /leagueoflegends/images/e/e6/Caitlyn.png
/leagueoflegends/images/e/ef/WillumpSquare.png/revision/latest?cb=20150402222523 -> /leagueoflegends/images/e/ef/Willump.png
/leagueoflegends/images/d/d8/MorganaSquare.png/revision/latest?cb=20150402220702 -> /leagueoflegends/images/d/d8/Morgana.png
/leagueoflegends/images/d/d1/UdyrSquare.png/revision/latest?cb=20150402221631 -> /leagueoflegends/images/d/d1/Udyr.png
The -n option is for testing the command. Once you are satisfied with the results, run without it.
Related
I would like to rename several files picked by find in some directory, then use xargs and mv to rename the files, with parameter expansion. However, it did not work...
example:
mkdir test
touch abc.txt
touch def.txt
find . -type f -print0 | \
xargs -I {} -n 1 -0 mv {} "${{}/.txt/.tx}"
Result:
bad substitution
[1] 134 broken pipe find . -type f -print0
Working Solution:
for i in ./*.txt ; do mv "$i" "${i/.txt/.tx}" ; done
Although I finally got a way to fix the problem, I still want to know why the first find + xargs way doesn't work, since I don't think the second way is very general for similar tasks.
Thanks!
Remember that shell variable substitution happens before your command runs. So when you run:
find . -type f -print0 | \
xargs -I {} -n 1 -0 mv {} "${{}/.txt/.tx}"
The shell tries to expan that ${...} construct before xargs even
runs...and since that contents of that expression aren't a valid shell variable reference, you get an error. A better solution would be to use the rename command:
find . -type f -print0 | \
xargs -I {} -0 rename .txt .tx {}
And since rename can operate on multiple files, you can simplify
that to:
find . -type f -print0 | \
xargs -0 rename .txt .tx
it should display only the file names without the.sh
Example
$>./file.sh | cat -e
you can do :
find . -maxdepth 1 -type f -name \*.sh | sed 's|.sh$||'
remove the maxdepth 1 option if you want to find all the files recursively
You can just do ls | grep -v .sh to find in the current directory.
-v, --invert-match
Invert the sense of matching, to select non-matching lines.
If you need system-wide search, you'll need find:
find / -type f ! -name "*\.sh"
/ means root folder
type f = file
! -name = not sending with .sh
I have a makefile that concatenates JavaScript files together and then runs the file through uglify-js to create a .min.js version.
I'm currently using this command to find and concat my files
find src/js -type f -name "*.js" -exec cat {} >> ${jsbuild}$# \;
But it lists files in directories first, this makes heaps of sense but I'd like it to list the .js files in the src/js files above the directories to avoid getting my undefined JS error.
Is there anyway to do this or? I've had a google around and seen the sort command and the -s flag for find but it's a bit above my understanding at this point!
[EDIT]
The final solution is slightly different to the accepted answer but it is marked as accepted as it brought me to the answer. Here is the command I used
cat `find src/js -type f -name "*.js" -print0 | xargs -0 stat -f "%z %N" | sort -n | sed -e "s|[0-9]*\ \ ||"` > public/js/myCleverScript.js
Possible solution:
use find for getting filenames and directory depth, i.e find ... -printf "%d\t%p\n"
sort list by directory depth with sort -n
remove directory depth from output to use filenames only
test:
without sorting:
$ find folder1/ -depth -type f -printf "%d\t%p\n"
2 folder1/f2/f3
1 folder1/file0
with sorting:
$ find folder1/ -type f -printf "%d\t%p\n" | sort -n | sed -e "s|[0-9]*\t||"
folder1/file0
folder1/f2/f3
the command you need looks like
cat $(find src/js -type f -name "*.js" -printf "%d\t%p\n" | sort -n | sed -e "s|[0-9]*\t||")>min.js
Mmmmm...
find src/js -type f
shouldn't find ANY directories at all, and doubly so as your directory names will probably not end in ".js". The brackets around your "-name" parameter are superfluous too, try removing them
find src/js -type f -name "*.js" -exec cat {} >> ${jsbuild}$# \;
find could get the first directory level already expanded on commandline, which enforces the order of directory tree traversal. This solves the problem just for the top directory (unlike the already accepted solution by Sergey Fedorov), but this should answer your question too and more options are always welcome.
Using GNU coreutils ls, you can sort directories before regular files with --group-directories-first option. From reading the Mac OS X ls manpage it seems that directories are grouped always in OS X, you should just drop the option.
ls -A --group-directories-first -r | tac | xargs -I'%' find '%' -type f -name '*.js' -exec cat '{}' + > ${jsbuild}$#
If you do not have the tac command, you could easily implement it using sed. It reverses the order of lines. See info sed tac of GNU sed.
tac(){
sed -n '1!G;$p;h'
}
You could do something like this...
First create a variable holding the name of our output file:
OUT="$(pwd)/theLot.js"
Then, get all "*.js" in top directory into that file:
cat *.js > $OUT
Then have "find" grab all other "*.js" files below current directory:
find . -type d ! -name . -exec sh -c "cd {} ; cat *.js >> $OUT" \;
Just to explain the "find" command, it says:
find
. = starting at current directory
-type d = all directories, not files
-! -name . = except the current one
-exec sh -c = and for each one you find execute the following
"..." = go to that directory and concatenate all "*.js" files there onto end of $OUT
\; = and that's all for today, thank you!
I'd get the list of all the files:
$ find src/js -type f -name "*.js" > list.txt
Sort them by depth, i.e. by the number of '/' in them, using the following ruby script:
sort.rb:
files=[]; while gets; files<<$_; end
files.sort! {|a,b| a.count('/') <=> b.count('/')}
files.each {|f| puts f}
Like so:
$ ruby sort.rb < list.txt > sorted.txt
Concatenate them:
$ cat sorted.txt | while read FILE; do cat "$FILE" >> output.txt; done
(All this assumes that your file names don't contain newline characters.)
EDIT:
I was aiming for clarity. If you want conciseness, you can absolutely condense it to something like:
find src/js -name '*.js'| ruby -ne 'BEGIN{f=[];}; f<<$_; END{f.sort!{|a,b| a.count("/") <=> b.count("/")}; f.each{|e| puts e}}' | xargs cat >> concatenated
I have a task where I need to move a bunch of files from one directory to another. I need move all files with the same file name (i.e. blah.pdf, blah.txt, blah.html, etc...) at the same time, and I can move a set of these every four minutes. I had a short bash script to just move a single file at a time at these intervals, but the new name requirement is throwing me off.
My old script is:
find ./ -maxdepth 1 -type f | while read line; do mv "$line" ~/target_dir/; echo "$line"; sleep 240; done
For the new script, I basically just need to replace find ./ -maxdepth 1 -type f
with a list of unique file names without their extensions. I can then just replace do mv "$line" ~/target_dir/; with do mv "$line*" ~/target_dir/;.
So, with all of that said. What's a good way to get a unique list of files without their file names with bash script? I was thinking about using a regex to grab file names and then throwing them in a hash to get uniqueness, but I'm hoping there's an easier/better/quicker way. Ideas?
A weird-named files tolerant one-liner could be:
find . -maxdepth 1 -type f -and -iname 'blah*' -print0 | xargs -0 -I {} mv {} ~/target/dir
If the files can start with multiple prefixes, you can use logic operators in find. For example, to move blah.* and foo.*, use:
find . -maxdepth 1 -type f -and \( -iname 'blah.*' -or -iname 'foo.*' \) -print0 | xargs -0 -I {} mv {} ~/target/dir
EDIT
Updated after comment.
Here's how I'd do it:
find ./ -type f -printf '%f\n' | sed 's/\..*//' | sort | uniq | ( while read filename ; do find . -type f -iname "$filename"'*' -exec mv {} /dest/dir \; ; sleep 240; done )
Perhaps it needs some explaination:
find ./ -type f -printf '%f\n': find all files and print just their name, followed by a newline. If you don't want to look in subdirectories, this can be substituted by a simple ls;
sed 's/\..*//': strip the file extension by removing everything after the first dot. Both foo.tar ad foo.tar.gz are transformed into foo;
sort | unique: sort the filenames just found and remove duplicates;
(: open a subshell:
while read filename: read a line and put it into the $filename variable;
find . -type f -iname "$filename"'*' -exec mv {} /dest/dir \;: find in the current directory (find .) all the files (-type f) whose name starts with the value in filename (-iname "$filename"'*', this works also for files containing whitespaces in their name) and execute the mv command on each one (-exec mv {} /dest/dir \;)
sleep 240: sleep
): end of subshell.
Add -maxdepth 1 as argument to find as you see fit for your requirements.
Nevermind, I'm dumb. there's a uniq command. Duh. New working script is: find ./ -maxdepth 1 -type f | sed -e 's/.[a-zA-Z]*$//' | uniq | while read line; do mv "$line*" ~/target_dir/; echo "$line"; sleep 240; done
EDIT: Forgot close tag on code and a backslash.
i have bunch of dirs , say **a, b, c0, d, Z , foo, ** and so on.
I want to remove all the directories except dirs foo, foo2, a and b
can anyone provide me the syntax to do this shell?
Thanks
UPDATE
I just want to say Thank you to all of you for your responses!
echo `ls -1 -d */ | egrep -v '^(foo|foo2|a|b)/$'`
If you are satisfied with the output, replace echo with rmdir (or rm -r, if the directories still contain data).
Probably the easiest way;
mkdir ../tempdir
mv foo foo2 a b ../tempdir
rm *
mv ../tempdir/* .
rmdir ../tempdir
Please note that this deletes also all files, not just directories.
You can use find on a complicated command line, but perhaps the simplest and, more importantly, safest way is to create a file that lists all of the directories you want to remove. Then use the file as input to rm like this:
find . -maxdepth 1 -type d > dirs_to_remove
Now edit the file and take out any directories you want to keep, then use rm:
rm -ir $(<edited_dirs_to_remove)
Note the -i argument. It's optional and forces rm to ask you before deleting each file.
Also note the $(<filename) syntax, which is specific to bash and is equivalent to, but cheaper than $(cat filename).
One of the more powerful ways to do this sort of trick is using find + grep + xargs:
DONT_REMOVE='a|b|c0|d|Z|foo'
find . -type d -print | egrep -v "^\.$DONT_REMOVE\$" | xargs rm -r
The only trick here is making sure the pattern matches only those you don't want to remove.
The above pattern only matches files in the current directory. You can make it more or less
permissive, e.g:
IF_PATH_IS_IMMEDIATE_SUBDIR="^\./($DONT_REMOVE)$"
IF_PATH_ENDS_IN="/($DONT_REMOVE)$"
IF_PATH_CONTAINS="/($DONT_REMOVE)(/.*)?$"
Then pass one of these in your egrep, e.g:
find . -type d -print | egrep -v "$IF_PATH_ENDS_IN" | xargs rm -r
To invert the choice (ie. delete all those items) just remove the -v from the egrep
one way
find . -maxdepth 1 -type d \( ! -name "bar" -a ! -name "foo" -a ! -name "a" -a ! -name "b" \) -delete # to remove files as well, remove -type d
OR try using extglob
shopt -s extglob
rm -rf !(foo|bar|a|b)/ # to delete files as well, remove the last "/"
And yes, this assume you don't want the rest of the directories in your directory except the 4 directories you want.