I've got a file
sandeep_mems_SJ_23102003.txt which needs to be renamed sj_new_members_SJ_23102003.txt
I'll be getting these files daily so its vital that anything after _SJ remain the same.
So far I've got the following:-
for each in `/bin/ls -1`;do
sed -i 's/sandeep_mems_SJ/sj_new_members/g' $each ;
done
sed would help you if you were changing the contents of files. For renaming the file itself, you could do:
for each in *;do
mv $each sj_new_members_${each##sandeep_mems_SJ}
done
I used * rather than /bin/ls because it avoids spawning an extra process and uses Bash's built in matching (globbing) mechanism.
Each filename is assigned to $each.
mv renames $each to sj_new_members_ followed by the substring of $each that you want, using Bash's substring mechanism. More details on how to use Bash substrings are here:
http://tldp.org/LDP/abs/html/string-manipulation.html
Also, here's an alternative that uses the cut command, which splits along a specified character delimiter, in this case _. I don't like it as much because it spawns a new process, but it works. View the cut man page for more details. Note that $(command) is equalent to using backticks -- it runs a command in a subshell.
for each in *;do
mv $each sj_new_members_$(cut -d '_' -f 3- <<< $each)
done
for each in `/bin/ls -1`;do
mv $each sj_new_members_SJ${each##*SJ}
done
The ##*SJ is syntax for parameter expansion for removing everything up to the last SJ. Haven't tested the whole thing but it should work.
You can use rename utility:
rename 's/sandeep.*?_(\d+\.txt)$/sj_new_members_$1/' sandeep*txt
I tried to replicate your function as much as possible, so here's a solution that implements sed:
for each in *; do
new=$(echo "$each" | sed 's/.*_SJ/sj_new_members_SJ_/')
mv $each $new
done
I don't believe you actually need the ls -1 command, as sed will change the filenames of those files that contain the requirements stated above.
In essence, what my command does is save the new file name in a variable, new, and then mv renames it to the filename saved in the variable.
Related
Trying to remove a string that is located after the file name extension, on multiple files at once. I do not know where the files will be, just that they will reside in a subfolder of the one I am in.
Need to remove the last string, everything after the file extension. File name is:
something-unknown.js?ver=12234.... (last bit is unknown too)
This one (below) I found in this thread:
for nam in *sqlite3_done
do
newname=${nam%_done}
mv $nam $newname
done
I know that I have to use % to remove the bit from the end, but how do I use wildcards in the last bit, when I already have it as the "for any file" selector?
Have tried with a modifies bit of the above:
for nam in *.js*
do
newname=${ nam .js% } // removing all after .js
mv $nam $newname
done
I´m in MacOS Yosemite, got bash shell and sed. Know of rename and sed, but I´ve seen only topics with specific strings, no wildcards for this issue except these:
How to rename files using wildcard in bash?
https://unix.stackexchange.com/questions/227640/rename-first-part-of-multiple-files-with-mv
I think this is what you are looking for in terms of parameter substitution:
$ ls -C1
first-unknown.js?ver=111
second-unknown.js?ver=222
third-unknown.js?ver=333
$ for f in *.js\?ver=*; do echo ${f%\?*}; done
first-unknown.js
second-unknown.js
third-unknown.js
Note that we escape the ? as \? to say that we want to match the literal question mark, distinguishing it from the special glob symbol that matches any single character.
Renaming the files would then be something like:
$ for f in *.js\?ver=*; do echo "mv $f ${f%\?*}"; done
mv first-unknown.js?ver=111 first-unknown.js
mv second-unknown.js?ver=222 second-unknown.js
mv third-unknown.js?ver=333 third-unknown.js
Personally I like to output the commands, save it to a file, verify it's what I want, and then execute the file as a shell script.
If it needs to be fully automated you can remove the echo and do the mv directly.
for x in $(find . -type f -name '*.js*');do mv $x $(echo $x | sed 's/\.js.*/.js/'); done
I can remove file extensions if I know the extensions, for example to remove .txt from files:
foreach file (`find . -type f`)
mv $file `basename $file .txt`
end
However if I don't know what kind of file extension to begin with, how would I do this?
I tried:
foreach file (`find . -type f`)
mv $file `basename $file .*`
end
but it wouldn't work.
What shell is this? At least in bash you can do:
find . -type f | while read -r; do
mv -- "$REPLY" "${REPLY%.*}"
done
(The usual caveats apply: This doesn't handle files whose name contains newlines.)
You can use sed to compute base file name.
foreach file (`find . -type f`)
mv $file `echo $file | sed -e 's/^\(.*\)\.[^.]\+$/\1/'`
end
Be cautious: The command you seek to run could cause loss of data!
If you don't think your file names contain newlines or double quotes, then you could use:
find . -type f -name '?*.*' |
sed 's/\(.*\)\.[^.]*$/mv "&" "\1"/' |
sh
This generates your list of files (making sure that the names contain at least one character plus a .), runs each file name through the sed script to convert it into an mv command by effectively removing the material from the last . onwards, and then running the stream of commands through a shell.
Clearly, you test this first by omitting the | sh part. Consider running it with | sh -x to get a trace of what the shell's doing. Consider making sure you capture the output of the shell, standard output and standard error, into a log file so you've got a record of the damage that occurred.
Do make sure you've got a backup of the original set of files before you start playing with this. It need only be a tar file stored in a different part of the directory hierarchy, and you can remove it as soon as you're happy with the results.
You can choose any shell; this doesn't rely on any shell constructs except pipes and single quotes and double quotes (pretty much common to all shells), and the sed script is version neutral too.
Note that if you have files xyz.c and xyz.h before you run this, you'll only have a file xyz afterwards (and what it contains depends on the order in which the files are processed, which needn't be alphabetic order).
If you think your file names might contain double quotes (but not single quotes), you can play with the changing the quotes in the sed script. If you might have to deal with both, you need a more complex sed script. If you need to deal with newlines in file names, then it is time to (a) tell your user(s) to stop being silly and (b) fix the names so they don't contain newlines. Then you can use the script above. If that isn't feasible, you have to work a lot harder to get the job done accurately — you probably need to make sure you've got a find that supports -print0, a sed that supports -z and an xargs that supports -0 (installing the most recent GNU versions if you don't already have the right support in place).
It's very simple:
$ set filename=/home/foo/bar.dat
$ echo ${filename:r}
/home/foo/bar
See more in man tcsh, in "History substitution":
r
Remove a filename extension '.xxx', leaving the root name.
I have a bunch of files (more than 1000) on this like the followings
$ ls
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-dev.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-dev.lex
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-train.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm-train.lex
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lc
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lex
org.allenai.ari.solvers.termselector.ExpandedLearner.lc
org.allenai.ari.solvers.termselector.ExpandedLearner.lex
org.allenai.ari.solvers.termselector.ExpandedLearnerSVM.lc
org.allenai.ari.solvers.termselector.ExpandedLearnerSVM.lex
....
I have to rename these files files by adding a learners right before the capitalized name. For example
org.allenai.ari.solvers.termselector.BaselineLearnersurfaceForm.lex
would change to
org.allenai.ari.solvers.termselector.learners.BaselineLearnersurfaceForm.lex
and this one
org.allenai.ari.solvers.termselector.ExpandedLearner.lc
would change to
org.allenai.ari.solvers.termselector.learners.ExpandedLearner.lc
Any ideas how to do this automatically?
for f in org.*; do
echo mv "$f" "$( sed 's/\.\([A-Z]\)/.learner.\1/' <<< "$f" )"
done
This short loop outputs an mv command that renames the files in the manner that you wanted. Run it as-is first, and when you are certain it's doing what you want, remove the echo and run again.
The sed bit in the middle takes a filename ($f, via a here-string, so this requires bash) and replaces the first occurrence of a capital letter after a dot with .learner. followed by that same capital letter.
There is a tool called perl-rename, sometimes rename. Not to be confused with rename from util-linux.
It's very good for tasks like this as it takes a perl expression and renames accordingly:
perl-rename 's/(?=\.[A-Z])/.learners/' *
You can play with the regex online
Alternative you can a for loop and $BASH_REMATCH:
for file in *; do
[ -e "$file" ] || continue
[[ "$file" =~ ^([^A-Z]*)(.*)$ ]]
mv -- "$file" "${BASH_REMATCH[1]}learners.${BASH_REMATCH[2]}"
done
A very simple approach (useful if you only need to do this one time) is to ls >dummy them into a text file dummy, and then use find/replace in a text editor to make lines of the form mv xxx.yyy xxx.learners.yyy. Then you can simple execute the resulting file with ./dummy.
The exact find/replace commands depend on the text editor you use, but something like
replace org. with mv org.. That gets you the mv in the beginning.
replace mv org.allenai.ari.solvers.termselector.$1 with mv org.allenai.ari.solvers.termselector.$1 org.allenai.ari.solvers.termselector.learner.$1 to duplicate the filename and insert the learner.
There is also syntax with a for, which can do it probably in one line, (long) but I cannot explain it - try help for if you want to learn about it.
I need to rename 45 files, and I don't want to do it one by one. These are the file names:
chr10.fasta chr13_random.fasta chr17.fasta chr1.fasta chr22_random.fasta chr4_random.fasta chr7_random.fasta chrX.fasta
chr10_random.fasta chr14.fasta chr17_random.fasta chr1_random.fasta chr2.fasta chr5.fasta chr8.fasta chrX_random.fasta
chr11.fasta chr15.fasta chr18.fasta chr20.fasta chr2_random.fasta chr5_random.fasta chr8_random.fasta chrY.fasta
chr11_random.fasta chr15_random.fasta chr18_random.fasta chr21.fasta chr3.fasta chr6.fasta chr9.fasta
chr12.fasta chr16.fasta chr19.fasta chr21_random.fasta chr3_random.fasta chr6_random.fasta chr9_random.fasta
chr13.fasta chr16_random.fasta chr19_random.fasta chr22.fasta chr4.fasta chr7.fasta chrM.fasta
I need to change the extension ".fasta" to ".fa". I'm trying to write a bash script to do it:
for i in $(ls chr*)
do
NEWNAME = `echo $i | sed 's/sta//g'`
mv $i $NEWNAME
done
But it doesn't work. Can you tell me why, or give another quick solution?
Thanks!
Several mistakes here:
NEWNAME = should be without space. Here bash is looking for a command named NEWNAME and that fails.
you parse the output of ls. this is bad if you had files with spaces. Bash can build itself a list of files with the glob operator *.
You don't escape "$i" and "$NEWNAME". If any of them contains a space it makes two arguments for mv.
If a file name begins with a dash mv will believe it is a switch. Use -- to stop argument processing.
Try:
for i in chr*
do
mv -- "$i" "${i/%.fasta/.fa}"
done
or
for i in chr*
do
NEWNAME="${i/%.fasta/.fa}"
mv -- "$i" "$NEWNAME"
done
The "%{var/%pat/replacement}" looks for pat only at the end of the variable and replaces it with replacement.
for f in chr*.fasta; do mv "$f" "${f/%.fasta/.fa}"; done
If you have the rename command, you can do:
rename .fasta .fa chr*.fasta
How can Bash rename a series of packages to remove their version numbers? I've been toying around with both expr and %%, to no avail.
Examples:
Xft2-2.1.13.pkg becomes Xft2.pkg
jasper-1.900.1.pkg becomes jasper.pkg
xorg-libXrandr-1.2.3.pkg becomes xorg-libXrandr.pkg
You could use bash's parameter expansion feature
for i in ./*.pkg ; do mv "$i" "${i/-[0-9.]*.pkg/.pkg}" ; done
Quotes are needed for filenames with spaces.
If all files are in the same directory the sequence
ls |
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' |
sh
will do your job. The sed command will create a sequence of mv commands, which you can then pipe into the shell. It's best to first run the pipeline without the trailing | sh so as to verify that the command does what you want.
To recurse through multiple directories use something like
find . -type f |
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' |
sh
Note that in sed the regular expression grouping sequence is brackets preceded by a backslash, \( and \), rather than single brackets ( and ).
I'll do something like this:
for file in *.pkg ; do
mv $file $(echo $file | rev | cut -f2- -d- | rev).pkg
done
supposed all your file are in the current directory. If not, try to use find as advised above by Javier.
EDIT: Also, this version don't use any bash-specific features, as others above, which leads you to more portability.
We can assume sed is available on any *nix, but we can't be sure
it'll support sed -n to generate mv commands. (NOTE: Only GNU sed does this.)
Even so, bash builtins and sed, we can quickly whip up a shell function to do this.
sedrename() {
if [ $# -gt 1 ]; then
sed_pattern=$1
shift
for file in $(ls $#); do
mv -v "$file" "$(sed $sed_pattern <<< $file)"
done
else
echo "usage: $0 sed_pattern files..."
fi
}
Usage
sedrename 's|\(.*\)\(-[0-9.]*\.pkg\)|\1\2|' *.pkg
before:
./Xft2-2.1.13.pkg
./jasper-1.900.1.pkg
./xorg-libXrandr-1.2.3.pkg
after:
./Xft2.pkg
./jasper.pkg
./xorg-libXrandr.pkg
Creating target folders:
Since mv doesn't automatically create target folders we can't using
our initial version of sedrename.
It's a fairly small change, so it'd be nice to include that feature:
We'll need a utility function, abspath (or absolute path) since bash
doesn't have this build in.
abspath () { case "$1" in
/*)printf "%s\n" "$1";;
*)printf "%s\n" "$PWD/$1";;
esac; }
Once we have that we can generate the target folder(s) for a
sed/rename pattern which includes new folder structure.
This will ensure we know the names of our target folders. When we
rename we'll need to use it on the target file name.
# generate the rename target
target="$(sed $sed_pattern <<< $file)"
# Use absolute path of the rename target to make target folder structure
mkdir -p "$(dirname $(abspath $target))"
# finally move the file to the target name/folders
mv -v "$file" "$target"
Here's the full folder aware script...
sedrename() {
if [ $# -gt 1 ]; then
sed_pattern=$1
shift
for file in $(ls $#); do
target="$(sed $sed_pattern <<< $file)"
mkdir -p "$(dirname $(abspath $target))"
mv -v "$file" "$target"
done
else
echo "usage: $0 sed_pattern files..."
fi
}
Of course, it still works when we don't have specific target folders
too.
If we wanted to put all the songs into a folder, ./Beethoven/ we can do this:
Usage
sedrename 's|Beethoven - |Beethoven/|g' *.mp3
before:
./Beethoven - Fur Elise.mp3
./Beethoven - Moonlight Sonata.mp3
./Beethoven - Ode to Joy.mp3
./Beethoven - Rage Over the Lost Penny.mp3
after:
./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3
Bonus round...
Using this script to move files from folders into a single folder:
Assuming we wanted to gather up all the files matched, and place them
in the current folder, we can do it:
sedrename 's|.*/||' **/*.mp3
before:
./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3
after:
./Beethoven/ # (now empty)
./Fur Elise.mp3
./Moonlight Sonata.mp3
./Ode to Joy.mp3
./Rage Over the Lost Penny.mp3
Note on sed regex patterns
Regular sed pattern rules apply in this script, these patterns aren't
PCRE (Perl Compatible Regular Expressions). You could have sed
extended regular expression syntax, using either sed -r or sed -E
depending on your platform.
See the POSIX compliant man re_format for a complete description of
sed basic and extended regexp patterns.
Here is a POSIX near-equivalent of the currently accepted answer. This trades the Bash-only ${variable/substring/replacement} parameter expansion for one which is available in any Bourne-compatible shell.
for i in ./*.pkg; do
mv "$i" "${i%-[0-9.]*.pkg}.pkg"
done
The parameter expansion ${variable%pattern} produces the value of variable with any suffix which matches pattern removed. (There is also ${variable#pattern} to remove a prefix.)
I kept the subpattern -[0-9.]* from the accepted answer although it is perhaps misleading. It's not a regular expression, but a glob pattern; so it doesn't mean "a dash followed by zero or more numbers or dots". Instead, it means "a dash, followed by a number or a dot, followed by anything". The "anything" will be the shortest possible match, not the longest. (Bash offers ## and %% for trimming the longest possible prefix or suffix, rather than the shortest.)
I find that rename is a much more straightforward tool to use for this sort of thing. I found it on Homebrew for OSX
For your example I would do:
rename 's/\d*?\.\d*?\.\d*?//' *.pkg
The 's' means substitute. The form is s/searchPattern/replacement/ files_to_apply. You need to use regex for this which takes a little study but it's well worth the effort.
better use sed for this, something like:
find . -type f -name "*.pkg" |
sed -e 's/((.*)-[0-9.]*\.pkg)/\1 \2.pkg/g' |
while read nameA nameB; do
mv $nameA $nameB;
done
figuring up the regular expression is left as an exercise (as is dealing with filenames that include spaces)
This seems to work assuming that
everything ends with $pkg
your version #'s always start with a "-"
strip off the .pkg, then strip off -..
for x in $(ls); do echo $x $(echo $x | sed 's/\.pkg//g' | sed 's/-.*//g').pkg; done
I had multiple *.txt files to be renamed as .sql in same folder.
below worked for me:
for i in \`ls *.txt | awk -F "." '{print $1}'\` ;do mv $i.txt $i.sql; done
Thank you for this answers. I also had some sort of problem. Moving .nzb.queued files to .nzb files. It had spaces and other cruft in the filenames and this solved my problem:
find . -type f -name "*.nzb.queued" |
sed -ne "s/^\(\(.*\).nzb.queued\)$/mv -v \"\1\" \"\2.nzb\"/p" |
sh
It is based on the answer of Diomidis Spinellis.
The regex creates one group for the whole filename, and one group for the part before .nzb.queued and then creates a shell move command. With the strings quoted. This also avoids creating a loop in shell script because this is already done by sed.