How can Bash rename a series of packages to remove their version numbers? I've been toying around with both expr and %%, to no avail.
Examples:
Xft2-2.1.13.pkg becomes Xft2.pkg
jasper-1.900.1.pkg becomes jasper.pkg
xorg-libXrandr-1.2.3.pkg becomes xorg-libXrandr.pkg
You could use bash's parameter expansion feature
for i in ./*.pkg ; do mv "$i" "${i/-[0-9.]*.pkg/.pkg}" ; done
Quotes are needed for filenames with spaces.
If all files are in the same directory the sequence
ls |
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' |
sh
will do your job. The sed command will create a sequence of mv commands, which you can then pipe into the shell. It's best to first run the pipeline without the trailing | sh so as to verify that the command does what you want.
To recurse through multiple directories use something like
find . -type f |
sed -n 's/\(.*\)\(-[0-9.]*\.pkg\)/mv "\1\2" "\1.pkg"/p' |
sh
Note that in sed the regular expression grouping sequence is brackets preceded by a backslash, \( and \), rather than single brackets ( and ).
I'll do something like this:
for file in *.pkg ; do
mv $file $(echo $file | rev | cut -f2- -d- | rev).pkg
done
supposed all your file are in the current directory. If not, try to use find as advised above by Javier.
EDIT: Also, this version don't use any bash-specific features, as others above, which leads you to more portability.
We can assume sed is available on any *nix, but we can't be sure
it'll support sed -n to generate mv commands. (NOTE: Only GNU sed does this.)
Even so, bash builtins and sed, we can quickly whip up a shell function to do this.
sedrename() {
if [ $# -gt 1 ]; then
sed_pattern=$1
shift
for file in $(ls $#); do
mv -v "$file" "$(sed $sed_pattern <<< $file)"
done
else
echo "usage: $0 sed_pattern files..."
fi
}
Usage
sedrename 's|\(.*\)\(-[0-9.]*\.pkg\)|\1\2|' *.pkg
before:
./Xft2-2.1.13.pkg
./jasper-1.900.1.pkg
./xorg-libXrandr-1.2.3.pkg
after:
./Xft2.pkg
./jasper.pkg
./xorg-libXrandr.pkg
Creating target folders:
Since mv doesn't automatically create target folders we can't using
our initial version of sedrename.
It's a fairly small change, so it'd be nice to include that feature:
We'll need a utility function, abspath (or absolute path) since bash
doesn't have this build in.
abspath () { case "$1" in
/*)printf "%s\n" "$1";;
*)printf "%s\n" "$PWD/$1";;
esac; }
Once we have that we can generate the target folder(s) for a
sed/rename pattern which includes new folder structure.
This will ensure we know the names of our target folders. When we
rename we'll need to use it on the target file name.
# generate the rename target
target="$(sed $sed_pattern <<< $file)"
# Use absolute path of the rename target to make target folder structure
mkdir -p "$(dirname $(abspath $target))"
# finally move the file to the target name/folders
mv -v "$file" "$target"
Here's the full folder aware script...
sedrename() {
if [ $# -gt 1 ]; then
sed_pattern=$1
shift
for file in $(ls $#); do
target="$(sed $sed_pattern <<< $file)"
mkdir -p "$(dirname $(abspath $target))"
mv -v "$file" "$target"
done
else
echo "usage: $0 sed_pattern files..."
fi
}
Of course, it still works when we don't have specific target folders
too.
If we wanted to put all the songs into a folder, ./Beethoven/ we can do this:
Usage
sedrename 's|Beethoven - |Beethoven/|g' *.mp3
before:
./Beethoven - Fur Elise.mp3
./Beethoven - Moonlight Sonata.mp3
./Beethoven - Ode to Joy.mp3
./Beethoven - Rage Over the Lost Penny.mp3
after:
./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3
Bonus round...
Using this script to move files from folders into a single folder:
Assuming we wanted to gather up all the files matched, and place them
in the current folder, we can do it:
sedrename 's|.*/||' **/*.mp3
before:
./Beethoven/Fur Elise.mp3
./Beethoven/Moonlight Sonata.mp3
./Beethoven/Ode to Joy.mp3
./Beethoven/Rage Over the Lost Penny.mp3
after:
./Beethoven/ # (now empty)
./Fur Elise.mp3
./Moonlight Sonata.mp3
./Ode to Joy.mp3
./Rage Over the Lost Penny.mp3
Note on sed regex patterns
Regular sed pattern rules apply in this script, these patterns aren't
PCRE (Perl Compatible Regular Expressions). You could have sed
extended regular expression syntax, using either sed -r or sed -E
depending on your platform.
See the POSIX compliant man re_format for a complete description of
sed basic and extended regexp patterns.
Here is a POSIX near-equivalent of the currently accepted answer. This trades the Bash-only ${variable/substring/replacement} parameter expansion for one which is available in any Bourne-compatible shell.
for i in ./*.pkg; do
mv "$i" "${i%-[0-9.]*.pkg}.pkg"
done
The parameter expansion ${variable%pattern} produces the value of variable with any suffix which matches pattern removed. (There is also ${variable#pattern} to remove a prefix.)
I kept the subpattern -[0-9.]* from the accepted answer although it is perhaps misleading. It's not a regular expression, but a glob pattern; so it doesn't mean "a dash followed by zero or more numbers or dots". Instead, it means "a dash, followed by a number or a dot, followed by anything". The "anything" will be the shortest possible match, not the longest. (Bash offers ## and %% for trimming the longest possible prefix or suffix, rather than the shortest.)
I find that rename is a much more straightforward tool to use for this sort of thing. I found it on Homebrew for OSX
For your example I would do:
rename 's/\d*?\.\d*?\.\d*?//' *.pkg
The 's' means substitute. The form is s/searchPattern/replacement/ files_to_apply. You need to use regex for this which takes a little study but it's well worth the effort.
better use sed for this, something like:
find . -type f -name "*.pkg" |
sed -e 's/((.*)-[0-9.]*\.pkg)/\1 \2.pkg/g' |
while read nameA nameB; do
mv $nameA $nameB;
done
figuring up the regular expression is left as an exercise (as is dealing with filenames that include spaces)
This seems to work assuming that
everything ends with $pkg
your version #'s always start with a "-"
strip off the .pkg, then strip off -..
for x in $(ls); do echo $x $(echo $x | sed 's/\.pkg//g' | sed 's/-.*//g').pkg; done
I had multiple *.txt files to be renamed as .sql in same folder.
below worked for me:
for i in \`ls *.txt | awk -F "." '{print $1}'\` ;do mv $i.txt $i.sql; done
Thank you for this answers. I also had some sort of problem. Moving .nzb.queued files to .nzb files. It had spaces and other cruft in the filenames and this solved my problem:
find . -type f -name "*.nzb.queued" |
sed -ne "s/^\(\(.*\).nzb.queued\)$/mv -v \"\1\" \"\2.nzb\"/p" |
sh
It is based on the answer of Diomidis Spinellis.
The regex creates one group for the whole filename, and one group for the part before .nzb.queued and then creates a shell move command. With the strings quoted. This also avoids creating a loop in shell script because this is already done by sed.
Related
Trying to remove a string that is located after the file name extension, on multiple files at once. I do not know where the files will be, just that they will reside in a subfolder of the one I am in.
Need to remove the last string, everything after the file extension. File name is:
something-unknown.js?ver=12234.... (last bit is unknown too)
This one (below) I found in this thread:
for nam in *sqlite3_done
do
newname=${nam%_done}
mv $nam $newname
done
I know that I have to use % to remove the bit from the end, but how do I use wildcards in the last bit, when I already have it as the "for any file" selector?
Have tried with a modifies bit of the above:
for nam in *.js*
do
newname=${ nam .js% } // removing all after .js
mv $nam $newname
done
I´m in MacOS Yosemite, got bash shell and sed. Know of rename and sed, but I´ve seen only topics with specific strings, no wildcards for this issue except these:
How to rename files using wildcard in bash?
https://unix.stackexchange.com/questions/227640/rename-first-part-of-multiple-files-with-mv
I think this is what you are looking for in terms of parameter substitution:
$ ls -C1
first-unknown.js?ver=111
second-unknown.js?ver=222
third-unknown.js?ver=333
$ for f in *.js\?ver=*; do echo ${f%\?*}; done
first-unknown.js
second-unknown.js
third-unknown.js
Note that we escape the ? as \? to say that we want to match the literal question mark, distinguishing it from the special glob symbol that matches any single character.
Renaming the files would then be something like:
$ for f in *.js\?ver=*; do echo "mv $f ${f%\?*}"; done
mv first-unknown.js?ver=111 first-unknown.js
mv second-unknown.js?ver=222 second-unknown.js
mv third-unknown.js?ver=333 third-unknown.js
Personally I like to output the commands, save it to a file, verify it's what I want, and then execute the file as a shell script.
If it needs to be fully automated you can remove the echo and do the mv directly.
for x in $(find . -type f -name '*.js*');do mv $x $(echo $x | sed 's/\.js.*/.js/'); done
On building apps with the Angular 2 CLI, I get outputs which are named, for instance:
inline.d41d8cd.bundle.js
main.6d2e2e89.bundle.js
etc.
What I'm looking to do is create a bash script to rename the files, replacing the digits between the first two . with some given generic string. Tried a few things, including sed, but I couldn't get them to work. Can anyone suggest a bash script to get this working?
In pure bash regEx using the =~ variable (supported from bash 3.0 onwards)
#!/bin/bash
string_to_replace_with="sample"
for file in *.js
do
[[ $file =~ \.([[:alnum:]]+).*$ ]] && string="${BASH_REMATCH[1]}"
mv -v "$file" "${file/$string/$string_to_replace_with}"
done
For your given input files, running the script
$ bash script.sh
inline.d41d8cd.bundle.js -> inline.sample.bundle.js
main.6d2e2e89.bundle.js -> main.sample.bundle.js
Short, powerfull and efficient:
Use this (perl) tool. And use Perl Regular Expression:
rename 's/\.\X{4,8}\./.myString./' *.js
or
rename 's/\.\X+\./.myString./' *.js
A pure-bash option:
shopt -s extglob # so *(...) will work
generic_string="foo" # or whatever else you want between the dots
for f in *.bundle.js ; do
mv -vi "$f" "${f/.*([^.])./.${generic_string}.}"
done
The key is the replacement ${f/.*([^.]./.${generic_string}.}. The pattern /.*([^.])./ matches the first occurrence of .<some text>., where <some text> does not include a dot ([^.]) (see the man page). The replacement .${generic_string}. replaces that with whatever generic string you want. Other than that, double-quote in case you have spaces, and there you are!
Edit Thanks to F. Hauri - added -vi to mv. -v = show what is being renamed; -i = prompt before overwrite (man page).
I've got a file
sandeep_mems_SJ_23102003.txt which needs to be renamed sj_new_members_SJ_23102003.txt
I'll be getting these files daily so its vital that anything after _SJ remain the same.
So far I've got the following:-
for each in `/bin/ls -1`;do
sed -i 's/sandeep_mems_SJ/sj_new_members/g' $each ;
done
sed would help you if you were changing the contents of files. For renaming the file itself, you could do:
for each in *;do
mv $each sj_new_members_${each##sandeep_mems_SJ}
done
I used * rather than /bin/ls because it avoids spawning an extra process and uses Bash's built in matching (globbing) mechanism.
Each filename is assigned to $each.
mv renames $each to sj_new_members_ followed by the substring of $each that you want, using Bash's substring mechanism. More details on how to use Bash substrings are here:
http://tldp.org/LDP/abs/html/string-manipulation.html
Also, here's an alternative that uses the cut command, which splits along a specified character delimiter, in this case _. I don't like it as much because it spawns a new process, but it works. View the cut man page for more details. Note that $(command) is equalent to using backticks -- it runs a command in a subshell.
for each in *;do
mv $each sj_new_members_$(cut -d '_' -f 3- <<< $each)
done
for each in `/bin/ls -1`;do
mv $each sj_new_members_SJ${each##*SJ}
done
The ##*SJ is syntax for parameter expansion for removing everything up to the last SJ. Haven't tested the whole thing but it should work.
You can use rename utility:
rename 's/sandeep.*?_(\d+\.txt)$/sj_new_members_$1/' sandeep*txt
I tried to replicate your function as much as possible, so here's a solution that implements sed:
for each in *; do
new=$(echo "$each" | sed 's/.*_SJ/sj_new_members_SJ_/')
mv $each $new
done
I don't believe you actually need the ls -1 command, as sed will change the filenames of those files that contain the requirements stated above.
In essence, what my command does is save the new file name in a variable, new, and then mv renames it to the filename saved in the variable.
I want to remove the numbers from the file names in one directory:
file names:
89_Ajohn_text_phones
3_jpegs_directory
..
What I would like to have
Ajohn_text_phones
jpegs_directory
I tried:
rename 's/^\([0-9]|[0-9][0-9]\)_//g' *
but I did not work.
There are two rename tools. The one you appear to try to use is based on Perl, and as such uses perl-style regular expressions. The escaping rules are a little different from sed; in particular, parentheses for grouping aren't escaped (escaped parentheses are literal parentheses). Use
rename 's/^([0-9]|[0-9][0-9])_//' *
or, somewhat more concisely,
rename 's/^[0-9]{1,2}_//' *
This rename is the default on Debian-based distributions such as Ubuntu.
The other rename tool is from util-linux, and it is a very simple tool that does not work with regexes and cannot handle this particular requirement. It is the default on, for example, Fedora-based distributions, and what's worse, those often don't even package the Perl rename. You can find the Perl rename here on CPAN and put it in /usr/local/bin if you want, but otherwise, your simplest option is probably a shell loop with mv and sed:
for f in *; do
# set dest to the target file name
# Note: using sed's extended regex syntax (-r) because it is nice.
# Much less escaping of metacharacters is needed. Note that
# sed on FreeBSD and Mac OS X uses -E instead of -r, so there
# you'd have to change that or use regular syntax, in which
# the regex would have to be written as ^[0-9]\{1,2\}_
dest="$(echo "$f" | sed -r 's/^[0-9]{1,2}_//')"
if [ "$f" = "$dest" ]; then
# $f did not match pattern; do nothing
true;
elif [ -e "$dest" ]; then
# avoid losing files.
echo "$dest already exists!"
else
mv "$f" "$dest"
fi
done
You could put this into a shell script, say rename.sh:
#!/bin/sh
transform="$1"
shift
for f in "$#"; do
dest="$(echo "$f" | sed -r "$transform")"
if [ "$f" = "$dest" ]; then
## do nothing
true;
elif [ -e "$dest" ]; then
echo "$dest already exists!"
else
mv "$f" "$dest"
fi
done
and call rename.sh 's/^[0-9]{1,2}_//' *.
One caveat remains: in
dest="$(echo "$f" | sed -r "$transform")"
there is a possibility that "$f" could be something that echo considers a command line option, such as -n or -e. I do not know a way to solve this portably (if you read this and know one, please leave a comment), but if you are in a position where tethering yourself to bash is not a problem, you can use bash's here strings instead:
dest="$(sed -r "$transform" <<< "$f")"
(and replace the #!/bin/sh shebang with #!/bin/bash). Such files are rare, though, so as timebombs go, this is not unlikely to be a dud.
#!/bin/bash
for f in *
do
mv "$f" "${f#*_}"
done
So i am writing a script which gets the substring from the input which is a path to a file (/path/to/file.ext) and if the directory (/path/to) does not exist it will run mkdir -p /path/to and then touch file.ext.
my question is this, how can i use cut to get the /path/to if we have a potentially unknown length of /'s
my script currently looks like this
INPUT=$0
SUBSTRING_PATH=`$INPUT | cut -d'/' -f 2`
if [! -d $SUBSTRING_PATH]; then
mkdir -p $SUBSTRING_PATH
fi
touch $INPUT
Instead of cut, use dirname and basename:
input=/path/to/foo
dir=$(dirname "$input")
file=$(basename "$input")
Now $DIR is /path/to and $FILE is foo.
dirname will also give you a valid directory for relative paths to the working directory (I mean that $(dirname file.txt) is .). This means, for example, that you can write "$dir/some/stuff/foo" without having to worry that you end up in a completely different directory tree (such as /some/stuff rather than ./some/stuff).
As #ruakh mentions in the comments, if you didn't have a directory but a string of tokens of which you wanted to discard the last (a line of a csv file, perhaps), one way to do it would be "${input%,*}", where the comma can be replaced by any delimiter. To my knowledge this is a bash extension. I only edit this in because a stray visitor in the future might have better luck seeing it here than in the comments; for your particular use case, dirname and basename are a better fit.