How to rename a lot of files - bash

I have directory of files called:
foo--ext1
foo--ext2
foo--ext3
...
I would like to rename them to:
foo-bar-ext1
foo-bar-ext2
foo-bar-ext3
....
How can I do this renaming in bash?
I have attempted to understand mywiki.wooledge.org/BashFAQ/030 but I can't work what should go in place of ${f%.foo}.bar"; in the first example.
So I have started with:
for f in foo--*; do echo mv -- "$f"
but what do I put next?

There are several ways to approach this.
I recommend bookmarking this page and referencing it often.
I would use this:
$: for f in foo--*; do mv "$f" "${f//foo--/foo-bar-}"; done
This uses string substitution in the current filename to construct the new name, replacing foo-- with foo-bar-.
Note the // in the replacement. This will replace every occurrence of foo-- with foo-bar- in each filename.
$: ls -l foo-*
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext1
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext2
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:32 foo-bar-ext3
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:33 foo-bar-foo-bar-etx4
Remove one of the leading slashes to make it only handle the first occurrence -
$: for f in foo--*; do mv "$f" "${f/foo--/foo-bar-}"; done
$: ls -l foo-*
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext1
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext2
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-ext3
-rw-r--r-- 1 paul 1049089 0 Jul 26 14:34 foo-bar-foo--ext4

Another simple method avoiding the loop is to use rename (from the util-linux package). There all that is needed is:
rename "foo-" "foo-bar" foo--*
You can check what will be done before actually doing the rename with the -n (no-act) and -v (verbose) options. For your example files, that would be:
$ rename -nv "foo-" "foo-bar" foo--*
`foo--ext1' -> `foo-bar-ext1'
`foo--ext2' -> `foo-bar-ext2'
`foo--ext3' -> `foo-bar-ext3'
There are two versions of rename that you will find provided in Linux distributions, the rename above from the util-linux package and perl-rename, which some Linux distros use instead which will also install as rename. Both are capable of handling the rename, but the options will be different. Check which you have with rename --version before use.

Related

Creating an alias in bash that calls a script and an additional command

I was toying with the idea of creating an alias that would allow me to list the contents of a target subdirectory, without changing to that directory.
I have successfully been able to create both an alias and a script to change directory and display contents. I call the script leap and it is simply:
#!/bin/bash
PATH=/bin:/usr/bin:.
# script to change to a directory, then display current contents
cd $1 && ls -l -a
The alias I use to trigger leap is defined: alias lp='. ~/scripts/leap'
My hope was I could simply create an alias named pk (pk is for peek) and concatenate my leap script and a standard bash command using &&. I could not be more wrong.
Current Directory
For reference, here is the contents of my current home directory (my user id has been replaced with $$$$):
drwxr-xr-x 3 $$$$$$ 46374 23 Aug 30 11:40 Fall_2019
drwxr-xr-x 5 $$$$$$ 46374 66 Aug 28 09:01 PAST_COURSES
drwxr-xr-x 3 $$$$$$ 46374 22 Aug 30 12:03 repos
drwxr-xr-x 3 $$$$$$ students 117 Aug 31 09:06 scripts
The Attempt(s)
Using alias pk='lp $1 && cd ..'
Entering [$$$$#host ~]$ pk PAST_COURSES results in:
-rw------- 1 $$$$$$ students 1766 Feb 28 2018 ~
drwx------ 10 $$$$$$ students 4096 Aug 31 09:06 .
drwx--x--x 1232 root root 28672 Aug 30 16:03 ..
-rw------- 1 $$$$$$ students 11368 Aug 30 12:20 .bash_history
-rw------- 1 $$$$$$ students 18 Aug 21 2017 .bash_logout
-rw------- 1 $$$$$$ students 180 Mar 8 2018 .bash_profile
-rw------- 1 $$$$$$ students 526 Aug 30 11:19 .bashrc
-rw------- 1 $$$$$$ students 266 Aug 21 2017 .cshrc
drwxr-xr-x 3 $$$$$$ 46374 23 Aug 30 11:40 Fall_2019
drwxr-xr-x 8 $$$$$$ students 155 Aug 30 12:14 .git
-rw-r--r-- 1 $$$$$$ students 87 Apr 11 2018 .gitconfig
-rw-r--r-- 1 $$$$$$ students 12288 Jan 29 2018 .hello.swp
-rw------- 1 $$$$$$ students 172 Aug 21 2017 .kshrc
-rw------- 1 $$$$$$ students 189 Mar 13 2018 .lesshst
-rw-r--r-- 1 $$$$$$ students 20480 Jan 29 2018 .ls.swn
drwxr-xr-x 5 $$$$$$ 46374 66 Aug 28 09:01 PAST_COURSES
drwxr----- 3 $$$$$$ 46374 18 Aug 30 12:16 .pki
drwxr-xr-x 3 $$$$$$ 46374 22 Aug 30 12:03 repos
drwxr-xr-x 3 $$$$$$ students 117 Aug 31 09:06 scripts
drwx------ 3 $$$$$$ students 103 Aug 30 11:12 .ssh
-rw------- 1 $$$$$$ students 12288 Sep 6 2017 .swp
drwxr-xr-x 2 $$$$$$ 46374 23 Aug 31 09:06 .vim
-rw-r--r-- 1 $$$$$$ students 8129 Aug 31 09:06 .viminfo
-rw-r--r-- 1 $$$$$$ students 142 Feb 14 2018 .vimrc
[$$$$#host home]$
As you can see, this displays the current directory ( ~ ) rather than switching to PAST_COURSES and displaying it. Additionally, the alias jumps up one directory above the current directory ( ~ ) rather than returning to it from PAST_COURSES.
Incidentally, I also get this exact result when I try using the following aliases:
pk='. ~/scripts/leap $1 && cd ..'
(using the script for leap rather than the alias)
pk='cd $1 && ls -l -a && cd ..'
(using the exact code inside leap )
Findings
In my tinkering I have noticed a few things:
First, if I simply type $$$$#host ~]$ ~/scripts/leap *[dir-name]* I actually get EXACTLY what I want - a look into a directory without changing to it. All by omitting the leading ., which boggles me.
Second, I can fix the current pk alias by changing the trailing cd .. to cd $(pwd), though it will not display the target directory instead of the current one.
At this point, I'd like a little help - not just in a script or alias that will do the job. An explanation that explains this behavior that I'm seeing would be marvelous.
Don't use . (the source builtin)
alias pk='. ~/scripts/leap $1 && cd ..' (using the script for leap rather than the alias)
is not equivalent to
alias pk='cd $1 && ls -l -a && cd ..'
In the first one, the . builtin (also known as source) is used, while in the second it is not. . doesn't just execute a command, it executes it in the current shell context. From the documentation:
. (a period)
. filename [arguments]
Read and execute commands from the filename argument in the current shell context.
That means anything the script does effects your current shell context. If the script changes directories, so does your current context.
If, on the other hand, the first version omitted the . like this:
alias pk='~/scripts/leap $1 && cd ..'
then the contents of the leap script would run in it's own bash context, but your current context would move up one directory (since the cd .. isn't inside the leap script).
Additional Recommendation on Functions vs Aliases
You could implement pk using a function like this:
pk() {
pushd $1
if [[ $? == 0 ]]; then
# Successfully changed directories.
# Run command
$2
# Return from the pre-pushd directory.
popd
fi
}
From the Bash Manual | 6.6 Aliases:
For almost every purpose, shell functions are preferred over aliases.
Example Alias
alias foo="echo bar"
Equivalent Function
foo() {
echo bar
}

Redirection to a variable has weird behaviour

I have this small script:
#!/bin/bash
for file in "$(ls | grep -v $0)";do
cat $file > "${file}-test"
done
On this directory:
total 40
-rwxr-xr-x 1 root root 783 Dec 11 09:19 appendToLog.sh
-rwxr-xr-x 1 root root 3995 Dec 11 13:22 con2dd.py
-rwxr-xr-x 1 root root 362 Dec 11 13:26 dd.py
-rw-r--r-- 1 root root 566 Dec 11 13:26 dd.pyc
-rw-r--r-- 1 root root 18558 Dec 25 11:24 moshe.log
-rw------- 1 root root 0 Dec 11 09:20 nohup.out
-rwxr-xr-x 1 root root 88 Dec 25 11:28 task.sh
-rwxr-xr-x 1 root root 560 Dec 11 10:33 test.py
Nevermind that I can achieve that with cp, I want to understand why this exactly is producing this file:
-rw-r--r-- 1 root root 24912 Dec 25 11:28 appendToLog.sh?con2dd.py?dd.py?dd.pyc?moshe.log?nohup.out?task.sh?test.py-test
And nothing else.
The problem is parsing output of ls is just wrong (see Why you shouldn't parse the output of ls(1), filenames in unix can have almost any special characters including whitespace, newlines, commas, pipe symbols. Its because you've quoted the output of ls in one construct, you have a single list of all the files concatenated as one string in the value of "${file}-test" which is quite not what you wanted to do.
Also notice how ls sometimes garbles your filename data (in our case, it turned the \n character in between the words into a ? question mark (could indicate a character that cannot be displayed).
Just use the glob expansion in bash to list the files and do actions on them.
for f in *; do
[[ -e $f ]] || continue
...
done
That said, You could probably have some non-printable characters on end of lines (eg. CRLF imported rom Windows)
Run cat -A scriptname it'll show you all characters in your script. Then, you can convert to unix-like format running dos2unix scriptname.

Why the shell doesn't write the good hour and give me the year?

I'm a little bit confused, i changed my time on one file with the shell command :
touch -t = touch -t 201606012135 trial01
But after the ls -lt, I got this :
-rw-r--r-- 1 CharleyD staff 87 1 jun 2016 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
Why the shell doesn't write the hours like the "Hybrid_proj" directory for the "trial01" ? The trial01 file have the hour : 21:35, so itsn't empty.
Indeed, I search to get this in output :
-rw-r--r-- 1 CharleyD staff 87 1 jun 21:35 trial01
drwxr-xr-x 15 CharleyD staff 510 3 apr 12:57 Hybrid_proj
How I can do this ?
Thx a lot buddies to enlighten my way ! ;)
If a file is not from the current year, ls defaults to showing the year instead of the time. The time is still correctly set, just formatted differently.
To always show the full time, with GNU ls, you can use ls --time-style=long-iso -l:
$ ls --time-style=full-iso -l
total 0
-rw-r--r-- 1 user user 0 2017-04-04 13:20 newfile
-rw-r--r-- 1 user user 0 2016-04-03 12:34 oldfile
With BusyBox ls, you can use -e:
$ busybox ls -e
total 0
-rw-r--r-- 1 user user 0 Tue Apr 4 13:20:42 2017 newfile
-rw-r--r-- 1 user user 0 Sun Apr 3 12:34:00 2016 oldfile
With macOS ls, you can use -lT:
$ ls -lT
total 0
-rw-r--r-- 1 user group 0 Apr 4 13:19:35 2017 myfile
-rw-r--r-- 1 user group 0 Apr 3 12:34:00 2016 oldfile
In each case, you get a long timestamp with the same format for older and newer files.
Use the -T option if your ls supports it.

display a line every two line (osX) zsh

I'd to display every two line, a line from a file. I've seen the sed -n 'f~d' awk and perl method. But the sed one doesn't work on osX (As I understood) and the two others are are interpreted languages which i can't use.
Can you help me ?
Here's an exemple :
output before :
-rw-r--r-- 1 mfassi-f 2013 22 Jul 17 12:36 test.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test1.sh
-rw-r--r-- 1 mfassi-f 2013 22 Jul 17 12:36 test2.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test3.sh
-rw-r--r-- 1 mfassi-f 2013 22 Jul 17 12:36 test4.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test5.sh
-rw-r--r-- 1 mfassi-f 2013 22 Jul 17 12:36 test6.sh
output after :
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test1.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test3.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test3.sh
-rw-r--r-- 1 mfassi-f 2013 29 Jul 17 12:30 test5.sh
Here are two answers. One for a file, and one for command-line input.
['cause the question's changed ever so slightly, but these two seemed too similar to put as independent answers].
You can use zsh, ls, cut and paste to do this in a for loop. It's not the cleanest solution, but it does work (surprisingly).
for file in `ls -1 | paste - - | cut -f 1`
do
ls -l -d $file
done
We take the output of ls -1, then extract every second filename. (The way ls chooses to sort the files will have an impact here). Then, we do ls -l -d on each of these files. -d is necessary to stop ls from showing us the contents of $file, if $file is a directory. (Not sure if this is OS X specific, or if that's default POSIX ls behaviour).
Second answer: display every second line from a file.
If you're after a mostly zsh solution, you could do something like the following:
$ jot 8 0 7 >> sample.txt # Generate some numbers.
$ count=0 # Storage variable
$ for i in `cat sample.txt`
do
if [ $(( $count % 2 )) -eq 0 ] ; then
echo $i
fi
count=`expr $count + 1`
done
This displays every second line.
Notes:
- This leaves a variable count in your session afterwards (it's not clean).
- This fails badly if sample.txt does not contain a single word per line.
- I'm almost sure that the modulus comparison I do isn't the most efficient: I grabbed it from here.
- I say it's mostly zsh because it does rely on cat, but I'm not sure how to avoid that.
The OS X version of sed is frustrating. Using sed -n '0~2p' <filename> doesn't work because, in the BSD sed, -n does something different:
-n
By default, each line of input is echoed to the standard output after all of the commands have been applied to it. The -n option suppresses this behavior.
I'd highly recommend installing GNU sed, which can be done using Homebrew:
brew install gnu-sed
And then you can use:
gsed -n '0~2p' filename # Display the 2nd, 4th etc
gsed -n '1~2p' filename # Display the 1st, 3rd etc.

Bash/Shell scripting

What does the command cp $1/. $2 do? I know cp is used for copying from source(stored in variable $1) to destination(stored in variable $2). I am just confused with the /. used along with the variable. Can someone please help me understand this?
The command:
$ cp -R $1/. $2
copies contents of directory pointed by $1 to the directory $2.
Without -R switch this command would fail both when $1 is a file or directory.
In general, . points to the current directory. You can see that by comparing inode's shown by ls:
$ mkdir test
$ ls -ali
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 .
771046 drwxrwxrwt 21 root wheel 714 23 mar 12:30 ..
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 test
$ cd test
$ ls -ali
9525312 drwxr-xr-x 2 IU wheel 68 23 mar 12:31 .
9525121 drwxr-xr-x 3 IU wheel 102 23 mar 12:31 ..
Note that inode 9525312 points to test when viewed from the parent directory, and points to . when viewed from inside the test directory.

Resources