Lets say I've a directory /etc/php5/conf.d/, with the following hypothetical files in it:
mysql.ini
mysqli.ini
20-mysql.ini
20-mysqli.ini
20-pdo_mysql.ini
I would like to delete all these files except the last one (pdo), this is what I have at the moment:
for phpIni in mysql mysqli; do
if [[ -f /etc/php5/conf.d/$phpIni.ini ]]; then
rm /etc/php5/conf.d/$phpIni.ini
if [[ -f /etc/php5/conf.d/20-$phpIni.ini ]]; then
rm /etc/php5/conf.d/20-$phpIni.ini
fi
done
It works, but I can't help noticing that the above could be simplified with glob patterns, such as:
if [[ ! -z /etc/php5/conf.d/{,20-}mysql*ini ]]; then
rm /etc/php5/conf.d/{,20-}mysql*ini
fi
There's a problem though, if the any of the expansions doesn't exists, rm will complain about it:
$ if [[ ! -z /etc/php5/conf.d/{,20-}mysql*ini ]]; then rm /etc/php5/conf.d/{,20-}mysql*ini; fi
rm: cannot remove `/etc/php5/conf.d/20-mysql*ini': No such file or directory
How can I make rm only remove existing files? Or at least prevent it from throwing all these errors?
You can use rm -f, it will not complain if files don't exist: as seen in the man page,
"ignore nonexistent files, never prompt".
Adding a -f as an rm parameter will prevent that. On the flipside, though, you need to ensure your glob is solid, lest you delete too much.
Add shopt -s nullglob -- if there are no files matching the pattern, the null string is substituted instead of the pattern string.
To delete all except one, use
shopt -s extglob
rm !(*pdo*).ini
Related
I have these test files:
ABCD1234__12_Ab2_Hh_3P.mp4
ABCD1234__12_Ab2_Lw_3P.wmv
ABCD1234__12_Ab2_SSV.mov
EFGH56789__13_Mn1_SSV.avi
EFGH56789__13_Mn1_Ve1_3P.mp4
EFGH56789__13_Mn1_Ve2_3P.webm
I want to create a context service in Automator that makes directories based on the filename prefixes above like so:
ABCD1234__12_Ab2
EFGH56789__13_Mn1
...and move the files into those two accordingly. The only consistent variables in the names are underscores, so I was thinking I could delineate by those, preferably capturing the name before the fourth one.
I originally started with this very simple script:
for file in "$#"
do
mkdir "${file%.*}" && mv "$file" "${file%.*}"
done
Which makes a folder for every file and moves each file into its own folder.
I tried adding variables, various if/thens, etc. but to no avail (not a programmer by trade).
I also wrote another script to do it in a slightly different way, but with the same results to mess around with:
for folder in "$#"
do
cd "$1"
find . -type f -maxdepth 1 -exec bash -c 'mkdir -p "${0%.*}"' {} \; \
-exec bash -c 'mv "$0" "${0%.*}"' {} \;
done
I feel like there's something obvious I am missing.
Your script is splitting on dot, but you say you want to split on underscore. If the one you want to split on is the last one, the fix is trivial:
for file in "$#"
do
mkdir -p "${file%_*}" && mv "$file" "${file%_*}"
done
To get precisely the fourth, try
for file in "$#"
do
tail=${file#*_*_*_*_}
dir=${file%_"$tail"}
mkdir -p "$dir" && mv "$file" "$dir"
done
The addition of the -p option is a necessary bug fix if you want to use && here; mkdir without this option will fail if the directory already exists.
Perhaps see also the section about parameter expansions in the Bash Reference Manual which explains this syntax and its variations.
You could do something like this:
#!/bin/bash
shopt -s extglob
for file
do
if [[ $file =~ ^(([^_]*_){3}[^_]*) ]]
then
echo "${BASH_REMATCH[0]}"
else
echo "${file%.*}"
fi
done
In an attempt to rename the files in one directory with numbers at the front I made an error in my script so that this happened in the wrong directory. Therefore I now need to remove these numbers from the beginning of all of my filenames in a directory. These range from 1 to 3 digits. Examples of the filnames I am working with are:
706terrain_Slope1000m_Minimum_all_25PCs_bolt_all_25PCs_qq_bolt.png
680met_sfcWind_all_25PCs_bolt_number.txt
460greenness_NDVI_500m_min_all_25PCs_bolt_number.txt
I was thinking of using mv but I'm not really sure how to do it with varying numbers of digits at the beginning, so any advice would be appreciated!
A simple way in bash is making use of a regular expression test:
for file in *; do
[[ -f "${file}" ]] && [[ "${file}" =~ (^[0-9]+) ]] && mv ${file} ${file/${BASH_REMATCH[1]}}
done
This does the following:
[[ -f "${file}" ]]: test if file is a file, if so
[[ "${file}" =~ (^[0-9]+) ]]: check if file starts with a number
${file/${BASH_REMATCH[1]}}: remove the number from the string file by using BASH_REMATCH, a variable that matches the groupings from the regex match.
If you've got perl's rename installed, the following should work :
rename 's/^[0-9]{1,3}//' /path/to/files
/path/to/files can be a list of specific files, or probably in your case a glob (e.g. *.{png,txt}). You don't need to select only files starting with digits as rename won't modify those that do not.
Using bash parameter expansion:
shopt -s extglob
for i in +([0-9])*.{txt,png}; do
mv -- "$i" "${i##+([0-9])}"
done
This will remove starting digits (any number) in filenames having png and txt extension.
The ## is removing the longest matching prefix pattern.
The +(...) is path name expansion syntax for repeated characters.
And [0-9] is pattern matching digits.
Alternate method using GNU find:
#!/usr/bin/env bash
find ./ \
-maxdepth 1\
-type f\
-name '[[:digit:]]*'\
-exec bash -c 'shopt -s extglob; f="${1##*/}"; d="${1%%/*}"; mv -- "$1" "${d}/${f##+([[:digit:]])}"' _ {} \;
Find all actual files in current directory whose name start with a digit.
For each found file, execute the Bash script below:
shopt -s extglob # need for extended pattern syntax
f="${1##*/}" # Get file name without directory path
d="${1%%/*}" # Get directory path without file name
mv -- "$1" "${d}/${f##+([[:digit:]])}" # Rename without the leading digits
Using basic features of a POSIX-compliant shell:
#!/bin/sh
for f in [[:digit:]]*; do
if [ -f "$f" ]; then
pf="${f%${f#???}}" pf="${pf##*[[:digit:]]}"
mv "$f" "$pf${f#???}"
fi
done
I am writing a bash script that pulls files from another server to the current directory. The issue is that I get a lot of files and I only need ~3 of them; however all 3 might not be there.
For example, make server all:
server call --> file1.txt file2.txt file3.xls file4.json .... (etc)
Then compress files with tar:
tar zcf needed_files.tgz file4.json file23.doc *.txt
But file4.json was not there, so I would expect tar to compress file23.doc and all .txt files but the script fails with:
tar: file4.json: Cannot stat: No such file or directory
I have tried other combinations of tar commands like czvf but no luck.
tar should successfully compress the existing files despite the "no such file or directory" errors.
Anyway, you could also use nullglob in combination with extglob #() to get only the existing files:
shopt -s extglob nullglob
files=( "fileA"#() "fileB"#() *.txt )
(( ${#files[#]} )) && tar zcf needed_files.tgz -- "${files[#]}"
Try an extended glob.
shopt -s extglob # set extended globbing on
if echo file[1234].+(txt|xls|json) | grep -vq '\['
then tar cvzf needed_files.tgz file[1234].+(txt|xls|json)
else echo No matching files for extglob 'file[1234].+(txt|xls|json)'
fi
If matching files exist, it will list them.
If not, it will literally echo back the pattern.
grepping out the pattern metacharacters tells you whether there are any files in the set. If they do exist, use the same glob to provide the files to tar, and it will receive exactly the set of matching files. If they don't, the condition test lets you skip it.
Of course, it breaks if you make files with [ in the names, etc...
Or, you could do it in a loop....
for f in file[1234].+(txt|xls|json)
do if [[ -e "$f" ]]
then [[ -e needed_files.tar ]] && c=r || c=c
tar ${c}vf needed_files.tar "$f"
fi
done
Not perfect, but might suit your tastes better.
Neither is a great solution, but one of them ought to get you rolling.
tar zcf needed_files.tgz $(ls -d file4.json file23.doc *.txt 2>/dev/null)
Notice that prints only existing files
ls -d file4.json file23.doc *.txt 2>/dev/null
Also you can use --ignore-failed-read option, but it will also ignore other read errors.
I'm having problems creating an if statement to check the files in my directory for a certain string in their names.
For example, I have the following files in a certain directory:
file_1_ok.txt
file_2_ok.txt
file_3_ok.txt
file_4_ok.txt
other_file_1_ok.py
other_file_2_ok.py
other_file_3_ok.py
other_file_4_ok.py
another_file_1_not_ok.sh
another_file_2_not_ok.sh
another_file_3_not_ok.sh
another_file_4_not_ok.sh
I want to copy all files that contain 1_ok to another directory:
#!/bin/bash
directory1=/FILES/user/directory1/
directory2=/FILES/user/directory2/
string="1_ok"
cd $directory
for every file in $directory1
do
if [$string = $file]; then
cp $file $directory2
fi
done
UPDATE:
The simpler answer was made by Faibbus, but refer to Inian if you want to remove or simply move files that don't have the specific string you want.
The other answers are valid as well.
cp directory1/*1_ok* directory2/
Use find for that:
find directory1 -maxdepth 1 -name '*1_ok*' -exec cp -v {} directory2 \;
The advantage of using find over the glob solution posted by Faibbus is that it can deal with an unlimited number of files which contain 1_ok were the glob solution will lead to an argument list too long error when calling cp with too many arguments.
Conclusion: For interactive use with a limited number of input files the glob will be fine, for a shell script, which has to be stable, I would use find.
With your script I suggest:
#!/bin/bash
source="/FILES/user/directory1"
target="/FILES/user/directory2"
regex="1_ok"
for file in "$source"/*; do
if [[ $file =~ $regex ]]; then
cp -v "$file" "$target"
fi
done
From help [[:
When the =~ operator is used, the string to the right of the operator
is matched as a regular expression.
Please take a look: http://www.shellcheck.net/
Using extglob matching in bash with the below pattern,
+(pattern-list)
Matches one or more occurrences of the given patterns.
First enable extglob by
shopt -s extglob
cp -v directory1/+(*not_ok*) directory2/
An example,
$ ls *.sh
another_file_1_not_ok.sh another_file_3_not_ok.sh
another_file_2_not_ok.sh another_file_4_nnoot_ok.sh
$ shopt -s extglob
$ cp -v +(*not_ok*) somedir/
another_file_1_not_ok.sh -> somelib/another_file_1_not_ok.sh
another_file_2_not_ok.sh -> somelib/another_file_2_not_ok.sh
another_file_3_not_ok.sh -> somelib/another_file_3_not_ok.sh
To remove the files except the one containing this pattern, do
$ rm -v !(*not_ok*) 2>/dev/null
My question is more so a question about code efficiency and simplicity than it is about simply completing a task. The scenario is as such: I would like to create a bash script that uses a for loop to iterate through /Users when it is in each users home directory I want it to see if two different directories exist in the style of:
for USER in /Users/*; do
if [ -d "$USER/Library/Caches/com.spotify.Client" ]; then
rm -rf "$USER/Library/Caches/com.spotify.Client"
...but I need to check for multiple directories. How do I accomplish this in the most elegant way? I would like to avoid using a series of if statements but don't know the best way to accomplish this.
Finally, I would like to use the find command to find a file, then set the result of the find (i.e. the path to the found file) to a variable and input it into another command. Thank you.
From what I understand of your requirements, I would nest the for loops:
subdirs=(
"Library/Caches/com.spotify.client"
"some/other/subdir/"
)
for homedir in /Users/*; do
for subdir in "${subdirs[#]}"; do
dir="$homedir/$subdir"
if [[ -d "$dir" ]]; then
rm -rf "$dir"
fi
done
done
The following builds on #David Wolever's answer above; these versions are shorter and let the shell do more globbing up front.
Inline, single-loop version without use of intermediate (array) variable:
for dir in /Users/*/'Library/Caches/com.spotify.client/' /Users/*/'some/other/subdir/'; do
[[ -d "$dir" ]] && rm -rf "$dir"
done
Array-variable version:
subdirs=(
'Library/Caches/com.spotify.client/'
'some/other/subdir/'
)
for subdir in "${subdirs[#]}"; do
for dir in /Users/*/"$subdir"; do
[[ -d "$dir" ]] && rm -rf "$dir"
done
done