delete a numbered range of files - bash

I have a range of files in the format namen.txt and I'd like to remove all but 3 whos n immediately precedes a selected file.
ie. if I input file24.txt I want to delete any file lower than file20.txt
The following works, but is there a simpler way? perhaps using find -name -delete or similar?
file=file24.txt
num=$(sed 's/[^0-9]//g' <<< $file)
((num-=3))
while :
do
files=($(find $dir -name "*txt"))
count=${#files[#]}
if ((count < 1 ))
then
break
fi
rm file"$num".txt
((num--))
done

Here is one way of doing it:
#!/bin/bash
# Grab the number from file passed to the script. $1 holds the that value
num="${1//[!0-9]}"
# To prevent from matching no files enable this shell option
shopt -s nullglob
# Iterate over the desired path where files are
for file in *; do
# Capture the number from file in loop
n=${file//[!0-9]}
# If file has no number and matches your criteria, delete the file
[[ ! -z $n ]] && (( n < num - 3)) && rm "$file"
done
Run it as:
./script.sh file24.txt

I'd probably do this with a Perl script:
#!/usr/bin/env perl
use strict;
use warnings;
for my $arg (#ARGV)
{
my($prefix, $number, $suffix) = ($arg =~ m/^ (\D+) (\d+) (\D.*) $/x);
foreach my $i (1..$number-4)
{
my $file = "$prefix$i$suffix";
unlink $file;
print "$file\n";
}
}
For each of the arguments specified on the command line, the name is split into 3 bits: a non-empty prefix of non-digits, a non-empty number of digits, and a suffix consisting of a non-digit followed by any sequence of characters (so file1.bz2 is split into file, 1 and .bz2). Then, for each number from 1 to 4 less than the given number, generate a file name from the prefix, the current number, and the suffix. With that file name, unlink the file and print the name. You can tweak it to remove only files that exist, or not report the names, or whatever. There's no fixed limit on the maximum number of files.
You could omit the unlink and simply print the file names and send those to xargs rm -f or an equivalent. You could ensure that the names were terminated with a null byte so that names with newlines could be handled correctly by GNU xargs and the -0 option. Etc.
You could code this in pure Bash if you wished to, though the splitting into prefix, number, suffix will be messier than in Perl. I wouldn't use awk for this, though it could probably be forced to do the job if you chose to make it do so.

I think this might be one of the easiest ways of doing it:
shopt -s extglob
rm !(file21.txt|file22.txt|file23.txt)
Here's a simple function that does this in a more generic way:
function rmbut3() {
for ((n=0 ; n < $(($1 - 3)) ; n++))
do
rm file${n}.txt
done
}
rmbut3 24 # deletes files up to file20.txt

Related

How to touch file names containing date greater than specific date in bash?

I want to touch all the files which are having date GREATER THAN 20201009 in its name, inside specific directory recursively.
examples of input file names and its directory location
/tmp/data/gov/USA/NY/nyc_20201006.txt
/tmp/data/gov/USA/NY/nyc_20201008.txt
/tmp/data/gov/USA/NY/nyc_20201009.txt
/tmp/data/gov/USA/NY/nyc_20201010.txt
/tmp/data/gov/USA/NY/nyc_20201011.txt
/tmp/data/gov/USA/CA/san_mateo_20201006.txt
/tmp/data/gov/USA/CA/san_mateo_20201007.txt
/tmp/data/gov/USA/CA/san_mateo_20201008.txt
/tmp/data/gov/USA/CA/san_mateo_20201009.txt
/tmp/data/gov/USA/CA/san_mateo_20201010.txt
/tmp/data/gov/USA/CA/san_mateo_20201011.txt
/tmp/data/gov/USA/FL/tampa_bay_20201006.txt
/tmp/data/gov/USA/FL/tampa_bay_20201008.txt
/tmp/data/gov/USA/FL/tampa_bay_20201009.txt
/tmp/data/gov/USA/FL/tampa_bay_20201010.txt
/tmp/data/gov/USA/FL/tampa_bay_20201011.txt
Now, Inside USA directory I want to touch any file whose name is containing dt>=20201009
so the candidate files to be touched will be
/tmp/data/gov/USA/NY/nyc_20201010.txt
/tmp/data/gov/USA/NY/nyc_20201011.txt
/tmp/data/gov/USA/CA/san_mateo_20201010.txt
/tmp/data/gov/USA/CA/san_mateo_20201011.txt
/tmp/data/gov/USA/FL/tampa_bay_20201010.txt
/tmp/data/gov/USA/FL/tampa_bay_20201011.txt
Would you please try the following:
#!/bin/bash
find /tmp/data/gov/USA -type f -name "*.txt" -print0 | while IFS= read -d "" f; do
if [[ $f =~ ([[:digit:]]{8})\.txt ]] && (( ${BASH_REMATCH[1]} > 20201009 )); then
touch "$f"
fi
done
The find command searches for the specified files and prints the
filenames delimited by a null character to preserve filenames
which contain special characters.
The while loop iterates over the filenames passed by find.
The regex ([[:digit:]]{8})\.txt matches eight consecutive digits
followed by a suffix .txt assigning the shell variable ${BASH_REMATCH[1]}
to the digits.
The condition (( ${BASH_REMATCH[1]} > 20201009 )) returns true
if the bash variable is greater than 20201009.

How can I check if exists file with name according to "template" in the directory?

Given variable with name template , for example: template=*.txt.
How can I check if files with name like this template exist in the current directory?
For example, according to the value of the template above, I want to know if there is files with the suffix .txt in the current directory.
I would do it like this with just built-ins:
templcheck () {
for f in * .*; do
[[ -f $f ]] && [[ $f = $1 ]] && return 0
done
return 1
}
This takes the template as an argument (must be quoted to prevent premature expansion) and returns success if there was a match in the current directory. This should work for any filenames, including those with spaces and newlines.
Usage would look like this:
$ ls
file1.txt 'has space1.txt' script.bash
$ templcheck '*.txt' && echo yes
yes
$ templcheck '*.md' && echo yes || echo no
no
To use with the template contained in a variable, that expansion has to be quoted as well:
templcheck "$template"
Use find:
: > found.txt # Ensure the file is empty
find . -prune -exec find -name "$template" \; > found.txt
if [ -s found.txt ]; then
echo "No matching files"
else
echo "Matching files found"
fi
Strictly speaking, you can't assume that found.txt contains exactly one file name per line; a filename with an embedded newline will look the same as two separate files. But this does guarantee that an empty file means no matching files.
If you want an accurate list of matching file names, you need to disable field splitting while keeping pathname expansion.
[[ -v IFS ]] && OLD_IFS=$IFS
IFS=
shopt -s nullglob
files=( $template )
[[ -v OLD_IFS ]] && IFS=$OLD_IFS
printf "Found: %s\n" "${files[#]}"
This requires several bash extensions (the nullglob option, arrays, and the -v operator for convenience of restoring IFS). Each element of the array is exactly one match.

How can I use multiple Bash arguments in loop dynamically without using long regex strings?

I have a directory with the following files:
file1.jpg
file2.jpg
file3.jpg
file1.png
file2.png
file3.png
I have a bash function named filelist and it looks like this:
filelist() {
if [ "$1" ]
then
shopt -s nullglob
for filelist in *."$#" ; do
echo "$filelist" >> created-file-list.txt;
done
echo "file created listing: " $#;
else
filelist=`find . -type f -name "*.*" -exec basename \{} \;`
echo "$filelist" >> created-file-list.txt
echo "file created listing: All Files";
fi
}
Goal: Be able to type as many arguments as I want for example filelist jpg png and create a file with a list of files of only the extensions I used as arguments. So if I type filelist jpg it would only show a list of files that have .jpg.
Currently: My code works great with one argument thanks to $#, but when I use both jpg and png it creates the following list
file1.jpg
file2.jpg
file3.jpg
png
It looks like my for loop is only running once and only using the first argument. My suspicion is I need to count how many arguments and run the loop on each one.
An obvious fix for this is to create a long regex check like (jpg|png|jpeg|html|css) and all of the different extensions one could ever think to type. This is not ideal because I want other people to be free to type their file extensions without breaking it if they type one that I don't have identified in my regex. Dynamic is key.
You can rewrite your function as shown below - just loop through each extension and append the list of matching files to the output file:
filelist() {
if [ $# -gt 0 ]; then
shopt -s nullglob
for ext in "$#"; do
printf '%s\n' *."$ext" >> created-file-list.txt
echo "created listing for extension $ext"
done
else
find . -type f -name "*.*" -exec basename \{} \; >> created-file-list.txt
echo "created listing for all files"
fi
}
And you can invoke your function as:
filelist jpg png
Try this
#!/bin/bash
while [ -n "$1" ]
do
echo "Current Parameter: $1 , Remaining $#"
#Pass $1 to some bash function or do whatever
shift
done
Using the shift you shift the args left and get the next one by reading the $1 variable.
See man bash on what shift does.
shift [n]
The positional parameters from n+1 ... are renamed to $1 .... Parameters represented by the numbers $# down to $#-n+1 are
unset. n must
be a non-negative number less than or equal to $#. If n is 0, no parameters are changed. If n is not given, it is assumed to
be 1. If n
is greater than $#, the positional parameters are not changed. The return status is greater than zero if n is greater than
$# or less
than zero; otherwise 0.
Or you can iterate like as follows
for this in "$#"
do
echo "Param = $this";
done

Replace numbers in a file name in unix (BASH)

I have multiple files approximately 150 and there names do not match a requirement of a vendor. example file names are:
company_red001.p12
company_red002.p12
.
.
.
.
company_red150.p12
I need to rename all files so that 24 is added to each number sequentially and that there are no preceding zero's and that the company_ component is removed.
red25.p12
red26.p12
red27.p12
.
.
.
red150.p12
I have used a for loop in bash to remove the company_ component but would like something that executes all changes simultaneously as I have to perform this at a moments notice.
example:
#!/bin/bash
n = 24
for file in company_red*
do
new_name=$file$n
n=$(($+1))
mv -i $file $new_name
done
example 2
#!/bin/bash
for f in company_red*
do mv "$f" "${f/company_red/red}";
done
Most probably this one could be fine :)
# printf is used to emulate a lot of files
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
# get the filename
n="$f"
# remove extension
n="${n%.*}"
# remove leading letters
n="${n##*[[:alpha:]]}"
# add 24, 10# is used to consider the 10-based number
n="$(( 10#$n + 24 ))"
# construct new filename
g="red${n}.p12"
echo mv "$f" "$g"
done
And this could be simplified a bit
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
# take the number from the specific, fixed position
n="${f:11:3}"
# everything below is the same as in the previous example
n="$(( 10#$n + 24 ))"
g="red${n}.p12"
echo mv "$f" "$g"
done
And finally, this could be simplified yet twice -- just escape of using $n and $g:
for f in $( printf "company_red%03d.p12\n" {1..150} )
do
echo mv "$f" "red$(( 10#${f:11:3} + 24 )).p12"
done
But this could complicate understanding and supporting of the code.
Do:
for file in *.p12; do
name=${file#*_} ## Extracts the portion after `_` from filename, save as variable "name"
pre=${name%.*} ## Extracts the portion before extension, save as "pre"
num=${pre##*[[:alpha:]]} ## Extracts number from variable "pre"
pre=${pre%%[0-9]*} ## Extracts the alphabetic portion from variable "pre"
suf=${name##*.} ## Extracts the extension from variable "name"
echo mv -i "$file" "${pre}""$(printf '%d' $((10#$num+24)))"."${suf}" ## Doing arithmetic expansion for addition, and necessary formatting to get desired name
done
Outputs:
mv -i company_red001.p12 red25.p12
mv -i company_red002.p12 red26.p12
The above is dry-run, remove echo if you are satisfied with the renaming to be done:
for file in *.p12; do
name=${file#*_}
pre=${name%.*}
num=${pre##*[[:alpha:]]}
pre=${pre%%[0-9]*}
suf=${name##*.}
mv -i "$file" "${pre}""$(printf '%d' $((10#$num+24)))"."${suf}"
done

Creating a which command in bash script

For an assignment, I'm supposed to create a script called my_which.sh that will "do the same thing as the Unix command, but do it using a for loop over an if." I am also not allowed to call which in my script.
I'm brand new to this, and have been reading tutorials, but I'm pretty confused on how to start. Doesn't which just list the path name of a command?
If so, how would I go about displaying the correct path name without calling which, and while using a for loop and an if statement?
For example, if I run my script, it will echo % and wait for input. But then how do I translate that to finding the directory? So it would look like this?
#!/bin/bash
path=(`echo $PATH`)
echo -n "% "
read ans
for i in $path
do
if [ -d $i ]; then
echo $i
fi
done
I would appreciate any help, or even any starting tutorials that can help me get started on this. I'm honestly very confused on how I should implement this.
Split your PATH variable safely. This is a general method to split a string at delimiters, that is 100% safe regarding any possible characters (including newlines):
IFS=: read -r -d '' -a paths < <(printf '%s:\0' "$PATH")
We artificially added : because if PATH ends with a trailing :, then it is understood that current directory should be in PATH. While this is dangerous and not recommended, we must also take it into account if we want to mimic which. Without this trailing colon, a PATH like /bin:/usr/bin: would be split into
declare -a paths='( [0]="/bin" [1]="/usr/bin" )'
whereas with this trailing colon the resulting array is:
declare -a paths='( [0]="/bin" [1]="/usr/bin" [2]="" )'
This is one detail that other answers miss. Of course, we'll do this only if PATH is set and non-empty.
With this split PATH, we'll use a for-loop to check whether the argument can be found in the given directory. Note that this should be done only if argument doesn't contain a / character! this is also something other answers missed.
My version of which handles a unique option -a that print all matching pathnames of each argument. Otherwise, only the first match is printed. We'll have to take this into account too.
My version of which handles the following exit status:
0 if all specified commands are found and executable
1 if one or more specified commands is nonexistent or not executable
2 if an invalid option is specified
We'll handle that too.
I guess the following mimics rather faithfully the behavior of my which (and it's pure Bash):
#!/bin/bash
show_usage() {
printf 'Usage: %s [-a] args\n' "$0"
}
illegal_option() {
printf >&2 'Illegal option -%s\n' "$1"
show_usage
exit 2
}
check_arg() {
if [[ -f $1 && -x $1 ]]; then
printf '%s\n' "$1"
return 0
else
return 1
fi
}
# manage options
show_only_one=true
while (($#)); do
[[ $1 = -- ]] && { shift; break; }
[[ $1 = -?* ]] || break
opt=${1#-}
while [[ $opt ]]; do
case $opt in
(a*) show_only_one=false; opt=${opt#?} ;;
(*) illegal_option "${opt:0:1}" ;;
esac
done
shift
done
# If no arguments left or empty PATH, exit with return code 1
(($#)) || exit 1
[[ $PATH ]] || exit 1
# split path
IFS=: read -r -d '' -a paths < <(printf '%s:\0' "$PATH")
ret=0
# loop on arguments
for arg; do
# Check whether arg contains a slash
if [[ $arg = */* ]]; then
check_arg "$arg" || ret=1
else
this_ret=1
for p in "${paths[#]}"; do
if check_arg "${p:-.}/$arg"; then
this_ret=0
"$show_only_one" && break
fi
done
((this_ret==1)) && ret=1
fi
done
exit "$ret"
To test whether an argument is executable or not, I'm checking whether it's a regular file1 which is executable with:
[[ -f $arg && -x $arg ]]
I guess that's close to my which's behavior.
1 As #mklement0 points out (thanks!) the -f test, when applied against a symbolic link, tests the type of the symlink's target.
#!/bin/bash
#Get the user's first argument to this script
exe_name=$1
#Set the field separator to ":" (this is what the PATH variable
# uses as its delimiter), then read the contents of the PATH
# into the array variable "paths" -- at the same time splitting
# the PATH by ":"
IFS=':' read -a paths <<< $PATH
#Iterate over each of the paths in the "paths" array
for e in ${paths[*]}
do
#Check for the $exe_name in this path
find $e -name $exe_name -maxdepth 1
done
This is similar to the accepted answer with the difference that it does not set the IFS and checks if the execute bits are set.
#!/bin/bash
for i in $(echo "$PATH" | tr ":" "\n")
do
find "$i" -name "$1" -perm +111 -maxdepth 1
done
Save this as my_which.sh (or some other name) and run it as ./my_which java etc.
However if there is an "if" required:
#!/bin/bash
for i in $(echo "$PATH" | tr ":" "\n")
do
# this is a one liner that works. However the user requires an if statment
# find "$i" -name "$1" -perm +111 -maxdepth 1
cmd=$i/$1
if [[ ( -f "$cmd" || -L "$cmd" ) && -x "$cmd" ]]
then
echo "$cmd"
break
fi
done
You might want to take a look at this link to figure out the tests in the "if".
For a complete, rock-solid implementation, see gniourf_gniourf's answer.
Here's a more concise alternative that makes do with a single invocation of find [per name to investigate].
The OP later clarified that an if statement should be used in a loop, but the question is general enough to warrant considering other approaches.
A naïve implementation would even work as a one-liner, IF you're willing to make a few assumptions (the example uses 'ls' as the executable to locate):
find -L ${PATH//:/ } -maxdepth 1 -type f -perm -u=x -name 'ls' 2>/dev/null
The assumptions - which will hold in many, but not all situations - are:
$PATH must not contain entries that when used unquoted result in shell expansions (e.g., no embedded spaces that would result in word splitting, no characters such as * that would result in pathname expansion)
$PATH must not contain an empty entry (which must be interpreted as the current dir).
Explanation:
-L tells find to investigate the targets of symlinks rather than the symlinks themselves - this ensures that symlinks to executable files are also recognized by -type f
${PATH//:/ } replaces all : chars. in $PATH with a space each, causing the result - due to being unquoted - to be passed as individual arguments split by spaces.
-maxdepth 1 instructs find to only look directly in each specified directory, not also in subdirectories
-type f matches only files, not directories.
-perm -u=x matches only files and directories that the current user (u) can execute (x).
2>/dev/null suppresses error messages that may stem from non-existent directories in the $PATH or failed attempts to access files due to lack of permission.
Here's a more robust script version:
Note:
For brevity, only handles a single argument (and no options).
Does NOT handle the case where entries or result paths may contain embedded \n chars - however, this is extremely rare in practice and likely leads to bigger problems overall.
#!//bin/bash
# Assign argument to variable; error out, if none given.
name=${1:?Please specify an executable filename.}
# Robustly read individual $PATH entries into a bash array, splitting by ':'
# - The additional trailing ':' ensures that a trailing ':' in $PATH is
# properly recognized as an empty entry - see gniourf_gniourf's answer.
IFS=: read -r -a paths <<<"${PATH}:"
# Replace empty entries with '.' for use with `find`.
# (Empty entries imply '.' - this is legacy behavior mandated by POSIX).
for (( i = 0; i < "${#paths[#]}"; i++ )); do
[[ "${paths[i]}" == '' ]] && paths[i]='.'
done
# Invoke `find` with *all* directories and capture the 1st match, if any, in a variable.
# Simply remove `| head -n 1` to print *all* matches.
match=$(find -L "${paths[#]}" -maxdepth 1 -type f -perm -u=x -name "$name" 2>/dev/null |
head -n 1)
# Print result, if found, and exit with appropriate exit code.
if [[ -n $match ]]; then
printf '%s\n' "$match"
exit 0
else
exit 1
fi

Resources