I'm writing a bash shell script that uses a case with three options:
If the user enters "change -r txt doc *", a file extension gets changed in a subdirectory.
If a user enters "change -n -r doc ", it should rename files that end with .-r or .-n (this will rename all files in the current directory called *.-r as *.doc)
If the user enters nothing, as in "change txt doc *", it just changes a file extension in the current directory.
Here's the code i produced for it (the last two options, i'm not sure how to implement):
#!/bin/bash
case $1 in
-r)
export currectFolder=`pwd`
for i in $(find . -iname "*.$2"); do
export path=$(readlink -f $i)
export folder=`dirname $path`
export name=`basename $path .$2`
cd $folder
mv $name.$2 $name.$3
cd $currectFolder
done
;;
-n)
echo "-n"
;;
*)
echo "all"
esac
Can anyone fix this for me? Or at least tell me where i'm going wrong?
What you should brush up on are string substitutions. All kinds of them actually. Bash is very good with those. Page 105 (recipe 5.18) of the Bash Cookbook is excellent reading for that.
#!/bin/bash
# Make it more flexible for improving command line parsing later
SWITCH=$1
EXTENSIONSRC=$2
EXTENSIONTGT=$3
# Match different cases for the only allowed switch (other than file extensions)
case $SWITCH in
-r|--)
# If it's not -r we limit the find to the current directory
[[ "x$SWITCH" == "x-r" ]] || DONTRECURSE="-maxdepth 1"
# Files in current folder with particular pattern (and subfolders when -r)
find . $DONTRECURSE -iname "*.$EXTENSIONSRC"|while read fname; do
# We use a while to allow for file names with embedded blank spaces
# Get canonical name of the item into CFNAME
CFNAME=$(readlink -f "$fname")
# Strip extension through string substitution
NOEXT_CFNAME="${CFNAME%.$EXTENSIONSRC}"
# Skip renaming if target exists. This can happen due to collisions
# with case-insensitive matching ...
if [[ -f "$NOEXT_CFNAME.$EXTENSIONTGT" ]]; then
echo "WARNING: Skipping $CFNAME"
else
echo "Renaming $CFNAME"
# Do the renaming ...
mv "$CFNAME" "$NOEXT_CFNAME.$EXTENSIONTGT"
fi
done
;;
*)
# The -e for echo means that escape sequences like \n and \t get evaluated ...
echo -e "ERROR: unknown command line switch\n\tSyntax: change <-r|--> <source-ext> <target-ext>"
# Exit with non-zero (i.e. failure) status
exit 1
esac
The syntax is obviously given in the script. I took the freedom to use the convention of -- separating command line switches from file names. This way it looks cleaner and is easier to implement, actually.
NB: it is possible to condense this further. But here I was trying to get a point across, rather than win the obfuscated Bash contest ;)
PS: also handles the case-insensitive stuff now in the renaming part. However, I decided to make it skip if the target file already exists. Can perhaps be rewritten to be a command line option.
Related
I have several .jpg files in serveral folders about 20K actually. The filename are different like 123.jpg, abc.jpg, ab12.jpg. What is need is to rename all those files using bash script with leading 0 pattern.
I had used one code down below and while I do this the everytime I add new files the previous files get renamed again. Could anyone help me out from this situation and this would be really help full. I have searched entire web for this and not find one :(
#!/bin/bash
num=0
for i in *.jpg
do
a=`printf "%05d" $num`
mv "$i" "filename_$a.jpg"
let "num = $(($num + 1))"
done
To provide a concrete example of the problem, consider:
touch foo.jpg bar.jpg baz.jpg
The first time this script is run, bar.jpg is renamed to filename_00000.jpg; baz.jpg is renamed to filename_00001.jpg; foo.jpg is renamed to filename_00002.jpg. This behavior is acceptable.
If someone then runs:
touch a.jpg
...and runs the script again, then it renames a.jpg to filename_00000.jpg, renames filename_00000.jpg (now a.jpg, as the old version got overwritten!) to filename_00001.jpg, renames filename_00001.jpg to filename_00002.jpg, etc.
How can I make the program leave the files already matching filename_#####.jpg alone, and rename new files to have numbers after the last one that already exists?
#!/bin/bash
shopt -s extglob # enable extended globbing -- regex-like syntax
prefix="filename_"
# Find the largest-numbered file previously renamed
num=0 # initialize counter to 0
for f in "$prefix"+([[:digit:]]).jpg; do # Iterate only over names w/ prefix/suffix
f=${f#"$prefix"} # strip the prefix
f=${f%.jpg} # strip the suffix
if (( 10#$f > num )); then # force base-10 evaluation
num=$(( 10#$f ))
fi
done
# Second pass: Iterate over *all* names, and rename the ones that don't match the pattern
for i in *.jpg; do
[[ $i = "$prefix"+([[:digit:]]).jpg ]] && continue # Skip files already matching pattern
printf -v a '%05d' "$num" # More efficient than subshell use
until mv -n -- "$i" "$prefix$a.jpg"; do # "--" forces parse of "$i" as name
[[ -e "$i" ]] || break # abort if source file disappeared
num=$((num + 1)) # if we couldn't rename, increment num
printf -v a '%05d' "$num" # ...and try again with the next name
done
num=$((num + 1)) # modern POSIX math syntax
done
Note the use of mv -n to prevent overwrites -- that way two copies of this script running at the same time won't overwrite each others' files.
I have list of files at a location ${POWERCENTER_FILE_DIR} .
The files consist of row header and values.
MART_Apple.csv
MART_SAMSUNG.csv
MART_SONY.csv
MART_BlackBerry.csv
Requirements:
select only those files which has atleast 1 row.
Add time stamp to the files which has at least 1 row.
For example:
If all the files except MART_BlackBerry.csv has atleast one row then my output files names should be
MART_Apple_20170811112807.csv
MART_SAMSUNG_20170811112807.csv
MART_SONY_20170811112807.csv
Code tried so far
#!/bin/ksh
infilename=${POWERCENTER_FILE_DIR}MART*.csv
echo File name is ${infilename}
if [ wc -l "$infilename"="0" ];
then
RV=-1
echo "input file name cannot be blank or *"
exit $RV
fi
current_timestamp=`date +%Y%m%d%H%M%S`
filename=`echo $infilename | cut -d"." -f1 `
sftpfilename=`echo $filename`_${current_timestamp}.csv
cp -p ${POWERCENTER_FILE_DIR}$infilename ${POWERCENTER_FILE_DIR}$sftpfilename
RV=$?
if [[ $RV -ne 0 ]];then
echo Adding timestamp to ${POWERCENTER_FILE_DIR}$infilename failed ... Quitting
echo Return Code $RV
exit $RV
fi
Encountering errors like:
line 3: [: -l: binary operator expected
cp: target `MART_Apple_20170811121023.csv' is not a directory
failed ... Quitting
Return Code 1
to be frank, i am not able to apprehend the errors nor i am sure i am doing it right. Beginner in unix scripting.Can any experts guide me where to the correct way.
Here's an example using just find, sh, mv, basename, and date:
find ${POWERCENTER_FILE_DIR}MART*.csv ! -empty -execdir sh -c "mv {} \$(basename -s .csv {})_\$(date +%Y%m%d%H%M%S).csv" \;
I recommend reading Unix Power Tools for more ideas.
When it comes to shell scripting there is rarely a single/one/correct way to accomplish the desired task.
Often times you may need to trade off between readability vs maintainability vs performance vs adhering-to-some-local-coding-standard vs shell-environment-availability (and I'm sure there are a few more trade offs). So, fwiw ...
From your requirement that you're only interested in files with at least 1 row, I read this to also mean that you're only interested in files with size > 0.
One simple ksh script:
#!/bin/ksh
# define list of files
filelist=${POWERCENTER_FILE_DIR}/MART*.csv
# grab current datetime stamp
dtstamp=`date +%Y%m%d%H%M%S`
# for each file in our list ...
for file in ${filelist}
do
# each file should have ${POWERCENTER_FILE_DIR} as a prefix;
# uncomment 'echo' line for debugging purposes to display
# the contents of the ${file} variable:
#echo "file=${file}"
# '-s <file>' => file exists and size is greater than 0
# '! -s <file>' => file doesn't exist or size is equal to 0, eg, file is empty in our case
#
# if the file is empty, skip/continue to next file in loop
if [ ! -s ${file} ]
then
continue
fi
# otherwise strip off the '.csv'
filebase=${file%.csv}
# copy our current file to a new file containing the datetime stamp;
# keep in mind that each ${file} already contains the contents of the
# ${POWERCENTER_FILE_DIR} variable as a prefix; uncomment 'echo' line
# for debug purposes to see what the cp command looks like:
#echo "cp command=cp ${file} ${filebase}.${dtstamp}.csv"
cp ${file} ${filebase}.${dtstamp}.csv
done
A few good resources for learning ksh:
O'Reilly: Learning the Korn Shell
O'Reilly: Learning the Korn Shell, 2nd Edition (includes the newer ksh93)
at your UNIX/Linux command line: man ksh
A simplified script would be something like
#!/bin/bash
# Note I'm using bash above, can't guarantee (but I hope) it would work in ksh too.
for file in ${POWERCENTER_FILE_DIR}/*.csv # Check Ref [1]
do
if [ "$( wc -l "$file" | grep -Eo '^[[:digit:]]+' )" -ne 0 ] # checking at least one row? Check Ref [2]
then
mv "$file" "${file%.csv}$(date +'%Y%m%d%H%M%S').csv" # Check Ref [3]
fi
done
References
File Globbing [1]
Command Substitution [2]
Parameter Substitution [3]
I am writing a simple shell script to make automated backups, and I am trying to use basename to create a list of directories and them parse this list to get the first and the last directory from the list.
The problem is: when I use basename in the terminal, all goes fine and it gives me the list exactly as I want it. For example:
basename -a /var/*/
gives me a list of all the directories inside /var without the / in the end of the name, one per line.
BUT, when I use it inside a script and pass a variable to basename, it puts single quotes around the variable:
while read line; do
dir_name=$(echo $line)
basename -a $dir_name/*/ > dir_list.tmp
done < file_with_list.txt
When running with +x:
+ basename -a '/Volumes/OUTROS/backup/test/*/'
and, therefore, the result is not what I need.
Now, I know there must be a thousand ways to go around the basename problem, but then I'd learn nothing, right? ;)
How to get rid of the single quotes?
And if my directory name has spaces in it?
If your directory name could include spaces, you need to quote the value of dir_name (which is a good idea for any variable expansion, whether you expect spaces or not).
while read line; do
dir_name=$line
basename -a "$dir_name"/*/ > dir_list.tmp
done < file_with_list.txt
(As jordanm points out, you don't need to quote the RHS of a variable assignment.)
Assuming your goal is to populate dir_list.tmp with a list of directories found under each directory listed in file_with_list.txt, this might do.
#!/bin/bash
inputfile=file_with_list.txt
outputfile=dir_list.tmp
rm -f "$outputfile" # the -f makes rm fail silently if file does not exist
while read line; do
# basic syntax checking
if [[ ! ${line} =~ ^/[a-z][a-z0-9/-]*$ ]]; then
continue
fi
# collect targets using globbing
for target in "$line"/*; do
if [[ -d "$target" ]]; then
printf "%s\n" "$target" >> $outputfile
fi
done
done < $inputfile
As you develop whatever tool will process your dir_list.tmp file, be careful of special characters (including spaces) in that file.
Note that I'm using printf instead of echo so that targets whose first character is a hyphen won't cause errors.
This might work
while read; do
find "$REPLY" >> dir_list.tmp
done < file_with_list.txt
I am creating a bash shell script that will rename a file extension without having to specify the old file extension name. If I enter "change foo *" to the Terminal in Linux, it will change all file extension to foo.
So lets say I've got four files: "file1.txt", "file2.txt.txt", "file3.txt.txt.txt" and "file4."
When I run the command, the files should look like this: "file1.foo", "file2.txt.foo", "file3.txt.txt.foo" and "file4.foo"
Can someone look at my code and correct it. I would also appreciate it if someone can implement this for me.
#!/bin/bash
shift
ext=$1
for file in "$#"
do
cut=`echo $FILE |sed -n '/^[a-Z0-9]*\./p'`
if test "${cut}X" == 'X'; then
new="$file.$ext"
else
new=`echo $file | sed "s/\(.*\)\..*/\1.$ext/"`
fi
mv $file $new
done
exit
Always use double quotes around variable substitutions, e.g. echo "$FILE" and not echo $FILE. Without double quotes, the shell expands whitespace and glob characters (\[*?) in the value of the variable. (There are cases where you don't need the quotes, and sometimes you do want word splitting, but that's for a future lesson.)
I'm not sure what you're trying to do with sed, but whatever it is, I'm sure it's doable in the shell.
To check if $FILE contains a dot: case "$FILE" in *.*) echo yes;; *) echo no;; esac
To strip the last extension from $FILE: ${FILE%.*}. For example, if $FILE is file1.txt.foo, this produces file1.txt. More generally, ${FILE%SOME_PATTERN} expands to $FILE with a the shortest suffix matching SOME_PATTERN stripped off. If there is no matching suffix, it expands to $FILE unchanged. The variant ${FILE%%SOME_PATTERN} strips the longest suffix. Similarly, ${FILE#SOME_PATTERN} and ${FILE##SOME_PATTERN} strip a suffix.
test "${TEMP}X" == 'X' is weird. This looks like a misremembered trick from the old days. The normal way of writing this is [ "$TEMP" = "" ] or [ -z "$TEMP" ]. Using == instead of = is a bash extension. There used to be buggy shells that might parse the command incorrectly if $TEMP looked like an operator, but these have gone the way of the dinosaur, and even then, the X needs to be at the beginning, because the problematic operators begin with a -: [ "X$TEMP" == "X" ].
If a file name begins with a -, mv will think it's an option. Use -- to say “that's it, no more options, whatever follows is an operand”: mv -- "$FILE" "$NEW_FILE".
This is very minor, but a common (not universal) convention is to use capital letters for environment variables and lowercase letters for internal script variables.
Since you're using only standard shell features, you can start the script with #!/bin/sh (but #!/bin/bash works too, of course).
exit at the end of the script is useless.
Applying all of these, here's the resulting script.
#!/bin/sh
ext="$1"; shift
for file in "$#"; do
base="${file%.*}"
mv -- "$file" "$base.$ext"
done
Not exactly what you are asking about, but have a look at the perl rename utility. Very powerful! man rename is a good start.
Use: for file in *.gif; do mv $file ${file%.gif}.jpg; done
Or see How to rename multiple files
For me this worked
for FILE in `ls`
do
NEW_FILE=${FILE%.*}
NEW_FILE=${NEW_FILE}${EXT}
done
I just want to tell about NEW_FILE=${FILE%.*}.
Here NEW_FILE gets the file name as output. You can use it as you want.
I tested in bash with uname -a = "Linux 2.4.20-8smp #1 SMP Thu Mar 13 17:45:54 EST 2003 i686 i686 i386 GNU/Linux"
With /bin/bash, how would I detect if a user has a specific directory in their $PATH variable?
For example
if [ -p "$HOME/bin" ]; then
echo "Your path is missing ~/bin, you might want to add it."
else
echo "Your path is correctly set"
fi
Using grep is overkill, and can cause trouble if you're searching for anything that happens to include RE metacharacters. This problem can be solved perfectly well with bash's builtin [[ command:
if [[ ":$PATH:" == *":$HOME/bin:"* ]]; then
echo "Your path is correctly set"
else
echo "Your path is missing ~/bin, you might want to add it."
fi
Note that adding colons before both the expansion of $PATH and the path to search for solves the substring match issue; double-quoting the path avoids trouble with metacharacters.
There is absolutely no need to use external utilities like grep for this. Here is what I have been using, which should be portable back to even legacy versions of the Bourne shell.
case :$PATH: # notice colons around the value
in *:$HOME/bin:*) ;; # do nothing, it's there
*) echo "$HOME/bin not in $PATH" >&2;;
esac
Here's how to do it without grep:
if [[ $PATH == ?(*:)$HOME/bin?(:*) ]]
The key here is to make the colons and wildcards optional using the ?() construct. There shouldn't be any problem with metacharacters in this form, but if you want to include quotes this is where they go:
if [[ "$PATH" == ?(*:)"$HOME/bin"?(:*) ]]
This is another way to do it using the match operator (=~) so the syntax is more like grep's:
if [[ "$PATH" =~ (^|:)"${HOME}/bin"(:|$) ]]
Something really simple and naive:
echo "$PATH"|grep -q whatever && echo "found it"
Where whatever is what you are searching for. Instead of && you can put $? into a variable or use a proper if statement.
Limitations include:
The above will match substrings of larger paths (try matching on "bin" and it will probably find it, despite the fact that "bin" isn't in your path, /bin and /usr/bin are)
The above won't automatically expand shortcuts like ~
Or using a perl one-liner:
perl -e 'exit(!(grep(m{^/usr/bin$},split(":", $ENV{PATH}))) > 0)' && echo "found it"
That still has the limitation that it won't do any shell expansions, but it doesn't fail if a substring matches. (The above matches "/usr/bin", in case that wasn't clear).
Here's a pure-bash implementation that will not pick up false-positives due to partial matching.
if [[ $PATH =~ ^/usr/sbin:|:/usr/sbin:|:/usr/sbin$ ]] ; then
do stuff
fi
What's going on here? The =~ operator uses regex pattern support present in bash starting with version 3.0. Three patterns are being checked, separated by regex's OR operator |.
All three sub-patterns are relatively similar, but their differences are important for avoiding partial-matches.
In regex, ^ matches to the beginning of a line and $ matches to the end. As written, the first pattern will only evaluate to true if the path it's looking for is the first value within $PATH. The third pattern will only evaluate to true if if the path it's looking for is the last value within $PATH. The second pattern will evaluate to true when it finds the path it's looking for in-between others values, since it's looking for the delimiter that the $PATH variable uses, :, to either side of the path being searched for.
I wrote the following shell function to report if a directory is listed in the current PATH. This function is POSIX-compatible and will run in compatible shells such as Dash and Bash (without relying on Bash-specific features).
It includes functionality to convert a relative path to an absolute path. It uses the readlink or realpath utilities for this but these tools are not needed if the supplied directory does not have .. or other links as components of its path. Other than this, the function doesn’t require any programs external to the shell.
# Check that the specified directory exists – and is in the PATH.
is_dir_in_path()
{
if [ -z "${1:-}" ]; then
printf "The path to a directory must be provided as an argument.\n" >&2
return 1
fi
# Check that the specified path is a directory that exists.
if ! [ -d "$1" ]; then
printf "Error: ‘%s’ is not a directory.\n" "$1" >&2
return 1
fi
# Use absolute path for the directory if a relative path was specified.
if command -v readlink >/dev/null ; then
dir="$(readlink -f "$1")"
elif command -v realpath >/dev/null ; then
dir="$(realpath "$1")"
else
case "$1" in
/*)
# The path of the provided directory is already absolute.
dir="$1"
;;
*)
# Prepend the path of the current directory.
dir="$PWD/$1"
;;
esac
printf "Warning: neither ‘readlink’ nor ‘realpath’ are available.\n"
printf "Ensure that the specified directory does not contain ‘..’ in its path.\n"
fi
# Check that dir is in the user’s PATH.
case ":$PATH:" in
*:"$dir":*)
printf "‘%s’ is in the PATH.\n" "$dir"
return 0
;;
*)
printf "‘%s’ is not in the PATH.\n" "$dir"
return 1
;;
esac
}
The part using :$PATH: ensures that the pattern also matches if the desired path is the first or last entry in the PATH. This clever trick is based upon this answer by Glenn Jackman on Unix & Linux.
This is a brute force approach but it works in all cases except when a path entry contains a colon. And no programs other than the shell are used.
previous_IFS=$IFS
dir_in_path='no'
export IFS=":"
for p in $PATH
do
[ "$p" = "/path/to/check" ] && dir_in_path='yes'
done
[ "$dir_in_path" = "no" ] && export PATH="$PATH:/path/to/check"
export IFS=$previous_IFS
$PATH is a list of strings separated by : that describe a list of directories. A directory is a list of strings separated by /. Two different strings may point to the same directory (like $HOME and ~, or /usr/local/bin and /usr/local/bin/). So we must fix the rules of what we want to compare/check. I suggest to compare/check the whole strings, and not physical directories, but remove duplicate and trailing /.
First remove duplicate and trailing / from $PATH:
echo $PATH | tr -s / | sed 's/\/:/:/g;s/:/\n/g'
Now suppose $d contains the directory you want to check. Then pipe the previous command to check $d in $PATH.
echo $PATH | tr -s / | sed 's/\/:/:/g;s/:/\n/g' | grep -q "^$d$" || echo "missing $d"
A better and fast solution is this:
DIR=/usr/bin
[[ " ${PATH//:/ } " =~ " $DIR " ]] && echo Found it || echo Not found
I personally use this in my bash prompt to add icons when i go to directories that are in $PATH.