Writing a shell conditional for extensions - bash

I'm writing a quick shell script to build and execute my programs in one fell swoop.
I've gotten that part down, but I'd like to include a little if/else to catch bad extensions - if it's not an .adb (it's an Ada script), it won't let the rest of the program execute.
My two-part question is:
How do I grab just the extension? Or is it easier to just say *.adb?
What would the if/else statement look like? I have limited experience in Bash so I understand that's a pretty bad question.
Thanks!

There are ways to extract the extension, but you don't really need to:
if [[ $filename == *.adb ]] ; then
. . . # this code is run if $filename ends in .adb
else
. . . # this code is run otherwise
fi
(The trouble with extracting the extension is that you'd have to define what you mean by "extension". What is the extension of a file named foo? How about a file named report.2012.01.29? So general-purpose extension-extracting code is tricky, and not worth it if your goal is just to confirm that file has a specific extension.)

There are multiple ways to do it. Which is best depends in part on what the subsequent operations will be.
Given a variable $file, you might want to test what the extension is. In that case, you probably do best with:
extn=${file##*.}
This deletes everything up to the last dot in the name, slashes and all, leaving you with adb if the file name was adafile.adb.
If, on the other hand, you want to do different things depending on the extension, you might use:
case "$file" in
(*.adb) ...do things with .adb files;;
(*.pqr) ...do things with .pqr files;;
(*) ...cover the rest - maybe an error;;
esac
If you want the name without the extension, you can do things the more traditional way with:
base=$(basename $file .adb)
path=$(dirname $file)
The basename command gives you the last component of the file name with the extension .adb stripped off. The dirname command gives you the path leading to the last component of the file name, defaulting to . (the current directory) if there is no specified path.
The more recent way to do those last two operations is:
base=${file##/}
path=${file%/*}
The advantage of these is that they are built-in operations that do not invoke a separate executable, so they are quicker. The disadvantage of the built-ins is that if you have a name that ends with a slash, the built-in treats it as significant but the command does not (and the command is probably giving you the more desirable behaviour, unless you want to argue GIGO).
There are other techniques available too. The expr command is an old, rather heavy-weight mechanism that would not normally be used (but it is very standard). There may be other techniques using the (( ... )), $(( ... )) and [[ ... ]] operators to evaluate various sorts of expression.

To get just the extension from the file path and name, use parameter expansion:
${filename##*.} # deletes everything to the last dot
To compare it with the string adb, just do
if [[ ${filename##*.} != adb ]] ; then
echo Invalid extension at "$filename".
exit 1
fi
or, using 'else`:
if [[ ${filename##*.} != adb ]] ; then
echo Invalid extension at "$filename".
else
# Run the script...
fi

Extension:
fileext=`echo $filename | sed 's_.*\.__'`
Test
if [[ x"${fileext}" = "xadb" ]] ; then
#do something
fi

Related

BASH Shell Find Multiple Files with Wildcard and Perform Loop with Action

I have a script that I call with an application, I can't run it from command line. I derive the directory where the script is called and in the next variable go up 1 level where my files are stored. From there I have 3 variables with the full path and file names (with wildcard), which I will refer to as "masks".
I need to find and "do something with" (copy/write their names to a new file, whatever else) to each of these masks. The do something part isn't my obstacle as I've done this fine when I'm working with a single mask, but I would like to do it cleanly in a single loop instead of duplicating loop and just referencing each mask separately if possible.
Assume in my $FILESFOLDER directory below that I have 2 existing files, aaa0.csv & bbb0.csv, but no file matching the ccc*.csv mask.
#!/bin/bash
SCRIPTFOLDER=${0%/*}
FILESFOLDER="$(dirname "$SCRIPTFOLDER")"
ARCHIVEFOLDER="$FILESFOLDER"/archive
LOGFILE="$SCRIPTFOLDER"/log.txt
FILES1="$FILESFOLDER"/"aaa*.csv"
FILES2="$FILESFOLDER"/"bbb*.csv"
FILES3="$FILESFOLDER"/"ccc*.csv"
ALLFILES="$FILES1
$FILES2
$FILES3"
#here as an example I would like to do a loop through $ALLFILES and copy anything that matches to $ARCHIVEFOLDER.
for f in $ALLFILES; do
cp -v "$f" "$ARCHIVEFOLDER" > "$LOGFILE"
done
echo "$ALLFILES" >> "$LOGFILE"
The thing that really spins my head is when I run something like this (I haven't done it with the copy command in place) that log file at the end shows:
filesfolder/aaa0.csv filesfolder/bbb0.csv filesfolder/ccc*.csv
Where I would expect echoing $ALLFILES just to show me the masks
filesfolder/aaa*.csv filesfolder/bbb*.csv filesfolder/ccc*.csv
In my "do something" area, I need to be able to use whatever method to find the files by their full path/name with the wildcard if at all possible. Sometimes my network is down for maintenance and I don't want to risk failing a change directory. I rarely work in linux (primarily SQL background) so feel free to poke holes in everything I've done wrong. Thanks in advance!
Here's a light refactoring with significantly fewer distracting variables.
#!/bin/bash
script=${0%/*}
folder="$(dirname "$script")"
archive="$folder"/archive
log="$folder"/log.txt # you would certainly want this in the folder, not $script/log.txt
shopt -s nullglob
all=()
for prefix in aaa bbb ccc; do
cp -v "$folder/$prefix"*.csv "$archive" >>"$log" # append, don't overwrite
all+=("$folder/$prefix"*.csv)
done
echo "${all[#]}" >> "$log"
The change in the loop to append the output or cp -v instead of overwrite is a bug fix; otherwise the log would only contain the output from the last loop iteration.
I would probably prefer to have the files echoed from inside the loop as well, one per line, instead of collect them all on one humongous line. Then you can remove the array all and instead simply
printf '%s\n' "$folder/$prefix"*.csv >>"$log"
shopt -s nullglob is a Bash extension (so won't work with sh) which says to discard any wildcard which doesn't match any files (the default behavior is to leave globs unexpanded if they don't match anything). If you want a different solution, perhaps see Test whether a glob has any matches in Bash
You should use lower case for your private variables so I changed that, too. Notice also how the script variable doesn't actually contain a folder name (or "directory" as we adults prefer to call it); fixing that uncovered a bug in your attempt.
If your wildcards are more complex, you might want to create an array for each pattern.
tmpspaces=(/tmp/*\ *)
homequest=($HOME/*\?*)
for file in "${tmpspaces[#]}" "${homequest[#]}"; do
: stuff with "$file", with proper quoting
done
The only robust way to handle file names which could contain shell metacharacters is to use an array variable; using string variables for file names is notoriously brittle.
Perhaps see also https://mywiki.wooledge.org/BashFAQ/020

Bash check if path contains two folders

How can I search an arbitrary path and determine if it has two folder names? The folder names can appear in any position in either order. Not a shell expert so seeking help here.
if [ -p "$PATH" ]; then
echo "path is set"
else
echo "path is not set"
fi
I found this segment but I'm not sure it's useful. $PATH is a special variable correct?
First, let me make sure I understand the question right. You have some path (like "/home/sam/foo/bar/baz") and you want to test whether it contains two specific directory names (e.g. "foo" and "bar") in either order, right? So, looking for "foo" and "bar":
/home/sam/foo/bar/baz would match
/mnt/bar/subdir/foo would also match
/mnt/bar/foo2 would not match, because "foo2" is not "foo"
If that's correct, you can do this in bash as two tests:
dir1="foo"
dir2="bar"
if [[ "/$path/" = *"/$dir1/"* && "/$path/" = *"/$dir2/"* ]]; then
echo "$path" contains both $dir1 and $dir2"
else
echo "$path" does not contain both $dir1 and $dir2"
fi
Notes:
This is using the [[ ]] conditional expression, which is different from [ ] and not available in basic shells. If you use this type of expression, you need to start the shell script with a shebang that tells the OS to run it with bash, not a generic shell (i.e. the first line should be either #!/bin/bash or #!/usr/bin/env bash), and do not run it with the sh command (that will override the shebang).
The way the comparison works is that it sees whether the path matches both the patterns *"/$dir1/"* and *"/$dir2/"* -- that is, it matches those names, with a slash at each end, maybe with something else (*) before and after. But since the path might not start and/or end with a slash, we add them ("/$path/") to make sure they're there.
Do not use PATH as a variable in your script -- it's a very special variable that tells the shell where to find executable commands. If you ever use it for anything else, your script will suddenly start getting "command not found" errors. Actually, there are a bunch of all-caps special-meaning variables; to avoid conflicts with them, use lowercase or mixed-case variables for your things.

While loop in .sh file with condition to compare if a string contains a substring

I have a .sh file in which I have written the following function. The command that calls this function will have the arguments- file1.war, file2.war ... fileN.war and other arguments.
I want to do a certain operation to the .war files and something else for the arguments after it. So I have written a while loop that will run till the arguments are .war files, and when an argument is encountered without .war extention, it will exit the loop and run the code below it for the rest of the arguments.
Here is the function in .sh file :
copyWarFiles()
{
downloadFileName=$1
shift 1
extn=".war"
while [ condition ]
do
log "war file $downloadFileName .."
#some operation..
downloadFileName=$1
shift 1
done
#operations for the rest of the arguments...
}
What should I give as condition that will return true if $downloadFileName ends with .war? I tried giving
$downloadFileName==*".war" (following the accepted answer in this )
and I also tried this :
`test "${downloadFileName#*$extn}" != "$downloadFileName"`
(following the accepted answer here) where extn is another variable I declared and assigned to .war.
But in both the cases, I see that it never enters the while loop. I think I have gone wrong with the syntax or something. Thank you for your help in advance.
What should I give as condition that will return true if $downloadFileName ends with ".war"? I tried giving $downloadFileName==*".war" […]
Bash, unlike typical programming languages, doesn't recognize == as a special operator; it's just yet another argument to the [ command. So you need to set it off with spaces.
Also, the [ command doesn't support having a pattern on the right-hand-side of ==; you need to use the special [[ ... ]] notation.
So:
while [[ $downloadFileName == *".war" ]]
Note, though, that the double-quotes around .war don't actually have any effect: none of the characters in .war are special characters that need to be quoted. Conversely, it's a best practice to always put variable expansions in double-quotes, in case the variables contain special characters. ([[ actually negates most of the problematic behaviors, but it's just a good habit to be in.)
So:
while [[ "$downloadFileName" == *.war ]]
Why not just:
check=`echo $downloadFile | grep '\.war'`
if [ -n "$check" ]; then
echo $downloadFile ends in .war
fi

Substitute output from variable to give another output

I am trying to substitute output from variable to give another output. The variable i have problems with is the $apps. It gives me "syntax error: bad substitution".
$appletDir is a directory with desktop shortcuts. The problem is that some shortcuts do not have the same name as the icon(png). So i need to substitute the program name with the png linking to it. I got it working with the commented out if-statement below. If this substitution could work then my script would look better. Cause i need to put down a couple of this.
I want it to look for "general_call" instead of "rtcom-call-ui" when going through the icon folders. Cause the png is called "general_call". The icons folders are the variables $icoDir64 $icoDirSca.
for applet in $appletDir*
do
app=`basename $applet | sed -e 's/.*://g' -e 's/.*osso-//g' -e 's/\.desktop.*//g'`
apps="${app/rtcom-call-ui/general_call}"
#if [ "${app}" = "rtcom-call-ui" ]; then
# app="general_call"
#fi
#echo $apps
#done
#exit 0
found=`find ${icoDir64} ${icoDirSca} -name "*.png"`
for file in $found
do
base="`basename ${file}`"
if [ "${base}" = "${app}.png" -o "${base}" = "tasklaunch_${app}.png" -o "${base}" = "general_${app}.png" ]; then
echo "WORKING!!!!!!!!!!!!!!!!!! $file"
fi
done
done
I think you may have a shell version problem (your shell isn't as modern as the notation you are using). A previous incarnation of this post suggested:
apps="${app}/rtcom-call-ui/general_call"
Or, for substituting rtcom-call-ui with general_call, you need to use echo and sed (at least in classic shells - it might be that bash has something built-in to do it):
apps=$(echo "${app}" | sed s/rtcom-call-ui/general_call/)
The notation ${var|continuation} (where | represents an arbitrary punctuation character) is used to modify the value substituted. For example:
apps="${app:-/something/suitable/as/the/default}"
would copy the value of $app, unless $app is not set at all (not relevant here; useful with environment variables) or if $app is an empty string.
The error you are getting is because there is no valid substitution that starts with '/' in your version of the shell. This notation seems to be valid in some versions of Bash (including the one I have to play with); I don't know when it was added. But if the shell you are using is complaining about the notation, then clearly it is not correct for the version of the shell you are using.
Depending on the shebang line (#!/bin/sh vs #!/bin/bash), it might work differently. Failing that, the version of Bash on your machine may be too old.
You can check your shell(s) with:
for app in /some/location/rtcom-call-ui/where.png /another/location/nowhere/thing.png
do
apps=${app/rtcom-call-ui/general-call}
echo $app
echo $apps
done

BASH Expression to replace beginning and ending of a string in one operation?

Here's a simple problem that's been bugging me for some time. I often find I have a number of input files in some directory, and I want to construct output file names by replacing beginning and ending portions. For example, given this:
source/foo.c
source/bar.c
source/foo_bar.c
I often end up writing BASH expressions like:
for f in source/*.c; do
a="obj/${f##*/}"
b="${a%.*}.obj"
process "$f" "$b"
done
to generate the commands
process "source/foo.c" "obj/foo.obj"
process "source/bar.c "obj/bar.obj"
process "source/foo_bar.c "obj/foo_bar.obj"
The above works, but its a lot wordier than I like, and I would prefer to avoid the temporary variables. Ideally there would be some command that could replace the beginning and ends of a string in one shot, so that I could just write something like:
for f in source/*.c; do process "$f" "obj/${f##*/%.*}.obj"; done
Of course, the above doesn't work. Does anyone know something that will? I'm just trying to save myself some typing here.
Not the prettiest thing in the world, but you can use a regular expression to group the content you want to pick out, and then refer to the BASH_REMATCH array:
if [[ $f =~ ^source/(.*).c$ ]] ; then f="obj/${BASH_REMATCH[1]}.o"; fi
you shouldn't have to worry about your code being "wordier" or not. In fact, being a bit verbose is no harm, consider how much it will improve your(or someone else) understanding of the script. Besides, for performance, using bash's internal string manipulation is much faster than calling external commands. Lastly, you are not going to retype your commands every time you use it right? So why worry that its "wordier" since these commands are already in your script?
Not directly in bash. You can use sed, of course:
b="$(sed 's|^source/(.*).c$|obj/$1.obj|' <<< "$f")"
Why not simply using cd to remove the "source/" part?
This way we can avoid the temporary variables a and b:
for f in $(cd source; printf "%s\n" *.c); do
echo process "source/${f}" "obj/${f%.*}.obj"
done

Resources