(bash) What is the least redundant way to systematically apply changes to an array of variables? - bash

My goal is to check a list of file paths if they end in "/" and remove it if that is the case.
Ideally I would like to change the original FILEPATH variables to reflect this change, and I'd like this to work for a long list without unnecessary redundancy. I tried doing it as a loop, but the changes didn't alter the original variables, it just changed the iterating "EACH_PATH" variable. Can anyone think of a better way to do this?
Here is my code:
FILEPATH1="filepath1/file1"
FILEPATH2="filepath2/file2/"
PATH_ARRAY=(${FILEPATH1} ${FILEPATH2})
echo ${PATH_ARRAY[#]}
for EACH_PATH in ${PATH_ARRAY[#]}
do
if [ "${EACH_PATH:$((${#EACH_PATH}-1)):${#EACH_PATH}}"=="/" ]
then EACH_PATH=${EACH_PATH:0:$((${#EACH_PATH}-1))}
fi
done
edit: I'm happy to do this in a totally different way and scrap the code above, I just want to know the most elegant way to do this.

I'm not entirely clear on the actual goal here, but depending on the situation I can see several possible solutions. The best (if it'll work in the situation) is to dispense with the individual variables, and just use array entries. For example, you could use:
declare -a filepath
filepath[1]="filepath1/file1"
filepath[2]="filepath2/file2/"
for index in "${!filepath[#]}"; do
if [[ "${filepath[index]}" = *?/ ]]; then
filepath[index]="${filepath[index]%/}"
fi
done
...and then use "${filepath[x]}" instead of "$FILEPATHx" throughout. Some notes:
I've used lowercase names. It's generally best to avoid all-caps names, since there are a lot of them with special functions, and accidentally using one of those names can cause trouble.
"${!filepath[#]}" gets a list of the indexes of the array (in this case, "1" "2") rather than their values; this is necessary so we can set the entries rather than just look at them.
I changed the logic of the slash-trimming test -- it uses [[ = ]] to do pattern matching, to see if the entry ends with "/" and has at least one character before that (i.e. it isn't just "/", 'cause you don't want to trim that). Then it uses in the expansion %/ to just trim "/" from the end of the value.
If a numerically-indexed array won't work (and you have at least bash version 4), how about a string-indexed ("associative") array? It's very similar, but use declare -A and use $ on variables in the index (and generally quote them). Something like this:
declare -A filepath
filepath["foo"]="filepath1/file1"
filepath["bar"]="filepath2/file2/"
for index in "${!filepath[#]}"; do
if [[ "${filepath["$index"]}" = *?/ ]]; then
filepath["$index"]="${filepath["$index"]%/}"
fi
done
If you really need separate variables instead of array entries, you might be able to use an array of variable names, and indirect variable references. how this works varies quite a bit between different shells, and can easily be unsafe depending on what's in your data (in this case, specifically what's in path_array). Here's a way to do it in bash:
filepath1="filepath1/file1"
filepath2="filepath2/file2/"
path_array=(filepath1 filepath2)
for varname in "${path_array[#]}"; do
if [[ "${!varname}" = *?/ ]]; then
declare "$varname=${!varname%/}"
fi
done

Using sed
PATH_ARRAY=($(echo ${PATH_ARRAY[#]} | sed 's#\/ ##g;s#/$##g'))
Demo:
$FILEPATH1="filepath1/file1"
$FILEPATH2="filepath2/file2/"
$PATH_ARRAY=(${FILEPATH1} ${FILEPATH2})
$echo ${PATH_ARRAY[#]}
filepath1/file1 filepath2/file2/
$PATH_ARRAY=($(echo ${PATH_ARRAY[#]} | sed 's#\/ ##g;s#/$##g'))
$echo ${PATH_ARRAY[#]}
filepath1/file1 filepath2/file2
$

Related

Using a variable for associative array key in Bash

I'm trying to create associative arrays based on variables. So below is a super simplified version of what I'm trying to do (the ls command is not really what I want, just used here for illustrative purposes)...
I have a statically defined array (text-a,text-b). I then want to iterate through that array, and create associative arrays with those names and _AA appended to them (so associative arrays called text-a_AA and text-b_AA).
I don't really need the _AA appended, but was thinking it might be
necessary to avoid duplicate names since $NAME is already being used
in the loop.
I will need those defined and will be referencing them in later parts of the script, and not just within the for loop seen below where I'm trying to define them... I want to later, for example, be able to reference text-a_AA[NUM] (again, using variables for the text-a_AA part). Clearly what I have below doesn't work... and from what I can tell, I need to be using namerefs? I've tried to get the syntax right, and just can't seem to figure it out... any help would be greatly appreciated!
#!/usr/bin/env bash
NAMES=('text-a' 'text-b')
for NAME in "${NAMES[#]}"
do
NAME_AA="${NAME}_AA"
$NAME_AA[NUM]=$(cat $NAME | wc -l)
done
for NAME in "${NAMES[#]}"
do
echo "max: ${$NAME_AA[NUM]}"
done
You may want to use "NUM" as the name of the associative array and file name as the key. Then you can rewrite your code as:
NUM[${NAME}_AA]=$(wc -l < "$NAME")
Then rephrase your loop as:
for NAME in "${NAMES[#]}"
do
echo "max: ${NUM[${NAME}_AA]}"
done
Check your script at shellcheck.net
As an aside: all uppercase is not a good practice for naming normal shell variables. You may want to take a look at:
Correct Bash and shell script variable capitalization

Including Variables in Strings, Bash Scripting

I need to insert variables into a string to create a URL. Right now, I'm looping over an array of values and inserting them into the string.
year="2015"
for i in "${array[#]}"
do
url="https://www.my-website.com/place/PdfLinkServlet?file=\place\\$i\099%2Bno-display\\$year_$i.pdf"
echo $url
done
The $i is being replaced with the corresponding array element, but $year just leaves a blank space. Can someone explain why and how to get a url that looks like: url="https://www.my-website.com/place/PdfLinkServlet?file=\place\place_id\099%2Bno-display\2015_place_id.pdf"
Because variable names can legally contain _ characters, there's no way for Bash to know that you wanted $year instead of $year_. To disambiguate, you can use enclose the variable name in brackets like this:
${year}_${i}.pdf
It's not bad practise to do this any time you are shoving variable expansions together. As you can see, it actually makes them stand out better to human eyes too.
Use ${var} instead:
year="2015"
for i in "${array[#]}"; do
url="https://www.my-website.com/place/PdfLinkServlet?file=place${i}099%2Bno-display${year}_${i}.pdf"
echo "$url"
done
Here is a little hack that also works well. And I use it for my projects frequently. Add the _ to the end of 2015 in the variable year
like year="2015_"
and remove the _ from the url variable and join the two variables i and year together like $year$i
I added an arbitrary array so that the script can run.
#!/bin/bash
year="2015_"
array=(web03 web04 web05 web06 web07)
for i in "${array[#]}";
do
url="https://www.my-website.com/place/PdfLinkServlet?file=\place\\$i\099%2Bno-display\\$year$i.pdf"
echo $url
done

Pattern matching in bash with tuple-like arguments

I want to pass a variable number of 'tuples' as arguments into a bash script and go through them in a loop using pattern matching, something like this:
for *,* in "$#"; do
#do something with first part of tuple
#do something with second part of tuple
done
is this possible? If so, how do I access each part of the tuple?
For example I would like to call my script like:
bash bashscript.sh first_file.xls,1 second_file,2 third_file,2 ... nth_file,1
Since bash doesn't have a tuple datatype (it just has strings), you need would need to encode and decode them yourself. For example:
$ bash bashscript.sh first_file.xls,1 second_file,2 third_file,2 ... nth_file,1
In bashscript.sh:
for tuple in "$#"; do
IFS=, read first second <<< "$tuple"
...
done
Yes, it's possible, and there is more than one way to do it. You can use the prefix/suffix expansion syntax on variables (e.g. ${var#prefix}, ${var##prefix}, ${var%suffix}, ${var%%suffix} - these remove either the shortest or longest prefix/suffix matching the specified pattern). Or you can replace the positional parameters with e.g. IFS=, set -- ${var} (although you'd have to make sure to save the rest of the original parameters in some way first so you can continue your loop). You can use arrays, if your version of bash is new enough (and if it isn't it's pretty old...). Those are probably three of the better methods, but there are others...
Edit: some examples using the suffix/prefix expansions:
for tuple in first_file.xls,1
do
echo ${tuple%,*} # "first_file.xls"
echo ${tuple#*,} # "1"
done
If your tuples are more than 2-ary, that method's a little more complex; for example:
for tuple in x,y,z
do
first=${tuple%%,*}
rest=${tuple#${first}}
second=${rest%%,*}
last=${rest#*,}
done
In that case you might prefer #chepner's answer of IFS=, read first second third <<< "${tuple}"... Otherwise, the bookkeeping can get hairy for large tuples. Setting an array from the tuple would be an acceptable alternative as well.
For simple pairs, though, I tend to prefer just stripping off a prefix/suffix as appropriate...

variable substitution removing quotes

I seem to have some difficulty getting what I want to work. Basically, I have a series of variables that are assigned strings with some quotes and \ characters. I want to remove the quotes to embed them inside a json doc, since json hates quotes using python dump methods.
I figured it would be easy. Just determine how to remove the characters easy and then write a simple for loop for the variable substitution, well it didn't work that way.
Here is what I want to do.
There is a variable called "MESSAGE23", it contains the following "com.centrify.tokend.cac", I want to strip out the quotes, which to me is easy, a simple echo $opt | sed "s/\"//g". When I do this from the command line:
$> MESSAGE23="com."apple".cacng.tokend is present"
$> MESSAGE23=`echo $MESSAGE23 | sed "s/\"//g"`
$> com.apple.cacng.tokend is present
This works. I get the properly formatted string.
When I then try to throw this into a loop, all hell breaks loose.
for i to {1..25}; do
MESSAGE$i=`echo $MESSAGE$i | sed "s/\"//g"`
done
This doesn't work (either it throws a bunch of indexes out or nothing), and I'm pretty sure I just don't know enough about arg or eval or other bash substitution variables.
But basically I want to do this for another set of variables with the same problems, where I strip out the quotes and incidentally the "\" too.
Any help would be greatly appreciated.
You can't do that. You could make it work using eval, but that introduces another level of quoting you have to worry about. Is there some reason you can't use an array?
MESSAGE=("this is MESSAGE[0]" "this is MESSAGE[1]")
MESSAGE[2]="I can add more, too!"
for (( i=0; i<${#MESSAGE[#]}; ++i )); do
echo "${MESSAGE[i]}"
done
Otherwise you need something like this:
eval 'echo "$MESSAGE'"$i"'"'
and it just gets worse from there.
First, a couple of preliminary problems: MESSAGE23="com."apple".cacng.tokend is present" will not embed double-quotes in the variable value, use MESSAGE23="com.\"apple\".cacng.tokend is present" or MESSAGE23='com."apple".cacng.tokend is present' instead. Second, you should almost always put double-quotes around variable expansions (e.g. echo "$MESSAGE23") to prevent parsing oddities.
Now, the real problems: the shell doesn't allow variable substitution on the left side of an assignment (i.e. MESSAGE$i=something won't work). Fortunately, it does allow this in a declare statement, so you can use that instead. Also, when the sees $MESSAGE$i it replaces it will the value of $MESSAGE followed by the value of $i; for this you need to use indirect expansion (`${!metavariable}').
for i in {1..25}; do
varname="MESSAGE$i"
declare $varname="$(echo "${!varname}" | tr -d '"')"
done
(Note that I also used tr instead of sed, but that's just my personal preference.)
(Also, note that #Mark Reed's suggestion of an array is really the better way to do this sort of thing.)

BASH Expression to replace beginning and ending of a string in one operation?

Here's a simple problem that's been bugging me for some time. I often find I have a number of input files in some directory, and I want to construct output file names by replacing beginning and ending portions. For example, given this:
source/foo.c
source/bar.c
source/foo_bar.c
I often end up writing BASH expressions like:
for f in source/*.c; do
a="obj/${f##*/}"
b="${a%.*}.obj"
process "$f" "$b"
done
to generate the commands
process "source/foo.c" "obj/foo.obj"
process "source/bar.c "obj/bar.obj"
process "source/foo_bar.c "obj/foo_bar.obj"
The above works, but its a lot wordier than I like, and I would prefer to avoid the temporary variables. Ideally there would be some command that could replace the beginning and ends of a string in one shot, so that I could just write something like:
for f in source/*.c; do process "$f" "obj/${f##*/%.*}.obj"; done
Of course, the above doesn't work. Does anyone know something that will? I'm just trying to save myself some typing here.
Not the prettiest thing in the world, but you can use a regular expression to group the content you want to pick out, and then refer to the BASH_REMATCH array:
if [[ $f =~ ^source/(.*).c$ ]] ; then f="obj/${BASH_REMATCH[1]}.o"; fi
you shouldn't have to worry about your code being "wordier" or not. In fact, being a bit verbose is no harm, consider how much it will improve your(or someone else) understanding of the script. Besides, for performance, using bash's internal string manipulation is much faster than calling external commands. Lastly, you are not going to retype your commands every time you use it right? So why worry that its "wordier" since these commands are already in your script?
Not directly in bash. You can use sed, of course:
b="$(sed 's|^source/(.*).c$|obj/$1.obj|' <<< "$f")"
Why not simply using cd to remove the "source/" part?
This way we can avoid the temporary variables a and b:
for f in $(cd source; printf "%s\n" *.c); do
echo process "source/${f}" "obj/${f%.*}.obj"
done

Resources