My property file:
a.prop
user=abc
location=home
user=xyz
location=roamer
I need to read a.prop and keep user and location inside a variable so that I can pass them to my other script (check.sh) as an argument.
The check.sh needs to be called for all the list of user/location.
I don't want to use AWK
Here's an extremely fragile, ill-advised solution that invokes a function for each stanza of your config, but requires the exact format of your input and is vulnerable to many, many attack vectors. Use (or, preferrably not) at your own risk!
#!/bin/bash
foo() { echo "user is $user, location is $location"; }
eval "$(sed -e '/^$/s//foo/' input; echo foo)"
You would be much better off pre-processing the input, and a willingness to use awk would be helpful.
untested
while read -r line; do
key=${line%%=*} # the left-hand-side of the =
case $key in
user) user=${line#*=} ;;
location) location=${line#*=} ;;
*) continue ;; # skip this line
esac
if [[ -n $user ]] && [[ -n $location ]]; then
echo "have user=$user and location=$location"
check.sh "$user" "$location"
unset user location
fi
done < a.prop
This version is a little unsafer: just assume the properties are valid shell variable assignments.
while read -r line; do
[[ $line != *=* ]] && continue
declare "$line"
if [[ -n $user ]] && [[ -n $location ]]; then
echo "have user=$user and location=$location"
check.sh "$user" "$location"
unset user location
fi
done < a.prop
Or, assuming "user" always appears before "location"
grep -E '^(user|location)=' a.prop |
while read userline; read locline; do
declare "$userline"
declare "$locline"
echo "have user=$user and location=$location"
check.sh "$user" "$location"
done
Related
I am trying to review information about functions and aliases and use this function to show me the definitions of them. However, it does not show just the functions / aliases defined by my profile scripts, and shows a lot of other things (that I am not so interested in).
Would there be a way to just show the functions / aliases that were specifically setup in .bashrc .profile etc (i.e. user defined functions / aliases)?
Would it be possible to see the date/time that the function or aliase was defined in this shell and would it be possible to see the source like .bashrc or .profile etc (aybe such information is not retained, but all command history date/time stamps are so it's possible I guess)?
def ()
{
if [ -z "$1" ]; then
declare -F;
printf "\nAbove listing is all defined functions 'declare -F' (use def <func-name> to show function contents)\nType 'alias' to show all aliases (def <alias-nam> to show alias definition, where 'def' uses 'command -V <name>')\n\n";
else
command -V $1;
fi
}
One approach, caching the list of known functions at the top of your dotfiles (so anything not so cached can be considered as unknown):
declare -A -g predefined_commands
populate_predefined_commands() {
local line fndec_re alias_re
fndec_re='^declare -f ([^[:space:]]+)$'
alias_re='^alias ([^[:space:]=])+='
while IFS= read -r line; do
[[ $line =~ $fndec_re ]] && predefined_commands[${BASH_REMATCH[1]}]=1
done < <(declare -F)
while IFS= read -r line; do
[[ $line =~ $alias_re ]] && predefined_commands[${BASH_REMATCH[1]}]=1
done < <(alias -p)
}
def() {
if [[ $1 ]]; then
command -V -- "$1"
return
fi
local line fndec_re alias_re
fndec_re='^declare -f ([^[:space:]]+)$'
alias_re='^alias ([^[:space:]=])+='
while IFS= read -r line; do
[[ $line =~ $fndec_re ]] && [[ ${predefined_commands[${BASH_REMATCH[1]}]} ]] && continue
printf '%s\n' "$line"
done < <(declare -F)
while IFS= read -r line; do
[[ $line =~ $alias_re ]] && [[ ${predefined_commands[${BASH_REMATCH[1]}]} ]] && continue
printf '%s\n' "$line"
done < <(alias -p)
}
# do this AFTER calling def, or else def itself will show up as a new function
populate_predefined_commands; unset -f populate_predefined_commands
I am trying to read a parameter file in a shell script and would want to skip the lines which start with "#". Have been trying it on Ubuntu VM (default bash) and for something that I can't understand, it doesn't seem to work.
Following is the pseudo-code that I am using:
while read line
do
if [ grep -q "#" <<< "$line" ]; then
## Do nothing (Commented Out)
echo "$line Line is Commented out"
elif [ "$line" = "" ]; then
## Do nothing (Blank Line)
echo "Blank line"
else
#echo "read line is $line"
...some logic here
fi
done <input_file.ini
This yields the the following error: Syntax error: redirection unexpected
The if [[ $line == *#* ]] construct doesn't seem to work. My earlier experience was on AIX where everything worked fine.
Could someone guide me what I am doing wrong here?
PS: On a related note, how do I handle cases where I don't want to do anything? e.g. when there is no '#' character in the read line, I don't want to do anything. I can't leave my if block blank so I am just using echo 'some random' text. My task works good but just wanted to understand what's a good practice to handle this.
Your code is clearly running with /bin/sh, not bash.
An alternative to [[ $line = *"#"* ]] that works with /bin/sh is case.
Thus, the following will work with /bin/sh, or when invoked with sh yourscript:
#!/bin/sh
while read -r line; do : line="$line"
case $line in
*"#"*) echo "Line is commented out: $line";;
"") echo "Line is empty" ;;
*) key=${line%%=*}
value=${line#*=}
eval "$key="'$line' # unsafe, but works with /bin/sh, which doesn't have better
# indirect assignment approaches.
printf '%s\t\t-\t\t%s\n' "$key" "$value"
;;
esac
done <input_file.ini
Alternately, consider putting in a guard to handle the case when your script is invoked with a non-bash shell:
#!/bin/bash
case $BASH_VERSION in
'')
echo "ERROR: Run with a non-bash shell" >&2
if [ "$tried_reexec" ]; then
echo "ERROR: Already attempted reexec and failed" >&2
exit 1
fi
if [ -s "$0" ]; then
export tried_reexec=1
exec bash "$0" "$#"
fi
;;
esac
while read -r line; do
if [[ $line = *"#"* ]]; then
echo "Line is Commented out: $line"
elif [[ "$line" = "" ]]; then
echo "Blank line"
else
key=${line%%=*}; value=${line#*=}
printf -v "$key" %s "$value"
printf '%s\t\t-\t\t%s\n' "$key" "$value"
fi
done <input_file.ini
I really wasn't able to figure out the exact problem with double [[ ]] and the character search in a string. Thanks to everyone who tried to help me out. However this was acting as a deterrent and I didn't want to continue to fiddle for too long, I used a slightly different approach to handle my situation.
The following code works for me now:
while read line
do
first_char=`echo $line | cut -c 1`
if [ "$first_char" = "#" ]; then
: "do nothing here. Line is commented out"
elif [ "$line" = "" ]; then
: "do nothing here. Blank line"
else
KEY="$(echo $line | cut -d '=' -f1)"
VALUE="$(echo $line | cut -d '=' -f2)"
printf \v "$KEY" %s "$VALUE"
echo "$KEY\t\t-\t\t$VALUE"
fi
done < ${SCHEDULER_LOC}/inputs/script_params.ini
Also I was able to learn few things so incorporated them as well. I did get few negative scores for this question. Understandably so since this might be rudimentary for the experts but it was a genuine problem I was seeking some guidance on. Still, I am thankful that I learnt something new. Kudos to the community.
I have a password function that I borrowed from How do I echo stars (*) when reading password with read?
I tried to adapt it so that I can run through the function twice to do a password confirmation and then evaluate the 2 passwords to determine if they match but I seem to be missing some basics of how bash works in this case.
I tried replacing PASSWORD with $1 but kept getting command not found errors
passWord() {
unset PASSWORD
unset CHARCOUNT
stty -echo
CHARCOUNT=0
while IFS= read -p "$PROMPT" -r -s -n 1 CHAR; do
# Enter - accept password
if [[ $CHAR == $'\0' ]] ; then
break
fi
# Backspace
if [[ $CHAR == $'\177' ]] ; then
if [ $CHARCOUNT -gt 0 ] ; then
CHARCOUNT=$((CHARCOUNT-1))
PROMPT=$'\b \b'
PASSWORD="${PASSWORD%?}"
else
PROMPT=''
fi
else
CHARCOUNT=$((CHARCOUNT+1))
PROMPT='*'
PASSWORD+="$CHAR"
fi
done
stty echo; echo
${1}=${PASSWORD}
}
echo -n "Enter the password > "
passWord passOne
echo -n "Please re-enter the password > "
passWord passTwo
if [[ $passOne == $passTwo ]]; then
PASSWORD=$passOne
else
echo "Passwords did not match, please try again."
fi
Update
Here is the script with the latest updates
#!/bin/bash
passWord() {
unset password
local prompt char
stty -echo
charcount=0
while IFS= read -p "$prompt" -r -s -n 1 CHAR; do
# Enter - accept password
if [[ $char == $'\0' ]] ; then
break
fi
# Backspace
if [[ $char == $'\177' ]] ; then
if [ $charcount -gt 0 ] ; then
charcount=$((CHARCOUNT-1))
prompt=$'\b \b'
password="${password%?}"
else
prompt=''
fi
else
charcount=$((charcount+1))
prompt='*'
password+="$char"
fi
done
stty echo; echo
}
echo -n "Enter the password > "
passWord
pass1=$password
echo -n "Please re-enter the password > "
passWord
pass2=$password
if [[ "$pass1" == "$pass2" ]]; then
PassWord=$pass1
else
echo "Passwords did not match, please try again."
fi
You are missing a declaration of your shell.
Please add a shebang as the first line:
#!/bin/bash
The assignment of variables (the line ${1}=${PASSWORD}) doesn't work.
One way to solve it (not recomended) is to add eval:
eval "${1}=${PASSWORD}" # don't use quite risky.
But as that makes any input a security issue, you should use some other line.
One solution is to use declare (bash 4.2+):
declare -g "${1}=${PASSWORD}"
The -g is required (required and available since bash 4.2) to change General variables (not local to the function).
Or use printf (since bash 3.1):
printf -v "${1}" '%s' "${PASSWORD}"
Other than that, you should add a local command for variables used inside the function to avoid conflicts with external variables and should add a PROMPT='' just before the loop to avoid the printing of an initial asterisk when calling the function a second time.
It should be said that using variables in CAPS should be avoided. Variables in CAPS denote environment variables, the rest of variables use lower case to avoid conflicts.
Latest Version
#!/bin/bash
set -e
shopt -s nocasematch
#vars
redbgbold='\e[1;97;41m'
resetcolor='\e[0m'
RegExFQDN='(?=^.{4,253}$)(^((?!-)[a-zA-Z0-9-]{1,63}(?<!-)\.)+[a-zA-Z]{2,63}$)'
#functions
ask() {
local input
until
read -rp "$1 > " input >&2 || return 1
grep -q -P "$2" <<< "$input"
do
printf "ERROR - "${redbgbold}"\"$input\""${resetcolor}" is not a valid " >&2; sed "s/.*the //" <<< "$1" >&2
done
printf '%s\n' "$input"
}
#code
while [ -z $fqdn ]; do
fqdn=$(ask "Enter the FQDN" $RegExFQDN)
echo "FQDN is $fqdn"
done
The Question
I have a read line, and I want to take what the user entered and see if it matches my regex, if it matches we leave the loop, if it fails it prints an error and we do the loop again until we get a match. It looks redundant to me, and I assume there should be a better way but not sure what that should be.
Original Code
#!/bin/bash
set -e
shopt -s nocasematch
function RegexValidation() {
if [ "$2" = "fqdn" ]; then
if [ `echo $1 | grep -c -P '(?=^.{1,254}$)(^(?>(?!\d+\.)[a-z0-9_\-]{1,63}\.?)+(?:[a-z]{2,})$)'` == "0" ]; then
echo "ERROR - $1 is not a valid FQDN"
unset $!{1}
fi
fi
}
while [ -z $fqdn ]; do
read -e -r -p "Enter the Fully Qualified Domain Name > " fqdn
RegexValidation $fqdn fqdn
done
shopt -u nocasematch
any help is appreciated.
Update #1 - fixed formatting issues.
Update #2 - using that other guy's suggestions with a few additional tweaks
I would do basically the same thing, but split it differently to make it easier to reuse:
#!/bin/bash
set -e
ask() {
local input
until
read -rp "$1 > " input >&2 || return 1
grep -q -P "$2" <<< "$input"
do
echo "Invalid answer. Try again" >&2
done
printf '%s\n' "$input"
}
ask_fqdn() {
ask "$1" '(?=^.{1,254}$)(^(?>(?!\d+\.)[a-z0-9_\-]{1,63}\.?)+(?:[a-z]{2,})$)'
}
fqdn=$(ask_fqdn "Enter first FQDN")
echo "You wrote $fqdn"
fqdn=$(ask_fqdn "Enter second FQDN")
echo "This time it was $fqdn"
number=$(ask "And now a number because why not" '^\d+$')
echo "So $number"
Now you don't have to write a new loop every time you want new information, and you can easily ask for new things without modifying the existing functions.
Have the function return a status, which you can test with if in the loop.
And rather than use test to check the result of grep, just test it directly with if. grep returns a non-zero status if the input doesn't match.
function RegexValidation() {
if [ "$2" = "fqdn" ]; then
if ! echo "$1" | grep -q -P '(?=^.{1,254}$)(^(?>(?!\d+\.)[a-z0-9_\-]{1,63}\.?)+(?:[a-z]{2,})$)'; then
echo "ERROR - $1 is not a valid FQDN"
return 1
fi
return 0
fi
}
while :; do
read -e -r -p "Enter the Fully Qualified Domain Name > " fqdn
if RegexValidation "$fqdn" fqdn
then break
fi
done
Also, remember to quote your variables.
The script is:
#!/bin/bash
# Dynamic Menu Function
createmenu () {
select selected_option; do # in "$#" is the default
if [ 1 -le "$REPLY" ] && [ "$REPLY" -le $(($#)) ]; then
break;
else
echo "Please make a vaild selection (1-$#)."
fi
done
}
declare -a drives=();
# Load Menu by Line of Returned Command
mapfile -t drives < <(lsblk --nodeps -o name,serial,size | grep "sd");
# Display Menu and Prompt for Input
echo "Available Drives (Please select one):";
createmenu "${drives[#]}"
# Split Selected Option into Array and Display
drive=($(echo "${selected_option}"));
echo "Drive Id: ${drive[0]}";
echo "Serial Number: ${drive[1]}";
The older system doesn't have mapfile or readarray so I need to convert that line to some alternative that can read each line of the lsblk output into an array.
The line in question that creates the array is:
mapfile -t drives < <(lsblk --nodeps -o name,serial,size | grep "sd");
You can loop over your input and append to the array:
$ while IFS= read -r line; do arr+=("$line"); done < <(printf '%d\n' {0..5})
$ declare -p arr
declare -a arr='([0]="0" [1]="1" [2]="2" [3]="3" [4]="4" [5]="5")'
Or, for your specific case:
while IFS= read -r line; do
drives+=("$line")
done < <(lsblk --nodeps -o name,serial,size | grep "sd")
See the BashFAQ/001 for an excellent explanation why IFS= read -r is a good idea: it makes sure that whitespace is conserved and backslash sequences not interpreted.
Here's the solution I came up with a while back. This is better because it provides a substitute function for older versions of Bash that don't support mapfile/readarray.
if ! type -t readarray >/dev/null; then
# Very minimal readarray implementation using read. Does NOT work with lines that contain double-quotes due to eval()
readarray() {
local cmd opt t v=MAPFILE
while [ -n "$1" ]; do
case "$1" in
-h|--help) echo "minimal substitute readarray for older bash"; exit; ;;
-r) shift; opt="$opt -r"; ;;
-t) shift; t=1; ;;
-u)
shift;
if [ -n "$1" ]; then
opt="$opt -u $1";
shift
fi
;;
*)
if [[ "$1" =~ ^[A-Za-z_]+$ ]]; then
v="$1"
shift
else
echo -en "${C_BOLD}${C_RED}Error: ${C_RESET}Unknown option: '$1'\n" 1>&2
exit
fi
;;
esac
done
cmd="read $opt"
eval "$v=()"
while IFS= eval "$cmd line"; do
line=$(echo "$line" | sed -e "s#\([\"\`]\)#\\\\\1#g" )
eval "${v}+=(\"$line\")"
done
}
fi
You don't have to change your code one bit. It just works!
readarray -t services -u < <(lsblk --nodeps -o name,serial,size | grep "sd")
For those playing along at home, this one aims to provide a mapfile that's feature-compliant with Bash 5, but still runs as far back as Bash 3.x:
#!/usr/bin/env bash
if ! (enable | grep -q 'enable mapfile'); then
function mapfile() {
local DELIM="${DELIM-$'\n'}"; opt_d() { DELIM="$1"; }
local COUNT="${COUNT-"0"}"; opt_n() { COUNT="$1"; }
local ORIGIN="${ORIGIN-"0"}"; opt_O() { ORIGIN="$1"; }
local SKIP="${SKIP-"0"}"; opt_s() { SKIP="$1"; }
local STRIP="${STRIP-"0"}"; opt_t() { STRIP=1; }
local FROM_FD="${FROM_FD-"0"}"; opt_u() { FROM_FD="$1"; }
local CALLBACK="${CALLBACK-}"; opt_C() { CALLBACK="$1"; }
local QUANTUM="${QUANTUM-"5000"}"; opt_c() { QUANTUM="$1"; }
unset OPTIND; local extra_args=()
while getopts ":d:n:O:s:tu:C:c:" opt; do
case "$opt" in
:) echo "${FUNCNAME[0]}: option '-$OPTARG' requires an argument" >&2; exit 1 ;;
\?) echo "${FUNCNAME[0]}: ignoring unknown argument '-$OPTARG'" >&2 ;;
?) "opt_${opt}" "$OPTARG" ;;
esac
done
shift "$((OPTIND - 1))"; set -- ${extra_args[#]+"${extra_args[#]}"} "$#"
local var="${1:-MAPFILE}"
### Bash 3.x doesn't have `declare -g` for "global" scope...
eval "$(printf "%q" "$var")=()" 2>/dev/null || { echo "${FUNCNAME[0]}: '$var': not a valid identifier" >&2; exit 1; }
local __skip="${SKIP:-0}" __counter="${ORIGIN:-0}" __count="${COUNT:-0}" __read="0"
### `while read; do...` has trouble when there's no final newline,
### and we use `$REPLY` rather than providing a variable to preserve
### leading/trailing whitespace...
while true; do
if read -d "$DELIM" -r <&"$FROM_FD"
then [[ ${STRIP:-0} -ge 1 ]] || REPLY="$REPLY$DELIM"
elif [[ -z $REPLY ]]; then break
fi
(( __skip-- <= 0 )) || continue
(( COUNT <= 0 || __count-- > 0 )) || break
### Yes, eval'ing untrusted content is insecure, but `mapfile` allows it...
if [[ -n $CALLBACK ]] && (( QUANTUM > 0 && ++__read % QUANTUM == 0 ))
then eval "$CALLBACK $__counter $(printf "%q" "$REPLY")"; fi
### Bash 3.x doesn't allow `printf -v foo[0]`...
### and `read -r foo[0]` mucks with whitespace
eval "${var}[$((__counter++))]=$(printf "%q" "$REPLY")"
done
}
### Alias `readarray` as well...
readarray() { mapfile "$#"; }
fi
if [[ -z ${PS1+YES} ]]; then
echo "'mapfile' should only be called as a shell function; try \"source ${BASH_SOURCE[0]##*/}\" first..." >&2
exit 1
fi