I am looking for a shell script analog to something like Pythons's ConfigParser or Perl's Config::INI. I have sourced files in the past to accomplish this, but I'd prefer to read rather than execute my "config file". Does anyone know of anything comparable to the above modules available for shell (or bash) scripts?
Thanks,
Jerry
You don't want source it, so you should:
1.read the config, 2.verify lines 3.eval them
CONFIGFILE="/path/to/config"
echo "=$ADMIN= =$TODO= =$FILE=" #these variables are not defined here
eval $(sed '/:/!d;/^ *#/d;s/:/ /;' < "$CONFIGFILE" | while read -r key val
do
#verify here
#...
str="$key='$val'"
echo "$str"
done)
echo =$ADMIN= =$TODO= =$FILE= #here are defined
sample of config file
ADMIN: root
TODO: delete
var=badly_formtatted_line_without_colon
#comment
FILE: /path/to/file
if you run the above sample should get (not tested):
== == ==
=root= =delete= =/path/to/file=
sure this is not the best solution - maybe someone post a nicer one.
You might want to take a look at cfget which can be installed with sudo apt-get install cfget.
#!/bin/bash
# Author: CJ
# Date..: 01/03/2013
## sample INI file save below to a file, replace "^I" with tab
#^I [ SECTION ONE ]
#TOKEN_TWO^I ="Value1 two "
#TOKEN_ONE=Value1 One
#TOKEN_THREE=^I"Value1^I three" # a comment string
#TOKEN_FOUR^I=^I"^IValue1 four"
#
#[SECTION_TWO]
#TOKEN_ONE=Value1 One ^I^I^I# another comment string
#TOKEN_TWO^I ="Value1 two "
#TOKEN_THREE=^I"Value1^I three"
#TOKEN_FOUR^I=^I"^IValue1 four"
## sample INI file
export INI= # allows access to the parsed INI values in toto by children
iniParse() {
# Make word separator Linefeed(\n)
OIFS="${IFS}"
IFS=$(echo)
SECTION=_
while read LINE; do {
IFS="${OIFS}"
# Skip blank lines
TMP="$(echo "${LINE}"|sed -e "s/^[ \t]*//")"
if [ 0 -ne ${#TMP} ]; then
# Ignore comment lines
if [ '#' == "${LINE:0:1}" -o '*' == "${LINE:0:1}" ]; then
continue
fi # if [ '#' == "${LINE:0:1}" -o '*' == "${LINE:0:1}" ]; then
# Section label
if [ "[" == "${LINE:0:1}" ]; then
LINE="${LINE/[/}"
LINE="${LINE/]/}"
LINE="${LINE/ /_}"
SECTION=$(echo "${LINE}")_
else
LINE="$(echo "${LINE}"|sed -e "s/^[ \t]*//")"
LINE="$(echo "${LINE}"|cut -d# -f1)"
TOKEN="$(echo "${LINE:0}"|cut -d= -f1)"
EQOFS=${#TOKEN}
TOKEN="$(echo "${TOKEN}"|sed -e "s/[ \t]*//g")"
VALUE="${LINE:${EQOFS}}"
VALUE="$(echo "${VALUE}"|sed -e "s/^[ \t=]*//")"
VALUE="$(echo "${VALUE}"|sed -e "s/[ \t]*$//")"
if [ "${VALUE:0:1}" == '"' ]; then
echo -n "${SECTION}${TOKEN}=${VALUE}"
echo -e "\r"
else
echo -n "${SECTION}${TOKEN}="\"${VALUE}\"""
echo -e "\r"
fi # if [ "${VALUE:0:1}" == '"' ]; then
fi # if [ "[" == "${LINE:0:1}" ]; then
fi # if [ 0 -ne ${#TMP} ]; then
IFS=$(echo)
} done <<< "$1"
IFS="${OIFS}" # restore original IFS value
} # iniParse()
# call this function with the INI filespec
iniReader() {
if [ -z "$1" ]; then return 1; fi
TMPINI="$(<$1)"
TMPINI="$(echo "${TMPINI}"|sed -e "s/\r//g")"
TMPINI="$(echo "${TMPINI}"|sed -e "s/[ \t]*\[[ \t]*/[/g")"
TMPINI="$(echo "${TMPINI}"|sed -e "s/[ \t]*\][ \t]*/]/g")"
INI=`iniParse "${TMPINI}"`
INI="$(echo "${INI}"|sed -e "s/\r/\n/g")"
eval "${INI}"
return 0
} # iniReader() {
# sample usage
if iniReader $1 ; then
echo INI read, exit_code $? # exit_code == 0
cat <<< "${INI}"
cat <<< "${SECTION_ONE_TOKEN_FOUR}"
cat <<< "${SECTION_ONE_TOKEN_THREE}"
cat <<< "${SECTION_TWO_TOKEN_TWO}"
cat <<< "${SECTION_TWO_TOKEN_ONE}"
else
echo usage: $0 filename.ini
fi # if iniReader $1 ; then
grep based alternative seems to be more readable:
CONFIG_FILE='/your/config/file.ini'
eval $(grep '^\[\|^#' CONFIG_FILE -v | while read line
do echo $line
done)
Where:
-v grep option means exclude matching lines
^\[\|^# selects all lines which starts with [ or # (configparser sections and comments)
It will work ONLY if your config file doesn't have spaces around = (if you would like to generate config with Python use space_around_delimiters=False see https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.write)
Supported config example:
FIRST_VAR="a"
[lines started with [ will be ignored]
secondvar="b"
# some comment
anotherVar="c"
You can use bash it-self to interpret ini values, by:
$ source <(grep = file.ini)
Sample file:
[section-a]
var1=value1
var2=value2
See more examples: How do I grab an INI value within a shell script?
Or you can use bash ini-parser which can be found at The Old School DevOps blog site.
Related
i'm trying to generate a new output file from each existing file in a directory of .txt files. I want to check line by line in each file for two substrings. And append the lines that match that substring to each new output file.
I'm having trouble generating the new files.
This is what i currently have:
#!/bin/sh
# My first Script
success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
failure="(Compiling)\s\".*\"\s\-\s(Failed)"
count_success=0
count_failure=0
for i in ~/Documents/reports/*;
do
while read -r line;
do
if [[$success=~$line]]; then
echo $line >> output_$i
count_success++
elif [[$failure=~$]]; then
echo $line >> output_$i
count_failure++
fi
done
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
~
Any help would be appreciated, thanks
Please, use https://www.shellcheck.net/ to check your shell scripts.
If you use Visual Studio Code, you could install "ShellCheck" (by Timon Wong) extension.
About your porgram.
Assume bash
Define different extensions for input and output files (really important if there are in the same directory)
Loop on report, input, files only
Clear output file
Read input file
if sequence:
if [[ ... ]] with space after [[ and before ]]
spaces before and after operators (=~)
reverse operands order for operators =~
Prevent globbing with "..."
#! /bin/bash
# Input file extension
declare -r EXT_REPORT=".txt"
# Output file extension
declare -r EXT_OUTPUT=".output"
# RE
declare -r success="(Compiling)\s\".*\"\s\-\s(Succeeded)"
declare -r failure="(Compiling)\s\".*\"\s\-\s(Failed)"
# Counters
declare -i count_success=0
declare -i count_failure=0
for REPORT_FILE in ~/Documents/reports/*"${EXT_REPORT}"; do
# Clear output file
: > "${REPORT_FILE}${EXT_OUTPUT}"
# Read input file (see named file in "done" line)
while read -r line; do
# does the line match the success pattern ?
if [[ $line =~ $success ]]; then
echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
count_success+=1
# does the line match the failure pattern ?
elif [[ $line =~ $failure ]]; then
echo "$line" >> "${REPORT_FILE}${EXT_OUTPUT}"
count_failure+=1
fi
done < "$REPORT_FILE"
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
What about using grep?
success='Compiling\s".*"\s-\sSucceeded'
failure='Compiling\s".*"\s-\sFailed'
count_success=0
count_failure=0
for i in ~/Documents/reports/*; do
(( count_success += $(grep -E "$success" "$i" | tee "output_$i" | wc -l) ))
(( count_failure += $(grep -E "$failure" "$i" | tee -a "output_$i" | wc -l) ))
done
echo "$count_success of jobs ran succesfully"
echo "$count_failure of jobs didn't work"
I have an .xcconfig file that I want to access via the terminal to access variables in the file. Is there a command or is there some way to do this? For example I have a variable called Build_Code = 1234. How would I access that?
Create a script to read the value of a variable.
Ex: .xconfig
var1 = value1
var2 = value2
get_value.bash
#!/bin/bash
#
# get_value.bash <file> <variable>
#
usage()
{
echo "Usage: get_value.bash <file> <variable>"
exit 1
}
#################################################
# Arguments
if [[ $# -eq 2 ]]
then
file="$1"
var="$2"
else
usage
fi
# Check if the file exists
if [[ ! -f "$file" ]]
then
echo "ERROR: file $file does not exist."
exit 2
fi
# Get the variable's value
grep -w "$var" "$file" | cut -d'=' -f2 | tr -d ' '
This simple version assumes the format of the lines is VARIABLE\s*=\s*VALUE.
The tr is to remove spaces around the value.
The VALUE cannot contain spaces.
The <file> argument could be hard coded if you will only ever check .xconfig
Many other solutions could be conceived, depending on the exact requirements, but this does the basic need you put in your question.
This question already has an answer here:
bash4 read file into associative array
(1 answer)
Closed 3 years ago.
I'm currently creating a list of commands so for example by saying "directory install plugin-name" I can install all needed plugins specified in an external list. This list is just a txt file with all plugin names. But I'm struggling getting all names in an associative array.
I've tried this one:
while IFS=";" read line;
do " communtyList[ $line ]=1 " ;
done < community-list.txt;
The desired output should be
communityList[test1]=1
communityList[test2]=1....
It need to be an associative array because I want to access it by words and not by index. This word will be implemented as parameters/arguments.
For example "install plugin" instead of "1 plugin"
So I can ask for example this way:
if [ ! -z "${!communtyList[$2]}" ];
Update, here the whole code:
#!/usr/bin/env bash
community(){
declare -A communtyList
while IFS= read line;
do communtyList[$line]=1 ;
done < community-list.txt;
# communtyList[test1]=1
# communtyList[test2]=1
# communtyList[test3]=1
# communtyList[test4]=1
if { [ $1 = 'install' ] || [ $1 = 'activate' ] || [ $1 = 'uninstall' ] || [ $1 = 'deactivate' ] ; } && [ ! -z $2 ] ; then
if [ $2 = 'all' ];
then echo "$1 all community plugins....";
while IFS= read -r line; do echo "$1 $line "; done < community-list.txt;
elif [ ! -z "${!communtyList[$2]}" ];
then echo "$1 community plugin '$2'....";
else
echo -e "\033[0;31m Something went wrong";
echo " Plugin '$2' does not exist.";
echo " Here a list of all available community plugins: ";
echo ${!communtyList[#]}
echo -e " \e[m"
fi
else
echo -e "\033[0;31m Something went wrong";
if [ -z $2 ];
then echo -e "[Plugin name] required. [community][action][plugin name] \e[m"
else
echo " Action '$1' does not exist.";
echo -e " Do you mean some of this? \n install \n activate \n uninstall \e[m"
fi
fi
echo ${!communtyList[#]}
}
"$#"
To use asociative array you have to declare it first
declare -A communityList
Then you can add values
communityList[test1]=1
communityList[test2]=2
...
Or with the declaration
declare -A communityList=(
communityList[test1]=1
communityList[test2]=2
...
)
The quotes around " communtyList[ $line ]=1 " mean you try to evaluate a command whose first character is a space. You want to take out those quotes, and probably put quotes around "$line" instead.
It's also unclear why you have IFS=";" -- you are not splitting the line into fields anyway, so this is not doing anything useful. Are there semicolons in your input file? Where and why; what do they mean?
You should probably prefer read -r unless you specifically require read to do odd things with backslashes in the input.
Finally, as suggested by Ivan, you have to declare the array's type as associative before you try to use it.
With those things out of the way, try
declare -A communityList
while read -r line; do
communtyList["$line"]=1
done < community-list.txt
I have a function with a parameter file. And I want to read it line by line.
Condition
If the lines are between <?bash and ?> then I do bash -c '$line' else I display the line.
Here my file (file):
<html><head></head><body><p>Hello
<?bash
echo "world !"
?>
</p></body></html>
Here my Bash script (bashtml):
#!/bin/bash
function generation()
{
while read line
do
if [ $line = '<?bash' ]
then
while [ $line != '?>' ]
do
bash -c '$line'
done
else
echo $line
fi
done
}
generation $file
I execute this script:
./bashhtml
I am novice in Bash script and I'm lost
I think this is what you mean. However, this code is HIGHLY DANGEROUS! Any command inserted into those bash tags would be executed under your user id. It could change your password, delete all your files, read or alter data, and so on. Don't do it!
#!/bin/bash
function generation
{
# If you don't use local (or declare) then variables are global
local file="$1" # Parameter passed to function, in a local variable
local start=False # A flag to indicate tags
local line
while read -r line
do
if [[ $line == '<?bash' ]]
then
start=True
elif [[ $line == '?>' ]]
then
start=False
elif "$start"
then
bash -c "$line" # Double quotes needed here
else
echo "$line"
fi
done < "$file" # Notice how the filename is redirected into read
}
infile="$1" # This gets the filename from the command-line
generation "$infile" # This calls the function
A text file has the following structure:
paa pee pii poo puu
baa bee bii boo buu
gaa gee gii goo guu
maa mee mii moo muu
Reading it line by line in a script is done with
while read LINE; do
ACTION
done < FILE
I'd need to get parameters 3 and 4 of each line into variables for ACTION. If this was manual input, $3 and $4 would do the trick. I assume awk is the tool, but I just can't wrap my head around the syntax. Halp?
read does this just fine. Pass it multiple variables and it will split on $IFS into that many fields.
while read -r one two three four five; do
action "$three" "$four"
done <file
I added the -r option because that is usually what you want. The default behavior is a legacy oddity of limited use.
Thanks tripleee. In the meantime I managed a suitably versatile solution:
#!/bin/sh
if [ ! $1 ]; then
echo "Which inputfile?"
exit
elif [ ! $2 -o ! $3 ]; then
echo "Two position parameters required"
exit
fi
if [ -f outfile ]; then
mv outfile outfile.old
fi
while read -a LINE; do
STRING="${LINE[#]}"
if [ ${LINE[$2-1]} == ${LINE[$3-1]} ]; then # remove comment for strings
# if [ ${LINE[$(($2-1))]} -eq ${LINE[$(($3-1))]} ]; then # remove comment for integers
echo $STRING >> outfile
fi
done < $1