A text file has the following structure:
paa pee pii poo puu
baa bee bii boo buu
gaa gee gii goo guu
maa mee mii moo muu
Reading it line by line in a script is done with
while read LINE; do
ACTION
done < FILE
I'd need to get parameters 3 and 4 of each line into variables for ACTION. If this was manual input, $3 and $4 would do the trick. I assume awk is the tool, but I just can't wrap my head around the syntax. Halp?
read does this just fine. Pass it multiple variables and it will split on $IFS into that many fields.
while read -r one two three four five; do
action "$three" "$four"
done <file
I added the -r option because that is usually what you want. The default behavior is a legacy oddity of limited use.
Thanks tripleee. In the meantime I managed a suitably versatile solution:
#!/bin/sh
if [ ! $1 ]; then
echo "Which inputfile?"
exit
elif [ ! $2 -o ! $3 ]; then
echo "Two position parameters required"
exit
fi
if [ -f outfile ]; then
mv outfile outfile.old
fi
while read -a LINE; do
STRING="${LINE[#]}"
if [ ${LINE[$2-1]} == ${LINE[$3-1]} ]; then # remove comment for strings
# if [ ${LINE[$(($2-1))]} -eq ${LINE[$(($3-1))]} ]; then # remove comment for integers
echo $STRING >> outfile
fi
done < $1
Related
I would like to print a number of variables in a tab-delimited manor, and for the sake of code cleanliness, I would like it write it like the below:
for f in files*
do
echo -e "
$var1\t
$var2\t
$var3\t
$var4\t
"
done
output:
file1_var1 file1_var2 file1_var3 file1_var4
file2_var1 file2_var2 file2_var3 file2_var4
I can get it to work like this, but it is not very tidy:
echo -e "$var1\t$var2\t$var3\t$var4\t"
I have tried using something like this to replace the new lines with tabs, but it seems a bit cumbersome. Also I need to add a new line at the end to prevent multiple files being printed on the same line, and it adds a new line between each output line as well.
for i in files*
do
echo -e "
$var1\t
$var2\t
$var3\t
$var4\t
| sed '$!{:a;N;s/\n/\t/;ta}'
"
echo -e "\n"
done
output:
file1_var1 file1_var2 file1_var3 file1_var4
file2_var1 file2_var2 file2_var3 file2_var4
Basically never use echo -e. You can almost always make it more elegant with printf, or simply cat
If you don't mind getting one spurious tab,
printf "%s\t" "$var1" "$var2" "$var3" "$var4"
printf '\n'
With a here document, you can write a simple sed script to replace all internal newlines with tabs.
sed 'N;N;N;s/\n/\t/g' <<:
$var1
$var2
$var3
$var4
:
(This is probably not entirely portable, but it works on MacOS and Ubuntu, so it should work most places you care about.)
Perhaps also look into pr, which can arrange things in a matrix.
printf "%s\n" "$var1" "$var2" "$var3" "$var4" | pr -bt4
(Again, probably not entirely portable, but works e.g. on Linux. On MacOS I had to remove the b option.)
It's not clear where the variables should come from; perhaps write a function to wrap printf if you just want to print its arguments in groups of four:
fourcols () {
while [ $# -gt 4 ]; do
fourcols "$1" "$2" "$3" "$4"
shift 4 # not portable to sh
done
while [ $# -gt 0 ]; do
printf "%s" "$1"
shift
if [ $# -gt 0 ]; then
printf "\t%s" "$#"
done
printf "\n"
done
}
Then you can just call fourcols * if you want to print all files in four columns, for example.
Defining a function that joins its arguments as a TSV record:
tsvrecord() { first=$1; shift && printf '%s' "$first" "${#/#/$'\t'}"; echo; }
tsvrecord "$var1" "$var2" "$var3" "$var4"
remark: no TSV escaping is done here, so if a filepath contains \t or \n (which is allowed on Unix) then the resulting TSV will be broken.
This question already has an answer here:
bash4 read file into associative array
(1 answer)
Closed 3 years ago.
I'm currently creating a list of commands so for example by saying "directory install plugin-name" I can install all needed plugins specified in an external list. This list is just a txt file with all plugin names. But I'm struggling getting all names in an associative array.
I've tried this one:
while IFS=";" read line;
do " communtyList[ $line ]=1 " ;
done < community-list.txt;
The desired output should be
communityList[test1]=1
communityList[test2]=1....
It need to be an associative array because I want to access it by words and not by index. This word will be implemented as parameters/arguments.
For example "install plugin" instead of "1 plugin"
So I can ask for example this way:
if [ ! -z "${!communtyList[$2]}" ];
Update, here the whole code:
#!/usr/bin/env bash
community(){
declare -A communtyList
while IFS= read line;
do communtyList[$line]=1 ;
done < community-list.txt;
# communtyList[test1]=1
# communtyList[test2]=1
# communtyList[test3]=1
# communtyList[test4]=1
if { [ $1 = 'install' ] || [ $1 = 'activate' ] || [ $1 = 'uninstall' ] || [ $1 = 'deactivate' ] ; } && [ ! -z $2 ] ; then
if [ $2 = 'all' ];
then echo "$1 all community plugins....";
while IFS= read -r line; do echo "$1 $line "; done < community-list.txt;
elif [ ! -z "${!communtyList[$2]}" ];
then echo "$1 community plugin '$2'....";
else
echo -e "\033[0;31m Something went wrong";
echo " Plugin '$2' does not exist.";
echo " Here a list of all available community plugins: ";
echo ${!communtyList[#]}
echo -e " \e[m"
fi
else
echo -e "\033[0;31m Something went wrong";
if [ -z $2 ];
then echo -e "[Plugin name] required. [community][action][plugin name] \e[m"
else
echo " Action '$1' does not exist.";
echo -e " Do you mean some of this? \n install \n activate \n uninstall \e[m"
fi
fi
echo ${!communtyList[#]}
}
"$#"
To use asociative array you have to declare it first
declare -A communityList
Then you can add values
communityList[test1]=1
communityList[test2]=2
...
Or with the declaration
declare -A communityList=(
communityList[test1]=1
communityList[test2]=2
...
)
The quotes around " communtyList[ $line ]=1 " mean you try to evaluate a command whose first character is a space. You want to take out those quotes, and probably put quotes around "$line" instead.
It's also unclear why you have IFS=";" -- you are not splitting the line into fields anyway, so this is not doing anything useful. Are there semicolons in your input file? Where and why; what do they mean?
You should probably prefer read -r unless you specifically require read to do odd things with backslashes in the input.
Finally, as suggested by Ivan, you have to declare the array's type as associative before you try to use it.
With those things out of the way, try
declare -A communityList
while read -r line; do
communtyList["$line"]=1
done < community-list.txt
The output comes from a command I run from our netscaler. It outputs the following ... One thing to note is that the middle two numbers change but the even/odd criteria is always on the last digit. We never have more than 2 digits, so we'll never hit 10.
WC-01-WEB1
WC-01-WEB4
WC-01-WEB3
WC-01-WEB5
WC-01-WEB8
I need to populate a file called "even" and "odds." If we're dealing with numbers I can figure it out, but having the number within a string is throwing me off.
Example code but I'm missing the part where I need to match the string.
if [ $even_servers -eq 0 ]
then
echo $line >> evenfile
else
echo $line >> oddfile
fi
This is a simple awk command:
awk '/[02468]$/{print > "evenfile"}; /[13579]$/{print > "oddfile"}' input.txt
There must be better way.
How about this version:
for v in `cat <my_file>`; do export type=`echo $v | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $v >> evenfile ; else echo $v >> oddfile; fi; done
I assume your list of servers is stored in the filename <my_file>. The basic idea is to tokenize on WEB using awk and process the chars after WEB to determine even-ness. Once this is known, we export the value to a variable type and use this to selectively dump to the appropriate file.
For the case when the name is the output of another command:
export var=`<another command>`; export type=`echo $var | awk -F 'WEB' '{print $2%2}'`; if [ $type -eq 0 ]; then echo $var >> evenfile ; else echo $var >> oddfile; fi;
Replace <another command> with your perl script.
As always grep is your friend:
grep "[2468]$" input_file > evenfile
grep "[^2468]$" input_file > oddfile
I hope this helps.
I've spent 2 hours with an if statement, that never works like I want:
#should return true
if [ "$1" == "355258054414904" ]; then
Here is the whole script:
#!/bin/bash
param=$1
INPUT=simu_900_imei_user_pass.csv
OLDIFS=$IFS
IFS=,
[ ! -f $INPUT ] && { echo "$INPUT ime not found"; exit 99; }
while read imei email pass
do
echo "First Parameter-IMEI: $1"
if [ "$1" == "355258054414904" ]; then
echo "GOOD"
fi
done < $INPUT
IFS=$OLDIFS
This is the output of the script:
First Parameter-IMEI: 355258054414904
First Parameter-IMEI: 355258054414904
First Parameter-IMEI: 355258054414904
I have seen a lot of pages about the subject, but I can't make it work :(
EDIT: I Join the content of csv for better understanding ! Tx for your help !
4790057be1803096,user1,pass1
355258054414904,juju,capp
4790057be1803096,user2,pass2
358854053154579,user3,pass3
The reason $1 does not match is because $1 means the first parameter given to the script on the command line, while you want it to match the first field read from the file. That value is in $imei.
You probably meant:
if [ "$imei" == "355258054414904" ]; then
echo "GOOD"
fi
Since it is inside the loop where you read input file line by line.
To check content of $1 use:
cat -vet <<< "$1"
UPDATE: To strip \r from $1 have this at top:
param=$(tr -d '\r' <<< "$1")
And then use "$param" in rest of your script.
To test string equality with [ you want to use a single '=' sign.
I am looking for a shell script analog to something like Pythons's ConfigParser or Perl's Config::INI. I have sourced files in the past to accomplish this, but I'd prefer to read rather than execute my "config file". Does anyone know of anything comparable to the above modules available for shell (or bash) scripts?
Thanks,
Jerry
You don't want source it, so you should:
1.read the config, 2.verify lines 3.eval them
CONFIGFILE="/path/to/config"
echo "=$ADMIN= =$TODO= =$FILE=" #these variables are not defined here
eval $(sed '/:/!d;/^ *#/d;s/:/ /;' < "$CONFIGFILE" | while read -r key val
do
#verify here
#...
str="$key='$val'"
echo "$str"
done)
echo =$ADMIN= =$TODO= =$FILE= #here are defined
sample of config file
ADMIN: root
TODO: delete
var=badly_formtatted_line_without_colon
#comment
FILE: /path/to/file
if you run the above sample should get (not tested):
== == ==
=root= =delete= =/path/to/file=
sure this is not the best solution - maybe someone post a nicer one.
You might want to take a look at cfget which can be installed with sudo apt-get install cfget.
#!/bin/bash
# Author: CJ
# Date..: 01/03/2013
## sample INI file save below to a file, replace "^I" with tab
#^I [ SECTION ONE ]
#TOKEN_TWO^I ="Value1 two "
#TOKEN_ONE=Value1 One
#TOKEN_THREE=^I"Value1^I three" # a comment string
#TOKEN_FOUR^I=^I"^IValue1 four"
#
#[SECTION_TWO]
#TOKEN_ONE=Value1 One ^I^I^I# another comment string
#TOKEN_TWO^I ="Value1 two "
#TOKEN_THREE=^I"Value1^I three"
#TOKEN_FOUR^I=^I"^IValue1 four"
## sample INI file
export INI= # allows access to the parsed INI values in toto by children
iniParse() {
# Make word separator Linefeed(\n)
OIFS="${IFS}"
IFS=$(echo)
SECTION=_
while read LINE; do {
IFS="${OIFS}"
# Skip blank lines
TMP="$(echo "${LINE}"|sed -e "s/^[ \t]*//")"
if [ 0 -ne ${#TMP} ]; then
# Ignore comment lines
if [ '#' == "${LINE:0:1}" -o '*' == "${LINE:0:1}" ]; then
continue
fi # if [ '#' == "${LINE:0:1}" -o '*' == "${LINE:0:1}" ]; then
# Section label
if [ "[" == "${LINE:0:1}" ]; then
LINE="${LINE/[/}"
LINE="${LINE/]/}"
LINE="${LINE/ /_}"
SECTION=$(echo "${LINE}")_
else
LINE="$(echo "${LINE}"|sed -e "s/^[ \t]*//")"
LINE="$(echo "${LINE}"|cut -d# -f1)"
TOKEN="$(echo "${LINE:0}"|cut -d= -f1)"
EQOFS=${#TOKEN}
TOKEN="$(echo "${TOKEN}"|sed -e "s/[ \t]*//g")"
VALUE="${LINE:${EQOFS}}"
VALUE="$(echo "${VALUE}"|sed -e "s/^[ \t=]*//")"
VALUE="$(echo "${VALUE}"|sed -e "s/[ \t]*$//")"
if [ "${VALUE:0:1}" == '"' ]; then
echo -n "${SECTION}${TOKEN}=${VALUE}"
echo -e "\r"
else
echo -n "${SECTION}${TOKEN}="\"${VALUE}\"""
echo -e "\r"
fi # if [ "${VALUE:0:1}" == '"' ]; then
fi # if [ "[" == "${LINE:0:1}" ]; then
fi # if [ 0 -ne ${#TMP} ]; then
IFS=$(echo)
} done <<< "$1"
IFS="${OIFS}" # restore original IFS value
} # iniParse()
# call this function with the INI filespec
iniReader() {
if [ -z "$1" ]; then return 1; fi
TMPINI="$(<$1)"
TMPINI="$(echo "${TMPINI}"|sed -e "s/\r//g")"
TMPINI="$(echo "${TMPINI}"|sed -e "s/[ \t]*\[[ \t]*/[/g")"
TMPINI="$(echo "${TMPINI}"|sed -e "s/[ \t]*\][ \t]*/]/g")"
INI=`iniParse "${TMPINI}"`
INI="$(echo "${INI}"|sed -e "s/\r/\n/g")"
eval "${INI}"
return 0
} # iniReader() {
# sample usage
if iniReader $1 ; then
echo INI read, exit_code $? # exit_code == 0
cat <<< "${INI}"
cat <<< "${SECTION_ONE_TOKEN_FOUR}"
cat <<< "${SECTION_ONE_TOKEN_THREE}"
cat <<< "${SECTION_TWO_TOKEN_TWO}"
cat <<< "${SECTION_TWO_TOKEN_ONE}"
else
echo usage: $0 filename.ini
fi # if iniReader $1 ; then
grep based alternative seems to be more readable:
CONFIG_FILE='/your/config/file.ini'
eval $(grep '^\[\|^#' CONFIG_FILE -v | while read line
do echo $line
done)
Where:
-v grep option means exclude matching lines
^\[\|^# selects all lines which starts with [ or # (configparser sections and comments)
It will work ONLY if your config file doesn't have spaces around = (if you would like to generate config with Python use space_around_delimiters=False see https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.write)
Supported config example:
FIRST_VAR="a"
[lines started with [ will be ignored]
secondvar="b"
# some comment
anotherVar="c"
You can use bash it-self to interpret ini values, by:
$ source <(grep = file.ini)
Sample file:
[section-a]
var1=value1
var2=value2
See more examples: How do I grab an INI value within a shell script?
Or you can use bash ini-parser which can be found at The Old School DevOps blog site.