I have a csv file that i am reading with a "while read" statement and i want to run an if statement on one of the fields in the csv.
====================================
csv file
client1,admin,password,5.9
client2,admin,password,5.8
====================================
this is my script
while read clientid user pass version
do
if [ '$version' = "5.9" ];
then
echo "IS"
else
echo "NOT"
fi
done < $1
The problem is that the if statement does not work.
It does not echo IS when the version is 5.9, it just keeps saying NOT, unless i change it to !=
I have tried using single and double quotes, even without... still doesn't work as expected.
The goal is to run commands until the end of the file.
Is this script correct for doing this?
Obviously the IS and NOT would be replaced by actual command, this is just for testing.
The sample csv file provided has trailing whitespace on the line, which can be removed from the version variable using parameter expansion.
This should work:
while IFS=, read -r clientid user pass version; do
if [ "${version//[[:space:]]/}" = "5.9" ]; then
echo "IS"
else
echo "NOT"
fi
done < $1
And here's another:
while IFS=$' \t\r\n' read -r line; do
IFS=, read -r clientid user pass version __ <<< "$line"
if [[ $version == '5.9' ]]; then
echo "IS"
else
echo "NOT"
fi
done < "$1"
Quote variables in the open always to prevent word splitting and pathname expansion.
Prefer [[ ]] over [ ]. It doesn't do word splitting and pathname expansion.
IFS=$' \t\r\n' trims out leading and trailing spaces.
__ is added to store surplus values just in case.
You can add the IFS value comma and whitespace IFS=', ' . You will get the exact result.
#!/bin/bash
IFS=', '
while read clientid user pass version
do
if [ "$version" == "5.9" ] ; then
echo "IS"
else
echo "NOT"
fi
done < $1
Related
I'am trying to get the first character of each string using regex and BASH_REMATCH in shell script.
My input text file contain :
config_text = STACK OVER FLOW
The strings STACK OVER FLOW must be uppercase like that.
My output should be something like this :
SOF
My code for now is :
var = config_text
values=$(grep $var test_file.txt | tr -s ' ' '\n' | cut -c 1)
if [[ $values =~ [=(.*)]]; then
echo $values
fi
As you can see I'am using tr and cut but I'am looking to replace them with only BASH_REMATCH because these two commands have been reported in many links as not functional on MacOs.
I tried something like this :
var = config_text
values=$(grep $var test_file.txt)
if [[ $values =~ [=(.*)(\b[a-zA-Z])]]; then
echo $values
fi
VALUES as I explained should be :
S O F
But it seems \b does not work on shell script.
Anyone have an idea how to get my desired output with BASH_REMATCH ONLY.
Thanks in advance for any help.
A generic BASH_REMATCH solution handling any number of words and any separator.
local input="STACK OVER FLOW" pattern='([[:upper:]]+)([^[:upper:]]*)' result=""
while [[ $input =~ $pattern ]]; do
result+="${BASH_REMATCH[1]::1}${BASH_REMATCH[2]}"
input="${input:${#BASH_REMATCH[0]}}"
done
echo "$result"
# Output: "S O F"
Bash's regexes are kind of cumbersome if you don't know how many words there are in the input string. How's this instead?
config_text="STACK OVER FLOW"
sed 's/\([^[:space:]]\)[^[:space:]]*/\1/g' <<<"$config_text"
First Put a valid shebang and paste your script at https://shellcheck.net for validation/recommendation.
With the assumption that the line starts with config and ends with FLOW e.g.
config_text = STACK OVER FLOW
Now the script.
#!/usr/bin/env bash
values="config_text = STACK OVER FLOW"
regexp="config_text = ([[:upper:]]{1})[^ ]+ ([[:upper:]]{1})[^ ]+ ([[:upper:]]{1}).+$"
while IFS= read -r line; do
[[ "$line" = "$values" && "$values" =~ $regexp ]] &&
printf '%s %s %s\n' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" "${BASH_REMATCH[3]}"
done < test_file.txt
If there is Only one line or the target string/pattern is at the first line of the test_file.txt, the while loop is not needed.
#!/usr/bin/env bash
values="config_text = STACK OVER FLOW"
regexp="config_text = ([[:upper:]]{1})[^ ]+ ([[:upper:]]{1})[^ ]+ ([[:upper:]]{1}).+$"
IFS= read -r line < test_file.txt
[[ "$line" = "$values" && "$values" =~ $regexp ]] &&
printf '%s %s %s\n' "${BASH_REMATCH[1]}" "${BASH_REMATCH[2]}" "${BASH_REMATCH[3]}"
Make sure you have and running/using Bashv4+ since MacOS, defaults to Bashv3
See How can I read a file (data stream, variable) line-by-line (and/or field-by-field)?
Another option rather than bash regex would be to utilize bash parameter expansion substring ${parameter:offset:length} to extract the desired characters:
$ read -ra arr <text.file ; printf "%s%s%s\n" "${arr[2]:0:1}" "${arr[3]:0:1}" "${arr[4]:0:1}"
SOF
This question already has an answer here:
bash4 read file into associative array
(1 answer)
Closed 3 years ago.
I'm currently creating a list of commands so for example by saying "directory install plugin-name" I can install all needed plugins specified in an external list. This list is just a txt file with all plugin names. But I'm struggling getting all names in an associative array.
I've tried this one:
while IFS=";" read line;
do " communtyList[ $line ]=1 " ;
done < community-list.txt;
The desired output should be
communityList[test1]=1
communityList[test2]=1....
It need to be an associative array because I want to access it by words and not by index. This word will be implemented as parameters/arguments.
For example "install plugin" instead of "1 plugin"
So I can ask for example this way:
if [ ! -z "${!communtyList[$2]}" ];
Update, here the whole code:
#!/usr/bin/env bash
community(){
declare -A communtyList
while IFS= read line;
do communtyList[$line]=1 ;
done < community-list.txt;
# communtyList[test1]=1
# communtyList[test2]=1
# communtyList[test3]=1
# communtyList[test4]=1
if { [ $1 = 'install' ] || [ $1 = 'activate' ] || [ $1 = 'uninstall' ] || [ $1 = 'deactivate' ] ; } && [ ! -z $2 ] ; then
if [ $2 = 'all' ];
then echo "$1 all community plugins....";
while IFS= read -r line; do echo "$1 $line "; done < community-list.txt;
elif [ ! -z "${!communtyList[$2]}" ];
then echo "$1 community plugin '$2'....";
else
echo -e "\033[0;31m Something went wrong";
echo " Plugin '$2' does not exist.";
echo " Here a list of all available community plugins: ";
echo ${!communtyList[#]}
echo -e " \e[m"
fi
else
echo -e "\033[0;31m Something went wrong";
if [ -z $2 ];
then echo -e "[Plugin name] required. [community][action][plugin name] \e[m"
else
echo " Action '$1' does not exist.";
echo -e " Do you mean some of this? \n install \n activate \n uninstall \e[m"
fi
fi
echo ${!communtyList[#]}
}
"$#"
To use asociative array you have to declare it first
declare -A communityList
Then you can add values
communityList[test1]=1
communityList[test2]=2
...
Or with the declaration
declare -A communityList=(
communityList[test1]=1
communityList[test2]=2
...
)
The quotes around " communtyList[ $line ]=1 " mean you try to evaluate a command whose first character is a space. You want to take out those quotes, and probably put quotes around "$line" instead.
It's also unclear why you have IFS=";" -- you are not splitting the line into fields anyway, so this is not doing anything useful. Are there semicolons in your input file? Where and why; what do they mean?
You should probably prefer read -r unless you specifically require read to do odd things with backslashes in the input.
Finally, as suggested by Ivan, you have to declare the array's type as associative before you try to use it.
With those things out of the way, try
declare -A communityList
while read -r line; do
communtyList["$line"]=1
done < community-list.txt
I have a file named parameters.txt whose contents are as follows:
sheet_name:TEST
sheet_id:CST
sheet_access:YES
And I have a shell script which fetches this text from the parameters.txt file. It uses : as a delimiter for each line of the parameters.txt file and stores whatever is left of : in var1 and whatever is right of : in var2. I want to print matched when var1 stores sheet_name and not matched when it doesn't stores sheet_name. Following is my code which always prints matched irrespective of what var1 stores:
filename=parameters.txt
IFS=$'\n' # make newlines the only separator
for j in `cat $filename`
do
var1=${j%:*} # stores text before :
var2=${j#*:} # stores text after :
if [ “$var1” == “sheet_name” ]; then
echo ‘matched’
else
echo “not matched“
fi
done
What am I doing wrong? Kindly help.
You have useless use of cat. But how about some [ shell parameter expansion ] ?
while read line
do
if [[ "${line%:*}" = "sheet_name" ]] #double quote variables deals word splitting
then
echo "matched"
fi
done<parameters.txt
would do exactly what you're looking for.
Message for you
[ ShellCheck ] says,
"To read lines rather than words, pipe/redirect to a 'while read'
loop."
Check [ this ] note from shellcheck.
How about this?
filename=parameters.txt
while IFS=: read -r first second; do
if [ “$first” == “sheet_name” ]; then
echo ‘matched’
else
echo “not matched“
fi
done < $filename
I'm trying to get the first character of a variable, but I'm getting a Bad substitution error. Can anyone help me fix it?
code is:
while IFS=$'\n' read line
do
if [ ! ${line:0:1} == "#"] # Error on this line
then
eval echo "$line"
eval createSymlink $line
fi
done < /some/file.txt
Am I doing something wrong or is there a better way of doing this?
-- EDIT --
As requested - here's some sample input which is stored in /some/file.txt
$MOZ_HOME/mobile/android/chrome/content/browser.js
$MOZ_HOME/mobile/android/locales/en-US/chrome/browser.properties
$MOZ_HOME/mobile/android/components/ContentPermissionPrompt.js
To get the first character of a variable you need to say:
v="hello"
$ echo "${v:0:1}"
h
However, your code has a syntax error:
[ ! ${line:0:1} == "#"]
# ^-- missing space
So this can do the trick:
$ a="123456"
$ [ ! "${a:0:1}" == "#" ] && echo "doesnt start with #"
doesnt start with #
$ a="#123456"
$ [ ! "${a:0:1}" == "#" ] && echo "doesnt start with #"
$
Also it can be done like this:
$ a="#123456"
$ [ "$(expr substr $a 1 1)" != "#" ] && echo "does not start with #"
$
$ a="123456"
$ [ "$(expr substr $a 1 1)" != "#" ] && echo "does not start with #"
does not start with #
Update
Based on your update, this works to me:
while IFS=$'\n' read line
do
echo $line
if [ ! "${line:0:1}" == "#" ] # Error on this line
then
eval echo "$line"
eval createSymlink $line
fi
done < file
Adding the missing space (as suggested in fedorqui's answer ;) ) works for me.
An alternative method/syntax
Here's what I would do in Bash if I want to check the first character of a string
if [[ $line != "#"* ]]
On the right hand side of ==, the quoted part is treated literally whereas * is a wildcard for any sequence of character.
For more information, see the last part of Conditional Constructs of Bash reference manual:
When the ‘==’ and ‘!=’ operators are used, the string to the right of the operator is considered a pattern and matched according to the rules described below in Pattern Matching
Checking that you're using the right shell
If you are getting errors such as "Bad substitution error" and "[[: not found" (see comment) even though your syntax is fine (and works fine for others), it might indicate that you are using the wrong shell (i.e. not Bash).
So to make sure you are using Bash to run the script, either
make the script executable and use an appropriate shebang e.g. #!/bin/bash
or execute it via bash my_script
Also note that sh is not necessarily bash, sometimes it can be dash (e.g. in Ubuntu) or just plain ol' Bourne shell.
Try this:
while IFS=$'\n' read line
do
if ! [ "${line:0:1}" = "#" ]; then
eval echo "$line"
eval createSymlink $line
fi
done < /some/file.txt
or you can use the following for your if syntax:
if [[ ! ${line:0:1} == "#" ]]; then
TIMTOWTDI ^^
while IFS='' read -r line
do
case "${line}" in
"#"*) echo "${line}"
;;
*) createSymlink ${line}
;;
esac
done < /some/file.txt
Note: I dropped the eval, which could be needed in some (rare!) cases (and are dangerous usually).
Note2: I added a "safer" IFS & read (-r, raw) but you can revert to your own if it is better suited. Note that it still reads line by line.
Note3: I took the habit of using always ${var} instead of $var ... works for me (easy to find out vars in complex text, and easy to see where they begin and end at all times) but not necessary here.
Note4: you can also change the test to : *"#"*) if some of the (comments?) lines can have spaces or tabs before the '#' (and none of the symlink lines does contain a '#')
I'm attempting to read a config file that is formatted as follows:
USER = username
TARGET = arrows
I realize that if I got rid of the spaces, I could simply source the config file, but for security reasons I'm trying to avoid that. I know there is a way to read the config file line by line. I think the process is something like:
Read lines into an array
Filter out all of the lines that start with #
search for the variable names in the array
After that I'm lost. Any and all help would be greatly appreciated. I've tried something like this with no success:
backup2.config>cat ~/1
grep '^[^#].*' | while read one two;do
echo $two
done
I pulled that from a forum post I found, just not sure how to modify it to fit my needs since I'm so new to shell scripting.
http://www.linuxquestions.org/questions/programming-9/bash-shell-program-read-a-configuration-file-276852/
Would it be possible to automatically assign a variable by looping through both arrays?
for (( i = 0 ; i < ${#VALUE[#]} ; i++ ))
do
"${NAME[i]}"=VALUE[i]
done
echo $USER
Such that calling $USER would output "username"? The above code isn't working but I know the solution is something similar to that.
The following script iterates over each line in your input file (vars in my case) and does a pattern match against =. If the equal sign is found it will use Parameter Expansion to parse out the variable name from the value. It then stores each part in it's own array, name and value respectively.
#!/bin/bash
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line#*= }
((i++))
fi
done < vars
echo "total array elements: ${#name[#]}"
echo "name[0]: ${name[0]}"
echo "value[0]: ${value[0]}"
echo "name[1]: ${name[1]}"
echo "value[1]: ${value[1]}"
echo "name array: ${name[#]}"
echo "value array: ${value[#]}"
Input
$ cat vars
sdf
USER = username
TARGET = arrows
asdf
as23
Output
$ ./varscript
total array elements: 2
name[0]: USER
value[0]: username
name[1]: TARGET
value[1]: arrows
name array: USER TARGET
value array: username arrows
First, USER is a shell environment variable, so it might be better if you used something else. Using lowercase or mixed case variable names is a way to avoid name collisions.
#!/bin/bash
configfile="/path/to/file"
shopt -s extglob
while IFS='= ' read lhs rhs
do
if [[ $lhs != *( )#* ]]
then
# you can test for variables to accept or other conditions here
declare $lhs=$rhs
fi
done < "$configfile"
This sets the vars in your file to the value associated with it.
echo "Username: $USER, Target: $TARGET"
would output
Username: username, Target: arrows
Another way to do this using keys and values is with an associative array:
Add this line before the while loop:
declare -A settings
Remove the declare line inside the while loop and replace it with:
settings[$lhs]=$rhs
Then:
# set keys
user=USER
target=TARGET
# access values
echo "Username: ${settings[$user]}, Target: ${settings[$target]}"
would output
Username: username, Target: arrows
I have a script which only takes a very limited number of settings, and processes them one at a time, so I've adapted SiegeX's answer to whitelist the settings I care about and act on them as it comes to them.
I've also removed the requirement for spaces around the = in favour of ignoring any that exist using the trim function from another answer.
function trim()
{
local var=$1;
var="${var#"${var%%[![:space:]]*}"}"; # remove leading whitespace characters
var="${var%"${var##*[![:space:]]}"}"; # remove trailing whitespace characters
echo -n "$var";
}
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
setting_name=$(trim "${line%%=*}");
setting_value=$(trim "${line#*=}");
case "$setting_name" in
max_foos)
prune_foos $setting_value;
;;
max_bars)
prune_bars $setting_value;
;;
*)
echo "Unrecognised setting: $setting_name";
;;
esac;
fi
done <"$config_file";
Thanks SiegeX. I think the later updates you mentioned does not reflect in this URL.
I had to edit the regex to remove the quotes to get it working. With quotes, array returned is empty.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line##*= }
((i++))
fi
done < vars
A still better version is .
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2`
((i++))
fi
done < vars
The first version is seen to have issues if there is no space before and after "=" in the config file. Also if the value is missing, i see that the name and value are populated as same. The second version does not have any of these. In addition it trims out unwanted leading and trailing spaces.
This version reads values that can have = within it. Earlier version splits at first occurance of =.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2-`
((i++))
fi
done < vars