Through some here help I was to get only the text between quotation marks in a file like so:
text undefined but text
something something text something
shouldfindthis "dolphin"
butalsothis "elephant"
by doing:
i=0
regex='(".*?")' # Match all what is between "
while read line # read file line by line
do
if [[ $line =~ $regex ]]; then # If regex match
vars[$i]="${BASH_REMATCH[1]}" # store capturing group in a array
i=$((i + 1))
fi
done < txt # file "txt"
# Print what we found
if [ -v "vars" ]; then # if we found something , "vars" will exist
for var in "${vars[#]}"
do
echo "$var"
done
fi
Yielding an output:
$ ./script.sh
"dolphin"
"elephant"
Is there a way to mark these as variables? As such that dolphin is $text1 and elephant is $text2? in doing so that I can replace only the text within the paranthasis?
First: Using arrays, as your code already does, is the best-practice approach. Don't change it. However, if you really want to write to separate variables rather than array, then replace:
vars[$i]="${BASH_REMATCH[1]}" # this could also be vars+=( "${BASH_REMATCH[1]}" )
...with...
printf -v "text$i" '%s' "${BASH_REMATCH[1]}"
If you want to eliminate the quotes themselves, move them outside the grouping operator in your regex:
regex='"([^"]*)"'
Related
I have a variable text file, in which All of my variables are there. I wrote a script through which I get the content of the file, here's the script:
#!/bin/sh
value=`cat dev.txt`
echo "$value"
And I got this output on my terminal screen.
location = "centralus"
location_abr = "cus"
client_name = "fair"
client_name_prefix = "f"
resource_group_postfix = "rg"
project_name = "OTOZ"
environment_name = "devtest"
instance = "04"
Clusterabr= "aks"
But, I want to use the value of the variables which I got in the output. For example, I want to get the value of Clusterabr, But I cannot get it.
Would you please try the following:
#!/bin/bash
declare -A ary # associative array to store name-value pairs
pat='^([^[:space:]]+)[[:space:]]*=[[:space:]]*"([^"]+)"$'
while IFS= read -r line; do
if [[ $line =~ $pat ]]; then
ary[${BASH_REMATCH[1]}]="${BASH_REMATCH[2]}"
fi
done < dev.txt
echo "${ary[Clusterabr]}" # example
Output:
aks
The regex $pat matches the line assiging ${BASH_REMATCH[1]} to the
lvalue and ${BASH_REMATCH[2]} to the rvalue.
ary[$name]="$value" assigns an associative array ary indexed
by "$name" to "$value".
I have an even better solution, based on the source command, as you can see:
Prompt>source dev.txt
Prompt>echo $location
Prompt>centralus
I must admit that you might need to do a small modification in your code, removing the spaces around the = characters, like:
Prompt>cat dev.txt
location="centralus"
location_abr="cus"
...
As you might have guessed, the source command treats your dev.txt file as a piece of script source code.
With the given data/input. With the assumption that there is only one = in each line. With bash and sed.
source <(sed 's/[[:space:]]*=[[:space:]]*/=/' file.txt)
value=`cat dev.txt`
# echo "$value"
#get the value of Clusterabr in dev.txt
value1=`cat dev.txt | grep Clusterabr | cut -d "=" -f2`
echo $value1
#remove "" from the value
value2=`echo $value1 | sed 's/"//g'`
echo $value2
The result:
I have an exercise where I have a file and at the begin of it I have something like
#!usr/bin/bash
# tototata
#tititutu
#ttta
Hello world
Hi
Test test
#zabdazj
#this is it
And I have to take each first line starting with a # until the line where I don't have one and stock it in a variable. In case of a shebang, it has to skip it and if there's blank space between lines, it has to skip them too. We just want the comment between the shebang and the next character.
I'm new to bash and I would like to know if there's a way to do it please ?
Expected output:
# tototata
#tititutu
#ttta
Try in this easy way to better understand.
#!/bin/bash
sed 1d your_input_file | while read line;
do
check=$( echo $line | grep ^"[#;]" )
if ([ ! -z "$check" ] || [ -z "$line" ])
then
echo $line;
else
exit 1;
fi
done
This may be more correct, although your question was unclear about weather the input file had a script shebang, if the shebang had to be skipped to match your sample output, or if the input file shebang was just bogus.
It is also unclear for what to do, if the first lines of the input file are not starting with #.
You should really post your assignment's text as a reference.
Anyway here is a script that does collects first set of consecutive lines starting with a sharp # into the arr array variable.
It may not be an exact solution to your assignment (witch you should be able to solve with what your previous lessons taught you), but will get you some clues and keys to iterate reading lines from a file and testing that lines starts with a #.
#!/usr/bin/env bash
# Our variable to store parsed lines
# Is an array of strings with an entry per line
declare -a arr=()
# Iterate reading lines from the file
# while it matches Regex: ^[#]
# mean while lines starts with a sharp #
while IFS=$'\n' read -r line && [[ "$line" =~ ^[#] ]]; do
# Add line to the arr array variable
arr+=("$line")
done <a.txt
# Print each array entries with a newline
printf '%s\n' "${arr[#]}"
How about this (not tested, so you may have to debug it a bit, but my comments in the code should explain what is going on):
while read line
do
# initial is 1 one the first line, and 0 after this. When the script starts,
# the variable is undefined.
: ${initial:=1}
# Test for lines starting with #. Need to quote the hash
# so that it is not taken as comment.
if [[ $line == '#'* ]]
then
# Test for initial #!
if (( initial == 1 )) && [[ $line == '#!'* ]]
then
: # ignore it
else
echo $line # or do whatever you want to do with it
fi
fi
# stop on non-blank, non-comment line
if [[ $line != *[^\ ]* ]]
then
break
fi
initial=0 # Next line won't be an initial line
done < your_file
I am reading a text file line by line and taking the count of all lines as a part of my requirement.
When there is blank line then it get messed up. I tried with if condition for [ -z "$line" ] , however not able to succeed.
Here is my current code:
countNumberOfCases() {
echo "2. Counting number of test cases -----------"
cd $SCRIPT_EXECUTION_DIR
FILE_NAME=Features
while read line || [[ -n "$line" ]]
do
TEST_CASE="$line"
if [ "${TEST_CASE:0:1}" != "#" ] ; then
cd $MVN_EXECUTION_DIR
runTestCase
fi
done < $FILE_NAME
echo " v_ToalNoOfCases : = " $v_ToalNoOfCases
}
And below is Features file
web/sprintTwo/TC_002_MultipleLoginScenario.feature
#web/sprintOne/TC_001_SendMoneyTransaction_Spec.feature
web/sprintTwo/TC_003_MultipleLoginScenario.feature
#web/sprintOne/TC_004_SendMoneyTransaction_Spec.feature
When there is blank line it wont work properly so my requirement is that if there is blank line then it should be skipped and should not get considered.
You can write your loop in a little more robust way:
#!/bin/bash
while read -r line || [[ $line ]]; do # read lines one by one
cd "$mvn_execution_dir" # make sure this is an absolute path
# or move it outside the loop unless "runTestCase" function changes the current directory
runTestCase "$line" # need to pass the argument?
done < <(sed -E '/^[[:blank:]]*$/d; /^[[:blank:]]+#/d' "$file_name") # strip blanks and comments
A few things:
get your script checked at shellcheck for common mistakes
see this post for proper variable naming convention:
Correct Bash and shell script variable capitalization
see this discussion about [ vs [[ in Bash
Test for non-zero length string in Bash: [ -n “$var” ] or [ “$var” ]
about reading lines from a text file
Looping through the content of a file in Bash
I'm using bash on cygwin.
I have to take a .csv file that is a subset of a much larger set of settings and shuffle the new csv settings (same keys, different values) into the 1000-plus-line original, making a new .json file.
I have put together a script to automate this. The first step in the process is to "clean up" the csv file by extracting lines that start with "mme " and "sms ". Everything else is to pass through cleanly to the "clean" .csv file.
This routine is as follows:
# clean up the settings, throwing out mme and sms entries
cat extract.csv | while read -r LINE; do
if [[ $LINE == "mme "* ]]
then
printf "$LINE\n" >> mme_settings.csv
elif [[ $LINE == "sms "* ]]
then
printf "$LINE\n" >> sms_settings.csv
else
printf "$LINE\n" >> extract_clean.csv
fi
done
My problem is that this thing stubs its toe on the following string at the end of one entry: 100%." When it's done with the line, it simply elides the %." and the new-line marker following it, and smears the two lines together:
... 100next.entry.keyname...
I would love to reach in and simply manually delimit the % sign, but it's not a realistic option for my use case. Clearly I'm missing something. My suspicion is that I am in some wise abusing cat or read in the first line.
If there is some place I should have looked to find the answer before bugging you all, by all means point me in that direction and I'll sod off.
Syntax for printf is :
printf format [argument]...
In [ printf ] format string, anything followed by % is a format specifier as described in the link above. What you would like to do is :
while read -r line; do # Replaced LINE with line, full uppercase variable are reserved for the syste,
if [[ "$line" = "mme "* ]] # Here* would glob for anything that comes next
then
printf "%s\n" $line >> mme_settings.csv
elif [[ "$line" = "sms "* ]]
then
printf "%s\n" $line >> sms_settings.csv
else
printf "%s\n" $line >> extract_clean.csv
fi
done<extract.csv # Avoided the useless use of cat
As pointed out, your problem is expanding a parameter containing a formatting instruction in the formatting argument of printf, which can be solved by using echo instead or moving the parameter to be expanded out of the formatting string, as demonstrated in other answers.
I recommend not looping over your whole file with Bash in the first place, as it's notoriously slow; you're extracting lines starting with certain patterns, which is a job at which grep excels:
grep '^mme ' extract.csv > mme_settings.csv
grep '^sms ' extract.csv > sms_settings.csv
grep -v '^mme \|^sms ' extract.csv > extract_clean.csv
The third command uses the -v option (extract lines that don't match) and alternation to exclude lines both starting with mme and sms.
I'm attempting to read a config file that is formatted as follows:
USER = username
TARGET = arrows
I realize that if I got rid of the spaces, I could simply source the config file, but for security reasons I'm trying to avoid that. I know there is a way to read the config file line by line. I think the process is something like:
Read lines into an array
Filter out all of the lines that start with #
search for the variable names in the array
After that I'm lost. Any and all help would be greatly appreciated. I've tried something like this with no success:
backup2.config>cat ~/1
grep '^[^#].*' | while read one two;do
echo $two
done
I pulled that from a forum post I found, just not sure how to modify it to fit my needs since I'm so new to shell scripting.
http://www.linuxquestions.org/questions/programming-9/bash-shell-program-read-a-configuration-file-276852/
Would it be possible to automatically assign a variable by looping through both arrays?
for (( i = 0 ; i < ${#VALUE[#]} ; i++ ))
do
"${NAME[i]}"=VALUE[i]
done
echo $USER
Such that calling $USER would output "username"? The above code isn't working but I know the solution is something similar to that.
The following script iterates over each line in your input file (vars in my case) and does a pattern match against =. If the equal sign is found it will use Parameter Expansion to parse out the variable name from the value. It then stores each part in it's own array, name and value respectively.
#!/bin/bash
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line#*= }
((i++))
fi
done < vars
echo "total array elements: ${#name[#]}"
echo "name[0]: ${name[0]}"
echo "value[0]: ${value[0]}"
echo "name[1]: ${name[1]}"
echo "value[1]: ${value[1]}"
echo "name array: ${name[#]}"
echo "value array: ${value[#]}"
Input
$ cat vars
sdf
USER = username
TARGET = arrows
asdf
as23
Output
$ ./varscript
total array elements: 2
name[0]: USER
value[0]: username
name[1]: TARGET
value[1]: arrows
name array: USER TARGET
value array: username arrows
First, USER is a shell environment variable, so it might be better if you used something else. Using lowercase or mixed case variable names is a way to avoid name collisions.
#!/bin/bash
configfile="/path/to/file"
shopt -s extglob
while IFS='= ' read lhs rhs
do
if [[ $lhs != *( )#* ]]
then
# you can test for variables to accept or other conditions here
declare $lhs=$rhs
fi
done < "$configfile"
This sets the vars in your file to the value associated with it.
echo "Username: $USER, Target: $TARGET"
would output
Username: username, Target: arrows
Another way to do this using keys and values is with an associative array:
Add this line before the while loop:
declare -A settings
Remove the declare line inside the while loop and replace it with:
settings[$lhs]=$rhs
Then:
# set keys
user=USER
target=TARGET
# access values
echo "Username: ${settings[$user]}, Target: ${settings[$target]}"
would output
Username: username, Target: arrows
I have a script which only takes a very limited number of settings, and processes them one at a time, so I've adapted SiegeX's answer to whitelist the settings I care about and act on them as it comes to them.
I've also removed the requirement for spaces around the = in favour of ignoring any that exist using the trim function from another answer.
function trim()
{
local var=$1;
var="${var#"${var%%[![:space:]]*}"}"; # remove leading whitespace characters
var="${var%"${var##*[![:space:]]}"}"; # remove trailing whitespace characters
echo -n "$var";
}
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
setting_name=$(trim "${line%%=*}");
setting_value=$(trim "${line#*=}");
case "$setting_name" in
max_foos)
prune_foos $setting_value;
;;
max_bars)
prune_bars $setting_value;
;;
*)
echo "Unrecognised setting: $setting_name";
;;
esac;
fi
done <"$config_file";
Thanks SiegeX. I think the later updates you mentioned does not reflect in this URL.
I had to edit the regex to remove the quotes to get it working. With quotes, array returned is empty.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line##*= }
((i++))
fi
done < vars
A still better version is .
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2`
((i++))
fi
done < vars
The first version is seen to have issues if there is no space before and after "=" in the config file. Also if the value is missing, i see that the name and value are populated as same. The second version does not have any of these. In addition it trims out unwanted leading and trailing spaces.
This version reads values that can have = within it. Earlier version splits at first occurance of =.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2-`
((i++))
fi
done < vars