Read multiple variables from file - bash

I need to read a file that has lines like
user=username1
pass=password1
How can I read multiple lines like this into separate variables like username and password?
Would I use awk or grep? I have found ways to read lines into variables with grep but would I need to read the file for each individual item?
The end result is to use these variables to access a database via the command line. So I need to be able to read, store and use these values in other commands.

if the process which generates the file is safe and has shell syntax just source the file.
. ./file
Otherwise the file can be processes before to add quotes
perl -ne 'if (/^([A-Za-z_]\w*)=(.*)/) {$k=$1;$v=$2;$v=~s/\x27/\x27\\\x27\x27/g;print "$k=\x27$v\x27\n";}' <file >file2
. ./file2

If you want to use awk then
Input
$ cat file
user=username1
pass=password1
Reading
$ user=$(awk -F= '$1=="user"{print $2;exit}' file)
$ pass=$(awk -F= '$1=="pass"{print $2;exit}' file)
Output
$ echo $user
username1
$ echo $pass
password1

You could use a loop for your file perhaps, but this is probably the functionality you're looking for.
$ echo 'user=username1' | awk -F= '{print $2}'
username1
Using the -F flag sets the delimiter to = and we select the 2nd item from the row.

file.txt:
user=username1
pass=password1
user=username2
pass=password2
user=username3
pass=password3
Do to avoid browsing several times the file file.txt:
#!/usr/bin/env bash
func () {
echo "user:$1 pass:$2"
}
i=0
while IFS='' read -r line; do
if [ $i -eq 0 ]; then
i=1
user=$(echo ${line} | cut -f2 -d'=')
else
i=0
pass=$(echo ${line} | cut -f2 -d'=')
func "$user" "$pass"
fi
done < file.txt
Output:
user:username1 pass:password1
user:username2 pass:password2
user:username3 pass:password3

Related

How to find values ​in quotes using bash?

I have a file with the following content:
"X-Apple-I-MD-M" = "MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s";
I want to extract the returned results Output as:
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
Tks Everybody!
One awk idea, assuming this is the only line in the file:
$ awk -F'"' '{print $4}' file
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
If there are other lines and you wish to focus only on the line with the string "X-Apple-I-MD-M":
Input file:
$ cat file
some line to ignore
"X-Apple-I-MD-M" = "MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s";
other line to ignore and "with" some "quotes"
New awk idea:
$ pattern='X-Apple-I-MD-M'
$ awk -v ptn="${pattern}" -F'"' '$2==ptn {print $4}' file
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
And saving the awk result in a variable:
$ mystring=$(awk ... )
$ echo "${mystring}"
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
NOTE: keep in mind if there are multiple matching lines in file then ${mystring} will contain a multi-line value (eg, line1match\nline2match\nline3match
I always like sed.
$: echo '"X-Apple-I-MD-M" = "MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s";'| sed -E 's/^.*= *"([^"]+)" *; *$/\1/'
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
if it's a file,
$: sed -E 's/^.*= *"([^"]+)" *; *$/\1/' file
MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s
With GNU grep(1), something like.
grep -Po '(?<="X-Apple-I-MD-M" = ").*(?=";)' <<< '"X-Apple-I-MD-M" = "MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s";'
If it is in a file.
grep -Po '(?<="X-Apple-I-MD-M" = ").*(?=";)' file.txt
If your content is consistent, an ugly solution is:
VAL='"X-Apple-I-MD-M" = "MR7v7ctwW0yr3mAUY3rAluXgOReA4CIn1JWJS2ba1s";'
echo $VAL
echo $VAL | awk '{split($0, a, " = "); print(substr(a[2], 2, length(a[2]) - 3))}'
Guessing by the bash tag, this is probably supposed to be in pure Bash, without external processes…? Two (somewhat) random options:
while IFS='"' read _ _ _ code _; do
echo "$code"
done
while read line; do
line="${line#\"*\" = \"}"
line="${line%\";}"
echo "$line"
done

How to use variable with awk when being read from a file

I have a file with the following entries:
foop07_bar2_20190423152612.zip
foop07_bar1_20190423153115.zip
foop08_bar2_20190423152612.zip
foop08_bar1_20190423153115.zip
where
foop0* = host
bar* = fp
I would like to read the file and create 3 variables, the whole file name, host and fp (which stands for file_path_differentiator).
I am using read to take the first line and get my whole file name variable, I though I could then feed this into awk to grab the next two variables, however the first method of variable insertion creates an error and the second gives me all the variables.
I would like to loop each line, as I wish to use these variables to ssh to the host and grab the file
#!/bin/bash
while read -r FILE
do
echo ${FILE}
host=`awk 'BEGIN { FS = "_" } ; { print $1 }'<<<<"$FILE"`
echo ${host}
path=`awk -v var="${FILE}" 'BEGIN { FS = "_" } ; { print $2 }'`
echo ${path}
done <zips_not_received.csv
Expected Result
foop07_bar2_20190423152612.zip
foop07
bar2
foop07_bar1_20190423153115.zip
foop07
bar1
Actual Result
foop07_bar2_20190423152612.zip
/ : No such file or directoryfoop07_bar2_20190423152612.zip
bar2 bar1 bar2 bar1
You can do this alone with bash, without using any external tool.
while read -r file; do
[[ $file =~ (.*)_(.*)_.*\.zip ]] || { echo "invalid file name"; exit 1; }
host="${BASH_REMATCH[1]}"
path="${BASH_REMATCH[2]}"
echo "$file"
echo "$host"
echo "$path"
done < zips_not_received.csv
typical...
Managed to work a solution after posting...
#!/bin/bash
while read -r FILE
do
echo ${FILE}
host=`echo "$FILE" | awk -F"_" '{print $1}'`
echo $host
path=`echo "$FILE" | awk -F"_" '{print $2}'`
echo ${path}
done <zips_not_received.csv
not sure on the elegance or its correctness as i am using echo to create variable...but i have it working..
Assuming there is no space or _ in your "file name" that are part of the host or path
just separate line before with sed, awk, ... if using default space separator (or use _ as argument separator in batch). I add the remove of empty line value as basic security seeing your sample.
sed 's/_/ /g;/[[:blank:]]\{1,\}/d' zips_not_received.csv \
| while read host path Ignored
do
echo "${host}"
echo "${path}"
done

How to remove a filename from the list of path in Shell

I would like to remove a file name only from the following configuration file.
Configuration File -- test.conf
knowledgebase/arun/test.rf
knowledgebase/arunraj/tester/test.drl
knowledgebase/arunraj2/arun/test/tester.drl
The above file should be read. And removed contents should went to another file called output.txt
Following are my try. It is not working to me at all. I am getting empty files only.
#!/bin/bash
file=test.conf
while IFS= read -r line
do
# grep --exclude=*.drl line
# awk 'BEGIN {getline line ; gsub("*.drl","", line) ; print line}'
# awk '{ gsub("/",".drl",$NF); print line }' arun.conf
# awk 'NF{NF--};1' line arun.conf
echo $line | rev | cut -d'/' -f 1 | rev >> output.txt
done < "$file"
Expected Output :
knowledgebase/arun
knowledgebase/arunraj/tester
knowledgebase/arunraj2/arun/test
There's the dirname command to make it easy and reliable:
#!/bin/bash
file=test.conf
while IFS= read -r line
do
dirname "$line"
done < "$file" > output.txt
There are Bash shell parameter expansions that will work OK with the list of names given but won't work reliably for some names:
file=test.conf
while IFS= read -r line
do
echo "${line%/*}"
done < "$file" > output.txt
There's sed to do the job — easily with the given set of names:
sed 's%/[^/]*$%%' test.conf > output.txt
It's harder if you have to deal with names like /plain.file (or plain.file — the same sorts of edge cases that trip up the shell expansion).
You could add Perl, Python, Awk variants to the list of ways of doing the job.
You can get the path like this:
path=${fullpath%/*}
It cuts away the string after the last /
Using awk one liner you can do this:
awk 'BEGIN{FS=OFS="/"} {NF--} 1' test.conf
Output:
knowledgebase/arun
knowledgebase/arunraj/tester
knowledgebase/arunraj2/arun/test

extracting a variable's value from text file using bash

I am using Linux and bash.
I have a simple text file like below:
VAR1=100
VAR2=5
VAR3=0
VAR4=99
I want to extract by means of bash the value of VAR2, that is 5.
How could I do that?
Assuming the file is called vars.txt
sed -n 's/^VAR2=\(.*\)/\1/p' < vars.txt
You can use the value elsewhere like this using single back quotes
echo VAR2=`sed -n 's/^VAR2=\(.*\)/\1/p' < txt`
The simplest way might be to use source or simply . to read and execute the file. This would work with your example, because there are no spaces in the variable values. Otherwise you need to use grep + cut or awk, as stated in other answers.
. /path/to/your/file
echo $VAR2
[edit]
As stated by dawg, this would make the other variables available in your script too, and possibly overwrite existing variables.
Given:
$ echo "$txt"
VAR1=100
VAR2=5
VAR3=0
VAR4=99
You can use awk:
$ echo "$txt" | awk -F= '/^VAR2/ { print $2 }'
5
Or grep and cut:
$ echo "$txt" | egrep '^VAR2=\d+' | cut -d = -f 2
5
On Bash, you can insert the value of those assignments into the current shell using source and filter the lines you wish to use. In this case, only the line VAR2=5 will be used. You need to write that to a file and then source that file:
$ echo "$txt" | grep '^VAR2' > tmp && source tmp && rm tmp
$ echo $VAR2
5
For the files as described, you can just source the file as bash script which will run it's content and update you workspace environment with it. For example:
source file.txt
echo $VAR2
Assume this as your txt file, named test.txt
VAR2 = 5
VAR3 = 0
VAR4 = 99
you can cat test.txt | grep 'VAR2' | awk '{printf $3}'
and then your output will be: 5
Here, cat test.txt will display the content of test.txt in your terminal,grep 'VAR2' will list lines containing 'VAR2' and awk '{printf $3}' will print the value of the variable

Save output of awk to two different variables

Okay. I am kind of lost and google search isn't helping me much.
I have a command like:
filesize_filename=$(echo $line | awk ' ''{print $5":"$9}')
echo $filesize_filename
1024:/home/test
Now this one saves the two returns or awk'ed items into one variable. I'd like to achieve something like this:
filesize,filename=$(echo $line | awk ' ''{print $5":"$9}')
So I can access them individually like
echo $filesize
1024
echo $filename
/home/test
How to I achieve this?
Thanks.
Populate a shell array with the awk output and then do whatever you like with it:
$ fileInfo=( $(echo "foo 1024 bar /home/test" | awk '{print $2, $4}') )
$ echo "${fileInfo[0]}"
1024
$ echo "${fileInfo[1]}"
/home/test
If the file name can contain spaces then you'll have to adjust the FS and OFS in awk and the IFS in shell appropriately.
You may not need awk at all of course:
$ line="foo 1024 bar /home/test"
$ fileInfo=( $line )
$ echo "${fileInfo[1]}"
1024
$ echo "${fileInfo[3]}"
/home/test
but beware of globbing chars in $line matching on local file names in that last case. I expect there's a more robust way to populate a shell array from a shell variable but off the top of my head I can't think of it.
Use bash's read for that:
read size name < "$(awk '{print $5, $9}' <<< "$line")"
# Now you can output them separately
echo "$size"
echo "$name"
You can use process substitution on awk's output:
read filesize filename < <(echo "$line" | awk '{print $5,$9}')
You can totally avoid awk by doing:
read _ _ _ _ filesize _ _ _ filename _ <<< "$line"

Resources