I have to create multiple variable while reading the file in bash.
These variables need to have a dynamic name as per the content of file.
E.g:
File content:
abc: 20
1 apple a day
abc: 40
1 keeps the doctor away
now i have to create variables as:
day_20_id = 1
day_20_fruit = apple
away_40_id = 1
away_40_who = doctor
it would be like in all variable names, only the the value of $abc will be updated and the value of the variable will be as per the file content.
Can somebody help me out to figure out how to achieve this.
You can use the eval command to accomplish this as illustrated below
abc=20 # assuming you got this from the input file
val=1 # assuming you also got this from the input file
varName="day_${abc}_id"
command="${varName}=${val}"
eval $command
# now print to output file as you have stated in the comments
outputFile=output.txt # randomly-chosen name
command="echo $varName = $val > $outputFile"
eval $command
Related
this is probably a very simple question. I looked at other answers but couldn't come up with a solution. I have a 365 line date file. file as below,
01-01-2000
02-01-2000
I need to read this file line by line and assign each day to a separate variable. like this,
d001=01-01-2000
d002=02-01-2000
I tried while read commands but couldn't get them to work.It takes a lot of time to shoot one by one. How can I do it quickly?
Trying to create named variable out of an associative array, is time waste and not supported de-facto. Better use this, using an associative array:
#!/bin/bash
declare -A array
while read -r line; do
printf -v key 'd%03d' $((++c))
array[$key]=$line
done < file
Output
for i in "${!array[#]}"; do echo "key=$i value=${array[$i]}"; done
key=d001 value=01-01-2000
key=d002 value=02-01-2000
Assumptions:
an array is acceptable
array index should start with 1
Sample input:
$ cat sample.dat
01-01-2000
02-01-2000
03-01-2000
04-01-2000
05-01-2000
One bash/mapfile option:
unset d # make sure variable is not currently in use
mapfile -t -O1 d < sample.dat # load each line from file into separate array location
This generates:
$ typeset -p d
declare -a d=([1]="01-01-2000" [2]="02-01-2000" [3]="03-01-2000" [4]="04-01-2000" [5]="05-01-2000")
$ for i in "${!d[#]}"; do echo "d[$i] = ${d[i]}"; done
d[1] = 01-01-2000
d[2] = 02-01-2000
d[3] = 03-01-2000
d[4] = 04-01-2000
d[5] = 05-01-2000
In OP's code, references to $d001 now become ${d[1]}.
A quick one-liner would be:
eval $(awk 'BEGIN{cnt=0}{printf "d%3.3d=\"%s\"\n",cnt,$0; cnt++}' your_file)
eval makes the shell variables known inside your script or shell. Use echo $d000 to show the first one of the newly defined variables. There should be no shell special characters (like * and $) inside your_file. Remove eval $() to see the result of the awk command. The \" quoted %s is to allow spaces in the variable values. If you don't have any spaces in your_file you can remove the \" before and after %s.
I have a list of files named with this format:
S2_7-CHX-2-5_Chr5.bed
S2_7-CHX-2-13_Chr27.bed
S2_7-CHX-2-0_Chr1.bed
I need to loop through each file to perform a task. Previously, I had named them without the step 2 indicator ("S2"), and this format had worked perfectly:
for FASTQ in *_clean.bam; do
SAMPLE=${FASTQ%_clean.bam}
echo $SAMPLE
echo $(samtools view -c ${SAMPLE}_clean.bam)
done
But now that I have the S2 preceding what I would like to set as the variable, this returns a list of empty "SAMPLE" variables. How can I rewrite the following code to specify only S2_*.bed?
for FASTQ in S2_*.bed; do
SAMPLE=${S2_FASTQ%.bed}
echo $SAMPLE
done
Edit: I'm trying to isolate the unique name from each file, for example "7-CHX-2-13_Chr27" so that I can refer to it later. I can't use the "S2" as part of this because I want to rename the file with "S3" for the next step, and so on.
Example of what I'm trying to use it for:
for FASTQ in S2_*.bed; do
SAMPLE=${S2_FASTQ%.bed}
echo $SAMPLE
#rename each mapping position with UCSC chromosome name using sed
while IFS=, read -r f1 f2; do
#rename each file
echo " sed "s/${f1}.1/chr${f2}/g" S2_${SAMPLE}_Chr${f2}.bed > S3_${SAMPLE}_Chr${f2}.bed" >> $SCRIPT
done < $INPUT
done
The name of the variable is still $FASTQ, the S2_ is not part of the variable name, but its value.
sample=${FASTQ%.bed}
# ~~~~~|~~~~
# | | |
# Variable | What to remove
# name |
# Remove
# from the right
If you want to remove the S2_ from the $sample, use left hand side removal:
sample=${sample#S2_}
The removals can't be combined, you have to proceed in two steps.
Note that I use lower case variable names. Upper case should be reserved for environment and internal shell variables.
Assume there are several files like the following
CLIENT_1.csv
CLIENT_2.csv
CLIENT_3.csv
CLIENT_4.csv
Is there any shell command/awk that we can use to direct the 1,2,3,4 into a variable named "ID"? So that we can do the following
if [ ${ID} != 4 ]
then
blah blah blah
fi
To get the number part for each file:
for file in CLIENT_*.csv ; do
id=${file%.csv} # remove trailing '.csv'
id=${id#CLIENT_} # remove leading 'CLIENT_'
if [ "$id" != 4 ] ; then ... ; fi
done
This is pretty much a repeat of this question.
How to iterate over files in a directory with Bash?
It uses filename wildcarding to iterate through the list. In your case you would use:
for filename in /mydirectory/CLIENT_*.csv; do
#whatever you wanted to do
done
Here is another way to do it where the filename string pattern is a variable.
bash: filenames with wildcards in variables
The file names are stored in the variable file (looking at the accepted answer) after the wildcarding is expanded in the for loop initialization. Again using the power of wildcarding your example would look something like this:
putfile=CLIENT_*.csv
for file in $putfile; do #whatever you wanted to do
done
I wrote a simple bash function which would read value from an ini file (defined by variable CONF_FILE) and output it
getConfValue() {
#getConfValue section variable
#return value of a specific variable from given section of a conf file
section=$1
var="$2"
val=$(sed -nr "/\[$section\]/,/\[/{/$var/p}" $CONF_FILE)
val=${val#$var=}
echo "$val"
}
The problem is that it does not ignore comments and runs into troubles if multiple variables within a section names share common substrings.
Example ini file:
[general]
# TEST=old
; TEST=comment
TEST=new
TESTING=this will be output too
PATH=/tmp/test
Running getConfValue general PATH would output /tmp/test as expected, but running getConfValue general TEST shows all the problems this approach has.
How to fix that?
I know there are dedicated tools like crudini or tools for python, perl and php out in the wild, but I do not want to add extra dependencies for simple config file parsing. A solution incorporating awk instead of sed would be just fine too.
Sticking with sed you could anchor your var search to the start of the record using ^ and end it with an equal sign:
"/\[$section\]/,/\[/{/^$var=/p}"
If you are concerned about whitespace in front of your record you could account for that:
"/\[$section\]/,/\[/{/^(\W|)$var=/p}"
That ^(\W|)$var= says "if there is whitespace at the beginning (^(\W) or nothing (|)) before your variable concatenated with an equal sign ($var=)."
If you wanted to switch over to awk you could use something like:
val=$(awk -F"=" -v section=$section -v var=$var '$1=="["section"]"{secFound=1}secFound==1 && $1==var{print $2; secFound=0}' $CONF_FILE)
That awk command splits the record by equal -F"=". Then if the first field in the record is your section ($1=="["section"]") then set variable secFound to 1. Then... if secFound is 1 and the first field is exactly equal to your var variable (secFound==1 && $1==var) then print out the second field ({print $2}) and sets secFound to 0 so we don't pick up any other Test keys.
I encountered this problem and came up with a solution similar to others here.
The main difference is it uses a single awk call to get a response suitable for creating an associative array of the property/value pairs for a section.
This will not ignore the commented properties. Though adding something to do that should not be to hard.
Here's a testing script demonstrating the awk and declare statements used;
#!/bin/bash
#
# Parse a INI style properties file and extract property values for a given section
#
# Author: Alan Carlyle
# License: CC0 (https://creativecommons.org/about/cclicenses/)
#
# Example Input: (example.properties)
# [SEC1]
# item1=value1
# item2="value 2"
#
# [Section 2]
# property 1="Value 1 of 'Section 2'"
# property 2='Single "quoted" value'
#
# Usage:
# $ read_props example.properties "Section 2" property\ 2
# $ Single "quoted" value
#
# Section names and properties with spaces do not need to be quoted.
# Values with spaces must be quoted. Values can use single or double quotes.
# The following characters [ = ] can not be used in names or values.
#
# If the property is not provided the the whole section is outputed.
#
propertiesFile=$1
section=$2
property=$3
# Extract the propetites for the section formated as for associtive array
sectionData="( "$(awk -F'=' -v s="$section" '/^\[/{ gsub(/[\[\]]/, "", $1); f = ($1 == s); next }
NF && f{ print "["$1"]="$2 }' $propertiesFile)" )"
# Create associtive array from extracted section data
declare +x -A "properties=$sectionData"
if [ -z "$property" ]
then
echo $sectionData
else
echo ${properties[$property]}
fi
Would like to read multiple values from a property file using a shell script
My properties files looks something like below, the reason I added it following way was to make sure, if in future more students joins I just need to add in in the properties file without changing any thing in the shell script.
student.properties
total_student=6
student_name_1="aaaa"
student_name_2="bbbb"
student_name_3="cccc"
student_name_4="dddd"
student_name_5="eeee"
When I run below script I not getting the desired output, for reading the student names from properties file
student.sh
#!/bin/bash
. /student.properties
i=1
while [ $i -lt $total_student ]
do
{
std_Name=$student_name_$i
echo $std_Name
#****** my logic *******
} || {
echo "ERROR..."
}
i=`expr $i + 1`
done
Output is something like this
1
2
3
4
5
I understand the script is not getting anything for $student_name_ hence only $i value is getting printed.
Hence, wanted to know how to read values from the properties file.
You can do variable name interpolation with ${!foo}. If $foo is "bar", then ${!foo} gives you the value of $bar. In your code that means changing
std_Name=$student_name_$i
to
var=student_name_$i
std_Name=${!var}
Alternatively, you could store the names in an array. Then you wouldn't have to do any parsing.
student.properties
student_names=("aaaa" "bbbb" "cccc" "dddd" "eeee")
student.sh
#!/bin/bash
. /student.properties
for student_name in "${student_names[#]}"; do
...
done
You can use indirect expansion:
std_Name=student_name_$i
echo "${!std_Name}"
the expression ${!var} basically evaluates the variable twice:
first evaluation: student_name_1
second evaluation: foo
Note that this is rarely a good idea and that using an array is almost always preferred.