Reading from an archive using Bourne Shell - shell

I am trying to use Bourne shell scripting for the first time ever, and I cannot seem to figure out determining how to save text from a file to internal script variables. The file format is as follows:
acc.text
Jason Bourne 213.4
Alice Deweger 1
Mark Harrington 312
The current script that I have (which might be ENTIRELY incorrect as I am simply creating it in NotePad++ without using an actual shell console) is as follows:
#!/bin/sh
process_file()
{
FILE = $1;
SALARYARRAY;
NAMEARRAY;
COUNTER = 0;
while read line
do
$NAMEARRAY[$COUNTER] =
$SALARYARRAY[$COUNTER] =
$COUNTER + 1;
echo $NAMEARRAY[$COUNTER]:$SALARYARRAY[$COUNTER];
done < "$FILE"
order_Map
}
# Function is not complete as of now, will later order SALARYARRAY in descending order
order_Map()
{
i = 0;
for i in $COUNTER
do
if ($SALARYARRAY[
done
}
##
# Main Script Body
#
# Takes a filename input by user, passes to process_file()
##
PROGRAMTITLE = "Account Processing Shell Script (APS)"
FILENAME = "acc.$$"
echo $PROGRAMTITLE
echo Please specify filename for processing
read $FILENAME
while(! -f $FILE || ! -r $FILE)
do
echo Error while attempting to write to file. Please specify file for processing:
read $FILENAME
done
echo Processing the file...
process_file $FILENAME

I have fixed some of your script. You need to cut out the name and salary fields from each line before storing them into the array.
#!/bin/bash
process_file()
{
file=$1;
counter=0
while read line
do
#the name is the first two fields
name=`echo $line | cut -d' ' -f1,2`
NAMEARRAY[$counter]="$name"
#the salary is the third field
salary=`echo $line | cut -d' ' -f3`
SALARYARRAY[$counter]="$salary"
echo ${NAMEARRAY[$counter]}:${SALARYARRAY[$counter]}
counter=$(($counter+1))
done < $file
}
##
# Main Script Body
#
# Takes a filename input by user, passes to process_file()
##
PROGRAMTITLE="Account Processing Shell Script (APS)"
echo $PROGRAMTITLE
echo -n "Please specify filename for processing: "
read FILENAME
if [ -r $FILENAME ] #check that the file exists and is readable
then
process_file $FILENAME
else
echo "Error reading file $FILENAME"
fi

Given the file format you specified, each record has three fields first last and amount, then:
i=0
while read fistname lastname amount; do
NAMEARRAY[$i]="$firstname $lastname"
SALARYARRAY[$i]=$amount
i = `expr $i + 1`
done < "$FILE"
The shell read built-in, automatically splits input. See the variable IFS in the man page for sh(1). If you have data after the amount field, and you wish to ignore it, just create another variable after amount; but don't use it. It will collect everything after the 1st 3 fields into the extra variable.
You specified Bourne shell, so I used some pretty antiquated stuff:
i=`expr $x + 1`
is usually written
let $((i++)) # ksh, bash (POSIX, Solaris, Linux)
On modern systems, /bin/sh is usually ksh or something pretty compatible. You can probably use let $((i++))

Related

In POSIX shell, how to increment a variable each run? [duplicate]

Can I create a Bash script with persistent variable-values?
For example, I initialize a variable with 0 when the script runs for the first time (during a specific time limit), and the variable increases automatically with every time the script is running.
You can't, but you can use a file to do it
#!/bin/bash
valuefile="/tmp/value.dat"
# if we don't have a file, start at zero
if [ ! -f "$valuefile" ]; then
value=0
# otherwise read the value from the file
else
value=$(cat "$valuefile")
fi
# increment the value
value=$((value + 1))
# show it to the user
echo "value: ${value}"
# and save it for next time
echo "${value}" > "$valuefile"
I'm afraid you have to save the state in a file somewhere. The trick is to put it somewhere the user will be able to write to.
yourscriptvar=0
if [ -e "$HOME/.yourscriptvar" ] ; then
yourscriptvar=$(cat "$HOME/.yourscriptvar")
fi
# do something in your script
#save it output the file
echo $yourscriptvar > "$HOME/.yourscriptvar"
I had the same problem a few days ago and I've written my own tool to do the work because there is not other way to do something similar.
gvar is a pure Bash key-value store where each user has a different collection of data. The records are stored in the user's home directory.
Here it is the most interesting functions from the source code:
get_variable() {
cat $FILE | grep -w $1 | cut -d'=' -f2
}
set_variable() {
echo $1=$2 >> $FILE
}
remove_variable() {
sed -i.bak "/^$1=/d" $FILE
}

Count the Words in text file without using the 'wc' command in unix shell scripting

Here I could not find the number of words in the text file . What would be the possible changes do I need to make?
What is the use of tty in this program?
echo "Enter File name:"
read filename
terminal=`tty`
exec < $filename
num_line=0
num_words=0
while read line
do
num_lines=`expr $num_lines + 1`
num_words=`expr $num_words + 1`
done
There is a simple way using arrays to read the number of words in a file:
#!/bin/bash
[ -n "$1" ] || {
printf printf "error: insufficient input. Usage: %s\n" "${0//\//}"
exit 1
}
fn="$1"
[ -r "$fn" ] || {
printf "error: file not found: '%s'\n" "$fn"
exit 1
}
declare -i cnt=0
while read -r line || [ -n "$line" ]; do # read line from file
tmp=( $line ) # create tmp array of words
cnt=$((cnt + ${#tmp[#]})) # add no. of words to count
done <"$fn"
printf "\n %s words in %s\n\n" "$cnt" "$fn" # show results
exit 0
input:
$ cat dat/wordfile.txt
Here I could not find the number of words in the text file. What
would be the possible changes do I need to make? What is the use
of tty in this program?
output:
$bash wcount.sh dat/wordfile.txt
33 words in dat/wordfile.txt
wc -w confirmation:
$ wc -w dat/wordfile.txt
33 dat/wordfile.txt
tty?
The use of terminal=tty assigns the terminal device for the current interactive shell to the terminal variable. (It is a way to determine which tty device you are connected to e.g. /dev/pts/4)
tty command prints the name of the terminal connected to the standard output. In the context of your program, it does nothing significant really, you might as well remove that line and run.
Regarding the number of words calculation, you would need to parse each line and find it using space as the delimiter. Currently the program just finds the number of lines $num_lines and uses the same calculation for $num_words.

Incrementing a variable inside a Bash loop

I'm trying to write a small script that will count entries in a log file, and I'm incrementing a variable (USCOUNTER) which I'm trying to use after the loop is done.
But at that moment USCOUNTER looks to be 0 instead of the actual value. Any idea what I'm doing wrong? Thanks!
FILE=$1
tail -n10 mylog > $FILE
USCOUNTER=0
cat $FILE | while read line; do
country=$(echo "$line" | cut -d' ' -f1)
if [ "US" = "$country" ]; then
USCOUNTER=`expr $USCOUNTER + 1`
echo "US counter $USCOUNTER"
fi
done
echo "final $USCOUNTER"
It outputs:
US counter 1
US counter 2
US counter 3
..
final 0
You are using USCOUNTER in a subshell, that's why the variable is not showing in the main shell.
Instead of cat FILE | while ..., do just a while ... done < $FILE. This way, you avoid the common problem of I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?:
while read country _; do
if [ "US" = "$country" ]; then
USCOUNTER=$(expr $USCOUNTER + 1)
echo "US counter $USCOUNTER"
fi
done < "$FILE"
Note I also replaced the `` expression with a $().
I also replaced while read line; do country=$(echo "$line" | cut -d' ' -f1) with while read country _. This allows you to say while read var1 var2 ... varN where var1 contains the first word in the line, $var2 and so on, until $varN containing the remaining content.
Always use -r with read.
There is no need to use cut, you can stick with pure bash solutions.
In this case passing read a 2nd var (_) to catch the additional "fields"
Prefer [[ ]] over [ ].
Use arithmetic expressions.
Do not forget to quote variables! Link includes other pitfalls as well
while read -r country _; do
if [[ $country = 'US' ]]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
minimalist
counter=0
((counter++))
echo $counter
You're getting final 0 because your while loop is being executed in a sub (shell) process and any changes made there are not reflected in the current (parent) shell.
Correct script:
while read -r country _; do
if [ "US" = "$country" ]; then
((USCOUNTER++))
echo "US counter $USCOUNTER"
fi
done < "$FILE"
I had the same $count variable in a while loop getting lost issue.
#fedorqui's answer (and a few others) are accurate answers to the actual question: the sub-shell is indeed the problem.
But it lead me to another issue: I wasn't piping a file content... but the output of a series of pipes & greps...
my erroring sample code:
count=0
cat /etc/hosts | head | while read line; do
((count++))
echo $count $line
done
echo $count
and my fix thanks to the help of this thread and the process substitution:
count=0
while IFS= read -r line; do
((count++))
echo "$count $line"
done < <(cat /etc/hosts | head)
echo "$count"
USCOUNTER=$(grep -c "^US " "$FILE")
Incrementing a variable can be done like that:
_my_counter=$[$_my_counter + 1]
Counting the number of occurrence of a pattern in a column can be done with grep
grep -cE "^([^ ]* ){2}US"
-c count
([^ ]* ) To detect a colonne
{2} the colonne number
US your pattern
Using the following 1 line command for changing many files name in linux using phrase specificity:
find -type f -name '*.jpg' | rename 's/holiday/honeymoon/'
For all files with the extension ".jpg", if they contain the string "holiday", replace it with "honeymoon". For instance, this command would rename the file "ourholiday001.jpg" to "ourhoneymoon001.jpg".
This example also illustrates how to use the find command to send a list of files (-type f) with the extension .jpg (-name '*.jpg') to rename via a pipe (|). rename then reads its file list from standard input.

How to extract values from a string on ksh (korn shell)

I have the following input
MyComposite[2.1], partition=default, mode=active, state=on, isDefault=true, deployedTime=2012-05-07T15:35:22.473-07:00
MessageManager[1.0], partition=default, mode=active, state=on, isDefault=true, deployedTime=2012-05-07T15:37:14.137-07:00
SimpleApproval[1.0], partition=default, mode=active, state=on, isDefault=true, deployedTime=2012-05-07T15:28:39.599-07:00
and I have a script that parses the input line by line from a file but I don't have a clue on how I could extract individual parameters from each line into local variables so I can perform additional processes
So far I'm trying the following:
#!/bin/ksh
file="output"
compositeName="foo" ci=0
# while loop while read line do
# display line or do somthing on $line
if echo "$line" | egrep -q '\[[0-9]*\.[0-9]*\].*?(mode=active).*?
(state=on)' then compositeName=$( echo "$line" | egrep '[0-9]*' )
echo "$compositeName"
#echo "$line"
fi
done <"$file"
I'm somwhow lookint to extract only two values from this string, the first word and the float between brackets
ie:
name = MyComposite
version = 2.1
any ideas?
I'm not sure if those line numbers are in the file or not. If not, you can do this:
#!/usr/bin/env ksh
while IFS="," read nameVersion line; do
name="${nameVersion%%\[*}"
version="${nameVersion//*\[+([0-9.])\]*/\1}"
print "name=$name version=$version"
done < "$file"
If the line numbers are in the file, change the name assignment in the above script to name="${nameVersion//+([0-9]).+( )+(*)\[*/\3}"

Read a config file in BASH without using "source"

I'm attempting to read a config file that is formatted as follows:
USER = username
TARGET = arrows
I realize that if I got rid of the spaces, I could simply source the config file, but for security reasons I'm trying to avoid that. I know there is a way to read the config file line by line. I think the process is something like:
Read lines into an array
Filter out all of the lines that start with #
search for the variable names in the array
After that I'm lost. Any and all help would be greatly appreciated. I've tried something like this with no success:
backup2.config>cat ~/1
grep '^[^#].*' | while read one two;do
echo $two
done
I pulled that from a forum post I found, just not sure how to modify it to fit my needs since I'm so new to shell scripting.
http://www.linuxquestions.org/questions/programming-9/bash-shell-program-read-a-configuration-file-276852/
Would it be possible to automatically assign a variable by looping through both arrays?
for (( i = 0 ; i < ${#VALUE[#]} ; i++ ))
do
"${NAME[i]}"=VALUE[i]
done
echo $USER
Such that calling $USER would output "username"? The above code isn't working but I know the solution is something similar to that.
The following script iterates over each line in your input file (vars in my case) and does a pattern match against =. If the equal sign is found it will use Parameter Expansion to parse out the variable name from the value. It then stores each part in it's own array, name and value respectively.
#!/bin/bash
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line#*= }
((i++))
fi
done < vars
echo "total array elements: ${#name[#]}"
echo "name[0]: ${name[0]}"
echo "value[0]: ${value[0]}"
echo "name[1]: ${name[1]}"
echo "value[1]: ${value[1]}"
echo "name array: ${name[#]}"
echo "value array: ${value[#]}"
Input
$ cat vars
sdf
USER = username
TARGET = arrows
asdf
as23
Output
$ ./varscript
total array elements: 2
name[0]: USER
value[0]: username
name[1]: TARGET
value[1]: arrows
name array: USER TARGET
value array: username arrows
First, USER is a shell environment variable, so it might be better if you used something else. Using lowercase or mixed case variable names is a way to avoid name collisions.
#!/bin/bash
configfile="/path/to/file"
shopt -s extglob
while IFS='= ' read lhs rhs
do
if [[ $lhs != *( )#* ]]
then
# you can test for variables to accept or other conditions here
declare $lhs=$rhs
fi
done < "$configfile"
This sets the vars in your file to the value associated with it.
echo "Username: $USER, Target: $TARGET"
would output
Username: username, Target: arrows
Another way to do this using keys and values is with an associative array:
Add this line before the while loop:
declare -A settings
Remove the declare line inside the while loop and replace it with:
settings[$lhs]=$rhs
Then:
# set keys
user=USER
target=TARGET
# access values
echo "Username: ${settings[$user]}, Target: ${settings[$target]}"
would output
Username: username, Target: arrows
I have a script which only takes a very limited number of settings, and processes them one at a time, so I've adapted SiegeX's answer to whitelist the settings I care about and act on them as it comes to them.
I've also removed the requirement for spaces around the = in favour of ignoring any that exist using the trim function from another answer.
function trim()
{
local var=$1;
var="${var#"${var%%[![:space:]]*}"}"; # remove leading whitespace characters
var="${var%"${var##*[![:space:]]}"}"; # remove trailing whitespace characters
echo -n "$var";
}
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
setting_name=$(trim "${line%%=*}");
setting_value=$(trim "${line#*=}");
case "$setting_name" in
max_foos)
prune_foos $setting_value;
;;
max_bars)
prune_bars $setting_value;
;;
*)
echo "Unrecognised setting: $setting_name";
;;
esac;
fi
done <"$config_file";
Thanks SiegeX. I think the later updates you mentioned does not reflect in this URL.
I had to edit the regex to remove the quotes to get it working. With quotes, array returned is empty.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line##*= }
((i++))
fi
done < vars
A still better version is .
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2`
((i++))
fi
done < vars
The first version is seen to have issues if there is no space before and after "=" in the config file. Also if the value is missing, i see that the name and value are populated as same. The second version does not have any of these. In addition it trims out unwanted leading and trailing spaces.
This version reads values that can have = within it. Earlier version splits at first occurance of =.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2-`
((i++))
fi
done < vars

Resources