I want to search a repository file to see if I have a file in my directory. To do this, I loop the directory hopefully containing the file I am looking for, and I check to see if that file is in my repository.txt. If it is in there, I change a boolean, and then check the value of that boolean after I compared all the files in the repository.
for file in *
do
inThere=false
cut -d, -f1 $repo | while read line
do
echo "comparing "$line" == "$file" "
if [ "$line" == "$file" ]; then
inThere=true
echo "I got tripped!!" #inThere is true
echo "in there is $inThere"
fi
done
echo "in there is $inThere" #inThere is false
done
Is there a way I can persist the changing boolean value, or is there another, smarter way of doing this? Please let me know if you have any questions.
BASH FAQ entry #24: "I set variables in a loop that's in a pipeline. Why do they disappear after the loop terminates? Or, why can't I pipe data to read?"
while read ...
do
...
done < <(cut ...)
Related
I got a config file called user.conf that contains this list:
USER1_USERNAME="John"
#USER2_USERNAME="Mike"
USER3_USERNAME="David"
USER4_USERNAME="James"
USER5_USERNAME="Jenny"
Notice that user can comment out that line USER2_USERNAME so, only 4 users are used in source file in bash script test.sh here:
#!/bin/bash
#source the username
source "user.conf"
n=1
while :; do
((n++))
if [ -n "${USER${n}_USERNAME}"]; then
echo "This variable USER${n}_USERNAME is set: ${USER${n}_USERNAME}"
else
echo "This variable USER${n}_USERNAME is not set"
fi
done
I want to display which variable is set with its value and skip the commented variables but at this point my code just display an error:
./test.sh: line 8: ${USER${n}_USERNAME}: bad substitution
Is it possible to loop that variable like above?
Also, user can define much more USER in that user.conf, for example
USER6_USERNAME="George "
Expected output:
This variable USER1_USERNAME is set to John
This variable USER2_USERNAME is not set
This variable USER3_USERNAME is set to David
This variable USER4_USERNAME is set to James
This variable USER5_USERNAME is set to Jenny
You can use bash variable substitution like this:
#!/usr/bin/env bash
source "user.conf"
for i in "${!USER#}"
do
echo $i ${!i}
done
Like your code, this first sources the users file, as it looks like a bash script. This of course has the potentially dangerous side effect that any other code in user.conf runs as well, so be careful and don't let strangers modify that file.
Then it uses ${!var#}, which expands to the names of variables whose names begin with a prefix, here "var", or for you "USER". You could also use ${!var*}, depending on whether you want all values in one quoted variable or multiple ones. See shell parameter expansion for details.
The whole approach is tied to a common prefix for your config variables. In this case, you'll also see $USER in the output, which is the name of the currently logged in user. You can filter that with e.g., grep or a simple if [ "$i" != "USER" ] in the loop.
If you want undefined variables as well, sourcing the users file may not be a good solution. You could instead read the file line by line and check for a leading #:
#!/usr/bin/env bash
set -eu
while IFS= read -r line
do
var=$(echo "$line" | cut -d '=' -f 1)
name=$(echo "$line" | cut -d '=' -f 2)
if [[ "$var" =~ ^# ]]
then
var=$(echo "$var" | cut -c 2-)
echo "The variable $var is not set"
else
echo "The variable $var is set to $name"
fi
done
Output:
bash users.sh < users.conf
The variable USER1_USERNAME is set to "John"
The variable USER2_USERNAME is not set
The variable USER3_USERNAME is set to "David"
The variable USER4_USERNAME is set to "James"
The variable USER5_USERNAME is set to "Jenny"
However, this approach is brittle as it doesn't understand bash syntax. A leading space would be fine when sourcing, but would trip the comment detection on this code. Variables in user names would not get expanded, which may or may not be a good thing.
I liked what Robert suggested. But IMHO whole concept of your script is wrong. This should be done with array(s) not dynamic vars.
users=(
"John"
"Mike"
"David"
"James"
"Jenny"
)
And than you just loop over that array
for user in "${users[#]}"; { echo "$user"; }
If you want to make this config editable by users this can also be done, thay can comment, delete or add items in array:
users=(
"John"
# "David"
"James"
"Jenny"
"Lenny"
)
I also found my own way to deal with this by modifying the source file user.conf to have suffix at the end:
#user.conf
USER_USERNAME1="John"
#USER_USERNAME2="Mike"
USER_USERNAME3="David"
USER_USERNAME4="James"
USER_USERNAME5="Jenny"
#!/bin/bash
source "user.conf"
usernames="${!USER_USERNAME#}"
username_count=$(echo "${usernames}" | wc -w)
count=1
while [[ ${count} -le ${username_count} ]]; do
typeset -n "username"="USER_USERNAME${count}"
if [ -n "${username}" ]; then
echo "The variable USER_USERNAME${count} is set to ${username}"
else
echo "The variable USER_USERNAME${count} is not set"
fi
((count = count + 1))
done
Since it's restricted to put this in question, I post it here.
I am creating a simple phonebook using unix shell scripts. I have gotten all of my functions to work except the removal of a contact after it has been created. I have tried combining grep and sed in order to accomplish this, but cannot seem to get over the hump. The removal shell i've tried is as follows.
#!/bin/sh
#removeContact.sh
echo “Remove Submenu”
echo “Please input First Name:”
read nameFirst
echo “Please input Last Name:”
read nameLast
x=$(grep -e “$nameFirst” -e “$nameLast” ContactList)
echo $x
sed '/'$x'/ d' ContactList;
echo “$nameFirst $nameLast is removed from your contacts”
exit 0
I'm not sure if I am declaring x incorrectly, or if my syntax is wrong when sed is used.
Any help would be greatly appreciated. Thank you.
#!/bin/bash
ContactList="contacts.txt"
export ContactList
exit=0
while [ $exit -ne 1 ]
do
echo "Main Menu"
echo "(a) Add a Contact"
echo "(r) Remove a Contact"
echo "(s) Search a Contact"
echo "(d) Display All Contact’s Information"
echo "(e) Exit"
echo "Your Choice?"
read choice
if [ "$choice" = "a" ]
then
./addContact.sh
elif [ "$choice" = "r" ]
then
./removeContact.sh
elif [ "$choice" = "s" ]
then
./searchContact.sh
elif [ "$choice" = "d" ]
then
./displayContact.sh
elif [ "$choice" = "e" ]
then
exit=1
else
echo "Error"
sleep 2
fi
done
exit 0
#!/bin/sh
#addContact.sh
ContactList="contacts.txt"
echo “Please input First Name:”
read nameFirst
echo “Please input Last Name:”
read nameLast
echo “Please input Phone Number:”
read number
echo “Please Input Address”
read address
echo “Please input Email:”
read email
echo $nameFirst:$nameLast:$number:$address:$email>> ContactList;
echo "A new contact is added to your book."
exit 0
sed '/'$x'/ d' ContactList
won't remove anything from the file ContactList, it will simply output the changes to standard output.
If you want to edit the file in-place, you'll need the -i flag (easy) or to make a temporary file which is then copied back over ContactList (not so easy, but needed if your sed has no in-place editing option).
In addition, since ContactList is a shell variable referencing the real file contacts.txt, you'll need to use $ContactList.
And, as a final note, since you're using the full line content to do deletion, the presence of an address like 1/15 Station St is going to royally screw up your sed command by virtue of the fact it contains the / character.
I would suggest using awk rather than sed for this task since it's much better suited to field-based data. With the record layout:
$nameFirst:$nameLast:$number:$address:$email
you could remove an entry with something like (including my patented paranoid perfect protection policy):
cp contacts.txt contacts.txt.$(date +%Y.%m.%d.%H.%M.%S_$$)
awk <contacts.txt >tmp.$$ -F: "-vF=$nameFirst" "-vL=$nameLast" '
F != $1 || L != $2 {print}'
mv tmp.$$ contacts.txt
I am new to bash script trying to read a file and if the matching string exists, i am putting something in the variable value. i guess it is considering it as a local variable and i am not able to use the value in later part of code.
How to get this value outside?? Please give me all the possible ways
if(...)
then
..........
elif (file exists)
then
cat file | while read line
do
if [ "$line" = "something" ]
then
value="correct"
fi
done
elif()
.........
fi
echo "value is $value"
output:
value is
The while loop is being run in a separate process because it's on the right-side of the pipe, so it can't modify its parent environment. Use redirection instead:
while read line
do
# ...
done < file
Just define value earlier in your code, for instance on your first line:
value=""
if (file exists)
then
cat file | while read line
do
if [ "$line" = "something" ]
then
value="correct"
fi
done
elif()
.........
fi
echo "value is $value"
I'm attempting to read a config file that is formatted as follows:
USER = username
TARGET = arrows
I realize that if I got rid of the spaces, I could simply source the config file, but for security reasons I'm trying to avoid that. I know there is a way to read the config file line by line. I think the process is something like:
Read lines into an array
Filter out all of the lines that start with #
search for the variable names in the array
After that I'm lost. Any and all help would be greatly appreciated. I've tried something like this with no success:
backup2.config>cat ~/1
grep '^[^#].*' | while read one two;do
echo $two
done
I pulled that from a forum post I found, just not sure how to modify it to fit my needs since I'm so new to shell scripting.
http://www.linuxquestions.org/questions/programming-9/bash-shell-program-read-a-configuration-file-276852/
Would it be possible to automatically assign a variable by looping through both arrays?
for (( i = 0 ; i < ${#VALUE[#]} ; i++ ))
do
"${NAME[i]}"=VALUE[i]
done
echo $USER
Such that calling $USER would output "username"? The above code isn't working but I know the solution is something similar to that.
The following script iterates over each line in your input file (vars in my case) and does a pattern match against =. If the equal sign is found it will use Parameter Expansion to parse out the variable name from the value. It then stores each part in it's own array, name and value respectively.
#!/bin/bash
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line#*= }
((i++))
fi
done < vars
echo "total array elements: ${#name[#]}"
echo "name[0]: ${name[0]}"
echo "value[0]: ${value[0]}"
echo "name[1]: ${name[1]}"
echo "value[1]: ${value[1]}"
echo "name array: ${name[#]}"
echo "value array: ${value[#]}"
Input
$ cat vars
sdf
USER = username
TARGET = arrows
asdf
as23
Output
$ ./varscript
total array elements: 2
name[0]: USER
value[0]: username
name[1]: TARGET
value[1]: arrows
name array: USER TARGET
value array: username arrows
First, USER is a shell environment variable, so it might be better if you used something else. Using lowercase or mixed case variable names is a way to avoid name collisions.
#!/bin/bash
configfile="/path/to/file"
shopt -s extglob
while IFS='= ' read lhs rhs
do
if [[ $lhs != *( )#* ]]
then
# you can test for variables to accept or other conditions here
declare $lhs=$rhs
fi
done < "$configfile"
This sets the vars in your file to the value associated with it.
echo "Username: $USER, Target: $TARGET"
would output
Username: username, Target: arrows
Another way to do this using keys and values is with an associative array:
Add this line before the while loop:
declare -A settings
Remove the declare line inside the while loop and replace it with:
settings[$lhs]=$rhs
Then:
# set keys
user=USER
target=TARGET
# access values
echo "Username: ${settings[$user]}, Target: ${settings[$target]}"
would output
Username: username, Target: arrows
I have a script which only takes a very limited number of settings, and processes them one at a time, so I've adapted SiegeX's answer to whitelist the settings I care about and act on them as it comes to them.
I've also removed the requirement for spaces around the = in favour of ignoring any that exist using the trim function from another answer.
function trim()
{
local var=$1;
var="${var#"${var%%[![:space:]]*}"}"; # remove leading whitespace characters
var="${var%"${var##*[![:space:]]}"}"; # remove trailing whitespace characters
echo -n "$var";
}
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
setting_name=$(trim "${line%%=*}");
setting_value=$(trim "${line#*=}");
case "$setting_name" in
max_foos)
prune_foos $setting_value;
;;
max_bars)
prune_bars $setting_value;
;;
*)
echo "Unrecognised setting: $setting_name";
;;
esac;
fi
done <"$config_file";
Thanks SiegeX. I think the later updates you mentioned does not reflect in this URL.
I had to edit the regex to remove the quotes to get it working. With quotes, array returned is empty.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=${line%% =*}
value[i]=${line##*= }
((i++))
fi
done < vars
A still better version is .
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2`
((i++))
fi
done < vars
The first version is seen to have issues if there is no space before and after "=" in the config file. Also if the value is missing, i see that the name and value are populated as same. The second version does not have any of these. In addition it trims out unwanted leading and trailing spaces.
This version reads values that can have = within it. Earlier version splits at first occurance of =.
i=0
while read line; do
if [[ "$line" =~ ^[^#]*= ]]; then
name[i]=`echo $line | cut -d'=' -f 1`
value[i]=`echo $line | cut -d'=' -f 2-`
((i++))
fi
done < vars
My bash script doesn't work the way I want it to:
#!/bin/bash
total="0"
count="0"
#FILE="$1" This is the easier way
for FILE in $*
do
# Start processing all processable files
while read line
do
if [[ "$line" =~ ^Total ]];
then
tmp=$(echo $line | cut -d':' -f2)
count=$(expr $count + 1)
total=$(expr $total + $tmp)
fi
done < $FILE
done
echo "The Total Is: $total"
echo "$FILE"
Is there another way to modify this script so that it reads arguments into $1 instead of $FILE? I've tried using a while loop:
while [ $1 != "" ]
do ....
done
Also when I implement that the code repeats itself. Is there a way to fix that as well?
Another problem that I'm having is that when I have multiple files hi*.txt it gives me duplicates. Why? I have files like hi1.txt hi1.txt~ but the tilde file is of 0 bytes, so my script shouldn't be finding anything.
What i have is fine, but could be improved. I appreciate your awk suggestions but its currently beyond my level as a unix programmer.
Strager: The files that my text editor generates automatically contain nothing..it is of 0 bytes..But yeah i went ahead and deleted them just to be sure. But no my script is in fact reading everything twice. I suppose its looping again when it really shouldnt. I've tried to silence that action with the exit commands..But wasnt successful.
while [ "$1" != "" ]; do
# Code here
# Next argument
shift
done
This code is pretty sweet, but I'm specifying all the possible commands at one time. Example: hi[145].txt
If supplied would read all three files at once.
Suppose the user enters hi*.txt;
I then get all my hi files read twice and then added again.
How can I code it so that it reads my files (just once) upon specification of hi*.txt?
I really think that this is because of not having $1.
It looks like you are trying to add up the totals from the lines labelled 'Total:' in the files provided. It is always a good idea to state what you're trying to do - as well as how you're trying to do it (see How to Ask Questions the Smart Way).
If so, then you're doing in about as complicated a way as I can see. What was wrong with:
grep '^Total:' "$#" |
cut -d: -f2 |
awk '{sum += $1}
END { print sum }'
This doesn't print out "The total is" etc; and it is not clear why you echo $FILE at the end of your version.
You can use Perl or any other suitable program in place of awk; you could do the whole job in Perl or Python - indeed, the cut work could be done by awk:
grep "^Total:" "$#" |
awk -F: '{sum += $2}
END { print sum }'
Taken still further, the whole job could be done by awk:
awk -F: '$1 ~ /^Total/ { sum += $2 }
END { print sum }' "$#"
The code in Perl wouldn't be much harder and the result might be quicker:
perl -na -F: -e '$sum += $F[1] if m/^Total:/; END { print $sum; }' "$#"
When iterating over the file name arguments provided in a shell script, you should use '"$#"' in place of '$*' as the latter notation does not preserve spaces in file names.
Your comment about '$1' is confusing to me. You could be asking to read from the file whose name is in $1 on each iteration; that is done using:
while [ $# -gt 0 ]
do
...process $1...
shift
done
HTH!
If you define a function, it'll receive the argument as $1. Why is $1 more valuable to you than $FILE, though?
#!/bin/sh
process() {
echo "doing something with $1"
}
for i in "$#" # Note use of "$#" to not break on filenames with whitespace
do
process "$i"
done
while [ "$1" != "" ]; do
# Code here
# Next argument
shift
done
On your problem with tilde files ... those are temporary files created by your text editor. Delete them if you don't want them to be matched by your glob expression (wildcard). Otherwise, filter them in your script (not recommended).