The command:
value=${value%?}
will remove the last character from a variable.
Is there any logical reason why it would not work from within a script?
In my script it has no effect whatsoever.
if [[ $line =~ "What I want" ]]
then
if [[ $CURRENT -eq 3 ]]
then
echo "line is " $line
value=`echo "$line" | awk '{print $4}'`
echo "value = "$value
value=${value%?}
echo "value = $value "
break
fi
fi
I cant post the whole script, but this is the piece I refer to. The loop is being entered properly, but the 2 echo $value lines return the same thing.
Edit - this question still stands. The code works fine line bu line in a terminal, but all together in a script it fails.
Echo adds an extra line character to $value in this line:
value=`echo "$line" | awk '{print $4}'`
And afaik that extra char is removed with %?, so it seems it does not change anything at all.
Try echo -n instead, which does not add \n to the string:
value=`echo -n "$line" | awk '{print $4}'`
Since you have provided only the relevant part in the code and not the whole file, I'm going to assume that the first line of your file reads `#!/bin/sh'. This is your problem. What you are trying to do (parameter expansion) is specific to bash, so unless /bin/sh points to bash via a symlink, then you are running the script in a shell which does not understand bash parameter expansion.
To see what /bin/sh really is you can do: ls -l /bin/sh. And to remedy the situation, force the script to run in bash by changing the `shebang' at the top to read `#!/bin/bash'
Related
This part of my script is comparing each line of a file to find a preset string. If the string does NOT exist as a line in the file, it should append it to the end of the file.
STRING=foobar
cat "$FILE" | while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
However, it appears as if both $LINE and $ISLINEINFILE are both cleared upon finishing the do loop. How can I avoid this?
Using shell
If we want to make just the minimal change to your code to get it working, all we need to do is switch the input redirection:
string=foobar
while read line
do
if [ "$string" == "$line" ]; then
islineinfile="yes"
fi
done <"$file"
if [ ! "$islineinfile" == yes ]; then
echo "$string" >> "$file"
fi
In the above, we changed cat "$file" | while do ...done to while do...done<"$file". With this one change, the while loop is no longer in a subshell and, consequently, shell variables created in the loop live on after the loop completes.
Using sed
I believe that the whole of your script can be replaced with:
sed -i.bak '/^foobar$/H; ${x;s/././;x;t; s/$/\nfoobar/}' file*
The above adds line foobar to the end of each file that doesn't already have a line that matches ^foobar$.
The above shows file* as the final argument to sed. This will apply the change to all files matching the glob. You could list specific files individually if you prefer.
The above was tested on GNU sed (linux). Minor modifications may be needed for BSD/OSX sed.
Using GNU awk (gawk)
awk -i inplace -v s="foobar" '$0==s{f=1} {print} ENDFILE{if (f==0) print s; f=0}' file*
Like the sed command, this can tackle multiple files all in one command.
Why does my variable set in a do loop disappear?
It disappears because it is set in a shell pipeline component. Most shells run each part of a pipeline in a subshell. By Unix design, variables set in a subshell cannot affect their parent or any already running other shell.
How can I avoid this?
There are several ways:
The simplest is to use a shell that doesn't run the last component of a pipeline in a subshell. This is ksh default behavior, e.g. use that shebang:
#!/bin/ksh
This behavior can also be bash one when the lastpipe option is set:
shopt -s lastpipe
You might use the variable in the same subshell that set it. Note that your original script indentation is wrong and might lead to the incorrect assumption that the if block is inside the pipeline, which isn't the case. Enclosing the whole block with parentheses will rectify that and would be the minimal change (two extra characters) to make it working:
STRING=foobar
cat "$FILE" | ( while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
)
The variable would still be lost after that block though.
You might simply avoid the pipeline, which is straigthforward in your case, the cat being unnecessary:
STRING=foobar
while read LINE
do
if [ "$STRING" == "$LINE" ]; then
export ISLINEINFILE="yes"
fi
done < "$FILE"
if [ ! "$ISLINEINFILE" == yes ]; then
echo "$LINE" >> "$FILE"
fi
You might use another argorithmic approach, like using sed or gawk as suggested by John1024.
See also https://unix.stackexchange.com/a/144137/2594 for standard compliance details.
I posted earlier with something similar to this. I am trying to check if a user is online with the command $ ./user mburkhar which prints out mburkhar is logged on. My program works correctly but if I just type $ ./user mb is also states mb is logged on. What I have is fine, but is there a way to match what the user typed in exactly instead of slightly matching the first 2 characters..?
Here is my program so you can see what I did:
# Check if a user is logged on
if [ -z $1 ] ; then
read user
else
user=$1
fi
if [ `who | cut -d" " -f1 | grep $1` ] ; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
To complement the accepted answer:
-w works well and is widely supported (GNU Grep, BSD Grep), but it is not POSIX-compliant.
In your case, given that your output lines contain just a username each and nothing else, using -x - to match entire lines - would make sense too (and it is POSIX-compliant).
Also, since you're searching for a literal username, it's good practice to use grep's -F option to indicate just that.
Using [ $(...) ] (or [ `...` ]) to test for nonempty output from a command is somewhat fragile and inefficient; it's better and simpler to:
use commands directly
base the test on the exit code
and suppress stdout output, if needed
grep's -q option not only suppresses stdout, but also makes the search potentially more efficient by terminating once the first match is found (with exit code 0 to indicate success):
if who | cut -d' ' -f1 | grep -Fxq "$1"; then # ...
Similarly, [ -z $1 ] is fragile in that would break if an argument with embedded whitespace is passed - not likely in this case, but it's better to get in the habit of using [[ -z $1 ]] (or, if you must remain POSIX-compliant, [ -z "$1" ]).
Outside of [[ ... ]], it makes sense to habitually double-quote variable references, such as the $1 in the grep command.
If we put it all together:
# Check if a user is logged on
if [[ -z $1 ]]; then
read user
else
user=$1
fi
if who | cut -d' ' -f1 | grep -Fxq "$1"; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
You can use grep's -w (--word-regexp) option to match only entire words. Replacing grep $1 with grep -w $1 in your script should fix it.
Specify the start and end of string anchors like this:
if [ -z $1 ] ; then
read user
else
user="^$1$"
fi
if [ $(who | cut -d" " -f1 | grep $user | uniq ) ] ; then
echo "$1 is logged on"
else
echo "$1 is either not valid or logged on"
fi
I also added the uniq to remove duplicate matches, in case an user got more than one tty, or [ will exit with: [: too many arguments
In a bash script, I have a list of lines in a file I wish to grep and then display on standard out, which is easiest done with a while read:
grep "regex" "filepath" | while read line; do
printf "$line\n"
done
However, I would like to inform the user if no lines were matched by the grep. I know that one can do this by updating a variable inside the loop but it seems like a much more elegant approach (if possible) would be to try to read a line in an until loop, and if there were no output, an error message could be displayed.
This was my first attempt:
grep "regex" "filepath" | until [[ -z ${read line} ]]; do
if [[ -z $input ]]; then
printf "No matches found\n"
break
fi
printf "$line\n"
done
But in this instance the read command is malformed, and I wasn't sure of another way the phrase the query. Is this approach possible, and if not, is there a more suitable solution to the problem?
You don't need a loop at all if you simply want to display a message when there's no match. Instead you can use grep's return code. A simple if statement will suffice:
if ! grep "regex" "filepath"; then
echo "no match" >&2
fi
This will display the results of grep matches (since that's grep's default behavior), and will display the error message if it doesn't.
A popular alternative to if ! is to use the || operator. foo || bar can be read as "do foo or else do bar", or "if not foo then bar".
grep "regex" "filepath" || echo "no match" >&2
John Kugelman's answer is the correct and succinct one and you should accept it. I am addressing your question about syntax here just for completeness.
You cannot use ${read line} to execute read -- the brace syntax actually means (vaguely) that you want the value of a variable whose name contains a space. Perhaps you were shooting for $(read line) but really, the proper way to write your until loop would be more along the lines of
grep "regex" "filepath" | until read line; [[ -z "$line" ]]; do
... but of course, when there is no output, the pipeline will receive no lines, so while and until are both wrong here.
It is worth amphasizing that the reason you need a separate do is that you can have multiple commands in there. Even something like
while output=$(grep "regex filepath"); echo "grep done, please wait ...";
count=$(echo "$output" | wc -l); [[ $count -gt 0 ]]
do ...
although again, that is much more arcane than you would ever really need. (And in this particular case, you would want probably actually want if , not while.)
As others already noted, there is no reason to use a loop like that here, but I wanted to sort out the question about how to write a loop like this for whenever you actually do want one.
As mentioned by #jordanm, there is no need for a loop in the use case you mentioned.
output=$(grep "regex" "file")
if [[ -n $output ]]; then
echo "$output"
else
echo "Sorry, no results..."
fi
If you need to iterate over the results for processing (rather than just displaying to stdout) then you can do something like this:
output=$(grep "regex" "file")
if [[ -n $output ]]; then
while IFS= read -r line; do
# do something with $line
done <<< "$output"
else
echo "Sorry, no results..."
fi
This method avoids using a pipeline or subshell so that any variable assignments made within the loop will be available to the rest of the script.
Also, i'm not sure if this relates to what you are trying to do at all, but grep does have the ability to load patterns from a file (one per line). It is invoked as follows:
grep search_target -f pattern_file.txt
I am trying to escape spaces in a path name /Volumes/NO NAME. I need to access files present within this directory via a bash script. I have used sed and printf commands and neither seems to have worked. The code I have so far:
while read line
do
if [[ -d ${line} ]]; then
echo "${line} is a directory. Skipping this.";
elif [[ -f ${line} ]]; then
spacedOutLine=`echo ${line} | sed -e 's/ /\\ /g'`;
#spacedOutLine=$(printf %q "$line");
echo "line = ${line} and spacedOutLine = ${spacedOutLine}";
else
echo "Some other type of file. Unrecognized.";
fi
done < ${2}
Neither of this seems to have worked. For an input like: /Volumes/NO NAME/hello.txt, the output is:
/Volumes/NO: No such file or directory
NAME/hello.txt: No such file or directory
I am on a Mac and using bash via the terminal. I have also gone through a lot of other posts on SO about this which have not helped me.
Use double quotes while calling the ${line} variable.
"${line}"
The output is blank fr the below script. What is it missing? I am trying to grep a string
#!/bin/ksh
file=$abc_def_APP_13.4.5.2
if grep -q abc_def_APP $file; then
echo "File Found"
else
echo "File not Found"
fi
In bash, use the <<< redirection from a string (a 'Here string'):
if grep -q abc_def_APP <<< $file
In other shells, you may need to use:
if echo $file | grep -q abc_def_APP
I put my then on the next line; if you want your then on the same line, then add ; then after what I wrote.
Note that this assignment:
file=$abc_def_APP_13.4.5.2
is pretty odd; it takes the value of an environment variable ${abc_def_APP_13} and adds .4.5.2 to the end (it must be an env var since we can see the start of the script). You probably intended to write:
file=abc_def_APP_13.4.5.2
In general, you should enclose references to variables holding file names in double quotes to avoid problems with spaces etc in the file names. It is not critical here, but good practices are good practices:
if grep -q abc_def_APP <<< "$file"
if echo "$file" | grep -q abc_def_APP
Yuck! Use the shell's string matching
if [[ "$file" == *abc_def_APP* ]]; then ...