I am performing a grep on a file which is resulting in a single line output. This output has * as data in it. In the shell script I am trying to assign the value to a variable but * is being replaced with the file list in the current folder.
Eg:
My script name is script1.sh and I have another file script2.sh in the same directory.
The content of the script is
VAR1=`grep pattern search_file`
echo $VAR1
The intended output would be
The pattern is *
But the output I am getting is
The pattern is script1.sh script2.sh
Kindly let me know what is that I am doing wrong.
You simply need to quote the variable: echo "$VAR1"
If you look at the sequence of bash shell expansions, you'll notice that filename expansion occurs after parameter expansion. Unquoted variables will be subsequently subjected to word splitting and filename expansion.
use set -f shell option will disable globbing in sub-shells and interactive session.
use set +f to enable globbing again.
You need to escape the * with \, otherwise it treats it as a wild-card that matches filenames in the current directory.
\*
Related
What else could be going wrong? Sorry I'm pretty new to programming so I'm not sure if this is the proper way to frame my question.
Here is the code from the terminal file:
echo "Patcher Coded by _Retro_"
PLACE=`dirname $0`
ROM=`ls ${PLACE}/Rom/*.nds | head -n 1`
PATCH=`ls ${PLACE}/Patch/*.* | head -n 1`
NAME=${ROM%.[^.]*}
$PLACE/xdelta3 -dfs $ROM $PATCH $NAME-patched.nds
Your script says this:
PLACE=`dirname $0`
First, the shell performs parameter expansion. That means (in this case) it expands $0. The variable $0 expands to the path used by the shell to execute your script, so that line expands to this:
PLACE=`dirname /Users/ShakeyBanks/Desktop/Perfect Heart CE./DS_Rom_Patcher/Rom_Patcher`
Note that there are no backslashes in the expansion! The backslashes were consumed by your interactive shell before starting the script.
Then the shell performs command substitution: it executes the command enclosed in `...`. The shell splits the command on spaces, so the command contains four words. The first word is the program to run, and the remaining three words are arguments to that command:
dirname
/Users/ShakeyBanks/Desktop/Perfect
Heart
CE./DS_Rom_Patcher/Rom_Patcher
The problem here is that the dirname program only wants one argument, but you're passing it three. It detects this and fails with an error:
usage: dirname path
To fix this, quote the $0 with double-quotes, like this:
PLACE=`dirname "$0"`
You also need to quote all subsequent uses of $PLACE, ${PLACE}, $ROM, $PATCH, and $NAME with double-quotes, because they will have the same problem.
OR, rename your directory to not contain spaces.
This is a simple question but i am unable to find it in tutorials. Could anybody please explain what this statement does when executed in a bash shell within a folder containing .sh scripts. I know -i does in place editing, i understand that it will run sed on all scripts within the current directory. And i know that it does some sort of substitution. But what does this \(.*\) mean?
sed -i 's/MY_BASE_DIR=\(.*\)/MY_BASE_DIR=${MY_BASE_DIR-\1}/' *.sh
Thanks in advance.
You have an expression like:
sed -i 's/XXX=\(YYY\)/XXX=ZZZ/' file
This looks for a string XXX= in a file and captures what goes after. Then, it replaces this captured content with ZZZ. Since there is a captured group, it is accessed with \1. Finally, using the -i flag in sed makes the edition to be in-place.
For the replacement, it uses the following syntax described in Shell parameter expansion:
${parameter:-word}
If parameter is unset or null, the expansion of word is substituted.
Otherwise, the value of parameter is substituted.
Example:
$ d=5
$ echo ${d-3}
5
$ echo ${a-3}
3
So with ${MY_BASE_DIR-SOMETHING-\1} you are saying: print $MY_BAS_DIR. And if this variable is unset or null, print what is stored in \1.
All together, this is resetting MY_BASE_DIR to the value in the variable $MY_BASE_DIR unless this is not set; in such case, the value remains the same.
Note though that the variable won't be expanded unless you use double quotes.
Test:
$ d=5
$ cat a
d=23
blabla
$ sed "s/d=\(.*\)/d=${d-\1}/" a # double quotes -> value is replaced
d=5
blabla
$ sed 's/d=\(.*\)/d=${d-\1}/' a # single quotes -> variable is not expanded
d=${d-23}
blabla
Andd see how the value remains the same if $d is not set:
$ unset d
$ sed "s/d=\(.*\)/d=${d-\1}/" a
d=23
The scripts contain lines like this:
MY_BASE_DIR=/usr/local
The sed expression changes them to:
MY_BASE_DIR=${MY_BASE_DIR-/usr/local}
The effect is that /usr/local is not used as a fixed value, but only as the default value. You can override it by setting the environment variable MY_BASE_DIR.
For future reference, I would take a look at the ExplainShell website:
http://explainshell
that will give you a breakdown of the command structure etc. In this instance, let step through the details...Let's start with a simple example, let's assume that we were going to make the simple change - commenting out all lines by adding a "#" before each line. We can do this for all *.sh files in a directory with the ".sh" extension in the current directory:
sed 's/^/\#/' *.sh
i.e. Substitute beginning of line ^, with a # ...
Caveat: You did not specify the OS you are using. You may get different results with different versions of sed and OS...
ok, now we can drill into the substitution in the script. An example is probably easier to explain:
File: t.sh
MY_BASE_DIR="/important data/data/bin"
the command 's/MY_BASE_DIR=\(.*\)/MY_BASE_DIR=${MY_BASE_DIR-\1}/' *.sh
will search for "MY_BASE_DIR" in each .sh file in the directory.
When it encounters the string "MY_BASE_DIR=.*", in the file, it expands it to be MY_BASE_DIR="/important data/data/bin", this is now replaced on the right side of the expression /MY_BASE_DIR=${MY_BASE_DIR-\1}/ which becomes
MY_BASE_DIR=${MY_BASE_DIR-"/important data/data/bin"}
essentially what happens is that the substitute operation takes
MY_BASE_DIR="/important data/data/bin"
and inserts
MY_BASE_DIR=${MY_BASE_DIR-"/important data/data/bin"}
now if we run the script with the variable MY_BASE_DIR set
export MY_BASE_DIR="/new/import/dir"
the scripts modified by the sed script referenced will now substitute /important data/data/bin with /new/import/dir...
My problem boils down to this:
echo $(echo '*')
That outputs the names of all the files in the current directory.
I do not want that. I want a literal asterisk (*).
How do I do this in a generic way?
My above example is simplified. The asterisk is not literally written in my bash script - it comes from the result of another command.
So this is perhaps closer to my real situation:
echo $(my-special-command)
I just want to get the literal output of my-special-command; I do not want any embedded asterisks (or other special characters) to be expanded.
How do I do this in a general-purpose way?
I suppose I could do set -f before running the command, but how can I be sure that covers everything? That turns off pathname expansion, but what about other kinds? I have zero control over what output might be produced by my-special-command, so must be able to handle everything properly.
Just enclose the Command substitution with double quotes:
echo "$(my-special-command)"
Its called globbing, you have multiply ways to prevent it:
echo * # will expand to files / directories
echo "*" # Will print *
echo '*' # Will also print *
In your example you can simple write:
echo "$(echo '*')"
You can also turn off globbing in your script by calling it with bash -f script.sh or inside your code:
#!/usr/bin/env bash
set -f
echo *
From the "Command Substitution" section of the man page:
If the [command] substitution appears within double quotes, word splitting and
pathname expansion are not performed on the results.
By quoting the command expansion, you prevent its result, *, from undergoing pathname expansion.
$ echo "$(echo "*")"
I have a bash script and I want to use that for replacing some lines with a string and add a date to the end of the line:
#! /bin/bash
today=`date '+%Y_%m_%d__%H_%M_%S'`;
sed -i '3s/.*/CONFIG_LOCALVERSION=/' ~/Desktop/file1 file2 ...
Also, can I do this for a range of files that start with a string like "file"?
To use variable expansion in bash, the variables must be non-quoted or double-quoted. Single quotes will prevent the expansion. On the other hand, you'd want to avoid expansion of * in 3s/.*/ in case you have a directory 3s containing files starting with ..
Fortunately, you can just chain strings together, so you can do
#!/bin/bash
today=$(date '+%Y_%m_%d__%H_%M_%S');
sed -i '3s/.*/CONFIG_LOCALVERSION='"$today"'/' ~/Desktop/file{1,2,Foo}
and can i do this for a range of file that start with a string like "file" ?
The glob ~/Desktop/file{1,2,Foo} will expand to ~/Desktop/file1 ~/Desktop/file2 ~/Desktop/fileFoo. If instead you want to match all files on your Desktop with a name starting with 'file', use ~/Desktop/file* instead.
I'm struggling with Bash variable expansion. Please see the following code:
~/tmp 689$ a=~/Library/Application\ *; echo $a
/Users/foo/Library/Application *
~/tmp 690$ echo ~/Library/Application\ *
/Users/foo/Library/Application Scripts /Users/foo/Library/Application Support
As the order of expansion is brace->tilde->parameter->....->pathname,
why is pathname expansion not applied to $a in the same way that it is in the 2nd command?
[added]
Does whitespace escaping have hidden behaviour regarding the following output?
~/tmp 705$ a=~/Library/Application*; echo $a
/Users/foo/Library/Application Scripts /Users/foo/Library/Application Support
To do what you meant to do, you'd have to use the following:
a=(~/Library/Application\ *) # use an *array* to capture the pathname-expanded results
echo "${a[#]}" # output all array elements (without further expansion)
As for why your code didn't work:
In the context of variable assignment involving only literals or string interpolation (references to other variables), NO pathname expansion takes place, even with unquoted strings (e.g., a=*, a="*", and a='*' all assign literal *)[1].
(By contrast, pathname expansion is applied to unquoted strings inside an array definition (e.g., a=(*), or inside a command substitution (e..g, a=$(echo *)).)
Thus, the literal content of $a is /Users/foo/Library/Application *
Executing echo $a - i.e., NOT double-quoting the variable reference $a - then applies word splitting and does the following:
it prints literal '/Users/foo/Library/Application' (the 1st word - no expansion applied, due to its contents)
it prints the pathname expansion applied to * (the 2nd word - i.e., it expands to matching filenames in the current dir.)
The fact that the latter results in * in your case implies that you happen to be running the echo command from an empty directory (save for hidden files, assuming the default configuration).
[1]Whether the string is unquoted or not does, however, matter with respect to tilde expansion; e.g., a=~ expands ~ to the user's home directory, whereas a='~' or a="~" assign literal ~.