I am trying to create a script that replaces sub strings with an absolute path, so I'm using
sed s/TEMPDIR/$PWD/ file.py > newfile.py
So obviously I want to replace TEMP with the absolute path, but I keep getting
sed: -e expression #1, char 12: unknown option to `s'
I think it might be because the path contains '/', but I don't know how to solve it. Any ideas?
Also, deos PWD guaranties an absolute path, given that I don't deal with links?
Thanks!!
use a different delimiter:
sed "s#TEMPDIR#$PWD#" file.py > newfile.py
Using PWD probably does not guarantee an absolute path. (Depends on the shell). But pwd will return an absolute path, so use:
sed "s#TEMPDIR#$(pwd)#" file.py > newfile.py
Note that neither of these will work if your paths contain #. Pick a delimiter that does not appear in either. Note the following from the shell standard:
In cases where PWD is set to the pathname that would be output by pwd -P, if there is insufficient permission on the current working directory, or on any parent of that directory, to determine what that pathname would be, the value of PWD is unspecified. Assignments to this variable may be ignored. If an application sets or unsets the value of PWD , the behaviors of the cd and pwd utilities are unspecified.
So you can probably rely on $PWD. Note that neither pwd nor $PWD will resolve symbolic links, so you might prefer to use readlink -f .
Related
In tcsh I can extract second path element from the end of path by following way
cd /some/long/directory/structure/path/
set x=`pwd`
echo ${x:h:h:t}
directory
How can I do the same in bash?
I mean , does bash also have this kind of modifiers?
The csh-style modifiers can be used with history expansion (unsurprisingly, because history expansion was borrowed from csh).
$ cd /some/long/directory/structure/path/
$ echo !!:1:h:h:t
echo directory
directory
!!:1 selects word 1 (counting from zero) of the previous command, so the argument to cd.
(echo directory appears on standard error because the shell defaults to displaying the result of history expansion before actually executing the resulting command.)
In a non-interactive bash script, history expansion commands as in #chepner's answer won't normally be available. However, you do have parameter expansions like:
$ cd /some/long//directory///structure/path/
$ set x=$(pwd)
$ echo $x
/some/long/directory/structure/path
$ y=${y%/*/*} # each /* is equivalent to one :h
$ y=${y##*/} # equivalent to :t
$ echo $y
directory
cd /some/long/path/somewhere
x=$PWD
basename "$(dirname "$x")"
> path
dirname gets the absolute path of the parent folder of the argument. basename gets the name of the argument.
Edit: remembered the much better way than I was doing before.
I am trying to copy a .nii file (Gabor3.nii) path to a variable but even though the file is found by the find command, I can't copy the path to the variable.
find . -type f -name "*.nii"
Data= '/$PWD/"*.nii"'
output:
./Gabor3.nii
./hello.sh: line 21: /$PWD/"*.nii": No such file or directory
What went wrong
You show that you're using:
Data= '/$PWD/"*.nii"'
The space means that the Data= parts sets an environment variable $Data to an empty string, and then attempts to run '/$PWD/"*.nii"'. The single quotes mean that what is between them is not expanded, and you don't have a directory /$PWD (that's a directory name of $, P, W, D in the root directory), so the script "*.nii" isn't found in it, hence the error message.
Using arrays
OK; that's what's wrong. What's right?
You have a couple of options. The most reliable is to use an array assignment and shell expansion:
Data=( "$PWD"/*.nii )
The parentheses (note the absence of spaces before the ( — that's crucial) makes it an array assignment. Using shell globbing gives a list of names, preserving spaces etc in the names correctly. Using double quotes around "$PWD" ensures that the expansion is correct even if there are spaces in the current directory name.
You can find out how many files there are in the list with:
echo "${#Data[#]}"
You can iterate over the list of file names with:
for file in "${Data[#]}"
do
echo "File is [$file]"
ls -l "$file"
done
Note that variable references must be in double quotes for names with spaces to work correctly. The "${Data[#]}" notation has parallels with "$#", which also preserves spaces in the arguments to the command. There is a "${Data[*]}" variant which behaves analogously to "$*", and is of similarly limited value.
If you're worried that there might not be any files with the extension, then use shopt -s nullglob to expand the globbing expression into an empty list rather than the unexpanded expression which is the historical default. You can unset the option with shopt -u nullglob if necessary.
Alternatives
Alternatives involve things like using command substitution Data=$(ls "$PWD"/*.nii), but this is vastly inferior to using an array unless neither the path in $PWD nor the file names contain any spaces, tabs, newlines. If there is no white space in the names, it works OK; you can iterate over:
for file in $Data
do
echo "No white space [$file]"
ls -l "$file"
done
but this is altogether less satisfactory if there are (or might be) any white space characters around.
You can use command substitution:
Data=$(find . -type f -name "*.nii" -print -quit)
To prevent multiline output, the -quit option stop searching after the first file was found(unless you're sure only one file will be found or you want to process multiple files).
The syntax to do what you seem to be trying to do with:
Data= '/$PWD/"*.nii"'
would be:
Data="$(ls "$PWD"/*.nii)"
Not saying it's the best approach for whatever you want to do next of course, it's probably not...
I am trying to list all files located in specific sub-directories of a directory in my bash script. Here is what I tried.
root_dir="/home/shaf/data/"
sub_dirs_prefixes=('ab' 'bp' 'cd' 'cn' 'll' 'mr' 'mb' 'nb' 'nh' 'nw' 'oh' 'st' 'wh')
ls "$root_dir"{$(IFS=,; echo "${sub_dirs_prefixes[*]}")}"rrc/"
Please note that I do not want to expand value stored in $root_dir as it may contain spaces but I do want to expand sub-path contained in {} which is a comma delimited string of contents of $sub_dirs_prefixes. I stored sub-directories prefixes in an array variable, $sub_dirs_prefixes , because I have to use them later on for something else.
I am getting following error:
ls: cannot access /home/shaf/data/{ab,bp,cd,cn,ll,mr,mb,nb,nh,nw,oh,st,wh}rrc/
If I copy the path in error message and run ls from command line, it does display contents of listed sub-directories.
You can command substitution to generate an extended pattern.
shopt -s extglob
ls "$root_dir"/$(IFS="|"; echo "#(${sub_dirs_prefixes[*]})rrc")
By the time parameter can command substitutions have completed, the shell sees this just before performing pathname expansion:
ls "/home/shaf/data/"/#(ab|bp|cd|cn|ll|mr|mb|nb|nh|nw|oh|st|wh)rrc
The #(...) pattern matches one of the enclosed prefixes.
It gets a little trickier if the components of the directory names contain characters that need to be quoted, since we aren't quoting the command substitution.
This is a simple question but i am unable to find it in tutorials. Could anybody please explain what this statement does when executed in a bash shell within a folder containing .sh scripts. I know -i does in place editing, i understand that it will run sed on all scripts within the current directory. And i know that it does some sort of substitution. But what does this \(.*\) mean?
sed -i 's/MY_BASE_DIR=\(.*\)/MY_BASE_DIR=${MY_BASE_DIR-\1}/' *.sh
Thanks in advance.
You have an expression like:
sed -i 's/XXX=\(YYY\)/XXX=ZZZ/' file
This looks for a string XXX= in a file and captures what goes after. Then, it replaces this captured content with ZZZ. Since there is a captured group, it is accessed with \1. Finally, using the -i flag in sed makes the edition to be in-place.
For the replacement, it uses the following syntax described in Shell parameter expansion:
${parameter:-word}
If parameter is unset or null, the expansion of word is substituted.
Otherwise, the value of parameter is substituted.
Example:
$ d=5
$ echo ${d-3}
5
$ echo ${a-3}
3
So with ${MY_BASE_DIR-SOMETHING-\1} you are saying: print $MY_BAS_DIR. And if this variable is unset or null, print what is stored in \1.
All together, this is resetting MY_BASE_DIR to the value in the variable $MY_BASE_DIR unless this is not set; in such case, the value remains the same.
Note though that the variable won't be expanded unless you use double quotes.
Test:
$ d=5
$ cat a
d=23
blabla
$ sed "s/d=\(.*\)/d=${d-\1}/" a # double quotes -> value is replaced
d=5
blabla
$ sed 's/d=\(.*\)/d=${d-\1}/' a # single quotes -> variable is not expanded
d=${d-23}
blabla
Andd see how the value remains the same if $d is not set:
$ unset d
$ sed "s/d=\(.*\)/d=${d-\1}/" a
d=23
The scripts contain lines like this:
MY_BASE_DIR=/usr/local
The sed expression changes them to:
MY_BASE_DIR=${MY_BASE_DIR-/usr/local}
The effect is that /usr/local is not used as a fixed value, but only as the default value. You can override it by setting the environment variable MY_BASE_DIR.
For future reference, I would take a look at the ExplainShell website:
http://explainshell
that will give you a breakdown of the command structure etc. In this instance, let step through the details...Let's start with a simple example, let's assume that we were going to make the simple change - commenting out all lines by adding a "#" before each line. We can do this for all *.sh files in a directory with the ".sh" extension in the current directory:
sed 's/^/\#/' *.sh
i.e. Substitute beginning of line ^, with a # ...
Caveat: You did not specify the OS you are using. You may get different results with different versions of sed and OS...
ok, now we can drill into the substitution in the script. An example is probably easier to explain:
File: t.sh
MY_BASE_DIR="/important data/data/bin"
the command 's/MY_BASE_DIR=\(.*\)/MY_BASE_DIR=${MY_BASE_DIR-\1}/' *.sh
will search for "MY_BASE_DIR" in each .sh file in the directory.
When it encounters the string "MY_BASE_DIR=.*", in the file, it expands it to be MY_BASE_DIR="/important data/data/bin", this is now replaced on the right side of the expression /MY_BASE_DIR=${MY_BASE_DIR-\1}/ which becomes
MY_BASE_DIR=${MY_BASE_DIR-"/important data/data/bin"}
essentially what happens is that the substitute operation takes
MY_BASE_DIR="/important data/data/bin"
and inserts
MY_BASE_DIR=${MY_BASE_DIR-"/important data/data/bin"}
now if we run the script with the variable MY_BASE_DIR set
export MY_BASE_DIR="/new/import/dir"
the scripts modified by the sed script referenced will now substitute /important data/data/bin with /new/import/dir...
I'm just trying to understand what is happening here, so that I understand how to parse strings in shell scripts better.
I know that usually, when you try to pass a string of arguments separated by spaces directly to a command, they will be treated as a single string argument and therefore not recognized:
>check="FileA.txt FileB.txt"
>ls $check
ls: cannot access FileA.txt FileB.txt: No such file or directory
However, in this script two arguments are taken each as space separated strings. In this case, both strings are recognizes as lists of arguments that can be passed to different commands:
testscript.sh
while getopts o:p: arguments
do
case $arguments in
o) olist="$OPTARG";;
p) plist=$OPTARG;;
esac
done
echo "olist"
ls -l $olist
echo "plist"
ls -l $plist
the output is then as follows:
>testscript.sh -o "fileA.txt fileB.txt" -p "file1.txt file2.txt"
Olist
fileA.txt
fileB.txt
plist
file1.txt
file2.txt
What is different here? Why are the space separated strings suddenly recognized as lists?
Your script does not start with a #!-line and does therefore not specify an interpreter. In that case the default is used, which is /bin/sh and not your login shell or the shell you are starting the script from (unless that is /bin/sh of course). Chances are good that /bin/sh is not a zsh, as most distributions and Unices seem to use sh, bash, dash or ksh as default shell. All of which handle parameter expansion such that strings are handles as lists if the parameter was not quoted with double-quotes.
If you want to use zsh as interpreter for your scripts, you have to specify it in the first line of the script:
#!/usr/bin/zsh
Modify the path to wherever your zsh resides.
You can also use env as a wrapper:
#!/usr/bin/env zsh
This makes you more independent of the actual location of zsh, it just has to be in $PATH.
As a matter of fact (using bash)...
sh$ check="FileA.txt FileB.txt"
sh$ ls $check
ls: cannot access FileA.txt: No such file or directory
ls: cannot access FileB.txt: No such file or directory
When you write $check without quotes, the variable is substituted by its content. Insides paces (or to be precises inside occurrences of IFS) are considered as field separators. Just as you where expecting it first.
The only way I know to reproduce your behavior is to set IFS to something else than its default value:
sh$ export IFS="-"
sh$ check="FileA.txt FileB.txt"
sh$ ls $check
ls: cannot access FileA.txt FileB.txt: No such file or directory