Bash shell / Command: strip out arbitrary components in a given path? - bash

Say if I have:
MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
then I want to construct another path AllHeaders/NetworkUtil/network.h
I actually need to get different components out fro the original path, is there a way to do it?
I found in:
Bash: remove first directory component from variable (path of file)
I can something like
${MYPATH#../Library}
to strip out the specified part, but that assumes I know the structure already, what if in my case I need the 3rd and last components in the original path?
Thanks

You can use bash arrays to access individual elements:
$ MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
$ OLD="$IFS"
$ IFS='/' a=($MYPATH)
$ IFS="$OLD"
$ NEWPATH="AllHeaders/${a[2]}/${a[-1]}"
$ echo $NEWPATH
AllHeaders/NetworkUtil/network.h
ADDENDUM: For completeness, another way to make MYPATH into an array is to use bash's pattern substitution: a=(${MYPATH//\// }):
$ MYPATH=../Library/NetworkUtil/Classes/Headers/network.h
$ a=(${MYPATH//\// })
$ NEWPATH="AllHeaders/${a[2]}/${a[-1]}"
$ echo $NEWPATH
AllHeaders/NetworkUtil/network.h
This eliminates the need for messing with IFS but would break badly if MYPATH had spaces, tabs, or CR in it to begin with.

Related

How to expand macros in strings read from a file in a ksh script?

I want to read a list of file names stored in a file, and the top level directory is a macro, since this is for a script that may be run in several environments.
For example, there is a file file_list.txt holding the following fully qualified file paths:
$TOP_DIR/subdir_a/subdir_b/file_1
$TOP_DIR/subdir_x/subdir_y/subdir_z/file_2
In my script, I want to tar the files, but in order to do that, tar must know the actual path.
How can I get the string containing the file path to expand the macro to get the actual path?
In the code below the string value echoed is exactly as in the file above.
I tried using actual_file_path=`eval $file_path` and while eval does evaluate the macro, it returns a status, not the evaluated path.
for file_path in `cat $input_file_list`
do
echo "$file_path"
done
With the tag ksh I think you do not have the utility envsubst.
When the number of variables in $input_file_list is very limited, you can substitute vars with awk :
awk -v top_dir="${TOP_DIR}" '{ sub(/$TOP_DIR/, top_dir); print}' "${input_file_list}"
I was using eval incorrectly. The solution is to use an assignment on the right side of eval as follows:
for file_path in `cat $input_file_list`
do
eval myfile=$file_name
echo "myfile = $myfile"
done
$myfile now has the actual expansion of the macro.

Assigning a variable (with wildcard) with parentheses versus none

I have a simple naive question, I've figured out how to make my script run but I'd like to know why it didn't work previously.
I was assigning a variable with a wildcard using syntax similar to:
var=$dir/$subj/name*text*text.nii.gz
I could call the proper filename with ls $file, but when I tried to substitute in $file as an input into a command line (using FSL for image processing), I got an error saying it couldn't find the file with wildcards in place.
However, when I assign the variable with parentheses:
var=($dir/$subj/name*text*text.nii.gz)
It runs just fine. I'm assuming there are other and probably better ways to do this, but I'm just wondering why the initial variable assignment didn't work, and what the optimal way to assign variables in this manner is.
Thanks!
Let's consider a directory with three files:
$ ls
file1 file2 file3
Now define a variable:
$ var=file*
We can see what is in var by using declare -p:
$ declare -p var
declare -- var="file*"
As you can see, var still has the * in it. This is because pathname expansion is not performed for variable assignments. Consequently, var will not always work as you may have wanted. For example:
$ ls "$var"
ls: cannot access file*: No such file or directory
Next, let's try creating an array:
$ var=(file*)
$ declare -p var
declare -a var='([0]="file1" [1]="file2" [2]="file3")'
As you can see, pathname expansion is performed on arrays. Consequently, the following does work:
$ ls "$var"
file1
But, note that, for an array, $var refers only to the first element. If you wanted to access all its entries, a more complex notation is needed:
$ ls "${var[#]}"
file1 file2 file3

bash: get directory from certain part of path onwards

I have an arbitrary path which contains the directory mydir:
/some/path/to/mydir/further/path/file.ext
I want to get the part after mydir, in this example:
/further/path/file.ext
Please note that the levels of subdirectories are also arbitrary, so a path like
/yet/another/long/path/to/mydir/file.ext
is also possible (where the result would be "file.ext")
The first occurrence of mydir should be used, so the path
/path/mydir/some/other/path/mydir/path/file.ext
should result in
/some/other/path/mydir/path/file.ext
How can one do this with bash?
Note. It is assumed that mydir will always appear enclosed between slashes.
after=${mydir#*/mydir/}
if [ "$mydir" = "$after" ]; then
fail_with_error "Path does not contain /mydir/"
fi
after="/$after"
In line 1, the # means substring after, and the * is the usual placeholder. To be safe against directories like .../mydirectaccess/... I included the slashes at both ends of mydir. Line 5 just prepends the slash that had been taken off by line 1.
Using Shell Parameter Expansion:
$ mydir="/some/path/to/mydir/further/path/file.ext"
$ echo ${mydir#*mydir}
/further/path/file.ext
$ mydir="/path/mydir/some/other/path/mydir/path/file.ext"
$ echo ${mydir#*mydir}
/some/other/path/mydir/path/file.ext
Go through sed. Example:
echo /some/path/to/mydir/further/path/file.ext | sed 's/.*mydir/mydir/'
Using bash, you can do something like this:
V=/yet/another/long/path/to/mydir/file.ext
R=${V#*mydir/}
echo $R
file.ext

How to name output file according to a command line argument in a bash script?

These lines work when copy-pasted to the shell but don't work in a script:
ls -l file1 > /path/`echo !#:2`.txt
ls -l file2 > /path/`echo !#:2`.txt
 
ls -l file1 > /path/$(echo !#:2).txt
ls -l file2 > /path/$(echo !#:2).txt
What's the syntax for doing this in a bash script?
If possible, I would like to know how to do this for one file and for all files with the same extension in a folder.
Non-interactive shell has history expansion disabled.
Add the following two lines to your script to enable it:
set -o history
set -o histexpand
(UPDATE: I misunderstood the original question as referring to arguments to the script, not arguments to the current command within the script; this is a rewritten answer.)
As #choroba said, history is disabled by default in scripts, because it's not really the right way to do things like this in a script.
The preferred way to do things like this in a script is to store the item in question (in this case the filename) in a variable, then refer to it multiple times in the command:
fname=file1
ls -l "$fname" > "/path/$fname.txt"
Note that you should almost always put variable references inside double-quotes (as I did above) to avoid trouble if they contain spaces or other shell metacharacters. If you want to do this for multiple files, use a for loop:
for fname in *; do # this will repeat for each file (or directory) in the current directory
ls -l "$fname" > "/path/$fname.txt"
done
If you want to operate on files someplace other than the current directory, things are a little more complicated. You can use /inputpath/*, but it'll include the path along with each filename (e.g. it'd run the loop with "/inputpath/file1", "/inputpath/file2", etc), and if you use that directly in the output redirect you'll get something like > /path/inputpath/file1.txt (i.e. the two different paths will get appended together), probably not what you want. In this case, you can use the basename command to strip off the unwanted path for output purposes:
for fpath in /inputpath/*; do
ls -l "$fpath" > "/path/$(basename "$fpath").txt"
done
If you want a list of files with a particular extension, just use *.foo or /inputpath/*.foo as appropriate. However, in this case you'll wind up with the output going to files named e.g. "file1.foo.txt"; if you don't want stacked extensions, basename has an option to trim that as well:
for fpath in /inputpath/*.foo; do
ls -l "$fpath" > "/path/$(basename "$fpath" .foo).txt"
done
Finally, it might be neater (depending how complex the actual operation is, and whether it occurs multiple times in the script) to wrap this in a function, then use that:
doStuffWithFile() {
ls -l "$1" > "/path/$(basename "$1" "$2").txt"
}
for fpath in /inputpath/*.foo; do
doStuffWithFile "$fpath" ".foo"
done
doStuffWithFile /otherpath/otherfile.bar .bar

Shell script : changing working dir and spaces in folder name

I want to make a script that takes a file path for argument, and cds into its folder.
Here is what I made :
#!/bin/bash
#remove the file name, and change every space into \space
shorter=`echo "$1" | sed 's/\/[^\/]*$//' | sed 's/\ /\\\ /g'`
echo $shorter
cd $shorter
I actually have 2 questions (I am a relative newbie to shell scripts) :
How could I make the cd "persistent" ? I want to put this script into /usr/bin, and then call it from wherever in the filesystem. Upon return of the script, I want to stay in the $shorter folder. Basically, if pwd was /usr/bin, I could make it by typing . script /my/path instead of ./script /my/path, but what if I am in an other folder ?
The second question is trickier. My script fails whenever there is a space in the given argument. Although $shorter is exactly what I want (for instance /home/jack/my\ folder/subfolder), cd fails whith the error /usr/bin/script : line 4 : cd: /home/jack/my\: no file or folder of this type. I think I have tried everything, using things like cd '$shorter' or cd "'"$shorter"'" doesn't help. What am I missing ??
Thanks a lot for your answers
in your .bashrc add the following line:
function shorter() { cd "${1%/*}"; }
% means remove the smaller pattern from the end
/* is the patern
Then in your terminal:
$ . ~/.bashrc # to refresh your bash configuration
$ type shorter # to check if your new function is available
shorter is a function
shorter ()
{
cd "${1%/*}"
}
$ shorter ./your/directory/filename # this will move to ./your/directory
The first part:
The change of directory won't be “persistent” beyond the lifetime of your script, because your script runs in a new shell process. You could, however, use a shell alias or a shell function. For example, you could embed the code in a shell function and define it in your .bash_profile or other source location.
mycdfunction () {
cd /blah/foo/"$1"
}
As for the “spaces in names” bit:
The general syntax for referring to a variable in Bourne shells is: "$var" — the "double quotes" tell the shell to expand any variables inside of them, but to group the outcome as a single parameter.
Omitting the double quotes around $var tells the shell to expand the variable, but then split the results into parameters (“words”) on whitespace. This is how the shell splits up parameters, normally.
Using 'single quotes' causes the shell to not expand any contents, but group the parameters togethers.
You can use \ (backslash-blank) to escape a space when you're typing (or in a script), but that's usually harder to read than using 'single quotes' or "double quotes"…
Note that the expansion phase includes: $variables wild?cards* {grouping,names}with-braces $(echo command substitution) and other effects.
| expansion | no expansion
-------------------------------------------------------
grouping | " " | ' '
splitting | (no punc.) | (not easily done)
For the first part, there is no need for the shorter variable at all. You can just do:
#!/bin/bash
cd "${1%/*}"
Explanation
Most shells, including bash, have what is called Parameter Expansion and they are very powerful and efficient as they allow you to manipulate variables nativly within the shell that would normally require a call to an external binary.
Two common examples of where you can use Parameter Expansion over an external call would be:
${var%/*} # replaces dirname
${var##*/} # replaces basename
See this FAQ on Parameter Expansion to learn more. In fact, while you're there might as well go over the whole FAQ
When you put your script inside /usr/bin you can call it anywhere. And to deal with whitespace in the shell just put the target between "" (but this doesn't matter !!).
Well here is a demo:
#!/bin/bash
#you can use dirname but that's not apropriate
#shorter=$(dirname $1)
#Use parameter expansion (too much better)
shorter=${1%/*}
echo $shorter
An alternate way to do it, since you have dirname on your Mac:
#!/bin/sh
cd "$(dirname "$1")"
Since you mentioned in the comments that you wanted to be able to drag files into a window and cd to them, you might want to make your script allow file or directory paths as arguments:
#!/bin/sh
[ -f "$1" ] && set "$(dirname "$1")" # convert a file to a directory
cd "$1"

Resources