I had an issue with third party ksh script.
Found out, that it was failing because of file named "\" in user home directory.
Here is a simple testcase:
$ mkdir -p ~/dir1 && cd ~/dir1 && touch '\' && x="\* a" && echo $x
\ a
$ mkdir -p ~/dir2 && cd ~/dir2 && x="\* a" && echo $x
\* a
The question is, why the presence of "\" file in a working directory changes the result.
Is this expected?
Thanks.
T.
Looks like the expected behaviour.
If you want the same behaviour in both cases, either use set -o noglob inside your script, or run the script with the -f option to disable file name substitution.
The default is that the * is a special character when interpolating so will match whatever file exists (in your case dir1 will contain only one real file with the name of the backslash character.)
The second directory dir2 has no real files so ksh just shows the pattern exactly as you typed it.
I find a list of files that I need to cd to (obviously to the parent directory).
If I do cd ./src/components/10-atoms/fieldset/package.json I get the error cd: not a directory:, which makes sense.
But isn't there a way to allow for that? Because manipulating the path-string is pretty cumbersome and to me that would make total sense to have an option for that, since cd is a directory function and it would be cool that if the path would not end up in a file, it would recursively jump higher and find the "first dir" from the given path.
So cd ./src/components/10-atoms/fieldset/package.json would put me into ./src/components/10-atoms/fieldset/ without going on my nerves, telling me that I have chosen a file rather than a dir.
You could write a shell function to do it.
cd() {
local args=() arg
for arg in "$#"; do
if [[ $arg != -* && -e $arg && ! -d $arg ]]; then
args+=("$(dirname "$arg")")
else
args+=("$arg")
fi
done
builtin cd ${args[0]+"${args[#]}"}
}
Put it in your ~/.bashrc if you want it to be the default behavior. It won't be inherited by shell scripts or other programs so they won't be affected.
It modifies cd's arguments, replacing any file names with the parent directory. Options with a leading dash are left alone. command cd calls the underlying cd builtin so we don't get trapped in a recursive loop.
(What is this unholy beast: ${args[0]+"${args[#]}"}? It's like "${args[#]}", which expands the array of arguments, but it avoids triggering a bash bug with empty arrays on the off chance that your bash version is 4.0-4.3 and you have set -u enabled.)
This function should do what you need:
cdd() { test -d "$1" && cd "$1" || cd $(dirname "$1") ; }
If its first argument "$1" is a directory, just cd into it,
otherwise cd into the directory containing it.
This function should be improved to take into account special files such as devices or symbolic links.
You can if you enter a bit longer line (or create dedicated shell script)
cd $(dirname ./src/components/10-atoms/fieldset/package.json)
If you add it in script it can be :
cd $(dirname $1)
but you need to execute it on this way:
. script_name ./src/components/10-atoms/fieldset/package.json
You can put this function in your ~/.bashrc:
function ccd() {
TP=$1 # destination you're trying to reach
while [ ! -d $TP ]; do # if $TP is not a directory:
TP=$(dirname $TP) # remove the last part from the path
done # you finally got a directory
cd $TP # and jump into it
}
Usage: ccd /etc/postfix/strangedir/anotherdir/file.txt will get you to /etc/postfix.
In a bash shell script do-for.sh I want to perform a command inside all directories named in a glob using bash. This has been answered oodles of times, but I want to provide the command itself on the command line. In other words assuming I have the directories:
foo
bar
I want to enter
do-for * pwd
and have bash print the working directory inside foo and then inside bar.
From reading the umpteen answers on the web, I thought I could do this:
for dir in $1; do
pushd ${dir}
$2 $3 $4 $5 $6 $7 $8 $9
popd
done
Apparently though the glob * gets expanded into the other command line arguments variable! So the first time through the loop, for $2 $3 $4 $5 $6 $7 $8 $9 I expected foo pwd but instead it appears I get foo bar!
How can I keep the glob on the command line from being expanded into the other parameters? Or is there a better way to approach this?
To make this clearer, here is how I want to use the batch file. (This works fine on the Windows batch file version, by the way.)
./do-for.sh repo-* git commit -a -m "Added new files."
I will assume you are open to your users having to provide some kind of separator, like so
./do-for.sh repo-* -- git commit -a -m "Added new files."
Your script could do something like (this is just to explain the concept, I have not tested the actual code) :
CURRENT_DIR="$PWD"
declare -a FILES=()
for ARG in "$#"
do
[[ "$ARG" != "--" ]] || break
FILES+=("$ARG")
shift
done
if
[[ "${1-}" = "--" ]]
then
shift
else
echo "You must terminate the file list with -- to separate it from the command"
(return, exit, whatever you prefer to stop the script/function)
fi
At this point, you have all the target files in an array, and "$#" contains only the command to execute. All that is left to do is :
for FILE in "${FILES[#]-}"
do
cd "$FILE"
"$#"
cd "$CURRENT_DIR"
done
Please note that this solution has the advantage that if your user forgets the "--" separator, she will be notified (as opposed to a failure due to quoting).
In this case the problem is not the expansion of metacharacter, is just that your script has an undefined number of arguments of which the last one is the command to execute for all previous arguments.
#!/bin/bash
CMND=$(eval echo "\${$#}") # get the command as last argument without arguments or
while [[ $# -gt 1 ]]; do # execute loop for each argument except last one
( cd "$1" && eval "$CMND" ) # switch to each directory received and execute the command
shift # throw away 1st arg and move to the next one in line
done
Usage: ./script.sh * pwd or ./script.sh * "ls -l"
To have the command followed by arguments (ex. ./script.sh * ls -l) the script has to be longer because each argument has to be tested if it's a directory until the command is identified (or backwards until a dir is identified).
Here is an alternative script that would accept the syntax: ./script.sh <dirs...> <command> <arguments...>
For example: ./script.sh * ls -la
# Move all dirs from args to DIRS array
typeset -i COUNT=0
while [[ $# -gt 1 ]]; do
[[ -d "$1" ]] && DIRS[COUNT++]="$1" && shift || break
done
# Validate that the command received is valid
which "$1" >/dev/null 2>&1 || { echo "invalid command: $1"; exit 1; }
# Execute the command + it's arguments for each dir from array
for D in "${DIRS[#]}"; do
( cd "$D" && eval "$#" )
done
Here is how I would do it:
#!/bin/bash
# Read directory arguments into dirs array
for arg in "$#"; do
if [[ -d $arg ]]; then
dirs+=("$arg")
else
break
fi
done
# Remove directories from arguments
shift ${#dirs[#]}
cur_dir=$PWD
# Loop through directories and execute command
for dir in "${dirs[#]}"; do
cd "$dir"
"$#"
cd "$cur_dir"
done
This loops over the arguments as seen after expansion, and as long as they are directories, they are added to the dirs array. As soon as the first non-directory argument is encountered, we assume that now the command starts.
The directories are then removed from the arguments with shift, and we store the current directory in cur_dir.
The last loop visits each directory and executes the command consisting of the rest of the arguments.
This works for your
./do-for.sh repo-* git commit -a -m "Added new files."
example – but if repo-* expands to anything other than directories, the script breaks because it will try to execute the filename as part of the command.
It could be made more stable if, for example, the glob and the command were separated by an indicator such as --, but if you know that the glob will always be just directories, this should work.
I will begin with the Windows batch file that you mentioned twice as working. The big difference is that on Windows, the shell doesn’t make any globbing, leaving it to the various commands (and each of them does it differently), while on Linux/Unix the globbing is usually done by the shell, and can be prevented by quoting or escaping. Both the Windows approach and the Linux approach have their merits, and they compare differently in different use cases.
For regular bash users, quoting
./do-for.sh repo-'*' git commit -a -m "Added new files."
or escaping
./do-for.sh repo-\* git commit -a -m "Added new files."
are the simplest solution, because they are what they consistently use on a daily basis. If your users need a different syntax, you have all the solutions proposed so far, that I will classify into four categories before proposing my own (note that in each example below do-for.sh stands for a different script adopting the respective solution, which can be found in one of the other answers.)
Disable shell globbing. This is clumsy, because, even if you remember which shell option does it, you have to remember to reset it to default to have the shell working normally afterwards.
Use a separator:
./do-for.sh repo-* -- git commit -a -m "Added new files."
This works, is similar to the solution adopted in similar situations with other shell commands, and fails only if your expansion of directory names includes a directory name exactly equal to the separator (an unlikely event, which wouldn’t happen in the above example, but in general might happen.)
Have the command as the last argument, all the rest are directories:
./do-for.sh repo-* 'git commit -a -m "Added new files."'
This works, but again, it involves quoting, possibly even nested, and there is no point in preferring it to the more usual quoting of globbing characters.
Try to be smart:
./do-for.sh repo-* git commit -a -m "Added new files."
and consider to be dealing with directories till you hit a name which is not a directory. This would work in many cases, but might fail in obscure ways (e.g. when you have a directory named like the command).
My solution doesn’t belong to any of the mentioned categories. In fact, what I propose is not to use * as a globbing character in the first argument of your script. (This is similar to the syntax used by the split command where you provide a non-globbed prefix argument for the files to be generated.) I have two versions (code below). With the first version, you would do the following:
# repo- is a prefix: the command will be excuted in all
# subdirectories whose name starts with it
./do-for.sh repo- git commit -a -m "Added new files."
# The command will be excuted in all subdirectories
# of the current one
./do-for.sh . git commit -a -m "Added new files."
# If you want the command to be executed in exactly
# one subdirectory with no globbing at all,
# '/' can be used as a 'stop character'. But why
# use do-for.sh in this case?
./do-for.sh repo/ git commit -a -m "Added new files."
# Use '.' to disable the stop character.
# The command will be excuted in all subdirectories of the
# given one (paths have to be always relative, though)
./do-for.sh repos/. git commit -a -m "Added new files."
The second version involves using a globbing character the shell knows nothing about, such as SQL’s % character
# the command will be excuted in all subdirectories
# matching the SQL glob
./do-for.sh repo-% git commit -a -m "Added new files."
./do-for.sh user-%-repo git commit -a -m "Added new files."
./do-for.sh % git commit -a -m "Added new files."
The second version is more flexible, as it allows non-final globs, but is less standard for the bash world.
Here is the code:
#!/bin/bash
if [ "$#" -lt 2 ]; then
echo "Usage: ${0##*/} PREFIX command..." >&2
exit 1
fi
pathPrefix="$1"
shift
### For second version, comment out the following five lines
case "$pathPrefix" in
(*/) pathPrefix="${pathPrefix%/}" ;; # Stop character, remove it
(*.) pathPrefix="${pathPrefix%.}*" ;; # Replace final dot with glob
(*) pathPrefix+=\* ;; # Add a final glob
esac
### For second version, uncomment the following line
# pathPrefix="${pathPrefix//%/*}" # Add a final glob
tmp=${pathPrefix//[^\/]} # Count how many levels down we have to go
maxDepth=$((1+${#tmp}))
# Please note that this won’t work if matched directory names
# contain newline characters (comment added for those bash freaks who
# care about extreme cases)
declare -a directories=()
while read d; do
directories+=("$d")
done < <(find . -maxdepth "$maxDepth" -path ./"$pathPrefix" -type d -print)
curDir="$(pwd)"
for d in "${directories[#]}"; do
cd "$d";
"$#"
cd "$curDir"
done
As in Windows, you would still need to use quotes if the prefix contains spaces
./do-for.sh 'repository for project' git commit -a -m "Added new files."
(but if the prefix does not contain spaces, you can avoid quoting it and it will correctly deal with any space-containing directory names beginning with that prefix; with obvious changes, the same is true for %-patterns in the second version.)
Please note the other relevant differences between a Windows and a Linux environment, such as case sensitivity in pathnames, differences in which characters are considered special, and so on.
In bash you may execute "set -o noglob" which will inhibit the shell to expand path names (globs). But this has to be set on the running shell before you execute the script, otherwise you should quote any meta character which you provide in the arguments.
find-while-read combination is one of the safest combination to parse file names. Do something like below
#!/bin/bash
myfunc(){
cd "$2"
eval "$1" # Execute the command parsed as an argument
}
cur_dir=$(pwd) # storing the current directory
find . -type d -print0 | while read -rd '' dname
do
myfunc "pwd" "$dname"
cd "$cur_dir" #Remember myfunc changes the current working dir, so you need this
done
Why not keep it simple and create a shell function that uses find but eases the burden for your users of typing out its commands, for example:
do_for() { find . -type d \( ! -name . \) -not -path '*/\.*' -name $1 -exec bash -c "cd '{}' && "${#:2}" " \; }
So they can type something like do_for repo-* git commit -a -m "Added new files."
Note, if you want to use the * by itself, you'll have to escape it:
do_for \* pwd
Wildcards are evaluated by the shell before being passed to any program or script. There is nothing you can do about that.
But if you accept quoting the globbing expression then this script should to do the trick
#!/usr/bin/env bash
for dir in $1; do (
cd "$dir"
"${#:2}"
) done
I tried it out with two test directories and it seems to be working. Use it like this:
mkdir test_dir1 test_dir2
./do-for.sh "test_dir*" git init
./do-for.sh "test_dir*" touch test_file
./do-for.sh "test_dir*" git add .
./do-for.sh "test_dir*" git status
./do-for.sh "test_dir*" git commit -m "Added new files."
Nobody proposing a solution using find ? Why not try something like this:
find . -type d \( -wholename 'YOURPATTERN' \) -print0 | xargs -0 YOURCOMMAND
Look at man find for more options.
how I can use the second argument of previous command in a new command ?
example, with
$ mkdir test
I make a directory, how I can use the name of directory for change to this ?
$ mkdir test && cd use_var
$_ is the last (right-most) argument of the previous command.
mkdir gash && cd "$_"
(I don't create files or directories called test, that's the name of a shell built-in and can cause confusions)
With history expansion, you can refer to arbitrary words in the current command line
mkdir dir1 && cd "!#:1"
# 0 1 2 3 4
!# refers to the line typed so far, and :1 refers to word number one (with mkdir starting at 0).
If you use this in a script (i.e., a non-interactive shell), you need to turn history expansion on with set -H and set -o history.
Pressing Esc + . places the last argument of previous command on the current place of cursor. Tested in bash shell and ksh shell.
I use functions for this. Type this in your shell:
mkcd() { mkdir "$1" ; cd "$1" ; }
Now you have a new command mkcd.
If you need this repeatedly, put the line into the file ~/.bash_aliases (if you use bash; other shells use different names).
The KornShell (ksh) used to have a very useful option to cd for traversing similar directory structures; e.g., given the following directories:
/home/sweet/dev/projects/trunk/projecta/app/models
/home/andy/dev/projects/trunk/projecta/app/models
Then if you were in the /home/sweet... directory then you could change to the equivalent directory in andy's structure by typing
cd sweet andy
So if ksh saw 2 arguments then it would scan the current directory path for the first value, replace it with the second and cd there. Is anyone aware of similar functionality built into Bash? Or if not, a hack to make Bash work in the same way?
Other solutions offered so far suffer from one or more of the following problems:
Archaic forms of tests - as pointed out by Michał Górny
Incomplete protection from directory names containing white space
Failure to handle directory structures which have the same name used more than once or with substrings that match: /canis/lupus/lupus/ or /nicknames/Robert/Rob/
This version handles all the issues listed above.
cd ()
{
local pwd="${PWD}/"; # we need a slash at the end so we can check for it, too
if [[ "$1" == "-e" ]]
then
shift
# start from the end
[[ "$2" ]] && builtin cd "${pwd%/$1/*}/${2:-$1}/${pwd##*/$1/}" || builtin cd "$#"
else
# start from the beginning
[[ "$2" ]] && builtin cd "${pwd/\/$1\///$2/}" || builtin cd "$#"
fi
}
Issuing any of the other versions, which I'll call cdX, from a directory such as this one:
/canis/lupus/lupus/specimen $ cdX lupus familiaris
bash: cd: /canis/familiaris/lupus/specimen: No such file or directory
fails if the second instance of "lupus" is the one intended. In order to accommodate this, you can use the "-e" option to start from the end of the directory structure.
/canis/lupus/lupus/specimen $ cd -e lupus familiaris
/canis/lupus/familiaris/specimen $
Or issuing one of them from this one:
/nicknames/Robert/Rob $ cdX Rob Bob
bash: cd: /nicknames/Bobert/Rob: No such file or directory
would substitute part of a string unintentionally. My function handles this by including the slashes in the match.
/nicknames/Robert/Rob $ cd Rob Bob
/nicknames/Robert/Bob $
You can also designate a directory unambiguously like this:
/fish/fish/fins $ cd fish/fins robot/fins
/fish/robot/fins $
By the way, I used the control operators && and || in my function instead of if...then...else...fi just for the sake of variety.
cd "${PWD/sweet/andy}"
No, but...
Michał Górny's substitution expression works nicely. To redefine the built-in cd command, do this:
cd () {
if [ "x$2" != x ]; then
builtin cd ${PWD/$1/$2}
else
builtin cd "$#"
fi
}