I installed win-bash on Windows 7 and I'm getting the following strange behavior.
bash$ cat C:/Home/.bashrc
PATH="C:/Program\ Files/GnuWin32/bin:C:/Windows/system32"
bash$ . C:/Home/.bashrc
bash$ echo $PATH
C:/Program\ Files/GnuWin32/bin:C:/Windows/system32
bash$ which diff
which: no diff in (.;C;\Program\ Files\GnuWin32\bin;C:\Windows\system32)
bash$ which ls
which: no ls in (.;C;\Program\ Files\GnuWin32\bin;C:\Windows\system32)
Why are the PATH values different?
The PATH value returned by which contains .:C;\Program\ Files\GnuWin32\bin
Note:
the ".:" in the beginning that does not exist in the bash PATH value.
the "C;" (not C:) contains a semi-colon instead of a colon.
the which PATH value has back slashes (\\) instead of forward slashes (/)
Where is which sourcing these PATH values?
I can not find any other .bashrc or .profile or profile files anywhere on the machine.
In addition,
bash$ diff file-abc.txt file-xyz.txt
1c1
< abc
---
\> xyz
bash$ ls file-abc.txt
file-abc.txt
Both diff and ls work on the command line even though which can not find the diff or ls commands.
Both diff and ls are located in C:/Program\ Files/GnuWin32/bin
But which returns C;\Program\ Files\GnuWin32\bin (note C; not C:) which is why which can not find ls or diff.
Again, where is which sourcing these PATH values?
In my bash script named Try1.sh I have these lines.
\`diff $CURRENT_FILE $NEW_FILE\`
\`ls $CURRENT_FILE\`
The diff command fails with
Try1.sh: 21c21: command not found
The ls command succeeds. Why?
Both diff and ls live in the same PATH location C:/Program\ Files/GnuWin32/bin.
Windows has a different search algorithm to UNIX-like systems. On Windows the first directory to be searched is the directory which the parent program (.exe) was loaded from, then the current directory, then C:/Windows/system32 is searched. That's where the directory names are coming from.
The path environment variable is only used as a last resort!
For a full discussion on this, see MSDN entry for CreateProcess
which is also showing the Windows path directory separator as ;, rather than : which UNIX-like systems use. Also, / or \ are valid as a directory separator in a Windows path, but only / is valid on UNIX.
Also note that environment variables (like path) are not case sensitive on Windows, but on UNIX they are.
EDIT: I have been trying to track down the source code for win-bash but can't find it. I found some source code for which in GNUUtils, but can't be sure that it is the same version as you are using. The version I looked at, 2.4, makes assumptions about Windows which are not necessarily correct.
After downloading the binary for win-bash, I found that the bundled which is indeed version 2.4, and looks the same as the source code I have been looking at.
It is a separate program and not integrated with the rest of the shell code. To answer the question on directory separators and path separators, they are hard-coded for Windows (sys.h):
#define DIRSEP '\\'
#define PATHSEP ';'
The path is read from the environment variable using getenv.
Further edit:
The command
\`diff $CURRENT_FILE $NEW_FILE\`
is invalid. It is capturing the output from diff and then trying to execute it. 21c21 is the output from diff, and of course there is no such program as 21c21. Just use:
diff $CURRENT_FILE $NEW_FILE
Related
I am attempting to work with an existing library of code but have encountered an issue. In short, I execute a shell script (let's call this one A) whose first act is to call another script (B). Script B is in my current directory (a requirement of the program I'm using). The software's manual makes reference to bash, however comments in A suggest it was developed in ksh. I've been operating in bash so far.
Inside A, the line to execute B is simply:
. B
It uses the "dot space" syntax to call the program. It doesn't do anything unusual like sudo.
When I call A without dot space syntax, i.e.:
./A
it always errors saying it cannot find the file B. I added pwd, ls, whoami, echo $SHELL, and echo $PATH lines to A to debug and confirmed that B is in fact right there, the script is running with the same $SHELL as I am at the command prompt, the script is the same user as I am, and the script has the same search path $PATH as I do. I also verified if I do:
. B
at the command line, it works just fine. But, if I change the syntax inside A to:
./B
instead, then A executes successfully.
Similarly, if I execute A with dot space syntax, then both . B and ./B work.
Summarizing:
./A only works if A contains ./B syntax.
. A works for A with either ./B or . B syntax.
I understand that using dot space (i.e. . A) syntax executes without forking to a subshell, but I don't see how this could result in the behavior I'm observing given that the file is clearly right there. Is there something I'm missing about the nuances of syntax or parent/child process workspaces? Magic?
UPDATE1: Added info indicating that the script may have been developed in ksh, while I'm using bash.
UPDATE2: Added checking to verify $PATH is the same.
UPDATE3: The script says it was written for ksh, but it is running in bash. In response to Kenster's answer, I found that running bash -posix then . B fails at the command line. That indicates that the difference in environments between the command line and the script is that the latter is running bash in a POSIX-compliant mode, whereas the command line is not. Looking a little closer, I see this in the bash man page:
When invoked as sh, bash enters posix mode after the startup files are read.
The shebang for A is indeed #!/bin/sh.
In summary, when I run A without dot space syntax, it's forking to its own subshell, which is in POSIX-compliant mode because the shebang is #!/bin/sh (instead of, e.g., #!/bin/bash. This is the critical difference between the command line and script runtime environments that leads to A being unable to find B.
Let's start with how the command path works and when it's used. When you run a command like:
ls /tmp
The ls here doesn't contain a / character, so the shell searches the directories in your command path (the value of the PATH environment variable) for a file named ls. If it finds one, it executes that file. In the case of ls, it's usually in /bin or /usr/bin, and both of those directories are typically in your path.
When you issue a command with a / in the command word:
/bin/ls /tmp
The shell doesn't search the command path. It looks specifically for the file /bin/ls and executes that.
Running ./A is an example of running a command with a / in its name. The shell doesn't search the command path; it looks specifically for the file named ./A and executes that. "." is shorthand for your current working directory, so ./A refers to a file that ought to be in your current working directory. If the file exists, it's run like any other command. For example:
cd /bin
./ls
would work to run /bin/ls.
Running . A is an example of sourcing a file. The file being sourced must be a text file containing shell commands. It is executed by the current shell, without starting a new process. The file to be sourced is found in the same way that commands are found. If the name of the file contains a /, then the shell reads the specific file that you named. If the name of the file doesn't contain a /, then the shell looks for it in the command path.
. A # Looks for A using the command path, so might source /bin/A for example
. ./A # Specifically sources ./A
So, your script tries to execute . B and fails claiming that B doesn't exist, even though there's a file named B right there in your current directory. As discussed above, the shell would have searched your command path for B because B didn't contain any / characters. When searching for a command, the shell doesn't automatically search the current directory. It only searches the current directory if that directory is part of the command path.
In short, . B is probably failing because you don't have "." (current directory) in your command path, and the script which is trying to source B is assuming that "." is part of your path. In my opinion, this is a bug in the script. Lots of people run without "." in their path, and the script shouldn't depend on that.
Edit:
You say the script uses ksh, while you are using bash. Ksh follows the POSIX standard--actually, KSH was the basis for the POSIX standard--and always searches the command path as I described. Bash has a flag called "POSIX mode" which controls how strictly it follows the POSIX standard. When not in POSIX mode--which is how people generally use it--bash will check the current directory for the file to be sourced if it doesn't find the file in the command path.
If you were to run bash -posix and run . B within that bash instance, you should find that it won't work.
I am new to bash, and I saw people often add : after a directory when modifying PATH. After searching for a while, I did not find an answer for that, or I believe I did not search it correctly. So I hope I could get an answer here.
Example:
/Users/chengluli/anaconda/bin:/Users/chengluli/.rbenv/shims:/
What does the : after bin and shims do?
: is the separator. The PATH variable is itself a list of folders that are "walked" through when you run a command.
In this case, the folders on your PATH are:
/Users/chengluli/anaconda/bin
/Users/chengluli/.rbenv/shims
/
As others have said, the : is a separator (Windows uses a semi-colon ;). But you are probably thinking of a trailing colon : at the end of the PATH variable. For example:
/Users/chengluli/anaconda/bin:/Users/chengluli/.rbenv/shims:
From the bash man pages:
A zero-length (null) directory name in the value of PATH indicates the current directory. A null directory name may appear as two adjacent colons, or as an initial or trailing colon.
Putting the current directory in the PATH is generally considered a security risk and a bad idea. It is particularly dangerous when using the root user.
By the way, bash only uses $PATH on the first call of an external program, after that it uses a hash table. See man bash and the hash command
If you run ls -l 123 at the command line, you are telling bash to find the command called ls in the filesystem. However, ls is just a file name, bash needs the absolute path of ls in the filesystem. So bash searches for a file called ls in a list of default directories, one by one in order.
A list of default directories is stored in the PATH variable, separated by :.
Th quotation from output of man bash command
PATH
The search path for commands. It is a colon-separated list of directories in which the shell looks for commands (see COMMAND EXECUTION below). A zero-length (null) directory
name in the value of PATH indicates the current directory. A null directory name may appear as two adjacent colons, or as an initial or trailing colon. The default path is sys‐tem-dependent, and is set by the administrator who installs bash. A common value is ``/usr/gnu/bin:/usr/local/bin:/usr/ucb:/bin:/usr/bin''.
If you have questions about bash script or environment variable, please use man bash firstly.
I'm writing a shell script which parses the path of the current working directory (printing a like of all basenames above the current directory).
So far, I've been using the environment variable PWD to parse the path but I wonder if
I can count on PWD to be always set
to give the same result on every platform
Would it possibly be better to use the pwd shell-builtin? I need this script to run on as many platforms as possible, so I just wondered...
POSIX requires $PWD to be set in the following fashion:
PWD
This variable shall represent an absolute pathname of the current working directory. It shall not contain any components that are dot or dot-dot. The value is set by the cd utility, and by the sh utility during initialization.
So you can rely on that being set – but do note "... an absolute path...", not the absolute path.
bash (at least recent versions) will remember what symlinks you followed when setting $PWD (and the pwd builtin). command pwd (that is, the external command) will not. So you'll get different results there, which might, or might not, be important for you. Use pwd -P if you want a path without symlinks.
Do note that the pwd documentation states:
If an application sets or unsets the value of PWD, the behavior of pwd is unspecified.
So, don't do that :)
In short, there is no winner here. The environment variable will be there in POSIX shells, as will the external command and possibly a built-in too. Choose the one that best fits your need, the important thing being whether you care about symlinks or not.
From that forum article, "$PWD vs `pwd`" which compares AIX 4.2.1, AIX 6, Sparc Solaris 10 and Redhat 5 enterprise with this regard:
there is no difference between $PWD and builtin pwd,
there is no difference between builtin pwd -P and /usr/bin/pwd.
The former shows working directory with names of symbolic links whereas the latter displays actual path.
The only discrepancy is that external command is in /usr/bin in most systems and /bin in Redhat.
Another point to note is
command substitutions are not generally safe on trailing
newlines .
This is obviously fairly contrived, but if you're really concerned about safely
handling input you should be using "$PWD". See, for example:
$ my_dir=$'/tmp/trailing_newline\n'
$ mkdir -p "$my_dir"
$ cd "$my_dir"
$ pwd
/tmp/trailing_newline
$ printf "%q\n" "$(pwd)" "$PWD"
/tmp/trailing_newline
$'/tmp/trailing_newline\n'
$ cd "$(pwd)"
sh: cd: /tmp/trailing_newline: No such file or directory
$ cd "$PWD"
It is possible to work around the command substitution but it is by no means
pretty. You can append a trailing character and then strip it with a
parameter expansion:
$ pwd_guarded="$(pwd; printf '#')"
$ pwd_fixed="${pwd_guarded%$'\n'#}"
$ printf "%q\n" "$pwd_fixed"
$'/tmp/trailing_newline\n'
$ cd "$pwd_fixed"
This is particularly ugly because you then also have to strip the newline that
pwd adds, which would normally have been stripped by the command substitution.
This becomes a total mess if you don't resort to non-POSIX constructs like
$'', so basically, just use "$PWD" if you care about these things. Of course
it is perfectly reasonable to just not support trailing newlines in directory
names.
If you know that bash is available and the script is executed with it, PWD is safe.
If, on some systems, only plain sh is available, use pwd.
If it were me, I'd use pwd since it is a built-in both for bash and sh. That does not mean they work identically in all respects, but if you are invoking it without options, that shouldn't matter.
I've created a bash shell script file that I can run on my local bash (version 4.2.10) but not on a remote computer (version 3.2). Here's what I'm doing
A script file (some_script.sh) exists in a local folder
I've done $ chmod 755 some_script.sh to make it an executable
Now, I try $ ./some_script.sh
On my computer, this runs fine. On the remote computer, this returns a Command not found error:
./some_script.sh: Command not found.
Also, in the remote version, executable files have stars(*) following their names. Don't know if this makes any difference but I still get the same error when I include the star.
Is this because of the bash shell version? Any ideas to make it work?
Thanks!
The command not found message can be a bit misleading. The "command" in question can be either the script you're trying to execute or the shell specified on the shebang line.
For example, on my system:
% cat foo.sh
#!/no/such/dir/sh
echo hello
% ./foo.sh
./foo.sh: Command not found.
./foo.sh clearly exists; it's the interpreter /no/such/dir/sh that doesn't exist. (I find that the error message varies depending on the shell from which you invoke foo.sh.)
So the problem is almost certainly that you've specified an incorrect interpreter name on line one of some_script.sh. Perhaps bash is installed in a different location (it's usually /bin/bash, but not always.)
As for the * characters in the names of executable files, those aren't actually part of the file names. The -F option to the ls command causes it to show a special character after certain kinds of files: * for executables, / for directories, # for symlinks, and so forth. Probably on the remote system you have ls aliased to ls -F or something similar. If you type /bin/ls, bypassing the alias, you should see the file names without the append * characters; if you type /bin/ls -F, you should see the *s again.
Adding a * character in a command name doesn't do what you think it's doing, but it probably won't make any difference. For example, if you type
./some_script.sh*
the * is a wild card, and the command name expands to a list of all files in the current directory whose names match the pattern (this is completely different from the meaning of * as an executable file in ls -F output). Chances are there's only one such file, so
./some_script.sh* is probably equivalent to ./some_script.sh. But don't type the *; it's unnecessary and can cause unexpected results.
I need to set the environment variables, usually we do this by
source script.sh
But now, I am automating it during the boot process and it looks like the root boots by default with sh shell. How do I source this script in sh?
The dot command '.' is the equivalent of the C Shell (and Bash) source command. It is specified by POSIX (see dot), and supported by the Bourne and Korn shells (and zsh, I believe).
. somefile
Note that the shell looks for the file using $PATH, but the file only has to be readable, not executable.
As noted in the comments below, you can of course specify a relative or absolute pathname for the file — any name containing a slash will not be searched for using $PATH. So:
. /some/where/somefile
. some/where/somefile
. ./somefile
could all be used to find somefile if it existed in the three different specified locations (if you could replace . with ls -l and see a file listed).
Pedants of the world unite! Yes, if the current directory is the root directory, then /some/where/somefile and ./some/where/somefile would refer to the same file — with the same real path — even without links, symbolic or hard, playing a role (and so would ../../some/where/somefile).
tl;dr With sh (as opposed to bash) the argument must contain a slash: source ./script.sh, not just source script.sh. Unless script.sh can be found in PATH.
Dot (.) command exists in both bash and sh. Additionally, bash aliases it as source. From bash manual:
https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html#index-source
source
source filename
A synonym for . (see Bourne Shell Builtins).
https://www.gnu.org/software/bash/manual/html_node/Bourne-Shell-Builtins.html#index-_002e
. (a period)
. filename [arguments]
Read and execute commands from the filename argument in the current shell context. If filename does not contain a slash, the PATH variable is used to find filename. When Bash is not in POSIX mode, the current directory is searched if filename is not found in $PATH.
From POSIX:
If file does not contain a /, the shell shall use the search path specified by PATH to find the directory containing file. Unlike normal command search, however, the file searched for by the dot utility need not be executable.