I have noticed a bit of a strange behaviour of $# in CSH. For example
set a=(Hello world); echo $#a
will output 2 as expected.
Why does $# act differently with combination of environment variables?
echo $#PATH
will actually output PATH ignoring # I would expect it to output 1.
My guess is that this is just one of those CSH quirks. Nothing in man page about this. Can somebody explain this behaviour? (Please don't bother writing "don't use CSH" comments I wouldn't use it if I didn't have to)
$#foo, where $foo is a shell array variable, yields the number of elements in the array $foo. If $foo is an ordinary non-array variable, $#foo yields 1.
csh and tcsh treat shell variables (set by the set command) quite differently from environment variables (set by the setenv command or inherited from the parent process). The value of an environment variable is always just a string (which is why $PATH, for example, needs : characters to delimit the list elements).
It would probably be more consistent for $#FOO, where $FOO is an environment variable, to yield 1 rather than expanding to the value of $FOO. In any case, the behavior doesn't seem to be documented, and you should avoid relying on it.
If you specifically want the number of directories in your path, $#path will give you that; $path is a shell array variable that's automatically tied to the value of the $PATH environment variable.
For other variables, you just have to keep track of whether a variable is a shell variable or an environment variable.
Environment variables can't be a list, only variables can, so doing $#PATH makes littles sense. It should probably be an error.
If you want to get the length of the path, you can use $#path. This will use the special $path variable.
Related
I'm surprised hasn't been asked before, but…
What is the difference between
alias ⇢ alias EXPORT='alias'
function ⇢ function exporter() { echo $EXPORT }
and
export ⇢ export ALIAS='export'
and for that matter...
alias export=$(function) (j/k)
in bash (zsh, et al.)
Specifically, I'd be most interested in knowing the lexical/practical difference between
alias this=that
and
export that=this
I have both forms... all over the place - and would prefer to stop arbitrarily choosing one, over the other. 😂
I'm sure there is a great reference to a "scopes and use-cases for unix shells", somewhere... but thought I'd post the question here, in the name of righteous-canonicalicism.
You're asking about two very different categories of things: aliases and functions define things that act like commands; export marks a variable to be exported to child processes. Let me go through the command-like things first:
An alias (alias ll='ls -l') defines a shorthand for a command. They're intended for interactive use (they're actually disabled by default in shell scripts), and are simple but inflexible. For example, any arguments you specify after the alias simply get tacked onto the end of the command; if you wanted something like alias findservice='grep "$1" /etc/services', you can't do it, because $1 doesn't do anything useful here.
A function is like a more flexible, more powerful version of an alias. Functions can take & process arguments, contain loops, conditionals, here-documents, etc... Basically, anything you could do with a shell script can be done in a function. Note that the standard way to define a function doesn't actually use the keyword function, just parentheses after the name. For example: findservice() { grep "$1" /etc/services; }
Ok, now on to shell variables. Before I get to export, I need to talk about unexported variables. Basically, you can define a variable to have some (text) value, and then if you refer to the variable by $variablename it'll be substituted into the command. This differs from an alias or function in two ways: an alias or function can only occur as the first word in the command (e.g. ll filename will use the alias ll, but echo ll will not), and variables must be explicitly invoked with $ (echo $foo will use the variable foo, but echo foo will not). More fundamentally, aliases and functions are intended to contain executable code (commands, shell syntax, etc), while variables are intended to store non-executable data.
(BTW, you should almost always put variable references inside double-quotes -- that is, use echo "$foo" instead of just echo $foo. Without double-quotes the variable's contents get parsed in a somewhat weird way that tends to cause bugs.)
There are also some "special" shell variables, that are automatically set by the shell (e.g. $HOME), or influence how the shell behaves (e.g. $PATH controls where it looks for executable commands), or both.
An exported variable is available both in the current shell, and also passed to any subprocesses (subshells, other commands, whatever). For example, if I do LC_ALL=en_US.UTF-8, that tells my current shell use the "en_US.UTF-8" locale settings. On the other hand, if I did export LC_ALL=en_US.UTF-8 that would tell the current shell and all subprocesses and commands it executes to use that locale setting.
Note that a shell variable can be marked as exported separately from defining it, and once exported it stays exported. For example, $PATH is (as far as I know) always exported, so PATH=/foo:/bar has the same effect as export PATH=/foo:/bar (although the latter may be preferred just in case $PATH somehow wasn't already exported).
It's also possible to export a variable to a particular command without defining it in the current shell, by using the assignment as a prefix for the command. For example LC_ALL=en_US.UTF-8 sort filename will tell the sort command to use the "en_US.UTF-8" locale settings, but not apply that to the current shell (or any other commands).
TL;DR:
The shell evaluation order (per POSIX) for the entities in your question is:
aliases --> variables --> command substitutions --> special built-ins --> functions --> regular built-ins
Aliases do not persist across subshells, but variables (and in Bash, functions) can be made to do so with the export command.
Regular built-ins can be overridden by writing functions that have the same name as the regular built-in (since functions expand before regular built-ins). (NOTE: If you're trying to add functionality to the regular built-in, call the built-in with command in your function definition so you don't accidentally create a recursive function.)
Variables can be made readonly with the (special built-in) readonly command, but aliases cannot.
USE CASES:
Export a variable if you need to use a variable across subshells.
Make a variable readonly if you don't want it changed for the life of the parent shell (once performed, this cannot be undone with unset; you must restart the parent shell).
If you want to override or add functionality to a regular built-in, use a function.
NOTE: If you want to be sure that you're using a special or regular built-in and not someone else's function, use builtin the_builtin, or if the shell doesn't support the builtin command, use the POSIX comand command -p the_builtin, where the -p switch tells command to use the $PATH that ships with the shell by default (in case the user has overriden path).
NOTE: A variable can be made to act like an alias that also persists across subshells and cannot be changed. For example,
#! /bin/sh
my_cmd='ls -al'
export my_cmd
readonly my_cmd
will act like
#! /bin/sh
alias my_cmd='ls -al'
so long as
my_cmd is used without double-quotes (i.e. ${my_cmd}, NOT "${my_cmd}") so it isn't treated as a single string, and
IFS is the standard space-tab-newline and not switched to something else so that the elements of my_cmd are globbed and each part separated by a space is evaluated as a single token (otherwise it will be evaluated as a single string).
Each shell (e.g. bash, zsh, ksh, yash, etc.) is a bit different, so be sure to review the reference manual for it (they each implement POSIX in a unique way, or sometimes not at all).
#!/bin/bash
env
This spits out a bunch of environment variables. $0, $#, $_, etc. are not output. Is there a way to dump them out, similar to how env or set dumps the rest of the environment? I don't mean anything like echo $0 or something, I want to see all of the possible special variables in their current state without directly calling all of them.
No, there is no existing function to automatically do this. You'll have to enumerate the variables yourself--but that's easy because there are only a few: What are the special dollar sign shell variables?
Built-in variables are included in the output to the set command. ($0 and $# aren't variables at all, even special ones, so they aren't included -- but $_, $BASH_ALIASES, $BASH_SOURCE, $BASH_VERSINFO, $DIRSTACK, $PPID, and all the other things that are built-in variables are present).
$0, $*, $#, etc. are not built-in variables; they are special parameters instead. Semantics are quite different (you can use declare -p to print a variable's value, but not that of a special parameter; many built-in variables lose their special behavior if reassigned, whereas special parameters can never be the target of an assignment; etc).
http://wiki.bash-hackers.org/syntax/shellvars covers both built-in variables and special parameters.
If your goal is to generate a log of current shell state, I suggest the following:
(set; set -x; : "$0" "$#") &>trace.log
set dumps the things that are actually built-in variables (including $_), and the set -x log of running : "$0" "$#" will contain enough information to reproduce all special parameters which are based on your positional parameters ("$*", "$#", etc); whereas the output from set will include all other state.
I'm little confused the make variable and shell variable in recipe.
as the each line of recipe is interpret as the shell, can i do the shell variable assignment?
following is the example:
.ONESHELL:
all:
param="hello"
echo $(param)
//--------------------
no output...
and i know we can use eval to the variable assignment, but it looks as make variable.
how can i just perform the normal shell variable assignment which i want to hold the shell command return value.
Thanks.
$(param) is expanded by GNU make. To make it expanded by the shell do $${param}. Using Variables in Recipes:
Variable and function references in recipes have identical syntax and semantics to references elsewhere in the makefile. They also have the same quoting rules: if you want a dollar sign to appear in your recipe, you must double it (‘$$’). For shells like the default shell, that use dollar signs to introduce variables, it’s important to keep clear in your mind whether the variable you want to reference is a make variable (use a single dollar sign) or a shell variable (use two dollar signs).
I would like to extract the current path in a variable and use it later on in the script
Something like:
mypath="$pwd"
Later on:
cd "$mypath"
But I am getting a different directory when doing ls
Almost:
mypath=$PWD
This one saves a fork over mypath=$(pwd). While some consider it good practice to always double quote variable assignments, technically it is not needed here, since the shell does not perform word-splitting for variable assignments.
PS: Note that you are assigning to mypath and then use myvar... you should be consistent in your variable naming, otherwise it won't work.
What is the difference between:
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram
and
prompt$ TSAN_OPTIONS="suppressions=/somewhere/file"
prompt$ ./myprogram
The thread-sanitizer library gives the first case as how to get their library (used within myprogram) to read the file given in options. I read it, and assumed it was supposed to be two separate lines, so ran it as the second case.
The library doesn't use the file in the second case, where the environment variable and the program execution are on separate lines.
What's the difference?
Bonus question: How does the first case even run without error? Shouldn't there have to be a ; or && between them? The answer to this question likely answers my first...
The format VAR=value command sets the variable VAR to have the value value in the environment of the command command. The spec section covering this is the Simple Commands. Specifically:
Otherwise, the variable assignments shall be exported for the execution environment of the command and shall not affect the current execution environment except as a side-effect of the expansions performed in step 4.
The format VAR=value; command sets the shell variable VAR in the current shell and then runs command as a child process. The child process doesn't know anything about the variables set in the shell process.
The mechanism by which a process exports (hint hint) a variable to be seen by child processes is by setting them in its environment before running the child process. The shell built-in which does this is export. This is why you often see export VAR=value and VAR=value; export VAR.
The syntax you are discussing is a short-form for something akin to:
VAR=value
export VAR
command
unset -v VAR
only without using the current process environment at all.
To complement Etan Reisner's helpful answer:
It's important to distinguish between shell variables and environment variables:
Note: The following applies to all POSIX-compatible shells; bash-specific extensions are marked as such.
A shell variable is a shell-specific construct that is limited to the shell that defines it (with the exception of subshells, which get their own copies of the current shell's variables),
whereas an environment variable is inherited by any child process created by the current process (shell), whether that child process is itself a shell or not.
Note that all-uppercase variable names should only be used for environment variables.
Either way, a child process only ever inherits copies of variables, whose modification (by the child) does not affect the parent.
All environment variables are also shell variables (the shell ensures that),
but the inverse is NOT true: shell variables are NOT environment variables, unless explicitly designated or inherited as such - this designation is called exporting.
note that the off-by-default -a shell option (set with set -a, or passed to the shell itself as a command-line option) can be used to auto-export all shell variables.
Thus,
any variables you create implicitly by assignment - e.g., TSAN_OPTIONS="suppressions=/somewhere/file" - are ONLY shell variables, but NOT ALSO environment variables,
EXCEPT - perhaps confusingly - when prepended directly to a command - e.g. TSAN_OPTIONS="suppressions=/somewhere/file" ./myprogram - in which case they are ONLY environment variables, only in effect for THAT COMMAND.
This is what Etan's answer describes.
Shell variables become environment variables as well under the following circumstances:
based on environment variables that the shell itself inherited, such as $HOME
shell variables created explicitly with export varName[=value] or, in bash, also with declare -x varName[=value]
by contrast, in bash, using declare without -x, or using local in a function, creates mere shell variables
shell variables created implicitly while the off-by-default -a shell option is in effect (with limited exceptions)
Once a shell variable is marked as exported - i.e., marked as an environment variable - any subsequent changes to the shell variable update the environment variable as well; e.g.:
export TSAN_OPTIONS # creates shell variable *and* corresponding environment variable
# ...
TSAN_OPTIONS="suppressions=/somewhere/file" # updates *both* the shell and env. var.
export -p prints all environment variables
unset [-v] MYVAR undefines shell variable $MYVAR and also removes it as an environment variable, if applicable.
in bash:
You can "unexport" a given variable without also undefining it as a shell variable with export -n MYVAR - this removes MYVAR from the environment, but retains its current value as a shell variable.
declare -p MYVAR prints variable $MYVAR's current value along with its attributes; if the output starts with declare -x, $MYVAR is exported (is an environment variable)