Related
I've written a simple experiment which consists of cpp and shell. The main thing that cpp does is execve("./test.sh", NULL, NULL);, and shell tries to output environment variables as follows:
printenv
echo "PATH: $PATH"
It's quite expected that the environment is empty since I've sent no environment variables, but I get the following output:
PWD=/home/user/code/security_playground/display_path
PATH: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
So, first of all, the environment is not empty. Thus, the first question is how the environment is populated? What are defaults and minimal environment you could possibly get and where is it defined (if defined in config, etc.)?
Second question is where did PATH emerge from. It's not printed by printenv, but it's there if I print it with echo, and test.sh is still capable of calling utilities. Brief googling gave me an idea of /etc/environment, but it's contents is different:
└─$ cat /etc/environment
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
Which, by the way, is surprisingly empty. Why?
I do realize that much of that should be somewhere in docs, but all my googling lead me to quite generic environment descriptions, so any answers or search hints are welcome.
Where default PATH environment variable is defined
It depends. On the distribution, on the shell, on the environment, on OS, on configuration files.
I have some systems at hand, I see:
ArchLinux adds to PATH in /etc/profile.
OpenSuSE has yet a different code that adds to PATH from /etc/profile
CentOS7 has has yet a different code in /etc/profile
I suspect that each distribution places files in slightly different places, they will ship with slightly different configuration.
how the environment is populated?
Bash has DEFAULT_PATH_VALUE that is used when PATH is not set. But don't look at the default value too much - package distrbutors overwrite it.
There is /etc/environment (and pam_env.conf) that are read by PAM on login. So when you login. But not when you chroot.
The is /etc/profile and ~/.profile files read by the shell. These scripts may read other files. So usually there is drop-in dir /etc/profile.d/*.
In bash there is /etc/bashrc or /etc/bash.bashrc /etc/bash.login /etc/bash.logout or similar and user configuration files ``~/.bashrcetc. In bash also/etc/bash_completion.dand all scripts in/usr/share/bash_completionare also sourced, when completion is enabled. BUT these files are for Bourne shell - there are also other shells, notablyzshandcsh`, and people use them and they have different syntax and different set of startup files.
Each of these scripts can manipulate PATH, it can reset it, append to it, do whatever it wants to it.
And there are also different set of files read on login/non-login + interactive/non-interactive shells, and these files can include each other (it's common to have .bash_profile or other files that just source .bashrc or similar).
What are defaults
The /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin looks like a good default that should be safe on any linux.
by the way, is surprisingly empty. Why?
My /etc/environment file is just empty and has only the default comment in linux-pam package. /etc/environment has a very simple syntax and is read by pam (there may be no pam on your system...). To modify shell environment, one would prefer shell files - so /etc/profile is the place to keep system modifications. Also /etc/environment has simple key=value syntax, wherea's in /etc/profile you have, well, shell. And /etc/environment is read on login by PAM - so putting PATH just there would break I think for example docker, it just runs a shell in chroot, there is login.
To quote the POSIX specification:
If PATH is unset or is set to null, the path search is implementation-defined.
One way way an implementation can behave when PATH is initially unset is to simply set it to a hardcoded value. (It could also refuse to run any external executables at all, or throw monkeys at your face; "implementation defined" means the standard takes no position on what the computer does in the circumstance at hand, so it's unwise to make any assumption about what that behavior will be).
Similarly, for PWD, with emphasis added:
The value is set by the cd utility, and by the sh utility during initialization
...so initializing this at shell startup time is explicitly mandated by the standard.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I've used a number of different *nix-based systems of the years, and it seems like every flavor of Bash I use has a different algorithm for deciding which startup scripts to run. For the purposes of tasks like setting up environment variables and aliases and printing startup messages (e.g. MOTDs), which startup script is the appropriate place to do these?
What's the difference between putting things in .bashrc, .bash_profile, and .environment? I've also seen other files such as .login, .bash_login, and .profile; are these ever relevant? What are the differences in which ones get run when logging in physically, logging in remotely via ssh, and opening a new terminal window? Are there any significant differences across platforms (including Mac OS X (and its Terminal.app) and Cygwin Bash)?
The main difference with shell config files is that some are only read by "login" shells (eg. when you login from another host, or login at the text console of a local unix machine). these are the ones called, say, .login or .profile or .zlogin (depending on which shell you're using).
Then you have config files that are read by "interactive" shells (as in, ones connected to a terminal (or pseudo-terminal in the case of, say, a terminal emulator running under a windowing system). these are the ones with names like .bashrc, .tcshrc, .zshrc, etc.
bash complicates this in that .bashrc is only read by a shell that's both interactive and non-login, so you'll find most people end up telling their .bash_profile to also read .bashrc with something like
[[ -r ~/.bashrc ]] && . ~/.bashrc
Other shells behave differently - eg with zsh, .zshrc is always read for an interactive shell, whether it's a login one or not.
The manual page for bash explains the circumstances under which each file is read. Yes, behaviour is generally consistent between machines.
.profile is simply the login script filename originally used by /bin/sh. bash, being generally backwards-compatible with /bin/sh, will read .profile if one exists.
That's simple. It's explained in man bash:
/bin/bash
The bash executable
/etc/profile
The systemwide initialization file, executed for login shells
~/.bash_profile
The personal initialization file, executed for login shells
~/.bashrc
The individual per-interactive-shell startup file
~/.bash_logout
The individual login shell cleanup file, executed when a login shell exits
~/.inputrc
Individual readline initialization file
Login shells are the ones that are read one you login (so, they are not executed when merely starting up xterm, for example). There are other ways to login. For example using an X display manager. Those have other ways to read and export environment variables at login time.
Also read the INVOCATION chapter in the manual. It says "The following paragraphs describe how bash executes its startup files.", i think that's a spot-on :) It explains what an "interactive" shell is too.
Bash does not know about .environment. I suspect that's a file of your distribution, to set environment variables independent of the shell that you drive.
Classically, ~/.profile is used by Bourne Shell, and is probably supported by Bash as a legacy measure. Again, ~/.login and ~/.cshrc were used by C Shell - I'm not sure that Bash uses them at all.
The ~/.bash_profile would be used once, at login. The ~/.bashrc script is read every time a shell is started. This is analogous to /.cshrc for C Shell.
One consequence is that stuff in ~/.bashrc should be as lightweight (minimal) as possible to reduce the overhead when starting a non-login shell.
I believe the ~/.environment file is a compatibility file for Korn Shell.
I found information about .bashrc and .bash_profile here to sum it up:
.bash_profile is executed when you
login. Stuff you put in there might be
your PATH and other important
environment variables.
.bashrc is used for non login shells.
I'm not sure what that means. I know
that RedHat
executes it everytime you start
another shell (su to this user or
simply calling bash again) You might
want to put aliases in there but again
I am not sure what that means. I
simply ignore it myself.
.profile is the equivalent of
.bash_profile for the root. I think
the name is changed to let other
shells (csh, sh, tcsh) use it as well.
(you don't need one as a user)
There is also .bash_logout wich
executes at, yeah good guess...logout.
You might want to stop deamons or even
make a little housekeeping . You can
also add "clear" there if you want to
clear the screen when you log out.
Also there is a complete follow up on each of the configurations files here
These are probably even distro.-dependant, not all distros choose to have each configuraton with them and some have even more. But when they have the same name, they usualy include the same content.
According to Josh Staiger, Mac OS X's Terminal.app actually runs a login shell rather than a non-login shell by default for each new terminal window, calling .bash_profile instead of .bashrc.
He recommends:
Most of the time you don’t want to maintain two separate config files
for login and non-login shells — when you set a PATH, you want it to
apply to both. You can fix this by sourcing .bashrc from your
.bash_profile file, then putting PATH and common settings in .bashrc.
To do this, add the following lines to .bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
Now when you login to your
machine from a console .bashrc will be called.
A good place to look at is the man page of bash. Here's an online version. Look for "INVOCATION" section.
I have used Debian-family distros which appear to execute .profile, but not .bash_profile,
whereas RHEL derivatives execute .bash_profile before .profile.
It seems to be a mess when you have to set up environment variables to work in any Linux OS.
Can I have certain settings that are universal for all my users?
As well as /etc/profile which others have mentioned, some Linux systems now use a directory /etc/profile.d/; any .sh files in there will be sourced by /etc/profile. It's slightly neater to keep your custom environment stuff in these files than to just edit /etc/profile.
If your LinuxOS has this file:
/etc/environment
You can use it to permanently set environmental variables for all users.
Extracted from: http://www.sysadmit.com/2016/04/linux-variables-de-entorno-permanentes.html
man 8 pam_env
man 5 pam_env.conf
If all login services use PAM, and all login services have session required pam_env.so in their respective /etc/pam.d/* configuration files, then all login sessions will have some environment variables set as specified in pam_env's configuration file.
On most modern Linux distributions, this is all there by default -- just add your desired global environment variables to /etc/security/pam_env.conf.
This works regardless of the user's shell, and works for graphical logins too (if xdm/kdm/gdm/entrance/… is set up like this).
Amazingly, Unix and Linux do not actually have a place to set global environment variables. The best you can do is arrange for any specific shell to have a site-specific initialization.
If you put it in /etc/profile, that will take care of things for most posix-compatible shell users. This is probably "good enough" for non-critical purposes.
But anyone with a csh or tcsh shell won't see it, and I don't believe csh has a global initialization file.
Some interesting excerpts from the bash manpage:
When bash is invoked as an interactive
login shell, or as a non-interactive
shell with the --login option, it
first reads and executes commands from
the file /etc/profile, if that file
exists. After reading that file, it
looks for ~/.bash_profile,
~/.bash_login, and ~/.profile, in that
order, and reads and executes commands
from the first one that exists and is
readable. The --noprofile option may
be used when the shell is started to
inhibit this behavior.
...
When an
interactive shell that is not a login
shell is started, bash reads and
executes commands from
/etc/bash.bashrc and ~/.bashrc, if
these files exist. This may be
inhibited by using the --norc option.
The --rcfile file option will force
bash to read and execute commands from
file instead of /etc/bash.bashrc and
~/.bashrc.
So have a look at /etc/profile or /etc/bash.bashrc, these files are the right places for global settings. Put something like this in them to set up an environement variable:
export MY_VAR=xxx
Every process running under the Linux kernel receives its own, unique environment that it inherits from its parent. In this case, the parent will be either a shell itself (spawning a sub shell), or the 'login' program (on a typical system).
As each process' environment is protected, there is no way to 'inject' an environmental variable to every running process, so even if you modify the default shell .rc / profile, it won't go into effect until each process exits and reloads its start up settings.
Look in /etc/ to modify the default start up variables for any particular shell. Just realize that users can (and often do) change them in their individual settings.
Unix is designed to obey the user, within limits.
NB: Bash is not the only shell on your system. Pay careful attention to what the /bin/sh symbolic link actually points to. On many systems, this could actually be dash which is (by default, with no special invocation) POSIXLY correct. Therefore, you should take care to modify both defaults, or scripts that start with /bin/sh will not inherit your global defaults. Similarly, take care to avoid syntax that only bash understands when editing both, aka avoiding bashisms.
Using PAM is execellent.
# modify the display PAM
$ cat /etc/security/pam_env.conf
# BEFORE: $ export DISPLAY=:0.0 && python /var/tmp/myproject/click.py &
# AFTER : $ python $abc/click.py &
DISPLAY DEFAULT=${REMOTEHOST}:0.0 OVERRIDE=${DISPLAY}
abc DEFAULT=/var/tmp/myproject
In bash login shell, there are dozens of environment variables exists, such as follow:
HOSTNAME=myhost
TERM=screen
SHELL=/bin/bash
HISTSIZE=1000
SSH_TTY=/dev/pts/20
LC_ALL=en_US.UTF-8
USER=user
LD_LIBRARY_PATH=$:/usr:/usr/lib:/usr/local/lib:/lib:/usr/local/lib64
DRC_ROOT=/home/ds
PATH=/usr/local/mysql/bin:/usr/local/mysql/bin:/usr/local/mysql/bin:/usr/local/mysql/bin:/usr/local/mysql/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin
MAIL=/var/spool/mail/user
PWD=/data/user
JAVA_HOME=/usr/java/jdk1.7.0_67-cloudera
LANG=en_US.UTF-8
TMUX_PANE=%135
PS1=(dbrt_env) \[\e[37m\][\[\e[32m\]\u\[\e[32m\](\[\e[36m\]\[\e[37m\])\[\e[35m\]#\[\e[0m\]\h \[\e[33m\]\W\[\e[0m\]]\$
HISTCONTROL=ignoredups
SHLVL=2
HOME=/data/user
LOGNAME=user
REALUSERNAME=
CVS_RSH=ssh
HISTTIMEFORMAT=%F %T
G_BROKEN_FILENAMES=1
_=/bin/env
.....
but in crontab job, the envrioment variables is really few:
SHELL=/bin/sh
USER=user
PATH=/usr/bin:/bin
PWD=/data/user
LANG=en_US.UTF-8
SHLVL=1
HOME=/data/user
LOGNAME=user
_=/usr/bin/env
what's the designing intent of the difference?
Why not make them the same?
Cron uses a different shell (/bin/sh) to your login shell (/bin/bash). The bash shell has various different files that it uses to set things up (man bash will give you the full details).
The best thing to do is not to try to get them to be the same (why does cron need PS1 etc.) rather create a script that has everything that you need in a controlled way and have cron use that.
If the environment that you want is in /home/me/setupenv.sh then add the following to the cron script and it will run it:
. /home/me/setupend.sh
Don't forget the leading . otherwise it will run the script in a different environment and the changes will be lost when the script ends.
An interactive environment is expected to be heavily customized for an individual user's preferences. As such, there are several sources of extra environment variables present:
PAM modules and other login-time sources (including, in the above case, environment variables set by SSH)
~/.bash_profile, /etc/profile, and other scripts intended for login-shell use
~/.bashrc and other scripts intended for interactive-shell initialization
By contrast, the noninteractive environment exposed to daemons (including cron) is expected to be predictable and consistent -- meaning that a script written on one machine should work on another. Pulling in an individual user's settings is contrary to this goal. Moreover, since there is no login shell involved in running cron, scripts which are only sourced by a login shell are never executed.
If you take a close look at the environment list you provided, many of them make no sense at all in a noninteractive shell:
TERM only makes sense when there's a terminal.
TMUX_PANE only makes sense when that terminal is tmux.
HISTSIZE only makes sense when you have an interactive shell storing history.
SSH_TTY only makes sense when you connected via ssh.
LC_ALL specifies language and locale settings to use for communicating with the user who's operating the terminal.
PS1 specifies how to format an interactive prompt.
Etc. Etc. Etc.
If you need a variable exported when your crontab is run, specify it explicitly at the top of that crontab. This prevents settings made for the convenience of interactive users from breaking or modifying the behavior of background jobs.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
I've used a number of different *nix-based systems of the years, and it seems like every flavor of Bash I use has a different algorithm for deciding which startup scripts to run. For the purposes of tasks like setting up environment variables and aliases and printing startup messages (e.g. MOTDs), which startup script is the appropriate place to do these?
What's the difference between putting things in .bashrc, .bash_profile, and .environment? I've also seen other files such as .login, .bash_login, and .profile; are these ever relevant? What are the differences in which ones get run when logging in physically, logging in remotely via ssh, and opening a new terminal window? Are there any significant differences across platforms (including Mac OS X (and its Terminal.app) and Cygwin Bash)?
The main difference with shell config files is that some are only read by "login" shells (eg. when you login from another host, or login at the text console of a local unix machine). these are the ones called, say, .login or .profile or .zlogin (depending on which shell you're using).
Then you have config files that are read by "interactive" shells (as in, ones connected to a terminal (or pseudo-terminal in the case of, say, a terminal emulator running under a windowing system). these are the ones with names like .bashrc, .tcshrc, .zshrc, etc.
bash complicates this in that .bashrc is only read by a shell that's both interactive and non-login, so you'll find most people end up telling their .bash_profile to also read .bashrc with something like
[[ -r ~/.bashrc ]] && . ~/.bashrc
Other shells behave differently - eg with zsh, .zshrc is always read for an interactive shell, whether it's a login one or not.
The manual page for bash explains the circumstances under which each file is read. Yes, behaviour is generally consistent between machines.
.profile is simply the login script filename originally used by /bin/sh. bash, being generally backwards-compatible with /bin/sh, will read .profile if one exists.
That's simple. It's explained in man bash:
/bin/bash
The bash executable
/etc/profile
The systemwide initialization file, executed for login shells
~/.bash_profile
The personal initialization file, executed for login shells
~/.bashrc
The individual per-interactive-shell startup file
~/.bash_logout
The individual login shell cleanup file, executed when a login shell exits
~/.inputrc
Individual readline initialization file
Login shells are the ones that are read one you login (so, they are not executed when merely starting up xterm, for example). There are other ways to login. For example using an X display manager. Those have other ways to read and export environment variables at login time.
Also read the INVOCATION chapter in the manual. It says "The following paragraphs describe how bash executes its startup files.", i think that's a spot-on :) It explains what an "interactive" shell is too.
Bash does not know about .environment. I suspect that's a file of your distribution, to set environment variables independent of the shell that you drive.
Classically, ~/.profile is used by Bourne Shell, and is probably supported by Bash as a legacy measure. Again, ~/.login and ~/.cshrc were used by C Shell - I'm not sure that Bash uses them at all.
The ~/.bash_profile would be used once, at login. The ~/.bashrc script is read every time a shell is started. This is analogous to /.cshrc for C Shell.
One consequence is that stuff in ~/.bashrc should be as lightweight (minimal) as possible to reduce the overhead when starting a non-login shell.
I believe the ~/.environment file is a compatibility file for Korn Shell.
I found information about .bashrc and .bash_profile here to sum it up:
.bash_profile is executed when you
login. Stuff you put in there might be
your PATH and other important
environment variables.
.bashrc is used for non login shells.
I'm not sure what that means. I know
that RedHat
executes it everytime you start
another shell (su to this user or
simply calling bash again) You might
want to put aliases in there but again
I am not sure what that means. I
simply ignore it myself.
.profile is the equivalent of
.bash_profile for the root. I think
the name is changed to let other
shells (csh, sh, tcsh) use it as well.
(you don't need one as a user)
There is also .bash_logout wich
executes at, yeah good guess...logout.
You might want to stop deamons or even
make a little housekeeping . You can
also add "clear" there if you want to
clear the screen when you log out.
Also there is a complete follow up on each of the configurations files here
These are probably even distro.-dependant, not all distros choose to have each configuraton with them and some have even more. But when they have the same name, they usualy include the same content.
According to Josh Staiger, Mac OS X's Terminal.app actually runs a login shell rather than a non-login shell by default for each new terminal window, calling .bash_profile instead of .bashrc.
He recommends:
Most of the time you don’t want to maintain two separate config files
for login and non-login shells — when you set a PATH, you want it to
apply to both. You can fix this by sourcing .bashrc from your
.bash_profile file, then putting PATH and common settings in .bashrc.
To do this, add the following lines to .bash_profile:
if [ -f ~/.bashrc ]; then
source ~/.bashrc
fi
Now when you login to your
machine from a console .bashrc will be called.
A good place to look at is the man page of bash. Here's an online version. Look for "INVOCATION" section.
I have used Debian-family distros which appear to execute .profile, but not .bash_profile,
whereas RHEL derivatives execute .bash_profile before .profile.
It seems to be a mess when you have to set up environment variables to work in any Linux OS.