I once read that one way to obtain a unique filename in a shell for temp files was to use a double dollar sign ($$). This does produce a number that varies from time to time... but if you call it repeatedly, it returns the same number. (The solution is to just use the time.)
I am curious to know what $$ actually is, and why it would be suggested as a way to generate unique filenames.
$$ is the process ID (PID) in bash. Using $$ is a bad idea, because it will usually create a race condition, and allow your shell-script to be subverted by an attacker. See, for example, all these people who created insecure temporary files and had to issue security advisories.
Instead, use mktemp. The Linux man page for mktemp is excellent. Here's some example code from it:
tempfoo=`basename $0`
TMPFILE=`mktemp -t ${tempfoo}` || exit 1
echo "program output" >> $TMPFILE
In Bash $$ is the process ID, as noted in the comments it is not safe to use as a temp filename for a variety of reasons.
For temporary file names, use the mktemp command.
$$ is the id of the current process.
Every process in a UNIX like operating system has a (temporarily) unique identifier, the PID. No two processes running at the same time can have the same PID, and $$ refers to the PID of the bash instance running the script.
This is very much not a unique idenifier in the sense that it will never be reused (indeed, PIDs are reused constantly). What it does give you is a number such that, if another person runs your script, they will get a different identifier whilst yours is still running. Once yours dies, the PID may be recycled and someone else might run your script, get the same PID, and so get the same filename.
As such, it is only really sane to say "$$ gives a filename such that if someone else runs the same script whist my instance is still running, they will get a different name".
$$ is your PID. It doesn't really generate a unique filename, unless you are careful and no one else does it exactly the same way.
Typically you'd create something like /tmp/myprogramname$$
There're so many ways to break this, and if you're writing to locations other folks can write to it's not too difficult on many OSes to predict what PID you're going to have and screw around -- imagine you're running as root and I create /tmp/yourprogname13395 as a symlink pointing to /etc/passwd -- and you write into it.
This is a bad thing to be doing in a shell script. If you're going to use a temporary file for something, you ought to be using a better language which will at least let you add the "exclusive" flag for opening (creating) the file. Then you can be sure you're not clobbering something else.
$$ is the pid (process id) of the shell interpreter running your script. It's different for each process running on a system at the moment, but over time the pid wraps around, and after you exit there will be another process with same pid eventually.As long as you're running, the pid is unique to you.
From the definition above it should be obvious that no matter how many times you use $$ in a script, it will return the same number.
You can use, e.g. /tmp/myscript.scratch.$$ as your temp file for things that need not be extremely reliable or secure. It's a good practice to delete such temp files at the end of your script, using, for example, trap command:
trap "echo 'Cleanup in progress'; rm -r $TMP_DIR" EXIT
$$ is the pid of the current shell process. It isn't a good way to generate unique filenames.
It's the process ID of the bash process. No concurrent processes will ever have the same PID.
The $$ is the process id of the shell in which your script is running. For more details, see the man page for sh or bash. The man pages can be found be either using a command line "man sh", or by searching the web for "shell manpage"
Let me second emk's answer -- don't use $$ by itself as a "unique" anything. For files, use mktemp. For other IDs within the same bash script, use "$$$(date +%s%N)" for a reasonably good chance of uniqueness.
-k
In Fish shell (3.1.2):
The $ symbol can also be used multiple times, as a kind of "dereference" operator (the * in C or C++)
set bar bazz
set foo bar
echo $foo # bar
echo $$foo # same as echo $bar → bazz
Also, You can grab login username via this command. Eg.
echo $(</proc/$$/login id). After that, you need to use getent command.
Related
So, I need to run a program, not keep track of its PID (in memory, at least), and later kill that program. Any ideas? My immediate thought was to tag the process with something I could find later, but that seems a bust. My next thought was to store on disk the PID, but I've no idea what the convention for that sort of thing is. Any ideas? Thanks!
Your program can create a directory under /var/run/ to store such files. For instance, if your program is myprog it might store its PID on startup in
/var/run/myprog/PID
If your program could have multiple instances running at the same time, you might use the PID itself in the file name, along with its startup time, to ensure a unique file name for each instance.
/var/run/myprog/201410302306.1283.pid
(Note that if you use the PID in the file name, it's up to you if you actually write the PID in the file itself; an empty file would suffice.)
The most straight forward (and common case) is, as you and chepner already mentioned, to store it in a pid file. For this:
If it's a system wide programm:
/var/run/prog.pid
If it's a system installed program that might be run multiple times:
/var/run/prog/prog_instance.pid
If it's a user local program:
$HOME/.prog/prog.pid
Other options might be to use pgrep for finding it as long as you can uniquely define the calling command. For example pass a dummy argument not used and use it for retrieval:
$ prog.sg --instance_1234 &
$ pgrep -f -- '--instance_1234'
3523
you can use start-stop-daemon to start a program and kill it later. It has a lot of options to find the programm, the most usefull is the --pid that stores and retrive the pid from a file in filesystem, usually in /var/run (pay attention to file system privileges to write in it).
See the man pages for more tips.
In a shell, I run following commands without problem,
ls -al
!ls
the second invocation to ls also list files with -al flag. However, when I put the above script to a bash script, complaints are thrown,
!ls, command not found.
how to realise the same effects in script?
You would need to turn on both command history and !-style history expansion in your script (both are off by default in non-interactive shells):
set -o history
set -o histexpand
The expanded command is also echoed to standard error, just like in an interactive shell. You can prevent that by turning on the histverify shell option (shopt -s histverify), but in a non-interactive shell, that seems to make the history expansion a null-op.
Well, I wanted to have this working as well, and I have to tell everybody that the set -o history ; set -o histexpand method will not work in bash 4.x. It's not meant to be used there, anyway, since there are better ways to accomplish this.
First of all, a rather trivial example, just wanting to execute history in a script:
(bash 4.x or higher ONLY)
#!/bin/bash -i
history
Short answer: it works!!
The spanking new -i option stands for interactive, and history will work. But for what purpose?
Quoting Michael H.'s comment from the OP:
"Although you can enable this, this is bad programming practice. It will make your scripts (...) hard to understand. There is a reason it is disabled by default. Why do you want to do this?"
Yes, why? What is the deeper sense of this?
Well, THERE IS, which I'm going to demonstrate in the follow-up section.
My history buffer has grown HUGE, while some of those lines are script one-liners, which I really would not want to retype every time. But sometimes, I also want to alter these lines a little, because I probably want to give a third parameter, whereas I had only needed two in total before.
So here's an ideal way of using the bash 4.0+ feature to invoke history:
$ history
(...)
<lots of lines>
(...)
1234 while IFS='whatever' read [[ $whatever -lt max ]]; do ... ; done < <(workfile.fil)
<25 more lines>
So 1234 from history is exactly the line we want. Surely, we could take the mouse and move there, chucking the whole line in the primary buffer? But we're on *NIX, so why can't we make our life a bit easier?
This is why I wrote the little script below. Again, this is for bash 4.0+ ONLY (but might be adapted for bash 3.x and older with the aforementioned set -o ... stuff...)
#!/bin/bash -i
[[ $1 == "" ]] || history | grep "^\s*$1" |
awk '{for (i=2; i<=NF; i++) printf $i" "}' | tr '\n' '\0'
If you save this as xselauto.sh for example, you may invoke
$ ./xselauto.sh 1234
and the contents of history line #1234 will be in your primary buffer, ready for re-use!
Now if anyone still says "this has no purpose AFAICS" or "who'd ever be needing this feature?" - OK, I won't care. But I would no longer want to live without this feature, as I'm just too lazy to retype complex lines every time. And I wouldn't want to touch the mouse for each marked line from history either, TBH. This is what xsel was written for.
BTW, the tr part of the pipe is a dirty hack which will prevent the command from being executed. For "dangerous" commands, it is extremely important to always leave the user a way to look before he/she hits the Enter key to execute it. You may omit it, but ... you have been warned.
P.S. This scriptlet is in fact a workaround, simulating !1234 typed on a bash shell. As I could never make the ! work directly in a script (echo would never let me reveal the contents of history line 1234), I worked around the problem by simply greping for the line I wanted to copy.
History expansion is part of the interactive command-line editing features of a shell, not part of the scripting language. It's not generally available in the context of a script, only when interacting with a (pseudo-)human operator. (pseudo meaning that it can be made to work with things like expect or other keystroke repeating automation tools that generally try to play act a human, not implying that any particular operator might be sub-human or anything).
My Makefile look something like this:
setsid ./CppServer>daemon.log 2>&1 &
echo $!>daemon.pid
What I expect it to do is to store the PID of my_awesome_script in corresponding file. However there's nothing there. So where's the problem?
If your makefile really looks like this you will get an error, because I don't see any actual make syntax, just shell syntax. However, my crystal ball tells me that these two lines might be part of the recipe for a rule. If they are, you should realise how make executes recipes; for each line a separate subshell is created, in which that line's command is executed independently: your two commands don't know anything about each other's environment. If you want two commands to be executed in the same subshell, you should issue them as one line (using line continuation characters if necessary), or use make's ONESHELL directive.
The variable you're trying to use prints the pid of the last program run in the background. It is correctly written as echo $! > filename.extension. But since you are running it in the foregorund you have two choices. Run it in the background by appending an & to the end of the line ./script_to_run &, or you can have the script itself print to file the pid of the currently running process by using echo $$ > filename.extension (inside the script). Here is a link that might help you http://tldp.org/LDP/abs/html/internalvariables.html
I have written a script that relies on other server responses (uses wget to pull data), and I want it to always be run in the background unquestionably. I know one solution is to just write a wrapper script that will call my script with an & appended, but I want to avoid that clutter.
Is there a way for a bash (or zsh) script to determine if it was called with say ./foo.sh &, and if not, exit and re-launch itself as such?
The definition of a background process (I think) is that it has a controlling terminal but it is not part of that terminal's foreground process group. I don't think any shell, even zsh, gives you any access to that information through a builtin.
On Linux (and perhaps other unices), the STAT column of ps includes a + when the process is part of its terminal's foreground process group. So a literal answer to your question is that you could put your script's content in a main function and invoke it with:
case $(ps -o stat= -p $$) in
*+*) main "$#" &;;
*) main "$#";;
esac
But you might as well run main "$#" & anyway. On Unix, fork is cheap.
However, I strongly advise against doing what you propose. This makes it impossible for someone to run your script and do something else afterwards — one would expect to be able to write your_script; my_postprocessing or your_script && my_postprocessing, but forking the script's main task makes this impossible. Considering that the gain is occasionally saving one character when the script is invoked, it's not worth making your script markedly less useful in this way.
If you really mean for the script to run in the background so that the user can close his terminal, you'll need to do more work — you'll need to daemonize the script, which includes not just backgrounding but also closing all file descriptors that have the terminal open, making the process a session leader and more. I think that will require splitting your script into a daemonizing wrapper script and a main script. But daemonizing is normally done for programs that never terminate unless explicitly stopped, which is not the behavior you describe.
I do not know, how to do this, but you may set variable in parent script and check for it in child:
if [[ -z "$_BACKGROUNDED" ]] ; then
_BACKGROUNDED=1 exec "$0" "$#" & exit
fi
# Put code here
Works both in bash and zsh.
the "tty" command says "not a tty" if you're in the background, or gives the controlling terminal name (/dev/pts/1 for example) if you're in the foreground. A simple way to tell.
Remember that you can't (or, not recommended to) edit the running script. This question and the answers give workarounds.
I don't write shell scripts a long time ago, but I can give you a very good idea (I hope). You can check the value of $$ (this is the PID of the process) and compare with the output of the command "jobs -l". This last command will return the PID of all the backgrounded processes (jobs) and if the value of $$ is contained in the result of the "jobs -l", this means that the current script is running on background.
I am creating temporary files from a bash script. I am deleting them at the end of the processing, but since the script is running for quite a long time, if I kill it or simply CTRL-C during the run, the temp files are not deleted.
Is there a way I can catch those events and clean-up the files before the execution ends?
Also, is there some kind of best practice for the naming and location of those temp files?
I'm currently not sure between using:
TMP1=`mktemp -p /tmp`
TMP2=`mktemp -p /tmp`
...
and
TMP1=/tmp/`basename $0`1.$$
TMP2=/tmp/`basename $0`2.$$
...
Or maybe is there some better solutions?
I usually create a directory in which to place all my temporary files, and then immediately after, create an EXIT handler to clean up this directory when the script exits.
MYTMPDIR="$(mktemp -d)"
trap 'rm -rf -- "$MYTMPDIR"' EXIT
If you put all your temporary files under $MYTMPDIR, then they will all be deleted when your script exits in most circumstances. Killing a process with SIGKILL (kill -9) kills the process right away though, so your EXIT handler won't run in that case.
You could set a "trap" to execute on exit or on a control-c to clean up.
trap '{ rm -f -- "$LOCKFILE"; }' EXIT
Alternatively, one of my favourite unix-isms is to open a file, and then delete it while you still have it open. The file stays on the file system and you can read and write it, but as soon as your program exits, the file goes away. Not sure how you'd do that in bash, though.
BTW: One argument I'll give in favour of mktemp instead of using your own solution: if the user anticipates your program is going to create huge temporary files, he might want set TMPDIR to somewhere bigger, like /var/tmp. mktemp recognizes that, your hand-rolled solution (second option) doesn't. I frequently use TMPDIR=/var/tmp gvim -d foo bar, for instance.
You want to use the trap command to handle exiting the script or signals like CTRL-C. See the Greg's Wiki for details.
For your tempfiles, using basename $0 is a good idea, as well as providing a template that provides room for enough temp files:
tempfile() {
tempprefix=$(basename "$0")
mktemp /tmp/${tempprefix}.XXXXXX
}
TMP1=$(tempfile)
TMP2=$(tempfile)
trap 'rm -f $TMP1 $TMP2' EXIT
Just keep in mind that choosen answer is bashism, which means solution as
trap "{ rm -f $LOCKFILE }" EXIT
would work only in bash (it will not catch Ctrl+c if shell is dash or classic sh), but if you want compatibility then you still need to enumerate all signals that you want to trap.
Also keep in mind that when script exits the trap for signal "0"(aka EXIT) is always performed resulting in double execution of trap command.
That the reason not to stack all signals in one line if there is EXIT signal.
To better understand it look at following script that will work across different systems without changes:
#!/bin/sh
on_exit() {
echo 'Cleaning up...(remove tmp files, etc)'
}
on_preExit() {
echo
echo 'Exiting...' # Runs just before actual exit,
# shell will execute EXIT(0) after finishing this function
# that we hook also in on_exit function
exit 2
}
trap on_exit EXIT # EXIT = 0
trap on_preExit HUP INT QUIT TERM STOP PWR # 1 2 3 15 30
sleep 3 # some actual code...
exit
This solution will give you more control since you can run some of your code on occurrence of actual signal just before final exit (preExit function) and if it needed you can run some code at actual EXIT signal (final stage of exit)
GOOD HABITS ARE BEAUTIFUL
Avoid assuming the value of a variable is never going to be changed at some super distant time (especially if such a bug would raise an error).
Do cause trap to expand the value of a variable immediately if applicable to your code. Any variable name passed to trap in single quotes will delay the expansion of its value until after the catch.
Avoid the assumption that a file name will not contain any spaces.
Do use Bash ${VAR#Q} or $(printf '%q' "$VAR") to overcome issues caused by spaces and other special characters like quotes and carriage returns in file names.
zTemp=$(mktemp --tmpdir "$(basename "$0")-XXX.ps")
trap "rm -f ${zTemp#Q}" EXIT
The alternative of using a predictable file name with $$ is a gaping security hole and you should never, ever, ever think about using it. Even if it is just a simple personal script on your single user PC. It is a very bad habit you should not obtain. BugTraq is full of "insecure temp file" incidents. See here, here and here for more information on the security aspect of temp files.
I was initially thinking of quoting the insecure TMP1 and TMP2 assignments, but on second thought that would probably not be a good idea.
I prefer using tempfile which creates a file in /tmp in the safe manner and you do not have to worry about its naming:
tmp=$(tempfile -s "your_sufix")
trap "rm -f '$tmp'" exit
You don't have to bother removing those tmp files created with mktemp. They will be deleted anyway later.
Use mktemp if you can as it generates more unique files then '$$' prefix. And it looks like more cross platform way to create temp files then explicitly put them into /tmp.