Is there a flock command on Mac OS X that manages file lock?
http://linux.die.net/man/1/flock
There is a cross-platform flock command here:
https://github.com/discoteq/flock
I have tested it and it works well on OSX as a drop-in replacement for the util-linux flock.
Perl one-liner:
perl -MFcntl=:flock -e '$|=1; $f=shift; print("starting\n"); open(FH,$f) || die($!); flock(FH,LOCK_EX); print("got lock\n"); system(join(" ",#ARGV)); print("unlocking\n"); flock(FH,LOCK_UN); ' /tmp/longrunning.sh /tmp/longrunning.sh
As a script:
#!/usr/bin/perl
# emulate linux flock command line utility
#
use warnings;
use strict;
use Fcntl qw(:flock);
# line buffer
$|=1;
my $file = shift;
my $cmd = join(" ",#ARGV);
if(!$file || !$cmd) {
die("usage: $0 <file> <command> [ <command args>... ]\n");
}
print("atempting to lock file: $file\n");
open(FH,$file) || die($!);
flock(FH,LOCK_EX) || die($!);
print("got lock\n");
print("running command: $cmd\n");
system($cmd);
print("unlocking file: $file\n");
flock(FH,LOCK_UN);
I don't believe that the flock command exists on OS X, but it does exist on BSD which should make it reasonably easy to port to OS X.
The closest that is available is the shlock command (man page), but it isn't as robust or secure as flock.
Your best bet may be to look at porting either the Linux or BSD version of flock to OS X.
macOS does not ship with a flock command, no, but you can install one via Homebrew (brew install flock). Which is probably the way to go if you need a shell script that can share a lockable resource with programs that use the flock system call to manage access to that resource.
If you are just trying to synchronize access to something and don't require compatibility with things already using flock, you could alternatively install procmail and use lockfile instead.
Just for completeness sake, you can compile flock(2) for OSX with some minor changes, i have not run any tests, but basic functionality works.
You can get the source from ftp://ftp.kernel.org//pub/linux/utils/util-linux. You then need to replace some calls to string functions not available on OSX, and you're good to go.
Here: https://gist.github.com/Ahti/4962822 is my modified flock.c of version 2.22.1, you still need the other sources for headers though.
You can install flock via conda, for example:
conda create --name flock flock
or
conda install flock
To install conda, see here.
Are you looking for flock the command line utility or flock the feature?
flock(1) is unavailable on OS X. flock(2) (the C function for file locking), however is.
Writing a simple command line flock(1) utility using flock(2) should be trivial.
You cannot write a shell-level flock(1) command for use in shell programming because of how file locking working. The lock is on the descriptor, not on the inode or directory entry.
Therefore, if you implement a shell command that flocks something, as soon as the locking command exits and the shell script moves on to the next command, the descriptor that held the lock disappears and so there is no lock retained.
The only way to implement this would be as a shell builtin. Alternately, you have to rewrite in a programming language that actually supports flock(2) directly, such as Perl.
Related
I am writing a command line tool in python and using pip to distribute it. I have written some scripts (one for bash and one for zsh) to allow the tool to have tab completion. Is there a way to get pip to install these scripts when someone does a pip install?
For example:
I have a completion.bash file. When someone does
pip install mypackage
It will also source the bash file.
I'm pretty sure I can do this for linux and bash by putting the script in my data_files section in the setup.py script.
data_file=[
('/etc/bash_completion.d', ['bin/completion.bash'])
],
But how can I do this so it is platform and shell independent? I need it to work for mac/linux in both bash and zsh. If possible, even support windows.
Is this possible? And if so, how?
In case it matters, here is my bash script:
_foo_complete() {
COMPREPLY=()
local words=( "${COMP_WORDS[#]}" )
local word="${COMP_WORDS[COMP_CWORD]}"
words=("${words[#]:1}")
local completions="$(foo completer --cmplt=\""${words[*]}"\")"
COMPREPLY=( $(compgen -W "$completions" -- "$word") )
}
complete -F _foo_complete foo
I am currently installing it by just running source completion.bash
You are asking several different questions.
First, there's no cross-platform or cross-shell solution for defining custom shell-completions. The one you posted works for bash, but in tcsh, for example, you use tcsh's complete command, which works differently than bash's.
Second, sourcing the files which contain those completion-definitions at the time of pip install wouldn't do much good. The completions might work in that very session, but what you probably want is for them to take effect in future sessions (i.e. shell invocations) as well. For that, your files would have to be sourced each time the shell starts (e.g. from within user's .bashrc, in case of bash).
This measn that "installing" your files simply means placing them somewhere, and suggesting the users should source them from their respective dot-rc file. Even if you could, you shouldn't try to "force" it. Give your users the option to add that to their dot-rc file if they want.
The best approach would probably be to include in your package a completion-definitions file per supported shell, e.g. complete.bash, complete.tcsh, and god knows what for windows (sorry, I'm not a windows user).
I am running an externally supplied perl script (the OpenSSL configure script if you are interested). It contains the line:
my $vew=`nasmw -v 2>NUL`;
I am running on Windows, but I am using the version of perl which ships with git version 2.19.0.windows.1, and this appears to by a Cygwin perl which uses sh to execute backticks. The problem is that this creates a file called 'NUL' rather than throwing stderr away. Amongst other things you can't delete such a file from Explorer or the command line. (You can via bash).
I have read this question but setting PERL5SHELL to C:\Windows\System32\cmd.exe /C doesn't stop the file being created.
Is there a way of controlling the shell that a Cygwin perl uses?
Perl -v reports version 5.26.2
(The obvious workround is to patch NUL to /dev/null but I'd rather not do that if I can avoid it.)
I mean I can't use it in bash, is it not available on OS X, or is it just missing on my Mac?
It's not a PATH variable issue, because I searched with find command, and there's no file named setsid on my Mac at all.
If it's missing on OS X, is there any alternative to it?
Or if it's the case that I somehow deleted it accidentally, where can I find a copy of it?
use Brew:
brew install util-linux
Yes. /usr/bin/setsid is missing on Mac OS/X.
The OS interface is available, so based on the chapter 2 man page there may be some hope for porting the Linux source to Darwin.
While macOS does not come with a setsid command, it does come with scripting languages which support calling the setsid C function, such as Perl and Python. So, if you don't want to (or for some reason can't) install a setsid command via Homebrew (or MacPorts or whatever), another option is to write your own in a scripting language. As an example, try this Perl script (which I based off this with some minor changes):
#!/usr/bin/perl -w
use strict;
use POSIX qw(setsid);
fork() && exit(0);
setsid() or die "setsid failed: $!";
exec #ARGV;
If you don't like Perl, Python's os module has a setsid function too.
A simple demo, which relies on the fact that /dev/tty is an alias of your controlling terminal if you have one, but reads/writes to it fail with an IO error if you don't:
$ bash -c 'echo I have a controlling terminal. > /dev/tty'
I have a controlling terminal.
$ ./setsid.pl bash -c 'echo I have a controlling terminal. > /dev/tty'
bash: /dev/tty: Device not configured
$
(Warning: With the release of macOS Catalina (10.15) in 2019, Apple deprecated the Perl, Python, Ruby and Tcl language runtimes shipped with macOS – they say new software should not use them, and they may be removed in a future macOS version – and Apple is not going to update their versions, which are becoming increasingly outdated. However, they are still there in Monterey, and while I haven't upgraded to Ventura yet, I haven't heard anything about their removal in that version either. One obviously shouldn't rely on them for any supported applications – if such software needs one of these runtimes, it should install its own copy of them. However, if it is just for a quick hacky script to easily test how some program behaves without a controlling terminal, using these OS-bundled runtimes is still fine.)
I have installed ActivePerl 5.20.2 today on Mac OS X 10.9.5
Checking the version of perl in Terminal (perl -v) I see 5.20.2
So everything seems to be ok. But..
When I start my CGI scripts the script is running under built in perl (which is 5.16) (if using #!/usr/bin/perl).
If I use #!/usr/local/ActivePerl5.20.2/bin/perl then it runs under 5.20.2 that is required.
The question is: is it somehow possible to change the directory for using in my scripts from #!/usr/local/ActivePerl5.20.2/bin/perl to simple and familiar #!/usr/bin/perl keeping running under ActivePerl instead of built in.
I need to override the system's default version with the new ActivePerl.
I would be appreciated for your detailed answers (with name of files and directories where they are located) if ones are to be changed to implement salvation.
Thanks!
The question is: is it somehow possible to change the directory for using in my scripts from #!/usr/local/ActivePerl5.20.2/bin/perl to simple and familiar #!/usr/bin/perl keeping running under ActivePerl instead of built in.
Don't even try. That way lies damnation, not salvation. The ability to specify the specific interpreter that will handle your scripts is an important feature.
Instead, package your CGI script as a simple CPAN module. Then, install it using the familiar
$ /usr/local/ActivePerl5.20.2/bin/perl Makefile.PL
$ make install
routine. The shebang line will be automatically adjusted to reflect the perl that was used to build and install your package.
First, instead of specifying a particular path to your Perl interpreter in your script:
#! /usr/local/ActivePerl5.20.2/bin/perl
or
#! /usr/bin/perl
Specify this:
#! /usr/bin/env perl
This will find the first executable Perl interpreter in your $PATH and then use that to execute your Perl script. This way, instead of having to change your program, you only have to change the $PATH variable.
Next time, take a look at PerlBrew for installing a different version of Perl. PerlBrew will allow you to install multiple versions of Perl all under user control, and let you select which version of Perl you'd like to use.
I also recommend to put /usr/local/bin as the first entry in your $PATH. Then, link the executables you want to run to that directory. You can use something like this to create your links:
for file in $/usr/local/ActivePerl5.20.2/bin/*
do
basename=$(basename $file)
ln -s "$file" "/usr/local/bin/$basename"
done
This way, all programs you want to execute are in the same directory which makes setting $PATH so much easier. I even put /usr/local/bin in before /usr/bin and /bin because I want to be able to override the system's default version.
I've seen in a number of places, including recommendations on this site (What is the preferred Bash shebang?), to use #!/usr/bin/env bash in preference to #!/bin/bash. I've even seen one enterprising individual suggest using #!/bin/bash was wrong and bash functionality would be lost by doing so.
All that said, I use bash in a tightly controlled test environment where every drive in circulation is essentially a clone of a single master drive. I understand the portability argument, though it is not necessarily applicable in my case. Is there any other reason to prefer #!/usr/bin/env bashover the alternatives and, assuming portability was a concern, is there any reason using it could break functionality?
#!/usr/bin/env searches PATH for bash, and bash is not always in /bin, particularly on non-Linux systems. For example, on my OpenBSD system, it's in /usr/local/bin, since it was installed as an optional package.
If you are absolutely sure bash is in /bin and will always be, there's no harm in putting it directly in your shebang—but I'd recommend against it because scripts and programs all have lives beyond what we initially believe they will have.
The standard location of bash is /bin, and I suspect that's true on all systems. However, what if you don't like that version of bash? For example, I want to use bash 4.2, but the bash on my Mac is at 3.2.5.
I could try reinstalling bash in /bin but that may be a bad idea. If I update my OS, it will be overwritten.
However, I could install bash in /usr/local/bin/bash, and setup my PATH to:
PATH="/usr/local/bin:/bin:/usr/bin:$HOME/bin"
Now, if I specify bash, I don't get the old cruddy one at /bin/bash, but the newer, shinier one at /usr/local/bin. Nice!
Except my shell scripts have that !# /bin/bash shebang. Thus, when I run my shell scripts, I get that old and lousy version of bash that doesn't even have associative arrays.
Using /usr/bin/env bash will use the version of bash found in my PATH. If I setup my PATH, so that /usr/local/bin/bash is executed, that's the bash that my scripts will use.
It's rare to see this with bash, but it is a lot more common with Perl and Python:
Certain Unix/Linux releases which focus on stability are sometimes way behind with the release of these two scripting languages. Not long ago, RHEL's Perl was at 5.8.8 -- an eight year old version of Perl! If someone wanted to use more modern features, you had to install your own version.
Programs like Perlbrew and Pythonbrew allow you to install multiple versions of these languages. They depend upon scripts that manipulate your PATH to get the version you want. Hard coding the path means I can't run my script under brew.
It wasn't that long ago (okay, it was long ago) that Perl and Python were not standard packages included in most Unix systems. That meant you didn't know where these two programs were installed. Was it under /bin? /usr/bin? /opt/bin? Who knows? Using #! /usr/bin/env perl meant I didn't have to know.
And Now Why You Shouldn't Use #! /usr/bin/env bash
When the path is hardcoded in the shebang, I have to run with that interpreter. Thus, #! /bin/bash forces me to use the default installed version of bash. Since bash features are very stable (try running a 2.x version of a Python script under Python 3.x) it's very unlikely that my particular BASH script will not work, and since my bash script is probably used by this system and other systems, using a non-standard version of bash may have undesired effects. It is very likely I want to make sure that the stable standard version of bash is used with my shell script. Thus, I probably want to hard code the path in my shebang.
There are a lot of systems that don't have Bash in /bin, FreeBSD and OpenBSD just to name a few. If your script is meant to be portable to many different Unices, you may want to use #!/usr/bin/env bash instead of #!/bin/bash.
Note that this does not hold true for sh; for Bourne-compliant scripts I exclusively use #!/bin/sh, since I think pretty much every Unix in existence has sh in /bin.
For invoking bash it is a little bit of overkill. Unless you have multiple bash binaries like your own in ~/bin but that also means your code depends on $PATH having the right things in it.
It is handy for things like python though. There are wrapper scripts and environments which lead to alternative python binaries being used.
But nothing is lost by using the exact path to the binary as long as you are sure it is the binary you really want.
#!/usr/bin/env bash
is definitely better because it finds the bash executable path from your system environment variable.
Go to your Linux shell and type
env
It will print all your environment variables.
Go to your shell script and type
echo $BASH
It will print your bash path (according to the environment variable list) that you should use to build your correct shebang path in your script.
I would prefer wrapping the main program in a script like below to check all bash available on system. Better to have more control on the version it uses.
#! /usr/bin/env bash
# This script just chooses the appropriate bash
# installed in system and executes testcode.main
readonly DESIRED_VERSION="5"
declare all_bash_installed_on_this_system
declare bash
if [ "${BASH_VERSINFO}" -ne "${DESIRED_VERSION}" ]
then
found=0
all_bash_installed_on_this_system="$(\
awk -F'/' '$NF == "bash"{print}' "/etc/shells"\
)"
for bash in $all_bash_installed_on_this_system
do
versinfo="$( $bash -c 'echo ${BASH_VERSINFO}' )"
[ "${versinfo}" -eq "${DESIRED_VERSION}" ] && { found=1 ; break;}
done
if [ "${found}" -ne 1 ]
then
echo "${DESIRED_VERSION} not available"
exit 1
fi
fi
$bash main_program "$#"
Normally #!path/to/command will trigger bash to prepend the command path to the invoking script when executed. Example,
# file.sh
#!/usr/bin/bash
echo hi
./file.sh will start a new process and the script will get executed like /bin/bash ./file.sh
Now
# file.sh
#!/usr/bin/env bash
echo hi
will get executed as /usr/bin/env bash ./file.sh which quoting from the man page of env describes it as:
env - run a program in a modified environment
So env will look for the command bash in its PATH environment variable and execute in a separate environment where the environment values can be passed to env like NAME=VALUE pair.
You can test this with other scripts using different interpreters like python, etc.
#!/usr/bin/env python
# python commands
Your question is biased because it assumes that #!/usr/bin/env bash is superior to #!/bin/bash. This assumption is not true, and here's why:
env is useful in two cases:
when there are multiple versions of the interpreter that are incompatible.
For example python 2/3, perl 4/5, or php 5/7
when the location depends on the PATH, for instance with a python virtual environment.
But bash doesn't fall under any of these two cases because:
bash is quite stable, especially on modern systems like Linux and BSD which form the vast majority of bash installations.
there's typically only one version of bash installed under /bin.
This has been the case for the past 20+ years, only very old unices (that nobody uses any longer) had a different location.
Consequently going through the PATH variable via /usr/bin/env is not useful for bash.
Add to these three good resons to use #!/bin/bash:
for system scripts (when not using sh) for which the PATH variable may not contain /bin.
For example cron defaults to a very strict PATH of /usr/bin:/bin which is fine, sure, but other context/environments may not include /bin for some peculiar reason.
when the user screwed-up his PATH, which is very common with beginners.
for security when for example you're calling a suid program that invokes a bash script. You don't want the interpreter to be found via the PATH variable which is entirely under the user's control!
Finally, one could argue that there is one legitimate use case of env to spawn bash: when one needs to pass extra environment variables to the interpreter using #!/usr/bin/env -S VAR=value bash.
But this is not a thing with bash because when you're in control of the shebang, you're also in control of the whole script, so just add VAR=value inside the script instead and avoid the aforementioned problems introduced by env with bash scripts.