How to make a bash script execute on file saves? [duplicate] - macos

I want to watch a folder on my Mac and then execute a bash script, passing it the name of whatever file/folder was just moved into or created in the watched directory.

fswatch
fswatch is a small program using the Mac OS X FSEvents API to monitor a directory.
When an event about any change to that directory is received, the specified
shell command is executed by /bin/bash
If you're on GNU/Linux,
inotifywatch (part of the
inotify-tools package on most distributions) provides similar
functionality.
Update: fswatch can now be used across many platforms including BSD, Debian, and Windows.
Syntax / A Simple Example
The new way that can watch multiple paths - for versions 1.x and higher:
fswatch -o ~/path/to/watch | xargs -n1 -I{} ~/script/to/run/when/files/change.sh
Note: The number output by -o will get added to the end of the xargs command if not for the -I{}. If you do choose to use that number, place {} anywhere in your command.
The older way for versions 0.x:
fswatch ~/path/to/watch ~/script/to/run/when/files/change.sh
Installation with Homebrew
As of 9/12/13 it was added back in to homebrew - yay! So, update your formula list (brew update) and then all you need to do is:
brew install fswatch
Installation without Homebrew
Type these commands in Terminal.app
cd /tmp
git clone https://github.com/alandipert/fswatch
cd fswatch/
make
cp fswatch /usr/local/bin/fswatch
If you don't have a c compiler on your system you may need to install Xcode or Xcode command line tools - both free. However, if that is the case, you should probably just check out homebrew.
Additional Options for fswatch version 1.x
Usage:
fswatch [OPTION] ... path ...
Options:
-0, --print0 Use the ASCII NUL character (0) as line separator.
-1, --one-event Exit fsw after the first set of events is received.
-e, --exclude=REGEX Exclude paths matching REGEX.
-E, --extended Use exended regular expressions.
-f, --format-time Print the event time using the specified format.
-h, --help Show this message.
-i, --insensitive Use case insensitive regular expressions.
-k, --kqueue Use the kqueue monitor.
-l, --latency=DOUBLE Set the latency.
-L, --follow-links Follow symbolic links.
-n, --numeric Print a numeric event mask.
-o, --one-per-batch Print a single message with the number of change events.
in the current batch.
-p, --poll Use the poll monitor.
-r, --recursive Recurse subdirectories.
-t, --timestamp Print the event timestamp.
-u, --utc-time Print the event time as UTC time.
-v, --verbose Print verbose output.
-x, --event-flags Print the event flags.
See the man page for more information.

You can use launchd for that purpose. Launchd can be configured to automatically launch a program when a file path is modified.
For example the following launchd config plist will launch the program /usr/bin/logger when the desktop folder of my user account is modified:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>logger</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/logger</string>
<string>path modified</string>
</array>
<key>WatchPaths</key>
<array>
<string>/Users/sakra/Desktop/</string>
</array>
</dict>
</plist>
To activate the config plist save it to the LaunchAgents folder in your Library folder as "logger.plist".
From the shell you can then use the command launchctl to activate the logger.plist by running:
$ launchctl load ~/Library/LaunchAgents/logger.plist
The desktop folder is now being monitored. Every time it is changed you should see an output in the system.log (use Console.app).
To deactivate the logger.plist, run:
$ launchctl unload ~/Library/LaunchAgents/logger.plist
The configuration file above uses the WatchPaths option. Alternatively you can also use the
QueueDirectories option. See the launchd man page for more information.

Facebook's watchman, available via Homebrew, also looks nice. It supports also filtering:
These two lines establish a watch on a source directory and then set
up a trigger named "buildme" that will run a tool named "minify-css"
whenever a CSS file is changed. The tool will be passed a list of the
changed filenames.
$ watchman watch ~/src
$ watchman -- trigger ~/src buildme '*.css' -- minify-css
Notice that the path must be absolute.

You might want to take a look at (and maybe expand) my little tool kqwait. Currently it just sits around and waits for a write event on a single file, but the kqueue architecture allows for hierarchical event stacking...

watchdog is a cross-platform python API for watching files / directories, and it has builtin "tricks" tool that allows you to trigger actions (including shell commands) when events occur (including new added file, removed file and changed file).

This is just to mention entr as an alternative on OSX to run arbitrary commands when files change. I find it simple and useful.
brew install entr on macos
apt install entr on Debian/Ubuntu

Here's a one-liner using sschober's tool.
$ while true; do kqwait ./file-to-watch.js; script-to-execute.sh; done

Apple OSX Folder Actions allow you to automate tasks based on actions taken on a folder.

Edit: fsw has been merged into fswatch. In this answer, any reference to fsw should now read fswatch.
I wrote an fswatch replacement in C++ called fsw which features several improvements:
It's a GNU Build System project which builds on any supported platform (OS X v. >= 10.6) with
./configure && make && sudo make install
Multiple paths can be passed as different arguments:
fsw file-0 ... file-n
It dumps a detailed record with all the event information such as:
Sat Feb 15 00:53:45 2014 - /path/to/file:inodeMetaMod modified isFile
Its output is easy to parse so that fsw output can be piped to another process.
Latency can be customised with -l, --latency.
Numeric event flags can be written instead of textual ones with -n, --numeric.
The time format can be customised using strftime format strings with -t, --time-format.
The time can be the local time of the machine (by default) or UTC time with -u, --utc-time.
Getting fsw:
fsw is hosted on GitHub and can be obtained cloning its repository:
git clone https://github.com/emcrisostomo/fsw
Installing fsw:
fsw can be installed using the following commands:
./configure && make && sudo make install
Further information:
I also wrote an introductory blog post where you can find a couple of examples about how fsw works.

My fork of fswatch provides the functionality of inotifywait -m with slightly less (no wait, more! I have a lot more troubles on Linux with inotifywait...) parse-friendly output.
It is an improvement upon the original fswatch because it sends out the actual path of the changed file over STDOUT rather than requiring you to provide a program that it forks.
It's been rock solid as the foundation of a series of scary bash scripts I use to automate stuff.
(this is off-topic) inotifywait on Linux, on the other hand, requires a lot of kludges on top of it and I still haven't figured out a good way to manage it, though I think something based on node.js might be the ticket.

I have a GIST for this and the usage is pretty simple
watchfiles <cmd> <paths...>
To illustrate, the following command will echo Hello World every time that file1 OR file2 change; and the default interval check is 1 second
watchfiles 'echo Hello World' /path/to/file1 /path/to/file2
If I want to check every 5 seconds I can use the -t flag
watchfiles -t 'echo Hello World' /path/to/file1 /path/to/file2
-v enables the verbose mode which shows debug information
-q makes watchfiles execute quietly (# will be shown so the user can see the program is executing)
-qq makes watchfiles execute completely quietly
-h shows the help and usage
https://gist.github.com/thiagoh/5d8f53bfb64985b94e5bc8b3844dba55

I ended up doing this for macOS. I'm sure this is terrible in many ways:
#!/bin/sh
# watchAndRun
if [ $# -ne 2 ]; then
echo "Use like this:"
echo " $0 filename-to-watch command-to-run"
exit 1
fi
if which fswatch >/dev/null; then
echo "Watching $1 and will run $2"
while true; do fswatch --one-event $1 >/dev/null && $2; done
else
echo "You might need to run: brew install fswatch"
fi

If you want to use NodeJS, you can use a package called chokidar (or chokidar-cli actually) for the watching and then use rsync (included with Mac):
Rsync command:
$ rsync -avz --exclude 'some-file' --exclude 'some-dir' './' '/my/destination'
Chokidar cli (installed globally via npm):
chokidar \"**/*\" -c \"your-rsync-command-above\"

sudo fs_usage -f filesys | grep "interesting thing" ?

I can wholeheartedly recommend using watchexec. Built in Rust and It Just Works™ no matter which platform you're on! Straightforward CLI options as well.

Here's a simple single line alternative for users who don't have the watch command who want to execute a command every 3 seconds:
while :; do your-command; sleep 3; done
It's an infinite loop that is basically the same as doing the following:
watch -n3 your-command

Related

Rename/translate multiple files on Mac

I have a lot of files with names in one language and I need to rename them all to another one.
Is there any script for this? (preferably on Mac)
Figured it out. We can do it with translate-shell CLI utility.
Install it with brew install translate-shell
Then run next script in your folder:
for i in *.txt
do
sleep 5
mv -i "$i" "$(echo ${i%.txt} | trans -b nl:en).txt";
done
translate-shell makes a call to Google Translate server to do translation
sleep 5 is needed to avoid being blocked by Google's server for too many requests in a second
trans is actual translate command
-b stands for "brief", as we don't need verbose output
nl:en are the source and destination languages

How to save/log the output of the iex shell to get persistent command history?

I just started working with Elixir and have been using the iex shell quite a bit. Is it possible for me to save / log a "session" to a file?
Thank you.
In Erlang/OTP-20 and higher
Since Erlang/OTP-20rc2, Shell history is supported out of the box (although initially disabled by default) through a port of this library to the Erlang/OTP code base. Enable the shell in these versions by setting the shell_history kernel environment variable to enabled with export ERL_AFLAGS="-kernel shell_history enabled" added to your environment variables (see Configuration Options to see more options).
-- https://github.com/ferd/erlang-history
Trouble shooting
The history seems to be not updated (not written to the file)?
It seems that the process that is writing the history to the file does it asynchronously and it needs some time to do it before the IEx shell is closed. You need to wait a bit before you exit the shell (e.g. press <ctrl+\>).
Pre Erlang/OTP-20:
I have found 2 ways to do it.
1. Erlang History
erlang-history (eh) is a tiny pair of files that can be used to patch an Erlang-OTP system to add support for history in the Erlang
shell.
The history supported is the one available through up/down arrows on
the keyboard.
Installation in Ubuntu Linux:
sudo su
cd /usr/local/src
git clone https://github.com/ferd/erlang-history.git
cd erlang-history
make install
Now every now started Erlang based REPL (and that is IEx) should use erlang-history.
2. rlwrap
As an alternative you can try a more generic REPL enhancer/fixer rlwrap which is a "readline wrapper":
...a small utility that uses the GNU readline library to allow the
editing of keyboard input for any command.
rlwrap -a -A iex -S mix
(In case you are on Ubuntu Linux use: sudo apt-get install rlwrap)
It let's you add a lot more features to the REPL like e.g. the pipeto filter rlwrap -a -z pipeto iex that lets you pipe things to shell commands - very useful to read documentation i.e.: iex> h Stream | less (more)
Know downsides:
It breaks code completion (i.e. tab completion) in IEx
Why is this very useful feature - command history - not already included in Elixir/Erlang?
https://github.com/elixir-lang/elixir/issues/2945#issuecomment-66594851
https://github.com/elixir-lang/elixir/issues/1516#issuecomment-21772851
When using asdf see this.
Not currently. You could probably write a small iex plugin to do this for you though. For example, I have the following file in ~/.iex.exs:
# .iex.exs
defmodule IExHelpers do
def reload! do
Mix.Task.reenable "compile.elixir"
Mix.Task.run "compile.elixir"
end
end
iex = IExHelpers
# $ iex -S mix
# iex(2)> iex.reload!
# :noop
This recompiles the current project and reloads it while still inside a shell spawned with iex -S mix. You could probably write something to save the current shell's history to a file, and read it back in on startup, but I'm not sure where you would start with that.

Can't run TextMate from shell with symbolic link in bin (new alpha version) (OSX)

I have the latest textmate installed in Applications, the executable is here -
/Applications/TextMate.app/Contents/MacOS/TextMate
I can launch TextMate OK as normal from the icon in Applications
To be able to run from within shell (I use ZSH), I added a symbolic link in /usr/local/bin like so -
sudo ln -s /Applications/TextMate.app/Contents/MacOS/TextMate /usr/local/bin/mate
But I try to run mate from shell, I'm getting the following -
mate[22695:8403] No Info.plist file in application bundle or no NSPrincipalClass in the Info.plist file, exiting
Perhaps I've installed textmate wrong?
You need to symlink the 'mate' CLI, not the TextMate file.
sudo ln -s /Applications/TextMate.app/Contents/Resources/mate /usr/local/bin/mate
I agree with #shellter but…
I don't know if it works in zsh but TextMate has its own CLI wrapper (mate) that you can install from Preferences -> Terminal. It has worked well for years and is very convenient:
Usage: mate [-awl<number>rdnhv] [file ...]
Options:
-a, --async Do not wait for file to be closed by TextMate.
-w, --wait Wait for file to be closed by TextMate.
-l, --line <number> Place caret on line <number> after loading file.
-r, --recent Add file to Open Recent menu.
-d, --change-dir Change TextMates working directory to that of the file.
-n, --no-reactivation After edit with -w, do not re-activate the calling app.
-h, --help Show this information.
-v, --version Print version information.
If multiple files are given, a project is created consisting of these
files, -a is then default and -w will be ignored (e.g. "mate *.tex").
By default mate will not wait for the file to be closed
except when used as filter:
ls *.tex|mate|sh -w implied
mate -|cat -n -w implied (read from stdin)
An exception is made if the command is started as something which ends
with "_wait". So to have a command with --wait as default, you can
create a symbolic link like this:
ln -s mate mate_wait
Another cheap option would be to add an alias to whatever ~/.*rc file zsh executes at startup similar to this one for bash:
alias mate='open -a textmate'

Program to re-run, eg, `make` when files are modified?

Is there a program that will automatically re-run, eg, make, when files are modified?
For example, when I'm writing sphinx documentation, it would be nice if make html was run automatically each time I edit any relevant files.
For simple things, rerun could be a good fit: http://pypi.python.org/pypi/rerun
"Command-line executable Python script to re-run the given command every time files are modified in the current directory or its subdirectories."
It requires a Python interpreter, but does not care if your command or files are written in Python.
Usage
rerun [--help|-h] [--verbose|-v] [--ignore|-i=<file>] [--version] <command>
Where:
<command> Command to execute
--help|-h Show this help message and exit.
--ignore|-i=<file> File or directory to ignore. Any directories of the
given name (and their subdirs) are excluded from the
search for changed files. Any modification to files of
the given name are ignored. The given value is
compared to basenames, so for example, "--ignore=def"
will skip the contents of directory "./abc/def/" and
will ignore file "./ghi/def". Can be specified multiple
times.
--verbose|-v Display the names of changed files before the command
output.
--version Show version number and exit.
In Linux, you can use this command line:
while true; do inotifywait -e close_write *.py; make; done
Uses standard system command inotifywait, if not available, install with something like:
sudo apt install inotify-tools
Well, since make will not do anything if nothing has changed, how about
while true; do sleep 60; make html; done
or the equivalent in your shell of choice? I don't think the usual file system layers are event-driven in such a way that they will you notify you of file changes without doing some similar themselves, but it's possibly DBUS can do that sort of stuff.
You could use inotifywait in a loop: https://github.com/rvoicilas/inotify-tools/wiki/#info
As per answer https://stackoverflow.com/a/22907316/71522 watchman seems to work very well:
$ watchman-make -p '*.c' 'Makefile' -t all
Will re-run make all each time any *.c or Makefile file changes.
It can be installed with:
$ brew install watchman
You could use incron: http://inotify.aiken.cz/?section=incron&page=about&lang=en
This question was also asked here: https://superuser.com/questions/181517/how-to-execute-a-command-whenever-a-file-changes/
You could try reflex
# Rerun make whenever a .c file changes
reflex -r '\.c$' make

Is there a command like "watch" or "inotifywait" on the Mac?

I want to watch a folder on my Mac and then execute a bash script, passing it the name of whatever file/folder was just moved into or created in the watched directory.
fswatch
fswatch is a small program using the Mac OS X FSEvents API to monitor a directory.
When an event about any change to that directory is received, the specified
shell command is executed by /bin/bash
If you're on GNU/Linux,
inotifywatch (part of the
inotify-tools package on most distributions) provides similar
functionality.
Update: fswatch can now be used across many platforms including BSD, Debian, and Windows.
Syntax / A Simple Example
The new way that can watch multiple paths - for versions 1.x and higher:
fswatch -o ~/path/to/watch | xargs -n1 -I{} ~/script/to/run/when/files/change.sh
Note: The number output by -o will get added to the end of the xargs command if not for the -I{}. If you do choose to use that number, place {} anywhere in your command.
The older way for versions 0.x:
fswatch ~/path/to/watch ~/script/to/run/when/files/change.sh
Installation with Homebrew
As of 9/12/13 it was added back in to homebrew - yay! So, update your formula list (brew update) and then all you need to do is:
brew install fswatch
Installation without Homebrew
Type these commands in Terminal.app
cd /tmp
git clone https://github.com/alandipert/fswatch
cd fswatch/
make
cp fswatch /usr/local/bin/fswatch
If you don't have a c compiler on your system you may need to install Xcode or Xcode command line tools - both free. However, if that is the case, you should probably just check out homebrew.
Additional Options for fswatch version 1.x
Usage:
fswatch [OPTION] ... path ...
Options:
-0, --print0 Use the ASCII NUL character (0) as line separator.
-1, --one-event Exit fsw after the first set of events is received.
-e, --exclude=REGEX Exclude paths matching REGEX.
-E, --extended Use exended regular expressions.
-f, --format-time Print the event time using the specified format.
-h, --help Show this message.
-i, --insensitive Use case insensitive regular expressions.
-k, --kqueue Use the kqueue monitor.
-l, --latency=DOUBLE Set the latency.
-L, --follow-links Follow symbolic links.
-n, --numeric Print a numeric event mask.
-o, --one-per-batch Print a single message with the number of change events.
in the current batch.
-p, --poll Use the poll monitor.
-r, --recursive Recurse subdirectories.
-t, --timestamp Print the event timestamp.
-u, --utc-time Print the event time as UTC time.
-v, --verbose Print verbose output.
-x, --event-flags Print the event flags.
See the man page for more information.
You can use launchd for that purpose. Launchd can be configured to automatically launch a program when a file path is modified.
For example the following launchd config plist will launch the program /usr/bin/logger when the desktop folder of my user account is modified:
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
<key>Label</key>
<string>logger</string>
<key>ProgramArguments</key>
<array>
<string>/usr/bin/logger</string>
<string>path modified</string>
</array>
<key>WatchPaths</key>
<array>
<string>/Users/sakra/Desktop/</string>
</array>
</dict>
</plist>
To activate the config plist save it to the LaunchAgents folder in your Library folder as "logger.plist".
From the shell you can then use the command launchctl to activate the logger.plist by running:
$ launchctl load ~/Library/LaunchAgents/logger.plist
The desktop folder is now being monitored. Every time it is changed you should see an output in the system.log (use Console.app).
To deactivate the logger.plist, run:
$ launchctl unload ~/Library/LaunchAgents/logger.plist
The configuration file above uses the WatchPaths option. Alternatively you can also use the
QueueDirectories option. See the launchd man page for more information.
Facebook's watchman, available via Homebrew, also looks nice. It supports also filtering:
These two lines establish a watch on a source directory and then set
up a trigger named "buildme" that will run a tool named "minify-css"
whenever a CSS file is changed. The tool will be passed a list of the
changed filenames.
$ watchman watch ~/src
$ watchman -- trigger ~/src buildme '*.css' -- minify-css
Notice that the path must be absolute.
You might want to take a look at (and maybe expand) my little tool kqwait. Currently it just sits around and waits for a write event on a single file, but the kqueue architecture allows for hierarchical event stacking...
watchdog is a cross-platform python API for watching files / directories, and it has builtin "tricks" tool that allows you to trigger actions (including shell commands) when events occur (including new added file, removed file and changed file).
This is just to mention entr as an alternative on OSX to run arbitrary commands when files change. I find it simple and useful.
brew install entr on macos
apt install entr on Debian/Ubuntu
Here's a one-liner using sschober's tool.
$ while true; do kqwait ./file-to-watch.js; script-to-execute.sh; done
Apple OSX Folder Actions allow you to automate tasks based on actions taken on a folder.
Edit: fsw has been merged into fswatch. In this answer, any reference to fsw should now read fswatch.
I wrote an fswatch replacement in C++ called fsw which features several improvements:
It's a GNU Build System project which builds on any supported platform (OS X v. >= 10.6) with
./configure && make && sudo make install
Multiple paths can be passed as different arguments:
fsw file-0 ... file-n
It dumps a detailed record with all the event information such as:
Sat Feb 15 00:53:45 2014 - /path/to/file:inodeMetaMod modified isFile
Its output is easy to parse so that fsw output can be piped to another process.
Latency can be customised with -l, --latency.
Numeric event flags can be written instead of textual ones with -n, --numeric.
The time format can be customised using strftime format strings with -t, --time-format.
The time can be the local time of the machine (by default) or UTC time with -u, --utc-time.
Getting fsw:
fsw is hosted on GitHub and can be obtained cloning its repository:
git clone https://github.com/emcrisostomo/fsw
Installing fsw:
fsw can be installed using the following commands:
./configure && make && sudo make install
Further information:
I also wrote an introductory blog post where you can find a couple of examples about how fsw works.
My fork of fswatch provides the functionality of inotifywait -m with slightly less (no wait, more! I have a lot more troubles on Linux with inotifywait...) parse-friendly output.
It is an improvement upon the original fswatch because it sends out the actual path of the changed file over STDOUT rather than requiring you to provide a program that it forks.
It's been rock solid as the foundation of a series of scary bash scripts I use to automate stuff.
(this is off-topic) inotifywait on Linux, on the other hand, requires a lot of kludges on top of it and I still haven't figured out a good way to manage it, though I think something based on node.js might be the ticket.
I have a GIST for this and the usage is pretty simple
watchfiles <cmd> <paths...>
To illustrate, the following command will echo Hello World every time that file1 OR file2 change; and the default interval check is 1 second
watchfiles 'echo Hello World' /path/to/file1 /path/to/file2
If I want to check every 5 seconds I can use the -t flag
watchfiles -t 'echo Hello World' /path/to/file1 /path/to/file2
-v enables the verbose mode which shows debug information
-q makes watchfiles execute quietly (# will be shown so the user can see the program is executing)
-qq makes watchfiles execute completely quietly
-h shows the help and usage
https://gist.github.com/thiagoh/5d8f53bfb64985b94e5bc8b3844dba55
I ended up doing this for macOS. I'm sure this is terrible in many ways:
#!/bin/sh
# watchAndRun
if [ $# -ne 2 ]; then
echo "Use like this:"
echo " $0 filename-to-watch command-to-run"
exit 1
fi
if which fswatch >/dev/null; then
echo "Watching $1 and will run $2"
while true; do fswatch --one-event $1 >/dev/null && $2; done
else
echo "You might need to run: brew install fswatch"
fi
If you want to use NodeJS, you can use a package called chokidar (or chokidar-cli actually) for the watching and then use rsync (included with Mac):
Rsync command:
$ rsync -avz --exclude 'some-file' --exclude 'some-dir' './' '/my/destination'
Chokidar cli (installed globally via npm):
chokidar \"**/*\" -c \"your-rsync-command-above\"
sudo fs_usage -f filesys | grep "interesting thing" ?
I can wholeheartedly recommend using watchexec. Built in Rust and It Just Works™ no matter which platform you're on! Straightforward CLI options as well.
Here's a simple single line alternative for users who don't have the watch command who want to execute a command every 3 seconds:
while :; do your-command; sleep 3; done
It's an infinite loop that is basically the same as doing the following:
watch -n3 your-command

Resources