I have two iTerm windows running zsh: one I use to documents in vim; the other I use to execute shell commands. I would like to synchronize the current working directories of the two sessions. I thought I could do this by outputting to a file ~/.cwd the new directory every time I change directories
alias cd="cd; pwd > ~/.cwd"
and creating a shell script ~/.dirsync that monitors the contents of ~/.cwd every second and changes directory if the other shell has updated it.
#!/bin/sh
echo $(pwd) > ~/.cwd
alias cd="cd; echo $(pwd) > ~/.cwd"
while true
do
if [[ $(pwd) != $(cat ~/.cwd) ]]
then
cd $(cat ~/.cwd)
fi
sleep 1
done
I would then append the following line of code to the end of my ~/.zshrc.
~/.dirsync &
However, it did not work. I then found out that shell scripts always execute in its own subshell. Does anyone know of a way to make this work?
Caveat emptor: I'm doing this on Ubuntu 10.04 with gnome-terminal, but it should work on any *NIX platform running zsh.
I've also changed things slightly. Instead of mixing "pwd" and "cwd", I've stuck with "pwd" everywhere.
Recording the Present Working Directory
If you want to run a function every time you cd, the preferred way is to use the chpwd function or the more extensible chpwd_functions array. I prefer chpwd_functions since you can dynamically append and remove functions from it.
# Records $PWD to file
function +record_pwd {
echo "$(pwd)" > ~/.pwd
}
# Removes the PWD record file
function +clean_up_pwd_record {
rm -f ~/.pwd
}
# Adds +record_pwd to the list of functions executed when "cd" is called
# and records the present directory
function start_recording_pwd {
if [[ -z $chpwd_functions[(r)+record_pwd] ]]; then
chpwd_functions=(${chpwd_functions[#]} "+record_pwd")
fi
+record_pwd
}
# Removes +record_pwd from the list of functions executed when "cd" is called
# and cleans up the record file
function stop_recording_pwd {
if [[ -n $chpwd_functions[(r)+record_pwd] ]]; then
chpwd_functions=("${(#)chpwd_functions:#+record_pwd}")
+clean_up_pwd_record
fi
}
Adding a + to the +record_pwd and +clean_up_pwd_record function names is a hack-ish way to hide it from normal use (similarly, the VCS_info hooks do this by prefixing everything with +vi).
With the above, you would simply call start_recording_pwd to start recording the present working directory every time you change directories. Likewise, you can call stop_recording_pwd to disable that behavior. stop_recording_pwd also removes the ~/.pwd file (just to keep things clean).
By doing things this way, synchronization be easily be made opt-in (since you may not want this for every single zsh session you run).
First Attempt: Using the preexec Hook
Similar to the suggestion of #Celada, the preexec hook gets run before executing a command. This seemed like an easy way to get the functionality you want:
autoload -Uz add-zsh-hook
function my_preexec_hook {
if [[-r ~/.pwd ]] && [[ $(pwd) != $(cat ~/.pwd) ]]; then
cd "$(cat ~/.pwd)"
fi
}
add-zsh-hook preexec my_preexec_hook
This works... sort of. Since the preexec hook runs before each command, it will automatically change directories before running your next command. However, up until then, the prompt stays in the last working directory, so it tab completes for the last directory, etc. (By the way, a blank line doesn't count as a command.) So, it sort of works, but it's not intuitive.
Second Attempt: Using signals and traps
In order to get a terminal to automatically cd and re-print the prompt, things got a lot more complicated.
After some searching, I found out that $$ (the shell's process ID) does not change in subshells. Thus, a subshell (or background job) can easily send signals to its parent. Combine this with the fact that zsh allows you to trap signals, and you have a means of polling ~/.pwd periodically:
# Used to make sure USR1 signals are not taken as synchronization signals
# unless the terminal has been told to do so
local _FOLLOWING_PWD
# Traps all USR1 signals
TRAPUSR1() {
# If following the .pwd file and we need to change
if (($+_FOLLOWING_PWD)) && [[ -r ~/.pwd ]] && [[ "$(pwd)" != "$(cat ~/.pwd)" ]]; then
# Change directories and redisplay the prompt
# (Still don't fully understand this magic combination of commands)
[[ -o zle ]] && zle -R && cd "$(cat ~/.pwd)" && precmd && zle reset-prompt 2>/dev/null
fi
}
# Sends the shell a USR1 signal every second
function +check_recorded_pwd_loop {
while true; do
kill -s USR1 "$$" 2>/dev/null
sleep 1
done
}
# PID of the disowned +check_recorded_pwd_loop job
local _POLLING_LOOP_PID
function start_following_recorded_pwd {
_FOLLOWING_PWD=1
[[ -n "$_POLLING_LOOP_PID" ]] && return
# Launch signalling loop as a disowned process
+check_recorded_pwd_loop &!
# Record the signalling loop's PID
_POLLING_LOOP_PID="$!"
}
function stop_following_recorded_pwd {
unset _FOLLOWING_PWD
[[ -z "$_POLLING_LOOP_PID" ]] && return
# Kill the background loop
kill "$_POLLING_LOOP_PID" 2>/dev/null
unset _POLLING_LOOP_PID
}
If you call start_following_recorded_pwd, this launches +check_recorded_pwd_loop as a disowned background process. This way, you won't get an annoying "suspended jobs" warning when you go to close your shell. The PID of the loop is recorded (via $!) so it can be stopped later.
The loop just sends the parent shell a USR1 signal every second. This signal gets trapped by TRAPUSR1(), which will cd and reprint the prompt if necessary. I don't understand having to call both zle -R and zle reset-prompt, but that was the magic combination that worked for me.
There is also the _FOLLOWING_PWD flag. Since every terminal will have the TRAPUSR1 function defined, this prevents them from handling that signal (and changing directories) unless you actually specified that behavior.
As with recording the present working directory, you can call stop_following_posted_pwd to stop the whole auto-cd thing.
Putting both halves together:
function begin_synchronize {
start_recording_pwd
start_following_recorded_pwd
}
function end_synchronize {
stop_recording_pwd
stop_following_recorded_pwd
}
Finally, you will probably want to do this:
trap 'end_synchronize' EXIT
This will automatically clean up everything just before your terminal exits, thus preventing you from accidentally leaving orphaned signalling loops around.
Related
There is a command line tool at my company that many people will use, but we need to set some restrictions on the tool's usage based on the environment that the user is in.
For example, if the user is logged into our production environment and they run dbt run <some more flags>, I need to flash a warning and prompt them to continue or exit.
I use zsh myself and found the preexec() hook, but unfortunately, because of the way this is implemented, calling exit kills the whole shell and closes the terminal. This seems to be a non-starter for this option.
I'm looking for a pure bash alternative (many of my coworkers may not use zsh) that will allow me to evaluate a command before it's executed, run the command through some logic, and allow me to either kill the command or run it.
Here is the zsh function i wrote for reference:
preexec() {
command=$1
first_seven_chars=${command:0:7}
if [[ "$first_seven_chars" != "dbt run" ]]; then return; fi
if [[ "$AWS_ENVIRONMENT" != "production" ]]; then return; fi
if [[ "$AWS_ENVIRONMENT" = "production" ]]; then
read -k1 -s "key?WARNING: You are in Production. Do you want to continue? Enter y to continue or any key to exit"
if [[ "$key" = "y" ]]; then
echo "Continuing in Production..."
BRANCH=$(git rev-parse --abbrev-ref HEAD)
if [[ "$BRANCH" != "master" ]]; then
echo 'You are currently not on the master branch. To prevent models and data deviating, please merge your changes into master and rerun';
exit;
fi
else
echo "\n"
echo "\nStopping command from running in production..."
exit;
fi
exit;
fi
}
Note: I realize https://github.com/rcaloras/bash-preexec exists, but since it mirrors the zsh functionality it might not be the best to use.
What you need to do is create a wrapper around the original command, not mess with preexec and other shell-specific hooks. Of course, you can write this wrapper itself as a zsh script. So you'll create, e.g., dbt_safe.zsh which would internally call dbt, and your colleagues would just use dbt_safe.zsh.
I want to run a Bash script every minute (through a CRON entry) to launch a series of Python scripts in a granular (time-wise) fashion.
So far, this is the script I've made:
# set the current date
DATE=`date +%Y-%m-%d`
# set the current system time (HH:MM)
SYSTIME=`date +%H-%M`
# parse all .py script files in the 'daily' folder
for f in daily/*.py; do
if [[ -f $f ]]; then
# set the script name
SCRIPT=$(basename $f)
# get the script time
SCRTIME=`echo $SCRIPT | cut -c1-5`
# execute the script only if its intended execution time and the system time match
if [[ $SCRTIME == $SYSTIME ]]; then
# ensure the directory exists
install -v -m755 -d done/$DATE/failed
# execute the script
python3 evaluator.py $f > done/$DATE/daily-$SCRIPT.log &
# evaluate the result
if [ $? -eq 0 ]; then
# move the script to the 'done' folder
cp $f done/$DATE/daily-$SCRIPT
else
# log the failure
mv done/$DATE/daily-$SCRIPT.log done/$DATE/failed/
# move the script to the 'retry' folder for retrial
cp $f retry/daily-$SCRIPT
fi
fi
fi
done
Let's say we have the following files in a folder called daily/ (for daily execution):
daily/08-00-script-1.py
daily/08-00-script-2.py
daily/08-05-script-3.py
daily/09-20-script-4.py
The idea for granular execution is that a CRON task runs this script every minute. I fetch the system time and extract the execution time for each script and when the time matches between the system and the script file, it gets handed over to Python. So far, so good.
I know this script is not right in the sense that it gets a subshell for each script that's going to be executed but the following code is wrong as I know Bash automatically returns 0 on subshell invocation (if I did read correctly while searching on Google) and what I need is for each subshell to execute the code below so, if it fails, it gets sent to another folder (retry/) which is controlled by another Bash script running checks every 30 minutes for retrial (it's the same script as this one minus the checking part).
So, basically, I need to run this:
# evaluate the result
if [ $? -eq 0 ]; then
# move the script to the 'done' folder
cp $f done/$DATE/daily-$SCRIPT
else
# log the failure
mv done/$DATE/daily-$SCRIPT.log done/$DATE/failed/
# move the script to the 'retry' folder for retrial
cp $f retry/daily-$SCRIPT
fi
For every subshell-ed execution. How can I do this the right way?
Bash may return 0 for every sub-shell invocation, but if you wait for the result, then you will get the result (and I see no ampersand). If the python3 commands is relaying the exit code of the script, then your code will work. If your code does not catch an error, then it is the fault of python3 and you need to create error communication. Redirecting the output of stderr might be helpful, but first verify that your code does not work.
For example, I can do this with a subshell:
VAL=$( do_something )
but how do I achieve the same thing with curly braces so the command is NOT executing in a subshell? I.e. this does not work:
VAL={ do_something; }
TIA.
I'm not sure I understand the reasoning for what you're trying to accomplish, but if you can elaborate a bit more I might be able to help you.
I do recommend reading this fantastic write up about what's actually going on though, and why I don't think you want to invoke a process without a subshell.
However, to try and answer what you've asked:
You can't really run a command inside ${}, except in the fallback clause for when a value is not set (in POSIX sh or bash; might be feasible in zsh, which allows all manner of oddball syntax).
However, you can call cd like this if you really wanted this:
cdr() {
if (( $# )); then
command cd "$#"
else
local home
home=$(git rev-parse --show-toplevel 2>/dev/null) || home=$HOME
command cd "$home"
fi
}
Note
Using a function lets us test our argument list, use branching logic, have local variables, &c.
command cd is used to call through to the real cd implementation rather than recursing.
set -e is kinda stiff. Try something like
trap 'err=$?;
echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
exit $err;' ERR
This is a lot more informative when reading through your logs, and you can put a similar command inside the subshell. Yes, it means adding it inside the subshell... but I often do this sort of thing in function definitions that get called in subshells. Works well.
In use:
$ trap 'echo BOOM' ERR # parent shell trap for demo
$ false # trigger manually for demo
BOOM
$ x="$( trap 'err=$?;
> echo >&2 "ERROR $err in $0 at line $LINENO, Aborting";
> exit $err;' ERR
> date
> pwd
> false
> echo "I shan't"
> )"
ERROR 1 in bash at line 7, Aborting
BOOM
$ echo "$x"
Thu, Jan 10, 2019 8:35:57 AM
/c/Users/P2759474/repos/Old/deploy_microservices
$
If the outer shell had the same or a similar trap, it would have aborted too, with another message. (It's usually helpful to make the messages different.)
If you just don't like that, then as a clumsy workaround you can drop the data to a tempfile. Here's a script that will do it.
set -ex
{ pwd
date
false
echo "will this happen?"
} > foo
x=$(<foo)
echo "$x"
Put that in a script, it successfully bails.
$: ./sete
+ pwd
+ date
+ false
$: echo $?
1
I'd still use the trap, but the logic works.
I'd also use mktemp, and a trap to delete the temp on exit, etc.... but you get the idea.
I am using inotifywait to run a command when a filesystem event happens. I would like this to wait for 5 seconds to see if another filesystem event happens and if another one does, I would like to reset the timer back to five seconds and wait some more. Make sense?
My problem is I'm attacking this in Bash and I don't know how I would do this. In JavaScript, I'd use setTimeout with some code like this:
function doSomething() { ... }
var timer;
function setTimer() {
window.clearTimeout(timer)
timer = window.setTimeout(doSomething, 5000);
}
// and then I'd just plug setTimer into the inotifywait loop.
But are there addressable, clearable background timers in Bash?
One idea I've had rattling around is forking out a subshell that sleeps and then runs my desired end command, and then stuffing that in the background. If it's run again, it'll pick up the previous PID and try to nuke it.
As a safety feature, after the sleep has finished, the subshell clears $PID to avoid the command being killed mid-execution
PID=0
while inotifywait -r test/; do
[[ $PID -gt 0 ]] && kill -9 $PID
{ sleep 5; PID=0; command; } & PID=$!
done
It's a bit messy but I've tested it and it works. If I create new files in ./test/ it sees that and if $PID isn't zero, it'll kill the previous sleeping command and reset the timer.
I provide this answer to illustrate a similar but more complex use case. Note that the code provided by #Oli is included in my answer.
I want to post process a file when it has changed. Specifically I want to invoke dart-sass on a scss file to produce a css file and its map file. Then the css file is compressed.
My problem is that editing/saving the scss source file could be done directly through vim (which uses a backup copy when writing the file) or through SFTP (specifically using macOS Transmit). That means the change could be seen with inotifywait as a pair CREATE followed by CLOSE_WRITE,CLOSE or as a single CREATE (due to the RENAME cmd through SFTP I think). So I have to launch the processing if I see a CLOSE_WRITE,CLOSE or a CREATE which is not followed by something.
Remarks:
It has to handle multiple concurrent edit/save.
The temporary files used by Transmit of the form <filename>_safe_save_<digits>.scss must not be taken into account.
The version of inotify-tools is 3.20.2.2 and has been compiled from the source (no package manager) to get a recent version with the include option.
#!/usr/bin/bash
declare -A pids
# $1: full path to source file (src_file_full)
# $2: full path to target file (dst_file_full)
function launch_dart() {
echo "dart"
/opt/dart-sass/sass "$1" "$2" && /usr/bin/gzip -9 -f -k "$2"
}
inotifywait -e close_write,create --include "\.scss$" -mr assets/css |
grep -v -P '(?:\w+)_safe_save_(?:\d+)\.scss$' --line-buffered |
while read dir action file; do
src_file_full="$dir$file"
dst_dir="${dir%assets/css/}"
dst_file="${file%.scss}.css"
dst_file_full="priv/static/css/${dst_dir%/}${dst_file}"
echo "'$action' on file '$file' in directory '$dir' ('$src_file_full')"
echo "dst_dir='$dst_dir', dst_file='$dst_file', dst_file_full='$dst_file_full'"
# if [ "$action" == "DELETE" ]; then
# rm -f "$dst_file_full" "${dst_file_full}.gz" "${dst_file_full}.map"
if [ "$action" == "CREATE" ]; then
echo "create. file size: " $(stat -c%s "$src_file_full")
{ sleep 1; pids[$src_file_full]=0; launch_dart "$src_file_full" "$dst_file_full"; } & pids[$src_file_full]=$!
elif [ "$action" == "CLOSE_WRITE,CLOSE" ]; then
[[ ${pids[$src_file_full]} -gt 0 ]] && kill -9 ${pids[$src_file_full]}
launch_dart "$src_file_full" "$dst_file_full"
fi
done
I'm working on a script that has gotten so complex I want to include an easy option to update it to the most recent version. This is my approach:
set -o errexit
SELF=$(basename $0)
UPDATE_BASE=http://something
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $0)
chmod $OCTAL_MODE $0.tmp
# Overwrite old file with new
mv $0.tmp $0
exit 0
}
The script seems to work as intended, but I'm wondering if there might be caveats with this kind of approach. I just have a hard time believing that a script can overwrite itself without any repercussions.
To be more clear, I'm wondering, if, maybe, bash would read and execute the script line-by-line and after the mv, the exit 0 could be something else from the new script. I think I remember Windows behaving like that with .bat files.
Update: My original snippet did not include set -o errexit. To my understanding, that should keep me safe from issues caused by wget.
Also, in this case, UPDATE_BASE points to a location under version control (to ease concerns).
Result: Based on the input from these answers, I constructed this revised approach:
runSelfUpdate() {
echo "Performing self-update..."
# Download new version
echo -n "Downloading latest version..."
if ! wget --quiet --output-document="$0.tmp" $UPDATE_BASE/$SELF ; then
echo "Failed: Error while trying to wget new version!"
echo "File requested: $UPDATE_BASE/$SELF"
exit 1
fi
echo "Done."
# Copy over modes from old version
OCTAL_MODE=$(stat -c '%a' $SELF)
if ! chmod $OCTAL_MODE "$0.tmp" ; then
echo "Failed: Error while trying to set mode on $0.tmp."
exit 1
fi
# Spawn update script
cat > updateScript.sh << EOF
#!/bin/bash
# Overwrite old file with new
if mv "$0.tmp" "$0"; then
echo "Done. Update complete."
rm \$0
else
echo "Failed!"
fi
EOF
echo -n "Inserting update process..."
exec /bin/bash updateScript.sh
}
(At least it doesn't try to continue running after updating itself!)
The thing that makes me nervous about your approach is that you're overwriting the current script (mv $0.tmp $0) as it's running. There are a number of reasons why this will probably work, but I wouldn't bet large amounts that it's guaranteed to work in all circumstances. I don't know of anything in POSIX or any other standard that specifies how the shell processes a file that it's executing as a script.
Here's what's probably going to happen:
You execute the script. The kernel sees the #!/bin/sh line (you didn't show it, but I presume it's there) and invokes /bin/sh with the name of your script as an argument. The shell then uses fopen(), or perhaps open() to open your script, reads from it, and starts interpreting its contents as shell commands.
For a sufficiently small script, the shell probably just reads the whole thing into memory, either explicitly or as part of the buffering done by normal file I/O. For a larger script, it might read it in chunks as it's executing. But either way, it probably only opens the file once, and keeps it open as long as it's executing.
If you remove or rename a file, the actual file is not necessarily immediately erased from disk. If there's another hard link to it, or if some process has it open, the file continues to exist, even though it may no longer be possible for another process to open it under the same name, or at all. The file is not physically deleted until the last link (directory entry) that refers to it has been removed, and no processes have it open. (Even then, its contents won't immediately be erased, but that's going beyond what's relevant here.)
And furthermore, the mv command that clobbers the script file is immediately followed by exit 0.
BUT it's at least conceivable that the shell could close the file and then re-open it by name. I can't think of any good reason for it to do so, but I know of no absolute guarantee that it won't.
And some systems tend to do stricter file locking that most Unix systems do. On Windows, for example, I suspect that the mv command would fail because a process (the shell) has the file open. Your script might fail on Cygwin. (I haven't tried it.)
So what makes me nervous is not so much the small possibility that it could fail, but the long and tenuous line of reasoning that seems to demonstrate that it will probably succeed, and the very real possibility that there's something else I haven't thought of.
My suggestion: write a second script whose one and only job is to update the first. Put the runSelfUpdate() function, or equivalent code, into that script. In your original script, use exec to invoke the update script, so that the original script is no longer running when you update it. If you want to avoid the hassle of maintaining, distributing, and installing two separate scripts. you could have the original script create the update script with a unique in /tmp; that would also solve the problem of updating the update script. (I wouldn't worry about cleaning up the autogenerated update script in /tmp; that would just reopen the same can of worms.)
Yes, but ... I would recommend you keep a more layered version of your script's history, unless the remote host can also perform version-control with histories. That being said, to respond directly to the code you have posted, see the following comments ;-)
What happens to your system when wget has a hiccup, quietly overwrites part of your working script with only a partial or otherwise corrupt copy? Your next step does a mv $0.tmp $0 so you've lost your working version. (I hope you have it in version control on the remote!)
You can check to see if wget returns any error messages
if ! wget --quiet --output-document=$0.tmp $UPDATE_BASE/$SELF ; then
echo "error on wget on $UPDATE_BASE/$SELF"
exit 1
fi
Also, Rule-of-thumb tests will help, i.e.
if (( $(wc -c < $0.tmp) >= $(wc -c < $0) )); then
mv $0.tmp $0
fi
but are hardly foolproof.
If your $0 could windup with spaces in it, better to surround all references like "$0".
To be super-bullet proof, consider checking all command returns AND that Octal_Mode has a reasonable value
OCTAL_MODE=$(stat -c '%a' $0)
case ${OCTAL_MODE:--1} in
-[1] )
printf "Error : OCTAL_MODE was empty\n"
exit 1
;;
777|775|755 ) : nothing ;;
* )
printf "Error in OCTAL_MODEs, found value=${OCTAL_MODE}\n"
exit 1
;;
esac
if ! chmod $OCTAL_MODE $0.tmp ; then
echo "error on chmod $OCTAL_MODE %0.tmp from $UPDATE_BASE/$SELF, can't continue"
exit 1
fi
I hope this helps.
Very late answer here, but as I just solved this too, I thought it might help someone to post the approach:
#!/usr/bin/env bash
#
set -fb
readonly THISDIR=$(cd "$(dirname "$0")" ; pwd)
readonly MY_NAME=$(basename "$0")
readonly FILE_TO_FETCH_URL="https://your_url_to_downloadable_file_here"
readonly EXISTING_SHELL_SCRIPT="${THISDIR}/somescript.sh"
readonly EXECUTABLE_SHELL_SCRIPT="${THISDIR}/.somescript.sh"
function get_remote_file() {
readonly REQUEST_URL=$1
readonly OUTPUT_FILENAME=$2
readonly TEMP_FILE="${THISDIR}/tmp.file"
if [ -n "$(which wget)" ]; then
$(wget -O "${TEMP_FILE}" "$REQUEST_URL" 2>&1)
if [[ $? -eq 0 ]]; then
mv "${TEMP_FILE}" "${OUTPUT_FILENAME}"
chmod 755 "${OUTPUT_FILENAME}"
else
return 1
fi
fi
}
function clean_up() {
# clean up code (if required) that has to execute every time here
}
function self_clean_up() {
rm -f "${EXECUTABLE_SHELL_SCRIPT}"
}
function update_self_and_invoke() {
get_remote_file "${FILE_TO_FETCH_URL}" "${EXECUTABLE_SHELL_SCRIPT}"
if [ $? -ne 0 ]; then
cp "${EXISTING_SHELL_SCRIPT}" "${EXECUTABLE_SHELL_SCRIPT}"
fi
exec "${EXECUTABLE_SHELL_SCRIPT}" "$#"
}
function main() {
cp "${EXECUTABLE_SHELL_SCRIPT}" "${EXISTING_SHELL_SCRIPT}"
# your code here
}
if [[ $MY_NAME = \.* ]]; then
# invoke real main program
trap "clean_up; self_clean_up" EXIT
main "$#"
else
# update myself and invoke updated version
trap clean_up EXIT
update_self_and_invoke "$#"
fi