If I use set -x, then the commands are displayed just before they are executed. That way they are printed.
But I want to have a debug mode for my script where the user can actually see what commands are going to get printed but those are not executed.
I also tried using : (null command) to print the current command but that does not propagate the result. e.g.,
find /my/home -name "test*" | while read -r i;
do
rm -f $i
done
For this purpose, the out put expected is:
+find /my/home -name "test"
+ rm -f test1
+ rm -f test2 ...
and so on
Is there any way I can achieve this without repeating code (the obvious way being have 2 sections in the batch script for debug and normal mode)?
You can maybe create a wrapper function that either prints or evaluates the command you give to it:
#!/bin/bash
run_command () {
printf '%q ' "$#"
"$#"
}
run_command ls -l
run_command touch /tmp/hello
run_command rm /tmp/hello
This way, you prepend run_command to any single thing you want to do and comment the execution or the echo action as you wish.
You could also provide a parameter to the script that switches to either echo or execute mode:
debug_mode=$1
run_command () {
if [ "$debug_mode" = true ]; then
printf '%q ' "$#"
else
"$#"
fi
}
run_command ...
Most simplest way to run preview/dryrun:
for f in *; do echo "these files will be removed with rm -f $f"; done
Related
I want to create a utility function for bash to remove duplicate lines. I am using function
function remove_empty_lines() {
if ! command -v awk &> /dev/null
then
echo '[x] ERR: "awk" command not found'
return
fi
if [[ -z "$1" ]]
then
echo "usage: remove_empty_lines <file-name> [--replace]"
echo
echo "Arguments:"
echo -e "\t--replace\t (Optional) If not passed, the result will be redirected to stdout"
return
fi
if [[ ! -f "$1" ]]
then
echo "[x] ERR: \"$1\" file not found"
return
fi
echo $0
local CMD="awk '!seen[$0]++' $1"
if [[ "$2" = '--reload' ]]
then
CMD+=" > $1"
fi
echo $CMD
}
If I am running the main awk command directly, it is working. But when i execute the same $CMD in the function, I am getting this error
$ remove_empty_lines app.js
/bin/bash
awk '!x[/bin/bash]++' app.js
The original code is broken in several ways:
When used with --reload, it would truncate the output file's contents before awk could ever read those contents (see How can I use a file in a command and redirect output to the same file without truncating it?)
It didn't ever actually run the command, and for the reasons described in BashFAQ #50, storing a shell command in a string is inherently buggy (one can work around some of those issues with eval; BashFAQ #48 describes why doing so introduces security bugs).
It wrote error messages (and other "diagnostic content") to stdout instead of stderr; this means that if your function's output was redirected to a file, you could never see its errors -- they'd end up jumbled into the output.
Error cases were handled with a return even in cases where $? would be zero; this means that return itself would return a zero/successful/truthy status, not revealing to the caller that any error had taken place.
Presumably the reason you were storing your output in CMD was to be able to perform a redirection conditionally, but that can be done other ways: Below, we always create a file descriptor out_fd, but point it to either stdout (when called without --reload), or to a temporary file (if called with --reload); if-and-only-if awk succeeds, we then move the temporary file over the output file, thus replacing it as an atomic operation.
remove_empty_lines() {
local out_fd rc=0 tempfile=
command -v awk &>/dev/null || { echo '[x] ERR: "awk" command not found' >&2; return 1; }
if [[ -z "$1" ]]; then
printf '%b\n' >&2 \
'usage: remove_empty_lines <file-name> [--replace]' \
'' \
'Arguments:' \
'\t--replace\t(Optional) If not passed, the result will be redirected to stdout'
return 1
fi
[[ -f "$1" ]] || { echo "[x] ERR: \"$1\" file not found" >&2; return 1; }
if [ "$2" = --reload ]; then
tempfile=$(mktemp -t "$1.XXXXXX") || return
exec {out_fd}>"$tempfile" || { rc=$?; rm -f "$tempfile"; return "$rc"; }
else
exec {out_fd}>&1
fi
awk '!seen[$0]++' <"$1" >&$out_fd || { rc=$?; rm -f "$tempfile"; return "$rc"; }
exec {out_fd}>&- # close our file descriptor
if [[ $tempfile ]]; then
mv -- "$tempfile" "$1" || return
fi
}
First off the output from your function call is not an error but rather the output of two echo commands (echo $0 and echo $CMD).
And as Charles Duffy has pointed out ... at no point is the function actually running the $CMD.
As for the inclusion of /bin/bash in your function's echo output ... the main problem is the reference to $0; by definition $0 is the name of the running process, which in the case of a function is the shell under which the function is being called. Consider the following when run from a bash command prompt:
$ echo $0
-bash
As you can see from your output this generates /bin/bash in your environment. See this and this for more details.
On a related note, the reference to $0 within double quotes causes the $0 to be evaluated, so this:
local CMD="awk '!seen[$0]++' $1"
becomes
local CMD="awk '!seen[/bin/bash]++' app.js"
I'm thinking what you want is something like:
echo $1 # the name of the file to be processed
local CMD="awk '!seen[\$0]++' $1" # escape the '$' in '$0'
becomes
local CMD="awk '!seen[$0]++' app.js"
That should fix the issues shown in your function's output; as for the other issues ... you're getting a good bit of feedback in the various comments ...
I am missing my bash aliases in fish, and don't want to manually convert all of them to fish functions.
How to get them all from bash to fish?
Bonus points if:
the solution supports an iterative process, as in: i can easily change the aliases in bash, and re-convert/re-import them into fish
the solution also imports bash functions
Convert bash aliases to bash scripts
I decided to do this instead of approach below, and putting the scripts into ~/bin/, which is in my PATH. This allows me to use them from whatever shell i am currently executing, and it prevents potential problems with quoting.
use like so:
# converts all bash aliases to script files
convert_bash_aliases_to_scripts
# removes all scripts previously converted by this script
convert_bash_aliases_to_scripts clean
bash conversion script:
#!/bin/bash
# Convert bash aliases to bash scripts.
#
# Copyright 2018 <hoijui.quaero#gmail.com>, licensed under the GPL-3.0+
#
# Usage:
# convert_bash_aliases_to_scripts # converts all bash aliases to script files
# convert_bash_aliases_to_scripts clean # removes all scripts previously converted by this script
COLOR_RED=$'\e[0;31m'
COLOR_ORANGE=$'\e[0;33m'
COLOR_BLUE=$'\e[0;34m'
COLOR_BLUE_LIGHT=$'\e[1;34m'
COLOR_GREEN=$'\e[0;32m'
COLOR_BROWN=$'\e[0;33m'
COLOR_YELLOW=$'\e[1;33m'
COLOR_WHITE=$'\e[1;37m'
COLOR_CYAN=$'\e[0;36m'
COLOR_PURPLE=$'\e[0;35m'
COLOR_GRAY=$'\e[1;30m'
COLOR_GRAY_LIGHT=$'\e[0;37m'
COLOR_NONE=$'\e[m' # No Color
OUTPUT_DIR=~/bin/converted/aliases
LINKS_DIR=~/bin
README_FILE_NAME="README.md"
README_FILE="$OUTPUT_DIR/$README_FILE_NAME"
if [ "$1" = "clean" ]
then
for script_file in $(find "$LINKS_DIR" -maxdepth 1 -type l)
do
conv_script_file="$OUTPUT_DIR/$(basename $script_file)"
if [ -e $conv_script_file ] && [ "$(readlink --canonicalize $script_file)" = "$(realpath $conv_script_file)" ]
then
script_name=$(basename $script_file)
echo "removing converted bash alias-script: $script_name"
rm $conv_script_file \
&& rm $script_file
fi
done
rm $README_FILE 2> /dev/null
rmdir $OUTPUT_DIR 2> /dev/null
exit 0
fi
SOURCE_FILES="${HOME}/.bashrc ${HOME}/.bash_aliases"
mkdir -p $OUTPUT_DIR
echo -e "# Bash alias conversion scripts\n\nsee $0\n\nWARNING: Do NOT manually edit files in this directory. instead, copy them to $LINKS_DIR (replacing the symbolic link that already exists there), and edit that new file.\nIf you edit the files in this dir, it will be replaced on the next (re)conversion from aliases." \
> $README_FILE
AUTO_IMPORT_WARNING="# WARNING Do NOT edit this file by hand, as it was auto-generated from a bash alias, and may be overwritten in the future. please read ${README_FILE}"
function _is_link_to {
local file_link=$1
local file_target=$2
test -e $file_target \
&& test "$(readlink --canonicalize $file_link)" = "$(realpath $file_target)"
return $?
}
for source_file in $SOURCE_FILES
do
IFS=$'\n'
for a in $(cat $source_file | grep "^alias")
do
a_name="$(echo "$a" | sed -e 's/alias \([^=]*\)=.*/\1/')"
a_command="$(echo "$a" | sed -e 's/alias \([^=]*\)=//' -e 's/[ \t]*#.*$//')"
if echo "${a_command:0:1}" | grep -q -e "[\'\"]"
then
# unquote
a_command_start=1
let a_command_end="${#a_command} - 2"
else
# leave as is
a_command_start=0
let a_command_end="${#a_command}"
fi
script_file="$LINKS_DIR/$a_name"
conv_script_file="$OUTPUT_DIR/$a_name"
# Check whether the script already exists.
# If so, we skip importing it, unless it is just a link to a previously imported script.
log_action="none"
log_action_color="${COLOR_NONE}"
log_content=""
if [ -e $script_file ] && ! $(_is_link_to $script_file $conv_script_file)
then
log_action="skipped (exists)"
log_action_color="${COLOR_ORANGE}"
log_content=""
else
if [ -e $script_file ]
then
log_action="reimporting"
log_action_color="${COLOR_BLUE}"
else
log_action="importing"
log_action_color="${COLOR_GREN}"
fi
# write the script file to a temporary location
conv_script_file_tmp="${conv_script_file}_BAK"
echo "#!/bin/bash" > $conv_script_file_tmp
echo -e "$AUTO_IMPORT_WARNING" >> $conv_script_file_tmp
echo -e "#\n# Imported bash alias '$a_name' from file '$source_file'" >> $conv_script_file_tmp
cat >> "${conv_script_file_tmp}" <<EOF
${a_command:${a_command_start}:${a_command_end}} \${#}
EOF
if diff -N ${conv_script_file_tmp} ${conv_script_file} > /dev/null
then
log_content="no change"
log_content_color="${COLOR_NONE}"
else
log_content="changed"
log_content_color="${COLOR_GREEN}"
fi
log_content=$(printf "%s %10s -> %s${COLOR_NONE}" "${log_content_color}" "${log_content}" "$a_command")
mv "${conv_script_file_tmp}" "${conv_script_file}"
# make the script executable
chmod +x $conv_script_file
# remove the link if it already exists (in case of reimport)
rm $script_file 2> /dev/null
# .. and re-create it as local symbolic link
# to the function in the imports dir
ln --symbolic --relative $conv_script_file $script_file
fi
printf "%s%20s: %-25s${COLOR_NONE}%s\n" "${log_action_color}" "${log_action}" "$a_name" "${log_content}"
done
done
Deprecated: Creating fish wrappers that execute bash code
Below is a script that creates fish script wrappers for the local bash aliases: For each bash alias, it takes the contents, and creates a fish alias/script that executes the code in bash sub-shell.
It is not optimal, but is sufficient for most of my aliases.
WARNING It might happen that the imported function acts differently then in bash. You may loose data or accidentally DDOS your coworkers when using them.
use like so:
# imports (or reimports) all bash aliases into fish functions, permanently
import_bash_aliases
# removes all fish functions previously imported by this script
import_bash_aliases clean
save this in ~/.config/fish/functions/import_bash_aliases.fish:
#!/usr/bin/fish
# Fish function to import bash aliases
#
# Copyright 2018 <hoijui.quaero#gmail.com>, licensed under the GPL-3.0+
#
# This script is based on a script from Malte Biermann,
# see: https://glot.io/snippets/efh1c4aec0
#
# WARNING: There is no guarantee that the imported aliases work the same way
# as they do in bash, so be cautious!
#
# Usage:
# import_bash_aliases # imports (or reimports) all bash aliases into fish functions, permanently
# import_bash_aliases clean # removes all fish functions previously imported by this script from bash aliases
function import_bash_aliases --description 'Converts bash aliases to .fish functions.\nThis might be called repeatedly, and will not override functions that are already defined in fish, except they are merely an older import from this script.'
set -l FISH_FUNCTIONS_DIR ~/.config/fish/functions
set -l BASH_IMPORTS_DIR_NAME bash-imports
set -l BASH_IMPORTS_DIR $FISH_FUNCTIONS_DIR/$BASH_IMPORTS_DIR_NAME
set -l README_FILE $BASH_IMPORTS_DIR/README.md
if test "$argv[1]" = "clean"
for fun_file in (find $FISH_FUNCTIONS_DIR -maxdepth 1 -name '*.fish')
set -l imp_fun_file $BASH_IMPORTS_DIR/(basename $fun_file)
if test -e $imp_fun_file ; and test (readlink --canonicalize $fun_file) = (realpath $imp_fun_file)
set -l fun_name (basename $fun_file '.fish')
echo "removing imported bash alias/function $fun_name"
rm $imp_fun_file
and rm $fun_file
and functions --erase $fun_name
end
end
rm $README_FILE ^ /dev/null
rmdir $BASH_IMPORTS_DIR ^ /dev/null
return 0
end
set -l SOURCE_FILES ~/.bashrc ~/.bash_aliases
mkdir -p $BASH_IMPORTS_DIR
echo -e "# Bash alias imports\n\nsee `$argv[0]`\n\nWARNING: Do NOT manually edit files in this directory. instead, copy them to $FISH_FUNCTIONS_DIR (replacing the symbolic link that already exists there), and edit that new file.\nIf you edit the files in this dir, it will be replaced on the next (re)import from bash aliases." \
> $README_FILE
set -l UNUSED_STUB_MSG "The bash alias corresponding to this function was NOT imported, because a corresponding function already exists at %s\n"
set -l AUTO_IMPORT_WARNING "# WARNING Do NOT edit this file by hand, as it was auto-generated from a bash alias, and may be overwritten in the future. please read {$README_FILE}"
function _fish_func_exists
set -l fun_name $argv[1]
# This also detects in-memory functions
functions --query $fun_name
# This also detects script files in the functions dir
# that do not contain a function wiht the same name
or test -e "$FISH_FUNCTIONS_DIR/$fun_name.fish"
return $status
end
function _is_link_to
set -l file_link $argv[1]
set -l file_target $argv[2]
test -e $file_target
and test (readlink --canonicalize $file_link) = (realpath $file_target)
return $status
end
for source_file in $SOURCE_FILES
for a in (cat $source_file | grep "^alias")
set -l a_name (echo $a | sed -e 's/alias \([^=]*\)=.*/\1/')
set -l a_command (echo $a | sed -e 's/alias \([^=]*\)=//' -e 's/[ \t]*#[^\'\"]\+$//')
set -l fun_file "$FISH_FUNCTIONS_DIR/$a_name.fish"
set -l imp_fun_file "$BASH_IMPORTS_DIR/$a_name.fish"
# Check whether the function already exists.
# If so, we skip importing it, unless it is just a link to a previously imported function.
if _fish_func_exists $a_name; and not _is_link_to $fun_file $imp_fun_file
set_color red
printf "%20s: %-25s\n" "skipping (exists)" $a_name
set_color normal
#printf $UNUSED_STUB_MSG $fun_file > $imp_fun_file
else
set_color green
printf "%20s: %-25s -> %s\n" "(re-)importing" $a_name $a_command
set_color normal
# remove the link, in case of re-importing
rm $fun_file ^ /dev/null
# write the function file
echo "#!/usr/bin/fish" > $imp_fun_file
echo "\
$AUTO_IMPORT_WARNING
function $a_name -d 'bash alias "$a_name" import'
bash -c $a_command' '\$argv''
end
" \
>> $imp_fun_file
# make the script executable
chmod +x $imp_fun_file
# .. and re-create it as local symbolic link
# to the function in the imports dir
ln --symbolic --relative $imp_fun_file $fun_file
end
end
end
# (re-)load all the functions we just defined
exec fish
end
How can I see if a file exists using test -f and with a wildcard in the path?
This works:
test -f $PREFIX/lib/python3.6/some_file
This does not work (what am I doing wrong here?):
test -f $PREFIX/lib/python*/some_file
I need a non-zero exit code if the file does not exist.
Expand wildcard to an array and then check first element:
f=($PREFIX/lib/python*/some_file)
if [[ -f "${f[0]}" ]]; then echo "found"; else echo "not found"; fi
unset f
You need to iterate over the files as test -f only works with a single file. I would use a shell function for that:
#!/bin/sh
# test-f.sh
test_f() {
for fname; do
if test -f "$fname"; then
return 0
fi
done
}
test_f "$#"
Then a test run could be
$ sh -x test-f.sh
$ sh -x test-f.sh doesnotexist*
$ sh -x test-f.sh *
From the man page of test:
-f file True if file exists and is a regular file
meaning that test -f <arg> expects arg to be a single file. If your wildcard in path results in multiple files an error will be thrown.
Try iterating when using a wildcard :)
I am trying to call a command from a script (defined as functions inside .bashrc) as following
the first function hooks the rm command by deleting any file except a file named "forensicfile1"
#hooking rm command
function rm(){
if [[ $1 = "forensicfile1" ]]; then
echo "$1 file is deleted by user ">forensicaudit
else
rm $1
fi
}
the second hooking function is trying to hook the ls command by listing all files except the forensicfile1
#hooking ls command
function ls(){
if [[ $PWD = "/home/waleed/forensicdata" ]]
then
alias myls='ls "/home/waleed/forensicdata" -1 "forensicfile1" '
myls
else
ls $PWD
fi
}
however the following lines are not working
rm $1 // within the first function
alias myls='ls "/home/waleed/forensicdata" -1 "forensicfile1" '
myls
ls $PWD
can anyone help?
Sometimes, it happens to me that I'm executing certain commands, and only afterwards I realize that I sent the wrong parameter to a command ( like a restart of a Heroku application ). I'd like to modify bash in such a way that if it sees a command containing a certain string, it will prompt me whether I'm sure or not. For example ( imagine the string is tempus ):
$ heroku restart --app tempus
Now, I'd like bash to prompt me with a Y/N prompt, and if I type y, only then I'd like it to execute the command. If I type N, the command will not be executed. How could I handle this problem?
I don't know of any way to intercept all bash commands, but you can intercept predetermined commands using the following trick.
Create a directory (say ~/interception) and set it as the first entry in $PATH
Create the following script in that directory with a list of commands you wish to intercept and the full path to the actual command
[bash]$ cat intercept.sh
#!/bin/bash
# map commands to full path
declare -A COMMANDS
COMMANDS[heroku]=/usr/bin/heroku
COMMANDS[grep]=/bin/grep
# ... more ...
CMD=$(basename $0) # command used to call this script
if [[ ! -z "${COMMANDS[$CMD]+x}" ]]; then # mapping found
# Do what you wish here. You can even modify/inspect the params.
echo "intercepted $CMD command... "
${COMMANDS[$CMD]} $# # run actual command with all params
else
echo "Unknown command $CMD"
fi
In the same directory, create symlinks to that script using the name of the commands you wish to intercept
[bash]$ ln -s intercept.sh grep
[bash]$ ln -s intercept.sh heroku
Now, each time you call the command, that script is invoked via the symlink and it can then do your bidding before calling the actual command.
You can extend this further by sourcing the $COMMANDS from a config file and create helper commands to augment the config file and create/remove the sym links. You would then be able to manage the who setup using commands such as:
intercept_add `which heroku`
intercept_remove heroku
intercept_list
Because bash itself doesn't support command line filter, it's not possible to intercept commands.
Here is a dirty solution:
Find all executables in PATH and create wrapper functions for each of them.
The wrapper function then call prefilter() function if it's declared.
If prefilter() function failed, the command is canceled.
SOURCE: cmd-wrap.sh
#!/bin/bash # The shebang is only useful for debug. Don't execute this script.
function create_wrapper() {
local exe="$1"
local name="${exe##*/}"
# Only create wrappers for non-builtin commands
[ `type -t "$name"` = 'file' ] || return
# echo "Create command wrapper for $exe"
eval "
function $name() {\
if [ \"\$(type -t prefilter)\" = 'function' ]; then \
prefilter \"$name\" \"\$#\" || return; \
fi; \
$exe \"\$#\";
}"
}
# It's also possible to add pre/post hookers by install
# [ `type -t \"$name-pre\"` = 'function' ] && \"$name-pre\" \"\$#\"
# into the dynamic generated function body.
function _create_wrappers() {
local paths="$PATH"
local path
local f n
while [ -n "$paths" ]; do
path="${paths%%:*}"
if [ "$path" = "$paths" ]; then
paths=
else
paths="${paths#*:}"
fi
# For each path element:
for f in "$path"/*; do
if [ -x "$f" ]; then
# Don't create wrapper for strange command names.
n="${f##*/}"
[ -n "${n//[a-zA-Z_-]/}" ] || create_wrapper "$f"
fi
done
done
unset _create_wrappers # Remove the installer.
unset create_wrapper # Remove the helper fn, which isn't used anymore.
}
_create_wrappers
To utilize it for your problem:
source it in bash:
. ./cmd-wrap.sh
Create your version of prefilter() to check if any argument contains the string:
function prefilter() {
local a y
for a in "$#"; do
if [ "$a" != "${a/tempus}" ]; then
echo -n "WARNING: The command contains tempus. Continue?"
read y
[ "$y" = 'Y' ] || [ "$y" = 'y' ]
return $?
fi
done
return 0
}
Run
heroku restart --app tempus
but not
/usr/bin/heroku restart --app tempus
to make use of the wrapper function.
The easiest way is to use aliases. This simple example should work for you:
This protects from executing the heroku command with tempus in the arguments
function protect_heroku {
# use grep to determine if the bad string is in arguments
echo "$*" | grep tempus > /dev/null
# if string is not in arguments
if [ $? != 0 ]; then
# run the protected command using its full path, so as not to trigger alias
/path/to/heroku "$#"
else
# get user confirmation
echo -n Are you sure \(y/n\)?' '
read CONFIRM
if [ "$CONFIRM" = y ]; then
# run the protected command using its full path
/path/to/heroku "$#"
fi
fi
}
# This is the key:
# This alias command means that 'heroku' from now refers
# to the function protect_heroku, rather than /bin/heroku
alias heroku=protect_heroku
Put this code into your bash profile ~/.profile and then log out and log back in. From now on, bash will protect you from accidentally running heroku with tempus.
Simplest way is to replace heroku with a script that does the checking before executing the real heroku. Another way would be to add a bash alias for heroku.