I've been working on a series of bash scripts and I need to automate password entry for batch processing of files and actions.
This isn't for just one program, for example, sometimes it needs to be done for GPG, other times, steghide.
There is a specific reason this is being done and I understand the security elements behind it. This is considered and is negated by how the scripts are stored and working.
The passwords or passphrases are passed via command line arguments to the script and the password/phrase must be repeated many times programmatically.
Here is an example of what I am working with inside the script:
for f in $dir
do
steghide embed -cf $f -ef /path/to/secret.txt
done
This simply interactively asked this for every image however:
Enter Passphrase:
Re-enter Passphrase:
For every image in a directory, this password will be requested and so the password should be able to be stored in a variable and reused.
I have been working with steghide most recently but there will also be a need to automate passphrases with GPG at a later date, although there is no need for the methods to be the same.
man steghide:
-p, --passphrase
Use the string following this argument as the
passphrase. If your passphrase contains whitespace,
you have to enclose it in quotes, for example: -p
"a very long passphrase".
man gpg:
--passphrase string
Use string as the passphrase. This can only be used if only one
passphrase is supplied. Obviously, this is of very questionable
security on a multi-user system. Don't use this option if you can
avoid it.
It's untested publicly, rough around the edges, and can be improved... but here's a preview of some of my research scripts that haven't been merged into one of the GitHub projects I'm writing... definitely run shellcheck against the below script to catch any typos.
#/usr/bin/env bash
Var_stego_out_dir="${1}"
Var_stego_in_dir="${2}"
Var_stego_cover_dir="${3}"
Var_passphrase_file="${4}"
Var_passphrase="${5}"
Var_auto_pass_length="${6:-64}"
Func_build_array_of_paths(){
_dir="${1}"
_arr="${2}"
_file_extension_list="${3}"
if [ -d "${_dir}" ] && [ "${#${_arr}[#]}" = "0" ]; then
find "${_dir}" -xtype f | while read _path; do
case "${_path##*.}" in
${_file_extension_list//,/|})
declare -ag "${_arr}+=( ${_path} )"
;;
esac
done
fi
}
Func_enc_stego(){
_cover_file="${1}"
_enc_file="${2}"
_pass_file="${3}"
_output_file="${Var_stego_out_dir}/${_cover_file##*/}"
if [ -f "${_cover_file}" ] && [ -f "${_enc_file}" ]; then
_auto_passphrase="${Var_passphrase:-$(base64 /dev/random | tr -cd '[:print:]' head -c${Var_auto_pass_length})}"
if ! [ -f "${_output_file}" ]; then
steghide -p ${_auto_passphrase} -ef ${_enc_file} -cf ${_cover_file} -sf ${_output_file}
cat <<<"### ${_output_file} ### ${_auto_passphrase}" >> "${_pass_file}"
else
steghide -p ${_auto_passphrase} -ef ${_enc_file} -cf ${_cover_file} -sf ${_output_file}_$(date -u +%s)
cat <<<"### ${_output_file}_$(date -u +%s) ### ${_auto_passphrase}" >> "${_pass_file}"
fi
fi
}
Func_main(){
## Build array of file paths for cover file use
Func_build_array_of_paths "${Var_stego_cover_dir}" "Arr_cover_list" "au,AU,bmp,BMP,jpeg,JPEG,wav,WAV"
## Build array of file paths for embed file use
Func_build_array_of_paths "${Var_stego_in_dir}" "Arr_input_list" "gpg,GPG,txt,TXT"
let _arr_input_count=0
let _arr_cover_count=0
until [ "${_arr_input_count}" = "${#Arr_input_list}" ]; do
if [ -f "${Arr_cover_list[${_arr_cover_count}]}" ]; then
Func_enc_stego "${Arr_cover_list[${_arr_cover_count}]}" "${Arr_input_list[${_arr_input_count}]}" "${Var_passphrase_file}"
let _arr_cover_count++
let _arr_input_count++
elif [ -f "${Arr_cover_list[$((${_arr_cover_count}-1))]}" ]; then
Func_enc_stego "${Arr_cover_list[$((${_arr_cover_count}-1))]}" "${Arr_input_list[${_arr_input_count}]}" "${Var_passphrase_file}"
let _arr_input_count++
_arr_cover_count="$((${_arr_cover_count}-1))"
if
done
}
Func_main
Run above with the following portions filled-in
script.sh "/path/to/stego_out_dir" "/path/to/stego_in_dir" "/path/to/stego_cover_dir" "/path/to/passphrase_file"
## or define static passphrase
#script.sh "/path/to/stego_out_dir" "/path/to/stego_in_dir" "/path/to/stego_cover_dir" "/path/to/passphrase_file" "passphrase"
Note saving the passphrase and file in plain-text like the above does is very bad practice, and because the OP stated that they also where looking at GnuPG automation too, readers and the OP"s author should look-up Perinoid_Pipes; and for specifically the GnuPG_Gen_Key.sh script and functions starting with Func_dec_* within the Paranoid_Pipes.sh for working/tested examples of automation involving GnuPG passphrases; and for protecting the passphrases written by the above script look-up functions starting with Func_enc_* within the Paranoid_Pipes.sh script for how the mkfifo command and resulting named pipe is used to automate encryption of most data types. Hint the fourth example argument "/path/to/passphrase_file" would point to an encrypting named pipe made by the linked script to keep things a bit more secure ;-)
Related
Background
Python scripts, for example, can have several "levels" of documentation via docstrings. What's neat about it, is that they can be defined at per-function levels, per-method levels, per-class levels, and most importantly (in the context of my question): per-file levels. For example, the top of the file may look like so:
#!/usr/bin/env python
"""
#brief A script that does cool stuff.
"""
What's especially useful about this feature is that it's easy to extract and print at run-time.
Question
Do bash scripts support such a feature? i.e. is there a "standardized" approach to generating a file-level set of documentation (i.e. human-readable description of the purpose of the script, usage syntax, etc.; so that it's easy for another script to automatically parse/extract this information? My goal is to create several debug scripts that are self-documenting, and if there's already a standard/de-facto-best way to do this, I'd like to avoid re-inventing the wheel.
The "File Header" section of Google's Shell Style Guide is one way to add a 'docstring' to your bash scripts.
Basically, the answer is to use #, rather than quotes like you would with Python.
You can do this for Bash easily, it is a little more tricky if you need to ensure compatibility with POSIX only shells like /bin/sh or primarily busybox systems like Alpine.
The Linux Documentation Project has some great examples.
http://tldp.org/LDP/abs/html/here-docs.html
Yet another twist of this nifty trick makes "self-documenting" scripts
possible.
Example 19-12. A self-documenting script
#!/bin/bash
# self-document.sh: self-documenting script
# Modification of "colm.sh".
DOC_REQUEST=70
if [ "$1" = "-h" -o "$1" = "--help" ] # Request help.
then
echo; echo "Usage: $0 [directory-name]"; echo
sed --silent -e '/DOCUMENTATIONXX$/,/^DOCUMENTATIONXX$/p' "$0" |
sed -e '/DOCUMENTATIONXX$/d'; exit $DOC_REQUEST; fi
: <<DOCUMENTATIONXX
List the statistics of a specified directory in tabular format.
---------------------------------------------------------------
The command-line parameter gives the directory to be listed.
If no directory specified or directory specified cannot be read,
then list the current working directory.
DOCUMENTATIONXX
if [ -z "$1" -o ! -r "$1" ]
then
directory=.
else
directory="$1"
fi
echo "Listing of "$directory":"; echo
(printf "PERMISSIONS LINKS OWNER GROUP SIZE MONTH DAY HH:MM PROG-NAME\n" \
; ls -l "$directory" | sed 1d) | column -t
exit 0
Using a cat script is an alternate way of accomplishing this.
DOC_REQUEST=70
if [ "$1" = "-h" -o "$1" = "--help" ] # Request help.
then # Use a "cat script" . . .
cat <<DOCUMENTATIONXX
List the statistics of a specified directory in tabular format.
---------------------------------------------------------------
The command-line parameter gives the directory to be listed.
If no directory specified or directory specified cannot be read,
then list the current working directory.
DOCUMENTATIONXX
exit $DOC_REQUEST
fi
A slightly more elegant example using functions to handle the documentation and error messages.
#!/bin/sh
usage() {
cat << EOF
Usage:
$0 [-u [username]] [-p]
Options:
-u <username> : Optionally specify the new username to set password for.
-p : Prompt for a new password.
EOF
}
die() {
echo
echo "$1, so giving up. Sorry."
echo
exit 2
}
if [ -z "$USER" ] ; then
die "Could not identify the existing user"
fi
if $PSET ; then
passwd $USER || die "Busybox didn't like your password"
fi
https://github.com/jyellick/mficli/blob/master/util/changecreds.sh
There is no standard for docstrings for bash. It's always nice to have man pages though (eg. https://www.cyberciti.biz/faq/linux-unix-creating-a-manpage/), or info pages (https://unix.stackexchange.com/questions/164443/how-to-create-info-documentation).
I have written a code but I am having a problem to make the double loop in my bash script. This script should read all the files 1 by 1 in the given directory to upload but the value of "XYZ" changes for each file. Is there a way for me to make the code ask me to enter the "XYZ" every time it reads a new file to upload? (if possible with the name of the file read) like "please enter the XYZ value of 'read file's name'" I could not think of any possible ways of doing so. I also have the XYZ values listed in a file in a different directory so maybe can it be called like the do loop I did for the path? I might actually need to use both cases as well...
#!/bin/bash
FILES=/home/user/downloads/files/
for f in $FILES
do
curl -F pitch=9 -F Name='astn' -F
"path=#/home/user/downloads/files;$f" -F "pass 1234" -F "XYZ= 1.2" -
F time=30 -F outputFormat=json
"http://blablabla.com"
done
try following once.
#!/bin/bash
FILES=/home/user/downloads/files/
for f in $FILES
do
echo "Please enter the name variable value here:"
read Name
curl -F pitch=9 -F "$Name" -F
"path=#/home/user/downloads/files;$f" -F "pass 1234" -F "XYZ= 1.2" -
F time=30 -F outputFormat=json
"http://blablabla.com"
done
I have entered a read command inside loop so each time it will prompt user for a value, since you haven't provided more details about your requirement so I haven't tested it completely.
The problem was actually the argument. By changing it to:
-F Name="$Name"
solved the problem. Trying to link the argument such as only $Name or "$Name" causes a bad reception.
I am trying to upload multiple files from one folder to a ftp site and wrote this script:
#!/bin/bash
for i in '/dir/*'
do
if [-f /dir/$i]; then
HOST='x.x.x.x'
USER='username'
PASSWD='password'
DIR=archives
File=$i
ftp -n $HOST << END_SCRIPT
quote USER $USER
quote PASS $PASSWD
ascii
put $FILE
quit
END_SCRIPT
fi
It is giving me following error when I try to execute:
username#host:~/Documents/Python$ ./script.sh
./script.sh: line 22: syntax error: unexpected end of file
I can't seem to get this to work. Any help is much appreciated.
Thanks,
Mayank
It's complaining because your for loop does not have a done marker to indicate the end of the loop. You also need more spaces in your if:
if [ -f "$i" ]; then
Recall that [ is actually a command, and it won't be recognized if it doesn't appear as such.
And... if you single quote your glob (at the for) like that, it won't be expanded. No quotes there, but double quotes when using $i. You probably also don't want to include the /dir/ part when you use $i as it's included in your glob.
If I'm not mistaken, ncftp can take wildcard arguments:
ncftpput -u username -p password x.x.x.x archives /dir/*
If you don't already have it installed, it's likely available in the standard repo for your OS.
First, the literal, fixing-your-script answer:
#!/bin/bash
# no reason to set variables that don't change inside the loop
host='x.x.x.x'
user='username'
password='password'
dir=archives
for i in /dir/*; do # no quotes if you want the wildcard to be expanded!
if [ -f "$i" ]; then # need double quotes and whitespace here!
file=$i
ftp -n "$host" <<END_SCRIPT
quote USER $user
quote PASS $password
ascii
put $file $dir/$file
quit
END_SCRIPT
fi
done
Next, the easy way:
lftp -e 'mput -a *.i' -u "$user,$password" "ftp://$host/"
(yes, lftp expands the wildcard internally, rather than expecting this to be done by the outer shell).
First of all my apologies in not making myself clear in the question. My actual task was to copy a file from local folder to a SFTP site and then move the file to an archive folder. Since the SFTP is hosted by a vendor I cannot use the key sharing (vendor limitation. Also, SCP will require password entering if used in a shell script so I have to use SSHPASS. SSHPASS is in the Ubuntu repo however for CentOS it needs to be installed from here
Current thread and How to run the sftp command with a password from Bash script? did gave me better understanding on how to write the script and I will share my solution here:
#!/bin/bash
#!/usr/bin
for i in /dir/*; do
if [ -f "$i" ]; then
file=$i
export SSHPASS=password
sshpass -e sftp -oBatchMode=no -b - user#ftp.com << !
cd foldername/foldername
put $file
bye
!
mv $file /somedir/test
fi
done
Thanks everyone for all the responses!
--Mayank
Is it possible to save last entered value of a variable by the user in the bash script itself so that I reuse value the next time while executing again?.
Eg:
#!/bin/bash
if [ -d "/opt/test" ]; then
echo "Enter path:"
read path
p=$path
else
.....
........
fi
The above script is just a sample example I wanted to give(which may be wrong), is it possible if I want to save the value of p permanently in the script itself to so that I use it somewhere later in the script even when the script is re-executed?.
EDIT:
I am already using sed to overwrite the lines in the script while executing, this method works but this is not at all good practice as said. Replacing the lines in the same file as said in the below answer is much better than what I am using like the one below:
...
....
PATH=""; #This is line no 7
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )";
name="$(basename "$(test -L "$0" && readlink "$0" || echo "$0")")";
...
if [ condition ]
fi
path=$path
sed -i '7s|.*|PATH='$path';|' $DIR/$name;
Someting like this should do the asked stuff :
#!/bin/bash
ENTERED_PATH=""
if [ "$ENTERED_PATH" = "" ]; then
echo "Enter path"
read path
ENTERED_PATH=$path
sed -i 's/ENTERED_PATH=""/ENTERED_PATH='$path'/g' $0
fi
This script will ask user a path only if not previously ENTERED_PATH were defined, and store it directly into the current file with the sed line.
Maybe a safer way to do this, would be to write a config file somewhere with the data you want to save and source it . data.saved at the begining of your script.
In the script itself? Yes with sed but it's not advisable.
#!/bin/bash
test='0'
echo "test currently is: $test";
test=`expr $test + 1`
echo "changing test to: $test"
sed -i "s/test='[0-9]*'/test='$test'/" $0
Preferable method:
Try saving the value in a seperate file you can easily do a
myvar=`cat varfile.txt`
And whatever was in the file is not in your variable.
I would suggest using the /tmp/ dir to store the file in.
Another option would be to save the value as an extended attribute attached to the script file. This has many of the same problems as editing the script's contents (permissions issues, weird for multiple users, etc) plus a few of its own (not supported on all filesystems...), but IMHO it's not quite as ugly as rewriting the script itself (a config file really is a better option).
I don't use Linux, but I think the relevant commands would be something like this:
path="$(getfattr --only-values -n "user.saved_path" "${BASH_SOURCE[0]}")"
if [[ -z "$path" ]]; then
read -p "Enter path:" path
setfattr -n "user.saved_path" -v "$path" "${BASH_SOURCE[0]}"
fi
this topic has been discussed at length, however, I have a variant on the theme that I just cannot crack. Two days into this now and decided to ping the community. THx in advance for reading..
Exec. summary is I have a script in OS X that runs fine and executes without issue or error when done manually. When I put the script in the crontab to run daily it still runs but it doesnt run all of the commands (specifically SFTP).
I have read enough posts to go down the path of environment issues, so as you will see below, I hard referenced the location of the SFTP in the event of a PATH issue...
The only thing that I can think of is the IdentityFile. NOTE: I am putting this in the crontab for my user not root. So I understand that it should pickup on the id_dsa.pub that I have created (and that has already been shared with the server)..
I am not trying to do any funky expect commands to bypass the password, etc. I dont know why when run from the cron that it is skipping the SFTP line.
please see the code below.. and help is greatly appreciated.. thx
#!/bin/bash
export DATE=`date +%y%m%d%H%M%S`
export YYMMDD=`date +%y%m%d`
PDATE=$DATE
YDATE=$YYMMDD
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
FEED="~/Dropbox/"
USER="user"
HOST="host.domain.tld"
A="/tmp/5nPR45bH"
>${A}.file1${PDATE}
>${A}.file2${PDATE}
BYEbye ()
{
rm ${A}.file1${PDATE}
rm ${A}.file2${PDATE}
echo "Finished cleaning internal logs"
exit 0
}
echo "get -r *" >> ${A}.file1${PDATE}
echo "quit" >> ${A}.file1${PDATE}
eval mkdir ${FEED}${YDATE}
eval cd ${FEED}${YDATE}
eval /usr/bin/sftp -b ${A}.file1${PDATE} ${USER}#${HOST}
BYEbye
exit 0
Not an answer, just comments about your code.
The way to handle filenames with spaces is to quote the variable: "$var" -- eval is not the way to go. Get into the habit of quoting all variables unless you specifically want to use the side effects of not quoting.
you don't need to export your variables unless there's a command you call that expects to see them in the environment.
you don't need to call date twice because the YYMMDD value is a substring of the DATE: YYMMDD="${DATE:0:6}"
just a preference: I use $HOME over ~ in a script.
you never use the "file2" temp file -- why do you create it?
since your sftp batch file is pretty simple, you don't really need a file for it:
printf "%s\n" "get -r *" "quit" | sftp -b - "$USER#$HOST"
Here's a rewrite, shortened considerably:
#!/bin/bash
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
FEED_DIR="$HOME/Dropbox/$(date +%Y%m%d)"
USER="user"
HOST="host.domain.tld"
mkdir "$FEED_DIR" || { echo "could not mkdir $FEED_DIR"; exit 1; }
cd "$FEED_DIR"
{
echo "get -r *"
echo quit
} |
sftp -b - "${USER}#${HOST}"