For the ease of use for ciphering and deciphering files and folder, and also to get some clues about when it's going to finish, I would like to set up some king of progress bar with gpg.
I'm using pv to do so, which works well for ciphering as I do not need a password.
However, I don't know how to do it for deciphering, as it may ask for the password of my private key.
Here is what I tried:
#!/bin/bash
if [ "$#" -ne 1 ] || ! [ -f "$1" ]; then
echo "Usage: $0 CIPHERED_FOLDER" >&2
exit 1
fi
echo "Deciphering $1"
pv $1 | gpg -d | tar -xz
It works well if the passphrase is already cached, but fails if it's not.
So I used a workaround, replacing the last line with gpg -d $1 | pv -W | tar -xz
However, as I can't know the exact size of the deciphered file, I can't have a percentage.
Is there a way to get this working? I tried looking at ways to manually ask in the script for the passphrase before trying to decipher to get it in the cache, but I couldn't find how to do so.
Thanks
Edit: I'm ciphering the files only for myself for security storage, and gpg doesn't ask me for a recipient anymore because I've set up default-recipient-self
Here is a better solution than my previous answer.
gpg -d <(pv $1) | tar -xz
The <() operator makes gpg read from a file descriptor created for pv while still allowing the passphrase to be read from stdin.
Here is where I took the solution from: https://serverfault.com/questions/40284/create-virtual-file-from-bash-command-output
Old answer for reference:
I managed to find a workaround.
I first sign a short text and throw the output directly in /dev/null.
This way, it makes sure that the passphrase is cached before the main deciphering.
It looks like this:
gpg -o /dev/null --sign <(echo "1234")
echo "Deciphering $1"
pv $1 | gpg -d -q | tar -xz
Related
I'm working on a script to cipher a bunch of folders through the use of tar, gzip and gpg, plus pv with du and awk to keep track of progress. Here is the line that causes problems
tar cf - "$f" | pv -s $(($(du -sk "$f" | awk '{print $1}') * 1024)) | gzip | gpg -e -o "$output/$(basename "$f").tar.gz.gpg"
This works well most of the time. However, if the output file already exists, gpg prompts the user, asking if we want to override the file or not. And in this case, when the script exits, the console kind of breaks: what I type does not appear anymore, pressing Enter does not create a new line, and so on.
The problem does not appear if the outfile does not exist yet, nor if the -s option of pv is missing or computed without du and awk (ex: $((500 * 500)). This won't break the console, but obviously the progress bar would be completely off)
The problem is reproducable even by using this command line outside of a script and replacing $f and $output with desired options.
Perhaps one or a combination of these changes will help.
Change the gpg command to write to stdout, redirected to the file you want: gpg -e -o - > "$output/$(basename "$f").tar.gz.gpg".
Calculate the file size with stat: stat -c "%s" "$f".
The whole line might then look like this:
tar cf - "$f" | pv -s $(stat -c "%s" "$f") | gzip | gpg -e -o - > "$output/$(basename "$f").tar.gz.gpg"
I prefer bat over pygmentize as bat has better language support and a better all round app in my opinion. However, one thing that I would like is the ability to retain it's syntax highlighted output even through more or grep or other programs. I guess that's because more or other apps do not support colour, but often in Linux, when something seems not possible, I find that there is some smart trick to achieve that thing, so if I pipe bat output to more or grep, is there a way to retain the colour that is part of the bat output?
e.g. bat ~/.bashrc | more
# Get latest bat release and install
bat_releases=https://github.com/sharkdp/bat/releases/
content=$(wget $bat_releases -q -O -)
firstlink=$(grep -oP 'href="/sharkdp/bat/releases/\K[^"]*_amd64\.deb' <<< "$content" | head -1)
DL=$bat_releases$firstlink
ver_and_filename=$(grep -oP 'https://github.com/sharkdp/bat/releases/download/\K[^"]*\.deb' <<< "$DL")
IFS='/' read -ra my_array <<< "$ver_and_filename"
ver=${my_array[0]}
filename=${my_array[1]}
IFS='.' read -ra my_array <<< "$filename"
extension=${my_array[-1]}
extension_with_dot="."$extension
filename_no_extension=${filename%%${extension_with_dot}*}
[ ! -f /tmp/$filename ] && exe wget -P /tmp/ $DL
sudo dpkg -i /tmp/$filename
I've been working on a series of bash scripts and I need to automate password entry for batch processing of files and actions.
This isn't for just one program, for example, sometimes it needs to be done for GPG, other times, steghide.
There is a specific reason this is being done and I understand the security elements behind it. This is considered and is negated by how the scripts are stored and working.
The passwords or passphrases are passed via command line arguments to the script and the password/phrase must be repeated many times programmatically.
Here is an example of what I am working with inside the script:
for f in $dir
do
steghide embed -cf $f -ef /path/to/secret.txt
done
This simply interactively asked this for every image however:
Enter Passphrase:
Re-enter Passphrase:
For every image in a directory, this password will be requested and so the password should be able to be stored in a variable and reused.
I have been working with steghide most recently but there will also be a need to automate passphrases with GPG at a later date, although there is no need for the methods to be the same.
man steghide:
-p, --passphrase
Use the string following this argument as the
passphrase. If your passphrase contains whitespace,
you have to enclose it in quotes, for example: -p
"a very long passphrase".
man gpg:
--passphrase string
Use string as the passphrase. This can only be used if only one
passphrase is supplied. Obviously, this is of very questionable
security on a multi-user system. Don't use this option if you can
avoid it.
It's untested publicly, rough around the edges, and can be improved... but here's a preview of some of my research scripts that haven't been merged into one of the GitHub projects I'm writing... definitely run shellcheck against the below script to catch any typos.
#/usr/bin/env bash
Var_stego_out_dir="${1}"
Var_stego_in_dir="${2}"
Var_stego_cover_dir="${3}"
Var_passphrase_file="${4}"
Var_passphrase="${5}"
Var_auto_pass_length="${6:-64}"
Func_build_array_of_paths(){
_dir="${1}"
_arr="${2}"
_file_extension_list="${3}"
if [ -d "${_dir}" ] && [ "${#${_arr}[#]}" = "0" ]; then
find "${_dir}" -xtype f | while read _path; do
case "${_path##*.}" in
${_file_extension_list//,/|})
declare -ag "${_arr}+=( ${_path} )"
;;
esac
done
fi
}
Func_enc_stego(){
_cover_file="${1}"
_enc_file="${2}"
_pass_file="${3}"
_output_file="${Var_stego_out_dir}/${_cover_file##*/}"
if [ -f "${_cover_file}" ] && [ -f "${_enc_file}" ]; then
_auto_passphrase="${Var_passphrase:-$(base64 /dev/random | tr -cd '[:print:]' head -c${Var_auto_pass_length})}"
if ! [ -f "${_output_file}" ]; then
steghide -p ${_auto_passphrase} -ef ${_enc_file} -cf ${_cover_file} -sf ${_output_file}
cat <<<"### ${_output_file} ### ${_auto_passphrase}" >> "${_pass_file}"
else
steghide -p ${_auto_passphrase} -ef ${_enc_file} -cf ${_cover_file} -sf ${_output_file}_$(date -u +%s)
cat <<<"### ${_output_file}_$(date -u +%s) ### ${_auto_passphrase}" >> "${_pass_file}"
fi
fi
}
Func_main(){
## Build array of file paths for cover file use
Func_build_array_of_paths "${Var_stego_cover_dir}" "Arr_cover_list" "au,AU,bmp,BMP,jpeg,JPEG,wav,WAV"
## Build array of file paths for embed file use
Func_build_array_of_paths "${Var_stego_in_dir}" "Arr_input_list" "gpg,GPG,txt,TXT"
let _arr_input_count=0
let _arr_cover_count=0
until [ "${_arr_input_count}" = "${#Arr_input_list}" ]; do
if [ -f "${Arr_cover_list[${_arr_cover_count}]}" ]; then
Func_enc_stego "${Arr_cover_list[${_arr_cover_count}]}" "${Arr_input_list[${_arr_input_count}]}" "${Var_passphrase_file}"
let _arr_cover_count++
let _arr_input_count++
elif [ -f "${Arr_cover_list[$((${_arr_cover_count}-1))]}" ]; then
Func_enc_stego "${Arr_cover_list[$((${_arr_cover_count}-1))]}" "${Arr_input_list[${_arr_input_count}]}" "${Var_passphrase_file}"
let _arr_input_count++
_arr_cover_count="$((${_arr_cover_count}-1))"
if
done
}
Func_main
Run above with the following portions filled-in
script.sh "/path/to/stego_out_dir" "/path/to/stego_in_dir" "/path/to/stego_cover_dir" "/path/to/passphrase_file"
## or define static passphrase
#script.sh "/path/to/stego_out_dir" "/path/to/stego_in_dir" "/path/to/stego_cover_dir" "/path/to/passphrase_file" "passphrase"
Note saving the passphrase and file in plain-text like the above does is very bad practice, and because the OP stated that they also where looking at GnuPG automation too, readers and the OP"s author should look-up Perinoid_Pipes; and for specifically the GnuPG_Gen_Key.sh script and functions starting with Func_dec_* within the Paranoid_Pipes.sh for working/tested examples of automation involving GnuPG passphrases; and for protecting the passphrases written by the above script look-up functions starting with Func_enc_* within the Paranoid_Pipes.sh script for how the mkfifo command and resulting named pipe is used to automate encryption of most data types. Hint the fourth example argument "/path/to/passphrase_file" would point to an encrypting named pipe made by the linked script to keep things a bit more secure ;-)
This script does not function properly when launched graphically (by double clicking script icon and selecting run), however, runs just fine if called from the terminal; will not save a file or load contents from an existing file. Please help! Thank you.
#!/bin/bash
# This script provides a simple and secure messaging system for users not
# familiar with gpg or using the terminal. The idea is to keep sensitive
# plaintext files off one's computer.
zenity --question --text="Select operation:" --ok-label="Compose" --cancel-label="Read"
if [[ $? == 0 ]]; then
usr=$(zenity --entry --text="Sender Key ID:")
rec=$(zenity --entry --text="Recipient Key ID:")
pwd=$(zenity --password)
outfile=$(zenity --file-selection --save --confirm-overwrite)
zenity --text-info --editable | gpg -aseu $usr -r $rec --passphrase $pwd --cipher-algo AES256 -o $outfile
else
infile=$(zenity --file-selection)
pwd=$(zenity --password)
gpg -d --passphrase $pwd $infile | zenity --text-info --height=600 --width=800
fi
A probable cause for the error is that you have different environments when executing via an interactive shell (thus sourcing your .bashrc) and double-clicking (non-interactive, and not sourcing .bashrc
You can compare the environments by doing an env > from_terminal vs. env > double_click and then using diff or something similar.
You could also (after doing the above) source from_terminal in your script to see if it works with the terminal environment. As stated in one of the comments, set -vx is your friend.
I want to create a shellscript that reads files from a .diz file, where information about various source files are stored, that are needed to compile a certain piece of software (imagemagick in this case). i am using Mac OSX Leopard 10.5 for this examples.
Basically i want to have an easy way to maintain these .diz files that hold the information for up-to-date source packages. i would just need to update these .diz files with urls, version information and file checksums.
Example line:
libpng:1.2.42:libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks:http://downloads.sourceforge.net/project/libpng/00-libpng-stable/1.2.42/libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks:9a5cbe9798927fdf528f3186a8840ebe
script part:
while IFS=: read app version file url md5
do
echo "Downloading $app Version: $version"
curl -L -v -O $url 2>> logfile.txt
$calculated_md5=`/sbin/md5 $file | /usr/bin/cut -f 2 -d "="`
echo $calculated_md5
done < "files.diz"
Actually I have more than just one question concerning this.
how to calculate and compare the checksums the best? i wanted to store md5 checksums in the .diz file and compare it with string comparison with "cut"ting out the string
is there a way to tell curl another filename to save to? (in my case the filename gets ugly libpng-1.2.42.tar.bz2?use_mirror=biznetnetworks)
i seem to have issues with the backticks that should direct the output of the piped md5 and cut into the variable $calculated_md5. is the syntax wrong?
Thanks!
The following is a practical one-liner:
curl -s -L <url> | tee <destination-file> |
sha256sum -c <(echo "a748a107dd0c6146e7f8a40f9d0fde29e19b3e8234d2de7e522a1fea15048e70 -") ||
rm -f <destination-file>
wrapping it up in a function taking 3 arguments:
- the url
- the destination
- the sha256
download() {
curl -s -L $1 | tee $2 | sha256sum -c <(echo "$3 -") || rm -f $2
}
while IFS=: read app version file url md5
do
echo "Downloading $app Version: $version"
#use -o for output file. define $outputfile yourself
curl -L -v $url -o $outputfile 2>> logfile.txt
# use $(..) instead of backticks.
calculated_md5=$(/sbin/md5 "$file" | /usr/bin/cut -f 2 -d "=")
# compare md5
case "$calculated_md5" in
"$md5" )
echo "md5 ok"
echo "do something else here";;
esac
done < "files.diz"
My curl has a -o (--output) option to specify an output file. There's also a problem with your assignment to $calculated_md5. It shouldn't have the dollar sign at the front when you assign to it. I don't have /sbin/md5 here so I can't comment on that. What I do have is md5sum. If you have it too, you might consider it as an alternative. In particular, it has a --check option that works from a file listing of md5sums that might be handy for your situation. HTH.