Unrar and Progress Bar Bash - bash

I'm trying to extract a RAR file in a destination directory with the unrar command, but I don't want to display the normal file list that unrar shows when executing. What I want to do instead, is show a progress bar that shows the process in percentages using a dialog --gauge, but so far I only get the error
syntax error: unexpected end of file
Here's the code I'm using:
#!/bin/bash
dest="testing"
file"example.rar"
cmd="$(unrar x "$file" "$dest")" &>/dev/null
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
(
for f in "$cmd"; do
pct="$(( 100*(++i)/fc ))" # calculate percentage
cat <<EOF
XXX
$pct
Extracting file "$f"...
XXX
EOF
done
) | dialog --title "Extracting Files" --gauge "Please wait" 7 70 0
So far, the code extracts the files in the dest folder, but they don't show the dialog box, and show the error unexpected end of file after completing the file extraction. It doesn't show any files being extracted which is to be expected by using &>/dev/null but I cannot get the dialog box to display properly.
I got this code from various examples I found online, but so far I cannot make it work. Can someone point me in the right direction?

Well, since no one has answered my question, I'm gonna answer it myself. I Finally managed to make the program work, it extracts the files, prints the pathnames of the files being extracted, and display the gauge bar filling correctly as each file is extracted. Here is the code:
#!/bin/bash
dest="testing" # destination directory
file"example.rar"
(
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
unrar x "$file" "$dest" | while IFS="" read -r in ; do
# extract the pathname of file being extracted
f=$(echo $in | cut -d ' ' -f 2)
echo "XXX"
echo "$f..." # Display file name in the dialog box
echo "XXX"
echo "$pct %" # Show percent
pct="$(( 100*(++i)/fc ))" # Calculate current percent
done
) | dialog --title "Extracting Files" --gauge "Please wait" 8 75 0
This next part is supposed to capture the result of executing the extraction of the .RAR file, however I'm unsure if at this point I'm just capturing the result of the execution of the dialog box or that of the unrar command. What I want to do is control the flow the script in case the unrar command fails, I'm not entirely sure of how I should proceed, but right now, after the line where I display the dialog, I put the following if-then:
if [ "$?" ]; then
clear
echo "Command successful"
fi
Is this correct or should I move the if-then block to somewhere else in the code?
In the meantime, Imma work in a way to make this script work like a kind of prettied up unrar command, where you can specify the file name to be extracted and an optional destination directory, that way I'll be able to eliminate the need to specify a destination directory
Edit:
This is the current code I'm using for the solution of my own question. So far the code seems to be handling errors just fine. I have tested it only with the "File already exist" kind of errors that halts the execution of the script to wait for input from the user, and CRC errors for the cases where you have partitioned the compressed files in various parts and one of them is missing.
#!/bin/bash
dest="testing" # destination directory
file="example.rar"
(
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
# I have to redirect the output from 0 to 1 because when there's a
# "file already exists" kind of error, unrar halts the execution of the
# script to wait for user input, so I have to process that message inside
# of the loop
unrar x "$file" "$dest" 0>&1 2>&1 | while IFS="" read -r in ; do
# we got a "file already exists" message?
if [[ "$f" == *"already"* ]]; then
echo "Error" # display an error message
break # exit the while loop
fi
# extract the pathname of file being extracted
f=$(echo $in | cut -d ' ' -f 2)
echo "XXX"
echo "$f..." # Display file name in the dialog box
echo "XXX"
echo "$pct %" # Show percent
pct="$(( 100*(++i)/fc ))" # Calculate current percent
done
exit ${PIPESTATUS[0]} # exit with status message we got from unrar
) | dialog --title "Extracting Files" --gauge "Please wait" 8 75 0
# if PIPESTATUS[0] is not equal to zero, it means there was an error
if [[ "${PIPESTATUS[0]}" -ne 0 ]]; then
echo "There was an error." # display a message
exit 1 # exit with status 1
fi
# if command was successful, display a message
if (( "$?" == 0 )); then
echo "Command executed correctly $?"
fi
So far, I'm not doing anything with the errors, I'm just interested in detecting them and returning messages to the scripts that call this one. It seems to be working just fine, but I know there has to be a better way to do this. What are your thoughts?

Related

Unable to break out of While loop

I would like to break out of a loop when it gets to a blank line in a file. The issue is that my regexp's used to condition my data create a line with characters so I need something from the beginning to check if a line is empty or not so I can break out. What am I missing?
#!/bin/bash
#NOTES: chmod this script with chmod 755 to run as regular local user
#This line allows for passing in a source file as an argument to the script (i.e: ./script.sh source_file.txt)
input_file="$1"
#This creates the folder structure used to mount the SMB Share and copy the assets over to the local machines
SOURCE_FILES_ROOT_DIR="${HOME}/operations/source"
DESTINATION_FILES_ROOT_DIR="${HOME}/operations/copied_files"
#This creates the fileshare mount point and place to copy files over to on the local machine.
echo "Creating initial folders..."
mkdir -p "${SOURCE_FILES_ROOT_DIR}"
mkdir -p "${DESTINATION_FILES_ROOT_DIR}"
echo "Folders Created! Destination files will be copied to ${DESTINATION_FILES_ROOT_DIR}/SHARE_NAME"
while read -r line;
do
if [ -n "$line" ]; then
continue
fi
line=${line/\\\\///}
line=${line//\\//}
line=${line%%\"*\"}
SERVER_NAME=$(echo "$line" | cut -d / -f 4);
SHARE_NAME=$(echo "$line" | cut -d / -f 5);
ASSET_LOC=$(echo "$line" | cut -d / -f 6-);
SMB_MOUNT_PATH="//$(whoami)#${SERVER_NAME}/${SHARE_NAME}";
if df -h | grep -q "${SMB_MOUNT_PATH}"; then
echo "${SHARE_NAME} is already mounted. Copying files..."
else
echo "Mounting it"
mount_smbfs "${SMB_MOUNT_PATH}" "${SOURCE_FILES_ROOT_DIR}"
fi
cp -a ${SOURCE_FILES_ROOT_DIR}/${ASSET_LOC} ${DESTINATION_FILES_ROOT_DIR}
done < $input_file
# cleanup
hdiutil unmount ${SOURCE_FILES_ROOT_DIR}
exit 0
Expected result was for the script to realize when it gets to a blank line and then stops. The script works works when i remove the
if [ -n "$line" ]; then
continue
fi
The script runs and pulls assets but just keeps on going and never breaks out. When I do it as is now I get :
Creating initial folders...
Folders Created! Destination files will be copied to /Users/baguiar/operations/copied_files
Mounting it
mount_smbfs: server connection failed: No route to host
hdiutil: unmount: "/Users/baguiar/operations/source" failed to unmount due to error 16.
hdiutil: unmount failed - Resource busy
cat test.txt
This is some file
There are lines in it
And empty lines
etc
while read -r line; do
if [[ -n "$line" ]]; then
continue
fi
echo "$line"
done < "test.txt"
will print out
That's because -n matches strings that are not null, i.e., non-empty.
It sounds like you have a misunderstanding of what continue means. It does not mean "continue on in this step of the loop", it means "continue to the next step of the loop", i.e., go to the top of the while loop and run it starting with the next line in the file.
Right now, your script says "go line by line, and if the line is not empty, skip the rest of the processing". I think your goal is actually "go line by line, and if the line is empty, skip the rest of the processing". This would be achieved by if [[ -z "$line" ]]; then continue; fi
TL;DR You are skipping all the non-empty lines. Use -z to check if your variable is empty instead of -n.

creating intrusion detection system with shell script

i have UNI work where i have to create a shell script that monitors a directory with a few text files that can inform you (can be manual check to be informed) if they have been modified. honestly im not sure where to start and im bad at linux and cant find anything to help. i can only use standard tools in ubuntu. any help would be great. thankyou
update - this is what i have so far and i need a way to verify that the values printed are the same after altering a file (if they are not the same then print whats files have been changed)
also sorry first time using the site trying to learn..
#!/bin/sh
echo "press 1 to check - press 2 to exit"
while :
do
read INPUT_STR
case $INPUT_STR in
1)
echo "checking sums"
md5sum Target/bob
md5sum Target/bec
md5sum Target/john
md5sum Target/mary
md5sum Target/mike
;;
2)
break
;;
*)
echo "incorrect input"
esac
done
echo "thankyou for using IDS"
You mean something like that?
#!/bin/bash
WORKDIR=/home
TARGETS=(
"$WORKDIR/bob"
"$WORKDIR/bec"
"$WORKDIR/john"
"$WORKDIR/mary"
"$WORKDIR/mike"
)
for target in "${TARGETS[#]}"; do
md5file="${target##*/}.md5"
if [[ -e "$md5file" ]]; then
md5sum --quiet -c "$md5file"
else
echo "create md5sum referenz file $md5file"
md5sum "$target"/* > "$md5file"
fi
done
At first run it creates for every directory a reference file. On the second run the directory will be compare with reference file. Modifications will be shown. When you delete one reference file, it will be created again on the next run.
explanation
# loop over the array with the targets
for target in "${TARGETS[#]}"; do
# remove all directories except the last, append .md5
# and assign it to the variable md5file
md5file="${target##*/}.md5"
# if md5file exists, use it as reference and check if
# modifications where happend
if [[ -e "$md5file" ]]; then
# use the md5file as reference and only show
# modified files, ignore ok messages
md5sum --quiet -c "$md5file"
else
# when no md5file exists
echo "create md5sum referenz file $md5file"
# use target, append /* to it and run md5sum
# redirect output to md5file (reference file for next run)
md5sum "$target"/* > "$md5file"
fi
done

How to delay a command substitution in a shell script?

I have a bash script that in a nutshell generates a file via other command (works fine) and then later in the script I do a wc command on the file generated. My problem is that i'm using command substitution on the wc command and when I execute the script it is executing this substitution immediately rather than waiting for the file to be generated earlier in the script. What are my options?
The script is a shell program to run Oracle SQLLoader to load data into an Oracle Table. The SQLLoader command generates a file with rejected records and I am trying to do a word count on it. This is my code:
#!/bin/sh
# Set variables
program_name="My Program Name"
input_dir="/interface/inbound/hr_datafile"
log_dir="/interface/inbound/log"
bad_dir="/interface/inbound/hr_bad"
archive_dir="/interface/inbound/hr_archive"
input_base="Import_20171213"
input_ext="csv"
control_file="data_loader.ctl"
exit_status=0
d=`date "+%Y%m%d"`
t=`date "+%H%M"`
# Check if data file exists, count records
if [ -f ${input_dir}/${input_base}*.${input_ext} ]; then
data_file_name=`ls -1rt ${input_dir}/${input_base}*.${input_ext} | head -1 | cut -c 32-100`
data_file_base=${data_file_name%.*}
echo "Data file name: " ${data_file_name}
echo "Data file base: " ${data_file_base}
no_of_recs=`expr $(wc -l ${input_dir}/${data_file_name} | cut -c 1-8) - 1`
echo "DEBUG no_of_recs: " ${no_of_recs}
no_of_errs=0
else
echo
echo
echo
echo "----------------------------- ${program_name} ------------------------------------------"
echo "----------------------------------- Error report : ------------------------------------------------"
echo
echo "Please place your Data files in the UNIX directory => "${input_dir}
echo "${program_name} Process exiting ..."
echo
echo "---------------------------------------------------------------------------------------------------"
echo
echo
echo
echo
echo
exit 1
fi
# Run SQL*Loader
echo
echo "==> Loading Data...into MY_TABLE table"
echo
# Create a temporary control file to pass the data file name in
cat $XX_TOP/bin/${control_file} | sed "s/:FILENAME/${data_file_name}/g" > $XX_TOP/bin/${data_file_base}_temp.ctl
# NOTE: The following sqlldr format is used to "hide" the oracle user name and password
sqlldr errors=100000 skip=1 control=$XX_TOP/bin/${data_file_base}_temp.ctl data=${input_dir}/${data_file_name} log=${log_dir}/${data_file_base}.log bad=${bad_dir}/${data_file_base}.bad <<-END_SQLLDR > /dev/null
apps/`cat ${DB_SCRIPTS}/.${ORACLE_SID}apps`
END_SQLLDR
exit_status=$?
echo "DEBUG exit_status " ${exit_status}
# Remove temporary control file
rm -rf $XX_TOP/bin/${data_file_base}_temp.ctl
# Check for Errors
if [ -f ${bad_dir}/${data_file_base}.bad ]; then
echo
echo "----------------------------- ${program_name} ------------------------------------------"
echo "----------------------------------- Error report : ------------------------------------------------"
echo
grep 'Record' ${log_dir}/${data_file_base}.log > ${log_dir}/${data_file_base}.rec
grep 'ORA' ${log_dir}/${data_file_base}.log > ${log_dir}/${data_file_base}.err
paste ${log_dir}/${data_file_base}.rec ${log_dir}/${data_file_base}.err ${bad_dir}/${data_file_base}.bad
echo
echo "<---------------------------------End of Error Report---------------------------------------------->"
echo
# Count error records
no_of_errs=$(wc -l ${bad_dir}/${data_file_base}.bad | cut -c 1-8)
no_of_recs=$(expr ${no_of_recs} - ${no_of_errs})
# Remove temp files
rm ${log_dir}/${data_file_base}.rec
rm ${log_dir}/${data_file_base}.err
rm ${bad_dir}/${data_file_base}.bad
else
echo "Bad File not found at ${bad_dir}/${data_file_base}.bad"
fi
if (( ${no_of_errs} > 0 )); then
echo "Error found in data file..."
exit_status=1
else
# Archive the data file if no errors
mv ${input_dir}/${data_file_name} ${archive_dir}/${data_file_base}_$d"_"$t.${input_ext}
echo "Data file archived to ${archive_dir}"
exit_status=0
fi
echo
echo
echo
echo "----------------------------- ${program_name} ------------------------------------------"
echo
echo "Total records errored out :" ${no_of_errs}
echo "Total records successfully read :" ${no_of_recs}
echo "---------------------------------------------------------------------------------------------------"
echo
# Final Exit Status
if [ ${exit_status} -eq 1 ]; then
echo "==> Exiting process...Status : ${exit_status}"
exit 1
fi
The file referenced in the if condition check is the file generated, so I'm checking if the file exists and then running the wc on it. I know that it's executing prematurely because this wc error appears in my script output before it should:
Data file name: Import_20171213.csv
Data file base: Import_20171213
DEBUG no_of_recs: 27
==> Loading Data...into MY_TABLE table
wc: 0653-755 Cannot open /interface/inbound/hr_bad/Import_20171213.bad.
Username:
SQL*Loader: Release 10.1.0.5.0 - Production on Thu Dec 21 12:42:39 2017
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 27
Program exited with status 2
In the code, the wc command on the .bad file is performed after the sqlldr section, yet the log shows the error occurring before sqlldr is invoked.
Any ideas would be much appreciated! Thanks!
You aren't seeing all of the output you expect - anything after the call to SQL*Loader is missing - and the error from wc is coming in the wrong place, before instead of after the Username; prompt. But it shouldn't be erroring giving the construct you've used - if the file doesn't exist the if test should stop that line being reach.
The issue is that it isn't seeing the end of the heredoc. All of the commands beyond the start of the heredoc are being executed as part of the heredoc processing but aren't going anywhere or being displayed to the terminal (as expected), and something in the way that is all being evaluated is causing the wc to run unexpectedly, even though the file doens't exist. You're seeing the stderr output from that.
And that is all happening before SQL*Loader starts, so you see the Username: prompt from that afterwards.
From comments it seems the heredoc-ending END_SQLLDR is really indented in your actual code, which isn't reflected in the posted question. As you used the <<- heredoc form that implies it's indented with spaces rather than tabs. That is causing that to not be recognised as the end of the here doc.
From The Linux Documentation Project:
The closing limit string, on the final line of a here document, must start in the first character position. There can be no leading whitespace. Trailing whitespace after the limit string likewise causes unexpected behavior. The whitespace prevents the limit string from being recognized.
Removing the whitespace so that is the first thing on the line will fix it.

Make script that reads argument from command line

I am running quantum chemical calculations by providing the command molcas -f file.input. I now have need for putting the molcas -f into a script that also tails the last 100 lines of the generated file.log, for me to quickly confirm that everything finished the way it's supposed to. So I want to run the script run.sh:
#!/bin/bash
molcas -f [here read the file.input from command line]
tail -100 [here read the file.log]
The question is how I can make the script read the argument I give, and then find on its own the output file (which has the same filename, but with a different extension).
Follow-up
Say I have a bunch of numbered files file-1, file-2, ..., file-n. I would save time if I instead of running
./run.sh file-1.input file-1.log
I run
./run.sh n n
or
./run.sh n.input n.log
assuming that the actual filename and placement of the number n is given in the script. Can that be done?
With this code:
#!/bin/bash
molcas -f "$1"
tail -100 "$2"
You will need to execute the script run.sh as follows:
./run.sh file.input file.log
to be hornest I have/had no clue over molcas, so I jumed to this side to get basic understandings.
The syntax shoould look like this ...
#!/bin/bash
# waiting for input
read -p "Enter a filename (sample.txt): " FILE
# checking for existing file
if [ -e "$FILE" ]; then
read -p "Enter a command for moculas: " CMD
else
echo "Sorry, \"${FILE}\" was not found. Exit prgramm."
exit 1
fi
# I am not sure how this command works.
# maybe you have to edit this line by your self.
molcas $FILE -f "$CMD"
# checking for programm-errors
ERRNO=$?
if [ "$ERRNO" != "" ] && [ "$ERRNO" -gt 0 ]; then
echo "Molcas returned an error. Number: ${ERRNO}"
exit 1
fi
# cuts off the fileending (For example: sample.txt gets sample)
FILENAME="${FILE%.*}"
# checking other files
ERRFILE="${FILENAME}.err"
tail -n 100 $ERRFILE
LOGFILE="${FILENAME}.log"
tail -n 100 $LOGFILE
exit 0
I would have posted more, but its not clear what to do with this data.
Hope this helps a bit.

Monitor folders using unix bash

I need a shell script that will monitor all the folders given in the command
and will notify the user if a certain file will be created inside them (the name of
the file will be read from keyboard).
I am allowed to use simple commands, so not inotify...
this is what i managed to do so far:
#!/bin/bash
echo "Please enter a file you want to monitor: "
read file_monitor
#this is an infinite while
while [ 1 ] ; do
#using test -e to search for the file
test -e $file_monitor && echo "The file has been created!"
sleep 5
done
I have to find a way to stop the while when the file has been created, and also to search for the file in the folders given in the command line. Can someone help me, please?
To exit the loop, use break:
test -e $file_monitor && echo "The file has been created!" && break
I would prefer to break first, and echo after the loop, or as #mkemp6 suggested, directly use the test as the condition for the loop.
To check the folders, simply loop through them, and check the file in each one.
break [n]
Exit from within a for, while, until, or select loop. If n is specified, break n
levels. n must be ≥ 1. If n is greater than the number of enclosing loops, all
enclosing loops are exited.
while ! test -e "$file_monitor"; do sleep 5; done
But you are much better off using something like inotify to monitor the appropriate directories.
#!/bin/bash
arr=$#
for i in $arr; do
if [ ! -d $i ]
then echo "The parameter $i is no a directory!"
exit 1
fi
done
echo -n "Please give file you want to monitor: "
read file_monitor
a=1
while [ $a -eq 1 ]; do
for i in $arr; do
cd $i
test -e $file_monitor && echo "The file has been created" && a=$((a+1))
cd ..
done
done
So this is what I have managed to do.

Resources