creating intrusion detection system with shell script - shell

i have UNI work where i have to create a shell script that monitors a directory with a few text files that can inform you (can be manual check to be informed) if they have been modified. honestly im not sure where to start and im bad at linux and cant find anything to help. i can only use standard tools in ubuntu. any help would be great. thankyou
update - this is what i have so far and i need a way to verify that the values printed are the same after altering a file (if they are not the same then print whats files have been changed)
also sorry first time using the site trying to learn..
#!/bin/sh
echo "press 1 to check - press 2 to exit"
while :
do
read INPUT_STR
case $INPUT_STR in
1)
echo "checking sums"
md5sum Target/bob
md5sum Target/bec
md5sum Target/john
md5sum Target/mary
md5sum Target/mike
;;
2)
break
;;
*)
echo "incorrect input"
esac
done
echo "thankyou for using IDS"

You mean something like that?
#!/bin/bash
WORKDIR=/home
TARGETS=(
"$WORKDIR/bob"
"$WORKDIR/bec"
"$WORKDIR/john"
"$WORKDIR/mary"
"$WORKDIR/mike"
)
for target in "${TARGETS[#]}"; do
md5file="${target##*/}.md5"
if [[ -e "$md5file" ]]; then
md5sum --quiet -c "$md5file"
else
echo "create md5sum referenz file $md5file"
md5sum "$target"/* > "$md5file"
fi
done
At first run it creates for every directory a reference file. On the second run the directory will be compare with reference file. Modifications will be shown. When you delete one reference file, it will be created again on the next run.
explanation
# loop over the array with the targets
for target in "${TARGETS[#]}"; do
# remove all directories except the last, append .md5
# and assign it to the variable md5file
md5file="${target##*/}.md5"
# if md5file exists, use it as reference and check if
# modifications where happend
if [[ -e "$md5file" ]]; then
# use the md5file as reference and only show
# modified files, ignore ok messages
md5sum --quiet -c "$md5file"
else
# when no md5file exists
echo "create md5sum referenz file $md5file"
# use target, append /* to it and run md5sum
# redirect output to md5file (reference file for next run)
md5sum "$target"/* > "$md5file"
fi
done

Related

How to access a target directory using bash scripting

I am relatively new to shell scripting. I am writing a script to compress all the files in current and target directory. I have found success in compressing the files of a current directory but I'm unable to write a script for compressing files in a target directory can anyone guide me?
I want to do something like this
% myCompress -t /home/users/bigFoot/ pdf ppt jpg
next time try to spread your code (it will make it easier to answer):
#!/bin/bash
if [[ $# == 0 ]]; then
echo "This shell script compress files with a specific extensions"
echo "Call Syntax: compress <extension_list>"
exit
fi
for ext in $*; do
for file in ls *.$ext; do
gzip -k $file
done
done
Mistakes made
1) $* - all args coming after command - so.... -t and path are not $ext variables
2) ls *.$ext is red in loop as 2 strings "ls and *.$ext" should be written as $(ls *.$ext) to get ls command executed
My script for your request
#!/bin/bash
script_name=`basename "$0"`
if [[ $# == 0 ]]; then
echo "This shell script compress files with a specific extensions"
echo "Call Syntax: $script_name <dirctories_list> <extension_list>"
exit
fi
# check if $1 is a directory
path=". "
file_type=""
for check_type in $* ; do
if [[ -d $check_type ]]; then
path=$path$check_type" "
else
file_type=$file_type"*."$check_type" "
fi
done
echo paths to gzip $path
echo files type to check "$file_type"
for x in $path; do
cd $x
for file in $(ls $file_type); do
gzip $file
done
cd -
done
Explanation
1) basename "$0" - get scripts name - it is more generic for usage - in case you change script's name
2) path=". " - variable hold a string of all directories to be compressed, your request is to run it also on current directory ". "
file_type="" - variable hold a string of all extensions to be compressed in $path string
3) running a loop on all input ARGS and concatenate directories names to $path string and other file types to $file_type
4) for each of the directories inserted to script:
i. cd $x - enter directorie
ii. gzip - compress all files with inserted extensions
iii. cd - - go back to base directories
Check gzip
I'm not familiar with the gzip command , check that you have -k flag

bash test for existence of files on SFTP server

The issue that I have is with the line: "does not work" - below. The last line does indeed work - but I need to understand why the second to last line does not. I need to check for file existence on the remote server.
Have a need to check for existence for files at the following location:
/home/remoteuser/files/
and when the files are processed, they are moved to:
/home/remoteuser/logs/archive/
Would like to create an alert if the files exist at - in other words, the files were not processed:
/home/remoteuser/logs/
Found the following page and seems to be what I am looking for:
http://www.cyberciti.biz/tips/find-out-if-file-exists-with-conditional-expressions.html
Testing this and I know there are files there, but does not work:
ssh remoteuser#1.2.3.4 [ ! -f /home/remoteuser/logs/archive/*.* ] && echo "File does not exist in the root" >> /home/localuser/files/dirlist.txt
Because we know this works and does indeed list files on the local server:
ssh remoteuser#1.2.3.4 ls -l /home/remoteuser/logs/archive/*.* >> /home/localuser/files/dirlist.txt
Wildcards and test construct in Bash
You cannot use the wildcards in the [ command to test the existence of multiple files. In fact, the wildcards will be expanded and all the files will be passed to the test. Te results is that it would complain that "-f" is given too many arguments.
Try this in any non empty directory to see the output:
[ ! -f *.* ]
The only situation in which the above command does not fail is when there is only one file matching the expression, in your case a non hidden file of the form "*.*" in /home/remoteuser/logs/archive/
Using Find
A possible solution is to use find in combination with grep:
ssh name#server find /path/to/the/files -type f -name "\*.\*" 2>/dev/null | grep -q . && echo "Not Empty" || echo "Empty"
find search for regular files (-type f) whose names are in the form . (-name) and return false if nothing is found, then "grep -q ." return 1 or 0 if something is found or not.
Your goal can be accomplished with only shell builtins -- and without any uses of those builtins which depend on their behavior when passed invalid syntax (as the [ ! -e *.* ] approach does). This removes the dependency on having an accessible, working find command on your remote system.
Consider:
rmtfunc() {
set -- /home/remoteuser/logs/*.* # put contents of directory into $# array
for arg; do # ...for each item in that array...
[ -f "$arg" ] && exit 0 # ...if it's a file that exists, success
done
exit 1 # if nothing matched above, failure
}
# emit text that defines that function into the ssh command, then run same
if ssh remoteuser#host "$(declare -f rmtfunc); rmtfunc"; then
echo "Found remote logfiles"
else
echo "No remote logfiles exist"
fi
ANSWER:
Did find the following about the use of -e for a regular file.
http://www.cyberciti.biz/faq/unix-linux-test-existence-of-file-in-bash/
Even though it says "too many arguments" it does seem to test out OK.
ssh remoteuser#1.2.3.4 [ ! -e /home/remoteuser/logs/archive/*.zip ] && echo "File does not exists in the root" >> /home/localuser/files/dirlist.txt || echo "File does exists in the root" >> /home/localuser/files/dirlist.txt
Your script will work simply using double parenthesis:
ssh remoteuser#1.2.3.4 [[ ! -f /home/remoteuser/logs/archive/*.* ]] && echo "File does not exist in the root" >> /home/localuser/files/dirlist.txt
From man bash
Word splitting and pathname expansion are not performed on the words between the [[ and ]].

Unrar and Progress Bar Bash

I'm trying to extract a RAR file in a destination directory with the unrar command, but I don't want to display the normal file list that unrar shows when executing. What I want to do instead, is show a progress bar that shows the process in percentages using a dialog --gauge, but so far I only get the error
syntax error: unexpected end of file
Here's the code I'm using:
#!/bin/bash
dest="testing"
file"example.rar"
cmd="$(unrar x "$file" "$dest")" &>/dev/null
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
(
for f in "$cmd"; do
pct="$(( 100*(++i)/fc ))" # calculate percentage
cat <<EOF
XXX
$pct
Extracting file "$f"...
XXX
EOF
done
) | dialog --title "Extracting Files" --gauge "Please wait" 7 70 0
So far, the code extracts the files in the dest folder, but they don't show the dialog box, and show the error unexpected end of file after completing the file extraction. It doesn't show any files being extracted which is to be expected by using &>/dev/null but I cannot get the dialog box to display properly.
I got this code from various examples I found online, but so far I cannot make it work. Can someone point me in the right direction?
Well, since no one has answered my question, I'm gonna answer it myself. I Finally managed to make the program work, it extracts the files, prints the pathnames of the files being extracted, and display the gauge bar filling correctly as each file is extracted. Here is the code:
#!/bin/bash
dest="testing" # destination directory
file"example.rar"
(
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
unrar x "$file" "$dest" | while IFS="" read -r in ; do
# extract the pathname of file being extracted
f=$(echo $in | cut -d ' ' -f 2)
echo "XXX"
echo "$f..." # Display file name in the dialog box
echo "XXX"
echo "$pct %" # Show percent
pct="$(( 100*(++i)/fc ))" # Calculate current percent
done
) | dialog --title "Extracting Files" --gauge "Please wait" 8 75 0
This next part is supposed to capture the result of executing the extraction of the .RAR file, however I'm unsure if at this point I'm just capturing the result of the execution of the dialog box or that of the unrar command. What I want to do is control the flow the script in case the unrar command fails, I'm not entirely sure of how I should proceed, but right now, after the line where I display the dialog, I put the following if-then:
if [ "$?" ]; then
clear
echo "Command successful"
fi
Is this correct or should I move the if-then block to somewhere else in the code?
In the meantime, Imma work in a way to make this script work like a kind of prettied up unrar command, where you can specify the file name to be extracted and an optional destination directory, that way I'll be able to eliminate the need to specify a destination directory
Edit:
This is the current code I'm using for the solution of my own question. So far the code seems to be handling errors just fine. I have tested it only with the "File already exist" kind of errors that halts the execution of the script to wait for input from the user, and CRC errors for the cases where you have partitioned the compressed files in various parts and one of them is missing.
#!/bin/bash
dest="testing" # destination directory
file="example.rar"
(
fc="$(unrar l "$file" | wc -l)" # get file count inside RAR file
fc="$((fc-10))" # file count has 10 extra lines, remove them
i=0 # index counter
# I have to redirect the output from 0 to 1 because when there's a
# "file already exists" kind of error, unrar halts the execution of the
# script to wait for user input, so I have to process that message inside
# of the loop
unrar x "$file" "$dest" 0>&1 2>&1 | while IFS="" read -r in ; do
# we got a "file already exists" message?
if [[ "$f" == *"already"* ]]; then
echo "Error" # display an error message
break # exit the while loop
fi
# extract the pathname of file being extracted
f=$(echo $in | cut -d ' ' -f 2)
echo "XXX"
echo "$f..." # Display file name in the dialog box
echo "XXX"
echo "$pct %" # Show percent
pct="$(( 100*(++i)/fc ))" # Calculate current percent
done
exit ${PIPESTATUS[0]} # exit with status message we got from unrar
) | dialog --title "Extracting Files" --gauge "Please wait" 8 75 0
# if PIPESTATUS[0] is not equal to zero, it means there was an error
if [[ "${PIPESTATUS[0]}" -ne 0 ]]; then
echo "There was an error." # display a message
exit 1 # exit with status 1
fi
# if command was successful, display a message
if (( "$?" == 0 )); then
echo "Command executed correctly $?"
fi
So far, I'm not doing anything with the errors, I'm just interested in detecting them and returning messages to the scripts that call this one. It seems to be working just fine, but I know there has to be a better way to do this. What are your thoughts?

Bash script with a loop not executing utility that has paramters passed in?

Anyone able to help me out? I have a shell script I am working on but for the loop below the command after "echo "first file is $firstbd" is not being executed.. the $PROBIN/proutil ?? Not sure why this is...
Basically I have a list of files in a directory (*.list), I grab them and read the first line and pass it as a parameter to the cmdlet then move the .list and the content of the .list to another directory (the .list has a list of files with full path).
for i in $(ls $STAGEDIR/*.list); do
echo "Working with $i"
# grab first .bd file
firstbd=`head -1 $i`
echo "First file is $firstbd"
$PROBIN/proutil $DBENV/$DBNAME -C load $firstbd tenant $TENANT -dumplist $STAGEDIR/$i.list >> $WRKDIR/$i.load.log
#move the list and its content to finished folder
binlist=`cat $i`
for movethis in $binlist; do
echo "Moving file $movethis to $STAGEDIR/finished"
mv $movethis $STAGEDIR/finished/
done
echo "Finished working with list $i"
echo "Moving it to $STAGEDIR/finished"
mv $i $STAGEDIR/finished/
done
The error I was getting is..
./tableload.sh: line 107: /usr4/dlc/bin/proutil /usr4/testdbs/xxxx2 -C load /usr4/dumpdir/xxxxx.bd tenant xxxxx -dumplist /usr4/dumpdir/PUB.xxxxx.list >> /usr4/dumpdir/PUB.xxxx.list.load.log: A file or directory in the path name does not exist... however if I run "/usr4/dlc/bin/proutil"
The fix was to remove ">> $WRKDIR/$i.load.log".. the binary utility wouldn't run when trying to output results to file.. strange..
A couple of really bad practices here
parse the output of ls
not quoting variables
iterating the lines of a file with cat and for
As shelter comments, you don't check that you've created all the directories in the path for your log file.
A rewrite:
for i in "$STAGEDIR"/*.list; do
echo "Working with $i"
# grab first .bd file
firstbd=$(head -1 "$i")
echo "First file is $firstbd"
# ensure the output directory exists
logfile="$WRKDIR/$i.load.log"
mkdir -p "$(dirname "$logfile")"
"$PROBIN"/proutil "$DBENV/$DBNAME" -C load "$firstbd" tenant "$TENANT" -dumplist "$STAGEDIR/$i.list" >> "$logfile"
# move the list and its content to finished folder
while IFS= read -r movethis; do
echo "Moving file $movethis to $STAGEDIR/finished"
mv "$movethis" "$STAGEDIR"/finished/
done < "$i"
echo "Finished working with list $i"
echo "Moving it to $STAGEDIR/finished"
mv "$i" "$STAGEDIR"/finished/
done

understanding a ksh script part of

Could someone help me understand the following piece of code which is deciding on the start and end dates to pick data out of a db.
# Get the current time as the stop time.
#
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
#
# Read the lasttime file to get the start time
#
if test -f $1/optlasttime
then
starttime=`cat $1/optlasttime`
# if the length of the chain is zero
# (lasttime is empty) It is updated properly
# (and I wait for the following hour)
if test -z "$starttime"
then
echo "Empty file lasttime"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 5
fi
else
# If lasttime does not exist I create, it with the present date
# and I wait for the following hour
echo "File lasttime does not exist"
echo $stoptime > $1/optlasttime
rm -f $1/.optpamo.pid
exit 6
fi
Thanks
The script checks to see if there's a non-empty file named optlasttime in the directory specified as an argument ($1). If so, the script exits successfully (status 0). If the file doesn't exist or is empty, the current hour formatted as 2010-01-07 14:00 is written to the file, another file named .optpamo.pid is deleted from the argument directory and the script exits unsuccessfully (status 5 or 6).
This script is obviously a utility being called by some outer process, to which you need to refer for full understanding.
1.) Sets stop time to current time
2.) Checks if file $1/optlasttime exists (where $1 is passed into the script)
a.) if $1/optlasttime exists it checks the contents of the file (which it is assumed that if it does have contents it is a timestamp)
b.) if $1/optlasttime does not exist it populates the $1/optlasttime file with the stoptime.
I copied and pasted a small snippet of this into a file I called test.ksh
stoptime=`date +"%Y-%m-%d %H:00"`
if test $? -ne 0
then
echo "Failed to get the date"
rm -f $1/.optpamo.pid
exit 4
fi
Then I ran it at the commandline, like so:
zhasper#berens:~$ ksh -x ./temp.ksh
+ date '+%Y-%m-%d %H:00'
+ stoptime='2010-01-08 18:00'
+ test 0 -ne 0
The -x flag to ksh makes it print out each commandline, in full, as it executes. Comparing what you see here with the snippet of shell script above should tell you something about how ksh is interpreting the file.
If you run this over the whole file you should get a good feel for what it's doing.
To learn more, you can read man ksh, or search for ksh scripting tutorial online.
Together, these three things should help you learn a lot more than us simply telling you what the script does.

Resources