Implement a Data Structure to Compare NTLM Hashfiles - shell

Question

Currently I'm in the process of simplifying a process for extracting Windows password hashes in security audits. Personally I want to make the process easier to generate a list of recovered users and their passwords when I do an audit. I think it would also be useful for other people who are trying to compare and generate large amounts of data.
So here's the gist:
When I extract all of the data from the Windows system files, I simplify them down to the format user:hash, where the hash is an NTLM hash such as "a87f3a357d73085c45f9416be5787e86."
I then will use oclHashcat and attempt to crack the hashes, whether it be dictionary or brute-force, it doesn't matter. I generate an output of all of the recovered hashes, however Hashcat generates them in the format hash:password.
Now here's my problem and what I would like some input on - I want to produce the output as user:password given the two input files. Considering that I can have hundreds of hashes yet only a few recovered passwords, there is no use of trying to order the lists.
I am unsure which data structure might benefit me the most. Arrays were too inefficient for large tables. I've looked into serialization and I've been exploring the use of Hash Maps and Hash Tables. Given the size of the hash, I haven't had any luck implementing either of these methods, or I'm doing so incorrectly.
Currently I'm running the program like so:
program [user:hash file] [hash:password file] -o [user:password output]
And I'm effectively trying to run the program like so (briefly):
Load Files
// user:hash file
For each line, split by ':' delimiter
before delimiter = table1.user
after delimiter = table1.hash
// hash:password file
For each line, split by ':' delimiter
before delimiter = table2.hash
after delimiter = table2.password
// generate user:password file
Check each entry of table1 vs table2
if table1.hash = table2.hash
table1.user = output.user
table2.password = output.password
print to output "output.user:output.password"
I am only trying to figure out an efficient method for tracing through each line and extracting the necessary data into a data structure that I can easily trace through.
If I need to clarify anything, please let me know. Any help is appreciated!

Answer 1

I would do this for my data structure
std::map<std::string,std::string> hash_user_map;
No go through all the users and hashes from table 1
For each user in table1
hash_user_map[table1.hash] = table1.user;
No go through all the cracked passwords with hashes from table2
std::string user = hash_user_map[table2.hash];
std::cout << user << ":" << table2.password << "\n;

Answer 2

I decided to go with a shell script, and I used an associated array to match the required data that I needed.
Note: Most of this program deals with the way my hash files are formatted. Look at the comments in the script.
#!/bin/bash
# Signus
# User-Password Hash Comparison Tool v1.0
# Simple utility that allows for the comparison between a file with a 'user:hash' format to a separate file with a 'hash:password' format. The comparison matches the hashes and returns an output file in the format 'user:password'
# Example of 'user:hash' -> george:c21cfaebe1d69ac9e2e4e1e2dc383bac
# Example of 'hash:password' -> c21cfaebe1d69ac9e2e4e1e2dc383bac:password
# 'user:hash' obtained from creddump suite: http://code.google.com/p/creddump/
# Note: Used custom 'dshashes.py' file: http://ptscripts.googlecode.com/svn/trunk/dshashes.py
# 'hash:password' obtained from ocl-Hashcat output
usage="Usage: $0 [-i user:hash input] [-t hash:password input] [-o output]"
declare -a a1
declare -a a2
declare -A o
index1=0
index2=0
countA1=0
countA2=0
matchCount=0
if [ -z "$*" ]; then
echo $usage
exit 1
fi
if [ $# -ne 6 ]; then
echo "Error: Invalid number of arguments."
echo $usage
exit 1
fi
echo -e
echo "---Checking Files---"
while getopts ":i:t:o:" option; do
case $option in
i) inputFile1="$OPTARG"
if [ ! -f $inputFile1 ]; then
echo "Unable to find or open file: $inputFile1"
exit 1
fi
echo "Reading...$inputFile1"
;;
t) inputFile2="$OPTARG"
if [ ! -f $inputFile2 ]; then
echo "Unable to find or open file: $inputFile2"
exit 1
fi
echo "Reading...$inputFile2"
;;
o) outputFile="$OPTARG"
echo "Writing...$outputFile"
;;
[?]) echo $usage >&2
exit 1
;;
esac
done
shift $(($OPTIND-1))
#First read the files and cut each line into an array
echo -e
echo "---Reading Files---"
while read LINE
do
a1[$index1]="$LINE"
index1=$(($index1+1))
countA1=$(($countA1+1))
done < $inputFile1
echo "Read $countA1 lines in $inputFile1"
while read LINE
do
a2[$index2]="$LINE"
index2=$(($index2+1))
countA2=$(($countA2+1))
done < $inputFile2
echo "Read $countA2 lines in $inputFile2"
#Then cut each item out of the array and store it into separate variables
echo -e
echo "Searching for Matches..."
for (( j=0; j<${#a2[#]}; j++ ))
do
hash2=$(echo ${a2[$j]} | cut -d: -f1)
pword=$(echo ${a2[$j]} | cut -d: -f2)
for (( i=0; i<${#a1[#]}; i++ ))
do
us=$(echo ${a1[$i]} | cut -d: -f1)
hash1=$(echo ${a1[$i]} | cut -d: -f2)
if [ "$hash2" = "$hash1" ]; then
matchCount=$(($matchCount+1))
o["$us"]=$pword
echo -e "Match Found[$matchCount]: \t Username:$us \t Password:$pword"
break
fi
done
done
echo -e "Matches Found: $matchCount\n" >> $outputFile
for k in ${!o[#]}
do
echo -e "Username: $k \t Password: ${o[$k]}" >> $outputFile
done
echo -e "\nWrote $matchCount lines to $outputFile"
unset o