I was trying to do do some more shell scripting in the last few days.
where I am trying to calculate the value of a function.
To start with I get an
echo "blah"
read blub
md5=`md5sum $blub | cut -d ' ' -f 1`
echo $md5
echo "secretkey"
read $SK
For example, If get the values like that.
Is it possible at-all to do a calculation withGPG like this
GPG(VALUE1,VALUE2) in my case it is GPG(md5,sk)
Later on I would like to take a modular function over that value. Which is possible via the shell script.
If its possible , May I ask how is it possible to do in GPG. or is there any other better public generating algorithm that can do this?
Thanks!
If I understand you correctly, you are trying to obtain a digest (hash, checksum) of the MD5 + secret key pair (much like the MD5 is a digest of the blub).
In any case, GPG won't help you here. It's mostly a tool for encryption and signing with PGP. It can do symmetric encryption of files as well but AFAICS it doesn't do stand-alone digests, and certainly doesn't do them in decimal form.
The simplest solution seems to be using one of the common shell checksumming tools which actually have numeric output — for example cksum.
Those tools take only a single blob as an argument, so you probably want to simply concatenate the input
I think a working example would be:
gpg=$(echo "$md5$sk" | cksum | cut -d' ' -f1)
Then gpg will contain some decimal value, unique for each md5 + secret key pair.
Related
I generate (dynamically) a script concatenating the following files:
testscript1
echo Writing File
cat > /tmp/test_file <<EOF
testcontent
line1
second line
testscript2
EOF
echo File is written
And I execute by calling
$ cat testscript1 testcontent testscript2 | ssh remote_host bash -s --
The effect is that the file /tmp/test_file is filled with the desired content.
Is there also a variant thinkable where binary files can be supplied in a similar fashion? Instead of cat of course dd could be used or other Tools, but the problem I see is 'telling' them that the STDIN now ended (can I send ^D through that stream?)
I am not able to get my head around that problem, but there is likely no comparable solution. However, I might be wrong, so I'd be happy to hear from you.
Regards,
Mazze
can I send ^D through that stream
Yes but you don't want to.
Control+D, commonly notated ^D, is just a character -- or to be pedantic (as I often am), a codepoint in the usual character code (ASCII or a superset like UTF-8) that we treat as a character. You can send that character/byte by a number of methods, most simply printf '\004', but the receiving system won't treat it as end-of-file; it will instead be stored in the destination file, just like any other data byte, followed by the subsequent data that you meant to be a new command and file etc.
^D only causes end-of-file when input from a terminal (more exactly, a 'tty' device) -- and then only in 'cooked' mode (which is why programs like vi and less can do things very different from ending a file when you type ^D). The form of ssh you used doesn't make the input a 'tty' device. ssh can make the input (and output) a 'tty' (more exactly a subclass of 'tty' called a pseudo-tty or 'pty', but that doesn't matter here) if you add the -t option (in some situations you may need to repeat it as -t -t or -tt). But then if your binary file contains any byte with the value \004 -- or several other special values -- which is quite possible, then your data will be corrupted and garbage commands executed (sometimes), which definitely won't do what you want and may damage your system.
The traditional approach to what you are trying to do, back in the 1980s and 1990s, was 'shar' (shell archive) and the usual solution to handling binary data was 'uuencode', which converts binary data into only printable characters that could go safely go through a link like this, matched by 'uudecode' which converts it back. See this surviving example from GNU. uuencode and uudecode themselves were part of a communication protocol 'uucp' used mostly for email and Usenet, which are (all) mostly obsolete and forgotten.
However, nearly all systems today contain a 'base64' program which provides equivalent (though not identical) functionality. Within a single system you can do:
base64 <infile | base64 -d >outfile
to get the same effect as cp infile outfile. In your case you can do something like:
{ echo "base64 -d <<END# >outfile"; base64 <infile; echo "END#"; otherstuff; } | ssh remote bash
You can also try:
cat testscript1 testcontent testscript2 | base64 | ssh <options> "base64 --decode | bash"
Don't worry about ^D, because when your input is exhausted, the next processes of the pipeline will notice that they have reached the end of the input file.
I have a big audio file (2 hours) foo.mp3. I want to enable a third party P to verify the date of its creation. However P shall only be able to view (hear) small certain parts of foo.mp3. On the date of the creation it is unclear which parts will be provided to P.
So my plan is to split foo.mp3 into small chunks, hash these chunks and publish the hashes somewhere on the net for later verification.
How would a bash script look like that splits foo.mp3 and creates hashes?
You don't have to split the file into chunks. Just publish the SHA256 of the whole file now and when the other party gets access to the file they will be able to see that it has the same SHA256 sum that you published earlier. (I'd recommend using at least SHA256 as the hash algorithm.)
Now if you still decide you have to do it like this for some reason, I suggest using split(1) and then running sha256sum:
mkdir split
split -b8192 foo split/
sha256sum split/*
This creates splits of the file into 8KB chunks in the split/ directory and prints the SHA256 hashes.
1. Splitting the file into several chunks
mp3splt -a -t 0.01 -o foo-#n -d outputDir foo.mp3
2. Create hashes
sha256sum outputDir/* > hashes.sha
3. Document results on the web
The outcome of a published file with sha256 hashes and/or sha256 hashes of big sha256 hashfiles could look like this:)
5b2718841d2f610ce264191760383dca309fd5a6f8c745e7466aa6c157e0b279 Files1.sha
0b568a4e802e9bc29740e8904c09f361c844dc4bd11d8855a188e81672f453ba Files2.sha
d8fec62a53228cf372ce81f4e3fd220b2df7bb5d5f8b144c8ffa4e41a4aeaad4 Camera.zip
ac0e59caa3f91f49824540fa452e285b9e47843f24dda309955a5287db4c89b5 Camerasmall.zip
e1f9b37f15eed99a8814f066021e6d370fd4e346b2779fc6494ee685b9e02467 SoundsSmall.zip
d054a2ac8adf069c2051659c68e38a7b6e26172afb2a6c377f7d067c666f9f6e Sounds.zip
If I use wbemcli to enumerate all the instances I get something similar to this:
wbemcli -nl -t -noverify ei 'https://aaa/aaa:aaa'
https://aaa/aaa:aaa.Version="",Vendor="",Name=""
-Version#=""
-Vendor#=""
-Name#=""
-Description=""
How can I call wbemcli to get only one item (i.e. the Name)? and not everything.
The -t option says:
-t
Append array ([]), reference (&) and key property (#) indicators to property names
but I wasn't able to utilize this in my favor.
Is there a way to retrieve this information in a key/value pair format?
Or maybe pipe the output into an array or something from which I can grab only what I need?
When I drop the output into an array all the data is stored in the first element ${a[0]}.
EDIT
Here's an output example:
$ wbemcli -nl -t -noverify ei 'https://user:pw#000.000.000.000:0000/root/aaa:AA_AaaAaaaAaaaa'
000.000.000.000:0000/root/aaa:AA_AaaAaaaAaaaa.ClassName="AA_AaaAaaaAaaaa",Name="123456a7ff890123"
-ClassName#="AA_AaaAaaaAaaaa"
-Name#="123456a7ff890123"
-Caption="aa aaa"
-Description="aa aa"
-ElementName="aa aaa aaaa"
-OperationalStatus[]=2
-HealthState=5
-CommunicationStatus=2
-DetailedStatus=1
-OperatingStatus=0
-PrimaryStatus=1
-EnabledState=5
-RequestedState=12
-EnabledDefault=2
-TransitioningToState=12
-PrimaryOwnerName="Uninitialized Contact"
-PrimaryOwnerContact="Uninitialized Contact"
The output is usually in this format.
If the query returns multiple objects they will be grouped and all will have the same members with their appropriate values.
http://linux.dell.com/files/whitepapers/WBEM_based_management_in_Linux.pdf has a number of examples which simply suggest to use grep to obtain the specific key and value you are looking for. There does not seem to be a way to directly query for a specific key within a result set.
Expanding on the comment by Etan Reisner, you could use something like
wbemcli <<query>> | grep -oP "^-$key=\K.*"
to obtain the value for the key named in $key, provided you have GNU grep which provides the -P option for Perl-compatible regular expressions (here, the \K "forget up through here" operator is useful). So for your specific example,
wbemcli -nl -t -noverify ei 'https://aaa/aaa:aaa' |
grep -oP '^-Name#=\K.*'
There is also a -dx option which produces XML output, which may be more robust if you are planning to write a major application on top of this protocol (but then perhaps you should be looking at a dedicated WBEM library such as the C or Java libraries listed in their wiki). It would not seem to be implausible to write a simple (e.g.) Python client to retrieve (part of?) the result tree and let you query or manipulate it locally, either.
We have a mail server and I am trying to write a script that will find all users with similar names to avoid malicious users from impersonating legitimate users. For example, a legit user may have the name of james2014#domain.com but a malicious user may register as james20l4#domain.com. The difference, if you notice carefully, is that I replaced the number 'one' with the letter 'l' (el). So I am trying to write something that can consult my /var/vmail/domain/* and find similar names and alert me (the administrator). I will then take the necessary steps to do what I need. Really appreciate any help.
One hacky way to do this is to derive "normalized" versions of your usernames, put those in an associative array as keys mapping to the original input, and use those to find problems.
The example I posted below uses bash associative arrays to store the mapping from normalized name to original name, and tr to switch some characters for other characters (and delete other characters entirely).
I'm assuming that your list of users will fit into memory; you'll also need to tweak the mapping of modified and removed characters to hit your favorite balance between effectiveness and false positives. If your list can't fit in memory, you can use a single file or the filesystem to approximate it, but honestly if you're processing that many names you're probably better off with a non-shell programming language.
Input:
doc
dopey
james2014
happy
bashful
grumpy
james20l4
sleepy
james.2014
sneezy
Script:
#!/bin/bash
# stdin: A list of usernames. stdout: Pairs of names that match.
CHARS_TO_REMOVE="._\\- "
CHARS_TO_MAP_FROM="OISZql"
CHARS_TO_MAP_TO="0152g1"
normalize() {
# stdin: A word. stdout: A modified version of the same word.
exec tr "$CHARS_TO_MAP_FROM" "$CHARS_TO_MAP_TO" \
| tr --delete "$CHARS_TO_REMOVE" \
| tr "A-Z" "a-z"
}
declare -A NORMALIZED_NAMES
while read NAME; do
NORMALIZED_NAME=$(normalize <<< "$NAME")
# -n tests for non-empty strings, as it would be if the name were set already.
if [[ -n ${NORMALIZED_NAMES[$NORMALIZED_NAME]} ]]; then
# This name has been seen before! Print both of them.
echo "${NORMALIZED_NAMES[$NORMALIZED_NAME]} $NAME"
else
# This name has not been seen before. Store it.
NORMALIZED_NAMES["$NORMALIZED_NAME"]="$NAME"
fi
done
Output:
james2014 james20l4
james2014 james.2014
For example under FreeBSD passwords are stored in /etc/master.passwd like this:
$1$7wtGfwgp$772bEQInetnJKUNtLM0Xt/
The password I used was "Test11". As I understand, $1$ means that it's a hashed MD5 crypt? How can one come up with the final hash "772bEQInetnJKUNtLM0Xt/" if he is aware of salt and password and uses md5sum? As I understand, it should be something like:
$ echo -n $(echo -n 7wtGfwgp)$(echo -n Test11) | md5sum
..but this doesn't quite match up. What am I doing wrong?
PS I'm asking this in order to understand the password hashing system under UNIX-like operating systems.
I'm on Linux and I am not sure whether FreeBSD actually uses the same algorithm, but you can take a look at these sources of information (hope the same is on FreeBSD):
man 3 crypt
https://unix.stackexchange.com/questions/21897/grub-md5-crypt-algorithm
http://s23.org/wiki/Crypt
Based on the last page, this PHP script will produce the output you expect (given your password and salt):
<?php
$password = 'Test11';
$salt = '$1$7wtGfwgp$';
echo 'Crypt hash: ' . crypt($password, $salt) . "\n"
?>
You can do the same using e.g. Python:
import crypt
password = 'Test11'
salt = '$1$7wtGfwgp$'
print(crypt.crypt(password, salt))
based on this Python doc page:
http://docs.python.org/library/crypt.html
Based on the Wikipedia article:
http://en.wikipedia.org/wiki/Crypt_(Unix)
you can see the source of crypt function e.g. here:
http://google.com/codesearch/p#ZWtxA-fyzBo/UnixArchive/PDP-11/Distributions/research/Henry_Spencer_v7/v7.tar.gz%7C118goTAkg2o/usr/src/libc/gen/crypt.c
As a side note, here's a nice online hash generator:
http://insidepro.com/hashes.php?lang=eng
Hope this helps.
Hashing and crypting is something different, even if hashing is a part of crypting ;)
So if you want to crypt it, do it like icyrock posted. md5sum (gmd5sum in freebsd coreutils) does only create a hash sum (RFC 1321) about some input. It's not using this hash to crypt in a further step as (several) crypt codes does.
That's why your code gives you something completely different as result.