We may notice many download sites provide md5 string. For example, when I download ABC.zip, along with an md5 string like: “2743a6a9fe6f873df1c7ed8ac91df5d7 *ABC.zip”. I know the idea behind it, it’s Digest algorithm to prevent file forge.
My question is how a user calculates md5 string for the ABC.zip, and compare it with value site provides? Any existing tool to generate md5 string?
It depends a bit on your operating system ofcourse. Under most Linux/Unix distributions you have an md5 or md5sum program available.
Example:
# md5sum eclipse-SDK-3.6RC3-linux-gtk.tar.gz
8eca528d2c0b33dae10ba8750b2e4b94 eclipse-SDK-3.6RC3-linux-gtk.tar.gz
It also has a check mode which does exactly what you're looking for:
# md5sum -c test.md5
eclipse-SDK-3.6RC3-linux-gtk.tar.gz: OK
(test.md5 has the output of the previous command)
On Linux systems, the program is usually named md5sum.
On BSD systems, the program is usually named md5.
On Windows systems, aim users to http://en.wikipedia.org/wiki/Md5sum
Note that the md5sum and md5 utilities have a command-line option that can verify all the MD5 hashes listed in an MD5SUM file automatically:
sarnold#haig:~/bin$ md5sum * > /tmp/MD5SUM
sarnold#haig:~/bin$ md5sum -c /tmp/MD5SUM
aa-change: OK
aa-change.c: OK
briss: OK
mkvtom2ts: OK
muxer: OK
muxer_orig: OK
Depends. Programming languages usually have a library to do this.
OS X has a command line utility 'md5'. Linux has something similar. For windows, no idea, but you can probably find something easily.
For windows, if you install Cygwin, md5sum becomes available.
Related
I am working on a battery of automatic tests which executes on 2 Unix virtual machines running with KSH. Those VMs are independant and they have practically the same .profile file. I would like to study their differences by launching:
tkdiff /usr/system/.profile system#{external_IP}:/usr/system/.profile
on the first VM but it doesn't work.
I suppose that directly accessing a hidden file is not possible. Is there a solution to my problem, or maybe an alternative?
If you want to compare different files on two remote machines, I suggest the following procedure:
1. Compare checksums:
First compare the checksums. Use sum, md5sum or sha256sum to compute a hash of the file. If the hash is the same, the probability of having the same file is extremely high! You can even increase that probability by check the total amount of characters, lines and words, in the file using wc.
$ file="/usr/system/.profile"
$ md5sum "$file" && wc "$file"
$ ssh user#host "md5sum '$file' && wc '$file'"
2. run a simple diff
Run a simple diff using the classic command line tools. They understand the POSIX standard to use - as /dev/stdin. This way you can do:
$ ssh user#host "cat -- '$file'" | diff "$file" -
note: with old versions of tkdiff or new versions of svn/git, it can be tricky here due to bugs in tkdiff. It will quickly throw errors of the form svn [XXXX] file .... is not a working copy or file xxxx is not part of a revision control system if one of the files might be under version control or you end up in a directory under version control. Stick to diff!
You are using the filename convention "user#host:/path/to/file" for the second argument to tkdiff.
That convention for naming is not native to Ksh, but instead is understood by some programs like scp and others (which can be interactive, e.g. to ask for a password for the remote system or other authentication related questions).
But from the tkdiff man page, it does not mention having built-in support for that filenaming convention userid#host:/path/to/file, and neither is such support built into ksh.
So you may need to use two steps, first to use scp or similar to copy the remote file locally then then use tkdiff with one argument the local file and the other the file-just-copied, or arrange to mount part of the other VM filesystem locally, and then use tkdiff with appropriate arguments.
Obviously, both files need to be readable by your userid or the user specified on the userid#host:/path/to/file for this to work.
You can directly made a remote ssh compare , run a remote display with help of cat command line, with this :
tkdiff <(ssh system#{external_IP}1 'cat /usr/system/.profile') <(ssh system#{external_IP}2 'cat /usr/system/.profile')
In your case to be able to compare with the local .profile file this :
tkdiff /usr/system/.profile <(ssh system#{external_IP} 'cat /usr/system/.profile')
Do you have just try with the simple diff command line (with -b -B option to remove blank line and space comparaison):
diff -b -B /usr/system/.profile <(ssh system#{external_IP} 'cat /usr/system/.profile')
Is there a short one-liner to get a file checksum, which works on both macos and ubuntu? It doesn't matter what algorithm or program, as long as I don't have to install or setup anything.
You could use OpenSSL, and the commands should be the same:
openssl sha256 filename | awk -F'= ' '{print $2}' # optional
Use whatever hashing algorithm you want, sha256, sha1, md5, etc.
Just try both of them:
md5 file 2>/dev/null; md5sum file 2>/dev/null;
That line will work on both OSs, running both commands and discarding the one that gives an error, it will print only the valid result.
With a quick OS check you can use either md5 (mac) or md5sum (ubuntu), alternatively you could alias one of them so you'd be using the same command on either OS.
On Linux, you can use md5sum file; on macOS, just md5 file. Both are default at a clean install, AFAIK. If you require that the command be the same, you can create an alias.
May I be so impertinent as to suggest writing your own?
python -c 'import sys, hashlib;
m = hashlib.sha256();
m.update(open(sys.argv[1]).read());
print("\t".join([m.hexdigest(), sys.argv[1]]))' file
The semicolons are gratuitous here, but necessary if you really want to force the issue and make this a literal one-liner.
I have some functions in my .bashrc file which are used to issue backup commands on remote websites. Right now, the username and password fields are stored as function-local strings in plain text within the function definition. Is there a better way of doing this?
My idea so far was to put a hashed version of the passwords in a file to which only my user account has read access, run a de-hashing command-line function on it and store the plain text result in memory, use it, then clear it.
Is there a better/safer or even a de-facto common way of accomplishing this?
Thank you.
There are 2 ways I can think of safely approaching this problem.
1. GPG
Keep a GPG encrypted file with your passwords in it in key=value format (shell parsable basically), one per line. Such as:
foo_pass='bar'
pop_pass='tart'
When you want to access them, just do:
eval "$(gpg -d /path/to/file | grep '^foo_pass=')"
SUPERSECRETPASSWORD="$foo_pass" somecmd
If the command needs the password as an argument (this is unsafe), just adjust that last line.
2. Keyring daemon
Depending on your OS, you might have access to a keyring which you can store your passwords in. On linux, this might be the gnome keyring daemon. Then this keyring can probably be accessed via CLI/script somehow.
For example, there is gkeyring for use with the gnome keyring daemon.
For example, typing compgen -u on an Ubuntu box returns a list of users. This includes more users than are listed in /etc/passwd . So the question is when using bash completion to list out users, where does the list come from?
/etc/passwd is the default source of user account information on UNIX-like systems, but it's often supplemented by other sources, paricularly on machines that are part of some larger organization that needs to keep that information consistent.
NIS (formerly known as "YP") is one common system. LDAP is another. I'm sure there are others.
The getent passwd command should show you all the relevant account information. On my machine (which doesn't use NIS or LDAP), it's equivalent to cat /etc/passwd; on yours, it will probably show additional information.
The various getpw*() functions (getpwuid(), getpwnam(), getpwent()) retrieve user account information, equivalent to what's in /etc/passwd plus whatever supplement your system uses. Presumably both the getent command and bash use this mechanism to obtain the relevant information.
You can run strace -o compgen.out bash -c 'compgen -u' and then look at the compgen.out file to try to find out what it is using.
On my machine that file ends with an open("/etc/passwd", O_RDONLY) followed by the writing of the output.
How do you extend the output of ps -fe in Solaris so that it displays more than 80 characters? My process has several arguments and the process name could not be displayed anymore.
You can't display them with the default ps (/usr/bin/ps) which is a SVR4 regular one.
To get the full argument line, use the BSD ps (UCB = University of California at Berkeley):
/usr/ucb/ps -alxwww
We have finally fixed this in Solaris; as of Solaris 11.3 SRU 5, all original argument vectors as well as the environment variables can be retrieved from /proc. ps will now print all of the command line.
Fixed in Solaris 11.3 SRU 5
The simple answer is that there is no way to reliably acquire the full arguments to processes on Solaris for processes owned by other users. If you have root or other privileged access you can use /usr/ucb/ps on older versions, and 'pargs' or similar tools on newer versions (there is no tool which works across all versions).
Essentially Solaris stores the original args at process start time, while most other platforms allow ps to access, via some means, the contents of argv at runtime for the process. This stored-copy is in a special kernel data structure with limited (80 byte) size. This also means that it's not possible for a program to modify the args post-start as displayed by ps for useful or nefarious means.
Thus, if you need to access the command line for portable purposes, such as pid checking, you will need to choose between enforcing a short command line via hacks like launching programs controlled execp paths without absolute paths, or you will need to forgo that portable functionality on Solaris.
you can use pargs PID
it will give you more information than ps
Try ps -efl. If that doesn't work (I don't have a Solaris box handy) you can also try ps -efl | cat (as some programs check whether they're outputting to a terminal to decide on their output width).
There are two sets of options available for ps. Others will chime in with the correct names ( ( maybe BSD and SRVn)?)
With the non-options-preceded-with-a-hyphen-version, you can do
ps auxww(w?) | grep ${PID} to extend the length of the command detail that is printed (again, notice NO leading '-' option indicator).
Note that in some cases you will see a lot of environment variable assignments before the actually command, i.e. myPath=... cfgFile=... /path/to/command ... args ...
I think that 'www' in some systems will print everything, regardless how long the command is.
Finally, in my experience using ps to do a lot of crazy things, I would ocassionally have a PID and the output would display the first 6? columns, but the space reserved for the command was empty or had some sort of place holder value. I eventually found out why that was true, by searching comp.unix.shell, but it's too long ago now to be sure and I don't have access to Solaris systems right now.
I hope this helps.