I'm looking for a method to generate true random bits/numbers on smartphones.
Create a custom TRNG seems to be tricky and a lot of people suggest to trust in already existing ones, widely tested and approved.
But I've found Jericho Comms, that uses a custom TRNG from the pictures as input.
How secure, strong and reliable is this? Could it be a valid method to generate truly random numbers/bits?
I know that smartphone cameras are not perfect for this purpose because of the JPEG compression and low quality... but omitting this, how correct is the algorithm proposed?
I paste the documentation:
This process describes the full algorithm:
Load the user's photograph into memory and store it in a sequential array of RGB values.
Get the red, green and blue (RGB) integer values for each pixel. This will return a number between 0 and 255 for each colour.
Remove sequentially repeating black (0, 0, 0) pixels, white (255, 255, 255) pixels and pixel colours that are exactly the same. This removes sections of the photograph with underexposed pixels, overexposed pixels and those that have have little entropy. Generally it is very unusual to have adjacent pixels that are exactly the same colour unless there is a hardware failure, or the section of the photo is under/overexposed. Usually in a good quality photograph there are at least very slight colour variations in adjact pixels. This step removes these low entropy areas.
Estimate 1 bit of input entropy per set of RGB values (1 bit per 24 bit pixel). This is a very conservative estimate. Users can increase this in the TRNG settings to 3 bits per pixel which would account for the entropy in the least significant bit of each colour.
Gather 512 RGB values to get an estimated entropy input of 512 bits.
Convert the 512 RGB values to their hexadecimal representation and input them into a cryptographic hash with a 512 bit output. The user can choose which hash to use and the program allows either Skein or Keccak [c=2d]. Both are very strong finalist algorithms from the NIST hash function competition. Store this hash output of 512 bits as the temporary seed into the next hash.
Start a loop:
- Check there is enough new input entropy for the new hash, or break out of the loop.
- Get the temporary seed from earlier.
- Get a new set of 512 RGB values (512 bits) as the new input entropy.
- Concatenate the seed and input entropy together and hash them using: Hash( seed || input entropy ).
- Append the first 256 bits of the hash output to the output random data.
- Update the temporary seed to be the last 256 bits of the hash output.
- Return to start of the loop.
It is important to note that the program does not use this collected entropy to seed a psuedo-random number generator to give an unlimited amount of random data. The program aims to be a true random number generator so only quality randomness is wanted and each uniformly random bit must be used solely to encrypt one bit of the plaintext. It's assumed that most pseudo-random number generators or even CSPRNGs that stretch out the entropy could produce a subtle bias in the output and allow the NSA or other governments to decrypt part or all of a message. With this program the aim is to avoid stretching the available entropy over more bits.
Related
I'm working on LSB-DCT based Image steganography in which i have to apply LSB to DCT coefficients of the image for data embedding to JPEG.i'm new to all this.so searched and read some research papers they all lack a lot of information regarding the process after DCT.i also read many questions and answers on stackoverflow too and got more confused.
here are the questions:
1-reasearch paper and in question on the web they all are using 8x8 block size from image for DCT..what i should do if the resolution of image does not completely divides into 8x8 blocks like 724 x 520.
520 / 8 = 65 but 724 / 8 = 90.5
2-if i have a lot of blocks and some information to hide which we suppose can fit into 5 blocks..do i still need to take dct of the remaining blocks and and idct.
3-do i need to apply quantization after dct and then apply lsb or i can apply lsb directly??
4-research papers are not mentioning anything about not to touch quantized dct coefficients with value 0 and 1 and the first value..now should i use them or not?? and why not?? i get it about the 0 because it's was high frequency components and is removed in JPEG for compression..and i'm not doing any compression..so can i use it and still produce the same JPEG file???
5-in quantization we divide the DCT Coefficients with quantization matrix and round off the values.in reverse,i have to multiply quantization matrix with DCT Coefficients just..no undo for round off???
For the Comment on DCT and then IDCT:
From different Research Papers:
JPEG steganography
If you want to save your image to jpeg, you have to follow the jpeg encoding process. Unfortunately, papers most I've read say don't do it justice. The complete process is the following (wiki summary of a 182-page specifications book):
RGB to YCbCr conversion (optional),
subsampling of the chroma channels (optional),
8x8 block splitting,
pixel value recentering,
DCT,
quantisation based on compression ratio/quality,
order the coefficients in a zigzag pattern, and
entropy encoding; most frequently involving Huffman coding and run-length encoding (RLE).
There are actually a lot more details involved, such as headers, section markers, specifics of how to store the DC and AC coefficients, etc. Then, there are aspects that the standard has only loosely defined and their implementation can vary between codecs, e.g., subsampling algorithm, quantisation tables and entropy encoding. That said, most pieces of software abide by the general JFIF standard and can be read by various software. If you want your jpeg file to do the same, be prepared to write hundreds (to about a thousand) lines of code just for an encoder. You're better off borrowing an encoder that has already been published on the internet than writing your own. You can start by looking into libjpeg which is written in C and forms the basis of many other jpeg codecs, its C# implementation or even a Java version inspired by it.
In some pseudocode, the encoding/decoding process can be described as follows.
function saveToJpeg(pixels, fileout) {
// pixels is a 2D or 3D array containing your raw pixel values
// blocks is a list of 2D arrays of size 8x8 each, containing pixel values
blocks = splitBlocks(pixels);
// a list similar to blocks, but for the DCT coefficients
coeffs = dct(blocks);
saveCoefficients(coeffs, fileout);
}
function loadJpeg(filein) {
coeffs = readCoefficients(filein);
blocks = idct(coeffs);
pixels = combineBlocks(blocks);
return pixels;
}
For steganography, you'd modify it as follows.
function embedSecretToJpeg(pixels, secret, fileout) {
blocks = splitBlocks(pixels);
coeffs = dct(blocks);
modified_coeffs = embedSecret(coeffs, secret);
saveCoefficients(modified_coeffs, fileout);
}
function extractSecretFromJpeg(filein) {
coeffs = readCoefficients(filein);
secret = extractSecret(coeffs);
return secret;
}
If your cover image is already in jpeg, there is no need to load it with a decoder to pixels and then pass it to an encoder to embed your message. You can do this instead.
function embedSecretToJpeg(pixels, secret, filein, fileout) {
coeffs = readCoefficients(filein);
modified_coeffs = embedSecret(coeffs, secret);
saveCoefficients(modified_coeffs, fileout);
}
As far as your questions are concerned, 1, 2, 3 and 5 should be taken care of by the encoder/decoder unless you're writing one yourself.
Question 1: Generally, you want to pad the image with the necessary number of rows/columns so that both the width and height are divisible by 8. Internally, the encoder will keep track of the padded rows/columns, so that the decoder will discard them after reconstruction. The choice of pixel value for these dummy rows/columns is up to you, but you're advised against using a constant value because it will result to ringing artifacts which has to do with the fact that the Fourier transform of a square wave being the sinc function.
Question 2: While you'll modify only a few blocks, the encoding process requires you to transform them all so they can be stored to a file.
Question 3: You have to quantise the float DCT coefficients as that's what's stored losslessly to a file. You can modify them to your heart's content after the quantisation step.
Question 4: Nobody prevents you from modifying any coefficient, but you have to remember each coefficient affects all 64 pixels in a block. The DC coefficient and the low frequency AC ones introduce the biggest distortions, so you might want to stay away from them. More specifically, because of the way the DC coefficients are stored, modifying one would propage the distortion to all following blocks.
Since most high frequency coefficients are 0, they are efficiently compressed with RLE. Modifying a 0 coefficient may flip it to a 1 (if you're doing basic LSB substitution), which disrupts this efficient compression.
Lastly, some algorithms store their secret in any non-zero coefficients and will skip any 0s. However, if you attempted to modify a 1, it might flip to a 0 and in the extraction process you'd blindly skip reading it. Therefore, such algorithms don't go near any coefficients with the value of 1 or 0.
Question 5: In decoding you just multiply the coefficient with the respective quantisation table value. For example, the DC coefficient is 309.443 and quantisation gives you round(309.443 / 16) = 19. The rounding off bit is the lossy part here, which doesn't allow you to reconstruct 309.433. So the reverse is simply 19 * 16 = 304.
Other uses of DCT in steganography
Frequency transforms, such as DCT and DWT can be used in steganography to embed the secret in the frequency domain but not necessarily store the stego image to jpeg. This process is pixels -> DCT -> coefficients -> modify coefficients -> IDCT -> pixels, which is what you send to the receiver. As such, the choice of format matters here. If you decide to save your pixels to jpeg, your secret in the DCT coefficients may be disturbed by another layer of quantisation from the jpeg encoding.
I am trying to compress a grayscale image using Huffman coding in MATLAB, and have tried the following code.
I have used a grayscale image with size 512x512 in tif format. My problem is that the size of the compressed image (length of the compressed codeword) is getting bigger than the size of the uncompressed image. The compression ratio is getting less than 1.
clc;
clear all;
A1 = imread('fig1.tif');
[M N]=size(A1);
A = A1(:);
count = [0:1:255]; % Distinct data symbols appearing in sig
total=sum(count);
for i=1:1:size((count)');
p(i)=count(i)/total;
end
[dict,avglen]=huffmandict(count,p) % build the Huffman dictionary
comp= huffmanenco(A,dict); %encode your original image with the dictionary you just built
compression_ratio= (512*512*8)/length(comp) %computing the compression ratio
%% DECODING
Im = huffmandeco(comp,dict); % Decode the code
I11=uint8(Im);
decomp=reshape(I11,M,N);
imshow(decomp);
There is a slight error in your code. I'm assuming you want to calculate the probability of encountering each pixel, which is the normalized histogram. You're not computing it properly. Specifically:
count = [0:1:255]; % Distinct data symbols appearing in sig
total=sum(count);
for i=1:1:size((count)');
p(i)=count(i)/total;
end
total is summing over [0,255] which is not correct. You're supposed to compute the probability distribution of your image. You should use imhist for that instead. As such, you should do this instead:
count = 0:255;
p = imhist(A1) / numel(A1);
This will correctly calculate your probability distribution for your image. Remember, when you're doing Huffman coding, you need to specify the probability of encountering a pixel. Assuming that each pixel can equally be likely to be chosen, this is captured by calculating the image's histogram, then normalizing by the total number of pixels in your image. Try that and see if you get any better results.
However, Huffman will only give you good compression ratios if you have frequently occurring symbols. Did you happen to take a look at the histogram or the spread of your pixels in your image?
If the spread is quite large, with very few entries per bin, then Huffman will not give you any compression savings. In fact it may give you a larger size as a result. Bear in mind that the TIFF compression standard only uses Huffman as part of the algorithm. There is also some pre- and post-processing done to further drive down the size.
As a further example, suppose I had an image that consisted of [0, 1, 2, ... 255; 0, 1, 2, ..., 255; 0, 1, 2, ..., 255]; I have 3 rows of [0,255], but really it could be any number of rows. This means that the probability of encountering each symbol is equiprobable, or 1/255, which means that for each symbol, we would need 8 bits per symbol... which is essentially the raw pixel value anyway!
The key behind Huffman is that a group of bits together generate one symbol. Frequently occurring symbols get assigned a smaller sequence of bits. Because this particular image that I talked about has intensities that are equiprobable, then you'd only generate one symbol per intensity rather than a group. With this, not only will you transmit the dictionary, you would effectively be sending one character at a time, and this is no better than sending the raw byte stream.
If you want your image to be compressed by raw Huffman, the distribution of pixels has to be skewed. For example, if most of the intensities in your image are dark, or are bright. If your image has good contrast or if the spread of the pixel intensities is flat throughout the image, then Huffman will not give you any compression savings.
I have a bunch (up to 255) of colors (of the total 256^3 possible values) and for compression purposes I want to think up some another color, that isn't among them.
For example, I have such a small color table: [0,0,0], [1,42,69] -- any one of the remaining 256^3-2 colors would be fine -- no matter whether it is [0,0,7] or [6,6,6].
Can someone provide me with an easy and efficient algorithm to find another color?
UPD: bad ideas are also welcome.
Make a hash table of all known colors, and put your colors into it.
Make an algorithm that takes a color, and produces its "successor" by incrementing the lowest byte, and continuing with the increment into higher-order bytes when there is a carry.
Start at [0,0,0] and check it against the hash table from step 1.
Loop until you find the first gap.
This algorithm is linear in the number of colors.
Since now we have two answers, I would like to post my own.
We don't want any color in existing color table to become transparent. That is why I stated, that color table can be maximum of 255 colors long.
Because of this there would be at least one Red (or Green or Blue, whatever) channel value left unused. So we don't have to use 256^3 large table of flags -- 256 (bits for memory or bytes for speed) would be enough.
Walk through your image, counting the number of times each of the 256 possible pixel values occurs. Use std::min_element (for one possibility) to find the smallest count, and use that color number. If you're really talking about 256 possible color values, that's about it.
If you really have 24 bits per pixel, then you probably want to use a sparse representation for the counts, since (for any reasonable size of picture) many of them will inevitably be zero (you'd need roughly a 16-megapixel picture to even theoretically use all the possible colors). OTOH, on a modern computer, even using the few dozen megabytes (or so) necessary for a dense representation of the count may be worthwhile--it'll probably make your processing faster (no hash codes to compute) and still little enough memory usage that the reduction in processing time is worth it.
Typically, the most common RGB format seems to be 24-bit RGB (8-bits for each channel). However, historically, RGB has been represented in many other formats, including 3-bit RGB (1-bit per channel), 6-bit RGB (2-bits per channel), 9-bit RGB (3-bits per channel), etc.
When an N-bit RGB file has a value of N that is not a multiple of 8, how are these bitmaps typically represented in memory? For example, if we have 6-bit RGB, it means each pixel is 6 bits, and thus each pixel is not directly addressable by a modern computer without using bitwise operations.
So, is it common practice to simply covert N-bit RGB files into bitmaps where each pixel is of an addressable size (e.g. convert 6-bit to 8-bit)? Or is it more common to simply use bitwise operations to manipulate bitmaps where the pixel size is not addressable?
And what about disk storage? How is, say, a 6-bit RGB file stored on disk, when the last byte of the bitmap may not even contain a full pixel?
Images are often heavy and bandwidth is critical when transferring them. So a 6 bit per channel image is a reasonable choice if some loss in chrominance is acceptable (usually unnoticeable with textures and photos)
How is, say, a 6-bit RGB file stored on disk, when the last byte of
the bitmap may not even contain a full pixel?
If the smallest unit of storage is a Byte then yes you need to add some padding to be 8 bit aligned. This is fine because the space saving compared to an 8 bit per channel image can be considerable.
A power of 2 value that is divisible by 6 is very large. Better numbers are 5 bits for the red and blue channels and 6 bits for the green channel and the total is 16 bits per pixel. R5G6B5 is a very common pixel format.
Apologies for the archeological dig, but I just couldn't resist, as there's no good answer imho.
In the old days memory was the most expensive part of a computer. These days memory is dirt-cheap, so the most sensible way to handle N-bit channel images in modern hardware is to blow up every channel to the number of bits that fits your API or hardware (usually 8 bits).
For files, you could do the same, as disk space is also cheap these days. (and you can use one of the many compressed file formats out there.)
That said, the way it worked in the old days when these formats were common, is this:
The total number of bits per pixel was not a multiple of 8, but the number of pixels per scan line always was a multiple of 8. In this case you can store your scan lines "a bit at a time" and not waste any memory space when storing it in bytes.
So if your pixels were 9 bits per pixel, and a scan line was 320 pixels, you would have 320/8 = 40 bytes containing bit #0 of each pixel, followed by 40 bytes containing all bit #1's etc. up to and including bit #8. Hence all pixel info for your scan line would be exactly 360 bytes.
The video chips would have a different hardware wiring to the memory, so rendering such scan lines was fast. In fact, this is the easiest way to implement a variable amount of bits/pixel hardware support: by pulling bits from N adresses at once.
Note that this method does not change the amount of 'bitshifting' required to find the bits for pixel number X in a scanline, based on the total number of bits you use. You just read less addresses ahead at once.
At the moment I am working on an on screen display project with black, white and transparent pixels. (This is an open source project: http://code.google.com/p/super-osd; that shows the 256x192 pixel set/clear OSD in development but I'm migrating to a white/black/clear OSD.)
Since each pixel is black, white or transparent I can use a simple 2 bit/4 state encoding where I store the black/white selection and the transparent selection. So I would have a truth table like this (x = don't care):
B/W T
x 0 pixel is transparent
0 1 pixel is black
1 1 pixel is white
However as can be clearly seen this wastes one bit when the pixel is transparent. I'm designing for a memory constrained microcontroller, so whenever I can save memory it is good.
So I'm trying to think of a way to pack these 3 states into some larger unit (say, a byte.) I am open to using lookup tables to decode and encode the data, so a complex algorithm can be used, but it cannot depend on the states of the pixels before or after the current unit/byte (this rules out any proper data compression algorithm) and the size must be consistent; that is, a scene with all transparent pixels must be the same as a scene with random noise. I was imagining something on the level of densely packed decimal which packs 3 x 4-bit (0-9) BCD numbers in only 10 bits with something like 24 states remaining out of the 1024, which is great. So does anyone have any ideas?
Any suggestions? Thanks!
In a byte (256 possible values) you can store 5 of your three-bit values. One way to look at it: three to the fifth power is 243, slightly less than 256. The fact that it's slightly less also shows that you're not wasting much of a fraction of a bit (hardly any, either).
For encoding five of your 3-bit "digits" into a byte, think of taking a number in base 3 made from your five "digits" in succession -- the resulting value is guaranteed to be less than 243 and therefore directly storable in a byte. Similarly, for decoding, do the base-3 conversion of a byte's value.