I'm working on an implementation of the JPEG compression algorithm in MATLAB. I've run into some issues when computing the discrete cosine transform(DCT) of the 8x8 image blocks(T = H * F * H_transposed, H is the matrix containing the DCT coefficients of an 8x8 matrix, generated with dctmtx(8) and F is an 8x8 image block). The code is bellow:
jpegCompress.m
function y = jpegCompress(x, quality)
% y = jpegCompress(x, quality) compresses an image X based on 8 x 8 DCT
% transforms, coefficient quantization and Huffman symbol coding. Input
% quality determines the amount of information that is lost and compression achieved. y is the encoding structure containing fields:
% y.size size of x
% y.numblocks number of 8 x 8 encoded blocks
% y.quality quality factor as percent
% y.huffman Huffman coding structure
narginchk(1, 2); % check number of input arguments
if ~ismatrix(x) || ~isreal(x) || ~ isnumeric(x) || ~ isa(x, 'uint8')
error('The input must be a uint8 image.');
end
if nargin < 2
quality = 1; % default value for quality
end
if quality <= 0
error('Input parameter QUALITY must be greater than zero.');
end
m = [16 11 10 16 24 40 51 61 % default JPEG normalizing array
12 12 14 19 26 58 60 55 % and zig-zag reordering pattern
14 13 16 24 40 57 69 56
14 17 22 29 51 87 80 62
18 22 37 56 68 109 103 77
24 35 55 64 81 104 113 92
49 64 78 87 103 121 120 101
72 92 95 98 112 100 103 99] * quality;
order = [1 9 2 3 10 17 25 18 11 4 5 12 19 26 33 ...
41 34 27 20 13 6 7 14 21 28 35 42 49 57 50 ...
43 36 29 22 15 8 16 23 30 37 44 51 58 59 52 ...
45 38 31 24 32 39 46 53 60 61 54 47 40 48 55 ...
62 63 56 64];
[xm, xn] = size(x); % retrieve size of input image
x = double(x) - 128; % level shift input
t = dctmtx(8); % compute 8 x 8 DCT matrix
% Compute DCTs pf 8 x 8 blocks and quantize coefficients
y = blkproc(x, [8 8], 'P1 * x * P2', t, t');
y = blkproc(y, [8 8], 'round(x ./ P1)', m); % <== nearly all elements from y are zero after this step
y = im2col(y, [8 8], 'distinct'); % break 8 x 8 blocks into columns
xb = size(y, 2); % get number of blocks
y = y(order, :); % reorder column elements
eob = max(x(:)) + 1; % create end-of-block symbol
r = zeros(numel(y) + size(y, 2), 1);
count = 0;
for j = 1:xb % process one block(one column) at a time
i = find(y(:, j), 1, 'last'); % find last non-zero element
if isempty(i) % check if there are no non-zero values
i = 0;
end
p = count + 1;
q = p + i;
r(p:q) = [y(1:i, j); eob]; % truncate trailing zeros, add eob
count = count + i + 1; % and add to output vector
end
r((count + 1):end) = []; % delete unused portion of r
y = struct;
y.size = uint16([xm xn]);
y.numblocks = uint16(xb);
y.quality = uint16(quality * 100);
y.huffman = mat2huff(r);
mat2huff is implemented as:
mat2huff.m
function y = mat2huff(x)
%MAT2HUFF Huffman encodes a matrix.
% Y = mat2huff(X) Huffman encodes matrix X using symbol
% probabilities in unit-width histogram bins between X's minimum
% and maximum value s. The encoded data is returned as a structure
% Y :
% Y.code the Huffman - encoded values of X, stored in
% a uint16 vector. The other fields of Y contain
% additional decoding information , including :
% Y.min the minimum value of X plus 32768
% Y.size the size of X
% Y.hist the histogram of X
%
% If X is logical, uintB, uint16 ,uint32 ,intB ,int16, or double,
% with integer values, it can be input directly to MAT2HUF F. The
% minimum value of X must be representable as an int16.
%
% If X is double with non - integer values --- for example, an image
% with values between O and 1 --- first scale X to an appropriate
% integer range before the call.For example, use Y
% MAT2HUFF (255 * X) for 256 gray level encoding.
%
% NOTE : The number of Huffman code words is round(max(X(:)))
% round (min(X(:)))+1. You may need to scale input X to generate
% codes of reasonable length. The maximum row or column dimension
% of X is 65535.
if ~ismatrix(x) || ~isreal(x) || (~isnumeric(x) && ~islogical(x))
error('X must be a 2-D real numeric or logical matrix.');
end
% Store the size of input x.
y.size = uint32(size(x));
% Find the range of x values
% by +32768 as a uint16.
x = round(double(x));
xmin = min(x(:));
xmax = max(x(:));
pmin = double(int16(xmin));
pmin = uint16(pmin+32768);
y.min = pmin;
% Compute the input histogram between xmin and xmax with unit
% width bins , scale to uint16 , and store.
x = x(:)';
h = histc(x, xmin:xmax);
if max(h) > 65535
h = 65535 * h / max(h);
end
h = uint16(h);
y.hist = h;
% Code the input mat rix and store t h e r e s u lt .
map = huffman(double(h)); % Make Huffman code map
hx = map(x(:) - xmin + 1); % Map image
hx = char(hx)'; % Convert to char array
hx = hx(:)';
hx(hx == ' ') = [ ]; % Remove blanks
ysize = ceil(length(hx) / 16); % Compute encoded size
hx16 = repmat('0', 1, ysize * 16); % Pre-allocate modulo-16 vector
hx16(1:length(hx)) = hx; % Make hx modulo-16 in length
hx16 = reshape(hx16, 16, ysize); % Reshape to 16-character words
hx16 = hx16' - '0'; % Convert binary string to decimal
twos = pow2(15 : - 1 : 0);
y.code = uint16(sum(hx16 .* twos(ones(ysize ,1), :), 2))';
Why is the block processing step generating mostly null values?
It is likely that multiplying the Quantization values you have by four is causing the DCT coefficients to go to zero.
Related
Context:
I have a hydraulic erosion algorithm that needs to receive an array of droplet starting positions. I also already have a pattern replicating algorithm, so I only need a good pattern to replicate.
The Requirements:
I need an algorism that produces a set of n^2 entries in a set of format (x,y) or [index] that describe cells in an nxn grid (where n = 2^i where i is any positive integer).
(as a set it means that every cell is mentioned in exactly one entry)
The pattern [created by the algorism ] should contain zero to none clustering of "visited" cells at any stage.
The cell (0,0) is as close to (n-1,n-1) as to (1,1), this relates to the definition of clustering
Note
I was/am trying to find solutions through fractal-like patterns built through recursion, but at the time of writing this, my solution is a lookup table of a checkerboard pattern(list of black cells + list of white cells) (which is bad, but yields fewer artifacts than an ordered list)
C, C++, C#, Java implementations (if any) are preferred
You can use a linear congruential generator to create an even distribution across your n×n space. For example, if you have a 64×64 grid, using a stride of 47 will create the pattern on the left below. (Run on jsbin) The cells are visited from light to dark.
That pattern does not cluster, but it is rather uniform. It uses a simple row-wide transformation where
k = (k + 47) mod (n * n)
x = k mod n
y = k div n
You can add a bit of randomness by making k the index of a space-filling curve such as the Hilbert curve. This will yield the pattern on the right. (Run on jsbin)
You can see the code in the jsbin links.
I have solved the problem myself and just sharing my solution:
here are my outputs for the i between 0 and 3:
power: 0
ordering:
0
matrix visit order:
0
power: 1
ordering:
0 3 2 1
matrix visit order:
0 3
2 1
power: 2
ordering:
0 10 8 2 5 15 13 7 4 14 12 6 1 11 9 3
matrix visit order:
0 12 3 15
8 4 11 7
2 14 1 13
10 6 9 5
power: 3
ordering:
0 36 32 4 18 54 50 22 16 52 48 20 2 38 34 6
9 45 41 13 27 63 59 31 25 61 57 29 11 47 43 15
8 44 40 12 26 62 58 30 24 60 56 28 10 46 42 14
1 37 33 5 19 55 51 23 17 53 49 21 3 39 35 7
matrix visit order:
0 48 12 60 3 51 15 63
32 16 44 28 35 19 47 31
8 56 4 52 11 59 7 55
40 24 36 20 43 27 39 23
2 50 14 62 1 49 13 61
34 18 46 30 33 17 45 29
10 58 6 54 9 57 5 53
42 26 38 22 41 25 37 21
the code:
public static int[] GetPattern(int power, int maxReturnSize = int.MaxValue)
{
int sideLength = 1 << power;
int cellsNumber = sideLength * sideLength;
int[] ret = new int[cellsNumber];
for ( int i = 0 ; i < cellsNumber && i < maxReturnSize ; i++ ) {
// this loop's body can be used for per-request computation
int x = 0;
int y = 0;
for ( int p = power - 1 ; p >= 0 ; p-- ) {
int temp = (i >> (p * 2)) % 4; //2 bits of the index starting from the begining
int a = temp % 2; // the first bit
int b = temp >> 1; // the second bit
x += a << power - 1 - p;
y += (a ^ b) << power - 1 - p;// ^ is XOR
// 00=>(0,0), 01 =>(1,1) 10 =>(0,1) 11 =>(1,0) scaled to 2^p where 0<=p
}
//to index
int index = y * sideLength + x;
ret[i] = index;
}
return ret;
}
I do admit that somewhere along the way the values got transposed, but it does not matter because of how it works.
After doing some optimization I came up with this loop body:
int x = 0;
int y = 0;
for ( int p = 0 ; p < power ; p++ ) {
int temp = ( i >> ( p * 2 ) ) & 3;
int a = temp & 1;
int b = temp >> 1;
x = ( x << 1 ) | a;
y = ( y << 1 ) | ( a ^ b );
}
int index = y * sideLength + x;
(the code assumes that c# optimizer, IL2CPP, and CPP compiler will optimize variables temp, a, b out)
I want to map a mX1 matrix X into mXp matrix Y where each row in the new matrix is as follows:
Y = [ X X.^2 X.^3 ..... X.^p]
I tried to use the following code:
Y = zeros(m, p);
for i=1:m
Y(i,:) = X(i);
for c=2:p
Y(i,:) = [Y(i,:) X(i).^p];
end
end
What you want do is called brodcasting. If you are using Octave 3.8 or later, the following will work fine:
octave> X = (1:5)'
X =
1
2
3
4
5
octave> P = (1:5)
P =
1 2 3 4 5
octave> X .^ P
ans =
1 1 1 1 1
2 4 8 16 32
3 9 27 81 243
4 16 64 256 1024
5 25 125 625 3125
The important thing to note is how X and P are a column and row vector respectively. See the octave manual on the topic.
For older of versions of Octave (without automatic broadcasting), the same can be accomplished with bsxfun (#power, X, P)
I am attempting to reclassify continuous data to categorical data using Matlab. The following script takes a 4-band (Red, Green, Blue, nIR) aerial image and calculates the normalized difference vegetation index (i.e. a vegetation index showing healthy green vegetation). The script then rescales the values from (-1 to 1) to (0 - 255). This is the matrix I am trying to reclassify in the third section of the script %% Reclassify Imag1 matrix. I am attempting to use conditional statements to perform the reclassification, although this may be the wrong approach. The reclassification step in the script does not have any apparent effect.
How can I reclassify continuous values (0 - 255) to categorical values (1, 2, 3, 4) on a cell by cell basis?
file = 'F:\path\to\naip\image\4112107_ne.tif';
[Z R] = geotiffread(file);
outputdir = 'F:\temp\';
%% Make NDVI calculations
NIR = im2single(Z(:,:,4));
red = im2single(Z(:,:,3));
ndvi = (NIR - red) ./ (NIR + red);
ndvi = double(ndvi);
%% Stretch NDVI to 0-255 and convert to 8-bit unsigned integer
ndvi = floor((ndvi + 1) * 128); % [-1 1] -> [0 256]
ndvi(ndvi < 0) = 0; % not really necessary, just in case & for symmetry
ndvi(ndvi > 255) = 255; % in case the original value was exactly 1
Imag1 = uint8(ndvi);
%% Reclassify Imag1 matrix
if (150 <= Imag1)
Imag1 = 1;
elseif (150 > Imag1) & (140 < Imag1)
Imag1 = 2;
elseif (140 > Imag1) & (130 < Imag1)
Imag1 = 3;
elseif (130 >= Imag1)
Imag1 = 4;
end
%% Write the results to disk
tiffdata = geotiffinfo(file);
outfilename = [outputdir 'reclass_ndvi' '.tif'];
geotiffwrite(outfilename, Imag1, R, 'GeoKeyDirectoryTag', tiffdata.GeoTIFFTags.GeoKeyDirectoryTag)
disp('Processing complete')
Try this:
Imag1 = [ 62 41 169 118 210;
133 158 96 149 110;
211 200 84 194 29;
209 16 15 146 28;
95 144 13 249 170];
Imag1(find(Imag1 <= 130)) = 4;
Imag1(find(Imag1 >= 150)) = 1;
Imag1(find(Imag1 > 140)) = 2;
Imag1(find(Imag1 > 130)) = 3;
Result:
Imag1 =
62 41 169 118 210
133 158 96 149 110
211 200 84 194 29
209 16 15 146 28
95 144 13 249 170
Imag1 =
4 4 1 4 1
3 1 4 2 4
1 1 4 1 4
1 4 4 2 4
4 2 4 1 1
I can go into the logic in detail if you like, but I wanted to confirm that this gives your expected results first.
Some updates based on comments on the follow-up question to eliminate the unnecessary find and make the code more robust and independent of execution order.
Imag2 = zeros(size(Imag1));
Imag2(Imag1 >= 150) = 1;
Imag2((Imag1 > 140) & (Imag1 < 150)) = 2;
Imag2((Imag1 > 130) & (Imag1 < 141)) = 3;
Imag2(Imag1 <= 130) = 4;
Note that the results are now in Imag2 instead of overwriting Imag1.
I want to randomly generate points. Well at least there should be a limitation on the y-axis. Later I connect the points to a line which should proceed in a simple animation. You can imagine this as a random walk of a drunken person, going uphill and downhill.
This sounds very simple. I searched around the web and found that this could be accomplished using the markov chain. I think this idea is really interesting.
You can create the first state of your scene by yourself and pass this state as input to the markov chain algorithm. The algorithm randomly changes this state and creates a walk.
However I cannot find any example of that algorithm and no source code. I just found an applet that demonstrates the markov chain algorithm: http://www.probability.ca/jeff/java/unif.html
Please suggest some code. Any other ideas how to accomplish this are appreciated too.
I painted an example
So I want the line to proceed in a similar way. There are valleys, slopes ... they are random but the randomness still apply to the initial state of the line. This is why I found makrov chain so interesting here: http://www.suite101.com/content/implementing-markov-chains-a24146
Here's some code in Lua:
absstepmax = 25
ymin = -100
ymax = 100
x = 0
y = 5
for i = 1, 20 do
y = y + (math.random(2*absstepmax) - absstepmax - 1)
y = math.max(ymin, math.min(ymax, y))
x = x + 5
print (x,y)
end
absstepmax limits the size of a y step per iteration
ymin and ymax limit the extent of y
There is no bias in the example, i.e., y can change symmetrically up or down. If you want your "drunk" tending more "downhill" you can change the offset after the call to random from absstepmax - 1 to absstepmax - 5 or whatever bias you like.
In this example, the x step is fixed. You may make this random as well using the same mechanisms.
Here are some sample runs:
> absstepmax = 25
> ymin = -100
> ymax = 100
> x = 0
> y = 5
> for i = 1, 20 do
>> y = y + (math.random(2*absstepmax) - absstepmax - 1)
>> y = math.max(ymin, math.min(ymax, y))
>> x = x + 5
>> print (x,y)
>> end
5 4
10 22
15 37
20 39
25 50
30 40
35 21
40 22
45 12
50 16
55 16
60 12
65 -1
70 -8
75 -14
80 -17
85 -19
90 -25
95 -37
100 -59
> absstepmax = 25
> ymin = -100
> ymax = 100
> x = 0
> y = 5
> for i = 1, 20 do
>> y = y + (math.random(2*absstepmax) - absstepmax - 1)
>> y = math.max(ymin, math.min(ymax, y))
>> x = x + 5
>> print (x,y)
>> end
5 -2
10 -15
15 -7
20 1
25 1
30 12
35 23
40 45
45 43
50 65
55 56
60 54
65 54
70 62
75 57
80 62
85 86
90 68
95 76
100 68
>
Painted result added from OP:
How can one generate say 1000 random points with a distribution like that of
towns and cities in e.g. Ohio ?
I'm afraid I can't define "distributed like cities" precisely;
uniformly distributed centres + small Gaussian clouds
are easy but ad hoc.
Added: There must be a family of 2d distributions
with a clustering parameter that can be varied to match a given set of points ?
Maybe you can take a look at Walter Christaller's Theory of Central Places. I guess there must be some generator somewhere, or you can cook up your own.
Start with a model of the water features in your target area (or make one up, if it's for an imaginary place), then cluster the cities near river junctions, along lakeshores, lake-river junctions. Then make imaginary highways connecting those major cities. Now sprinkle some intermediate cities along those highways at reasonable spacing, preferring to be near junctions in the highways. Now sprinkle some small towns through the empty spaces.
Gaussian clusters with Poisson cluster sizes work fairly well.
Problem: generate random points that cluster roughly like given cities, say in the USA.
Subproblems:
a) describe clusters with rows of numbers, so that "cluster A is like cluster B"
simplifies to "clusternumbers(A) is like "clusternumbers(B)".
Running N=100 then 1000 points through fcluster below, with ncluster=25, gives
N 100 ncluster 25: 22 + 3 r 117
sizes: av 4 10 9 8 7 6 6 5 5 4 4 4 ...
radii: av 117 202 198 140 134 64 62 28 197 144 148 132 ...
N 1000 cluster 25: 22 + 3 r 197
sizes: av 45 144 139 130 85 84 69 63 43 38 33 30 ...
radii: av 197 213 279 118 146 282 154 245 212 243 226 235 ...
b) find a combiation of random generators with 2 or 3 parameters
which can be varied to generate different clusterings.
Gaussian clusters with Poisson cluster sizes can match clustering of cities fairly well:
def randomclusters( N, ncluster=25, radius=1, box=box ):
""" -> N 2d points: Gaussian clusters, Poisson cluster sizes """
pts = []
lam = eval( str( N // ncluster ))
clustersize = lambda: np.random.poisson(lam - 1) + 1
# poisson 2: 14 27 27 18 9 4 %
# poisson 3: 5 15 22 22 17 10 %
while len(pts) < N:
u = uniformrandom2(box)
csize = clustersize()
if csize == 1:
pts.append( u )
else:
pts.extend( inbox( gauss2( u, radius, csize )))
return pts[:N]
# Utility functions --
import scipy.cluster.hierarchy as hier
def fcluster( pts, ncluster, method="average", criterion="maxclust" ):
""" -> (pts, Y pdist, Z linkage, T fcluster, clusterlists)
ncluster = n1 + n2 + ... (including n1 singletons)
av cluster size = len(pts) / ncluster
"""
# Clustering is pretty fast:
# sort pdist, then like Kruskal's MST, O( N^2 ln N )
# Many metrics and parameters are possible; these satisfice.
pts = np.asarray(pts)
Y = scipy.spatial.distance.pdist( pts ) # N*(N-1)/2
Z = hier.linkage( Y, method ) # N-1, like mst
T = hier.fcluster( Z, ncluster, criterion=criterion )
clusters = clusterlists(T)
return (pts, Y, Z, T, clusters)
def clusterlists(T):
""" T = hier.fcluster( Z, t ) e.g. [a b a b c a]
-> [ [0 2 5] [1 3] ] sorted by len, no singletons [4]
"""
clists = [ [] for j in range( max(T) + 1 )]
for j, c in enumerate(T):
clists[c].append( j )
clists.sort( key=len, reverse=True )
n1 = np.searchsorted( map( len, clists )[::-1], 2 )
return clists[:-n1]
def radius( x ):
""" rms |x - xmid| """
return np.sqrt( np.mean( np.var( x, axis=0 )))
# * 100 # 1 degree lat/long ~ 70 .. 111 km
In java this is provided through new Random().nextGaussian(). Since the java source is available, you can look at it:
synchronized public double nextGaussian() {
// See Knuth, ACP, Section 3.4.1 Algorithm C.
if (haveNextNextGaussian) {
haveNextNextGaussian = false;
return nextNextGaussian;
} else {
double v1, v2, s;
do {
v1 = 2 * nextDouble() - 1; // between -1 and 1
v2 = 2 * nextDouble() - 1; // between -1 and 1
s = v1 * v1 + v2 * v2;
} while (s >= 1 || s == 0);
double multiplier = StrictMath.sqrt(-2 * StrictMath.log(s)/s);
nextNextGaussian = v2 * multiplier;
haveNextNextGaussian = true;
return v1 * multiplier;
}
}
Plotting 30000 houses using
x = r.nextGaussian() * rad/4 + rad;
y = r.nextGaussian() * rad/4 + rad;
yields this beautiful city: