I am a little confused about the Huffman code. So as I read the books, it states that after the zigzag ordering, it will be the run length encoding and the Huffman for the run length. I have 3 questions:
1) Is it necessary to do both run length encoding and Huffman, or just Huffman for the whole image( which is gray scale). I mean like could I just scan the block 8x8 and count the frequency of appearance of characters, then create the codewords.
2) If I use the run length coding for each block, so the Huffman is also for each block of 8x8, or I have to scan through all the whole image.
3) In the book it states that we could just use the Table K.3 and Table K.5 in Annex K for the DC and AC coefficient encoding. Could I not use those tables and generate my own based on the theory in question 2 which Im also confused.
Thank you for helping me out
This is the link for the Annex K
https://www.w3.org/Graphics/JPEG/itu-t81.pdf
You COULD compress like you are saying but it would not be JPEG. The encoding process is rather complicated in JPEG. It is not really Huffman encoding of values. It is Huffman encoding of instructions on zero runs and the number of additional raw bits that have to be read.
1) Is it necessary to do both run length encoding and Huffman, or just Huffman for the whole image( which is gray scale).
For it to be a JPEG stream, you have to do both.
2) 2) If I use the run length coding for each block, so the Huffman is also for each block of 8x8, or I have to scan through all the whole image.
Some encoders do that to generate optimum Huffman tables.
3) In the book it states that we could just use the Table K.3 and Table K.5 in Annex K for the DC and AC coefficient encoding. Could I not use those tables and generate my own based on the theory in question 2 which Im also confused.
Some encoders do that to avoid having to make two passes over the DCT data to generate Huffman tables.
Related
I'm parsing HEVC [H.265] header and I noticed that many values are in Golomb code notation. One of them, for example, is the width.
Let's suppose a width value of 1600, in Golomb code is written as:
g=000000000001001000001
call "leadingZero" (lz) the first part of the string (from left to right).
LeadingZero is composed by 11 zeros. Let's call b the rest of the string of Golomb code.
to decode the Golomb code, where b=1001000001 (or decimal 577), you do:
a=2^(lz-1)-1;
n=a+to_decimal(b)
where to_decimal converts from binary to decimal value.
So you have 1023 + 577 = 1600.
Question:
With Golomb you're using 21 bits to represent 1600.
But 1600 in binary takes 11 bits (110 0100 0000).
Also the Golomb method does not allow for a custom number of bits to represent values.
So... Why Golomb code is used in compression algorithms like H.265?
Well, usually compression of High Level Syntax (HLS) is not a critical priority in video compression. If you do the math for a typical resolution (e.g. 1080p) in a typical bandwidth (e.g. 7 Mbps), you will see that saving a few bits to signal frame-level and sequence-level information is really negligible.
However, since ex Colomb code is also used in signalling large DCT coefficients, one might ask the same question in that context. And it would be a valid compression concern, as efficiency residual coding is everything! To answer that question, there are a lot of well stablished literature, dating back to AVC time.
I implement the canonical Huffman algorithm and there are several questions on the theoretical part, namely about storing of information for decoding. As a method, it's proposed to transmit alphabetical characters and lengths of their canonical codes together with encoded data, because for restoring the canonical table, we only need the code lengths.
Example: string "bbbaacd". Canonical codes: b 0 (1 bit) a 10 (2) c 110 (3) d 111 (3) i.e. decoding data: b1a2c3d3. This raises several questions.
1)Is it necessary to transfer this table in one file along with encoded data according to the table (at the end / beginning of the file)? Are there any real examples?
2)If yes, then if there are numbers in the data, how to understand where in our table is the alphabet symbol (number), and where is the number of bits (length)?
3)And finally, how to understand where is the border between the table and the encoded data?
If everything is stored in separate files (which, in my opinion, is simpler and more logical), then the last 2 questions disappear by themselves.
Normally the code lengths are stored in the same file as the data. If you set a maximum code length of 16 bits, for example, then the first 128 bytes in your file can be the lengths of the codes for all 256 bytes (4 bits each).
A slightly more sophisticated way would be to Huffman-encode those lengths. You can use a fixed tree for that... or you can use a dynamic tree and first write out the lengths of the symbols for each length.
I have a low-resource embedded system with a graphical user interface. The interface requires font data. To conserve read-only memory (flash), the font data needs to be compressed. I am looking for an algorithm for this purpose.
Properties of the data to be compressed
transparency data for a rectangular pixel map with 8 bits per pixel
there are typically around 200..300 glyphs in a font (typeface sampled in certain size)
each glyph is typically from 6x9 to 15x20 pixels in size
there are a lot of zeros ("no ink") and somewhat less 255's ("completely inked"), otherwise the distribution of octets is quite even due to the nature of anti-aliasing
Requirements for the compression algorithm
The important metrics for the decompression algorithm is the size of the data plus the size of the algorithm (as they will reside in the same limited memory).
There is very little RAM available for the decompression; it is possible to decompress the data for a single glyph into RAM but not much more.
To make things more difficult, the algorithm has to be very fast on a 32-bit microcontroller (ARM Cortex-M core), as the glyphs need to be decompressed while they are being drawn onto the display. Ten or twenty machine cycles per octet is ok, a hundred is certainly too much.
To make things easier, the complete corpus of data is known a priori, and there is a lot of processing power and memory available during the compression phase.
Conclusions and thoughts
The naïve approach of just packing each octet by some variable-length encoding does not give good results due to the relatively high entropy.
Any algorithm taking advantage of data decompressed earlier seems to be out of question as it is not possible to store the decompressed data of other glyphs. This makes LZ algorithms less efficient as they can only reference to a small amount of data.
Constraints on the processing power seem to rule out most bitwise operations, i.e. decompression should handle the data octet-by-octet. This makes Huffman coding difficult and arithmetic coding impossible.
The problem seems to be a good candidate for static dictionary coding, as all data is known beforehand, and the data is somewhat repetitive in nature (different glyphs share same shapes).
Questions
How can a good dictionary be constructed? I know finding the optimal dictionary for certain data is a np complete problem, but are there any reasonably good approximations? I have tried the zstandard's dictionary builder, but the results were not very good.
Is there something in my conclusions that I've gotten wrong? (Am I on the wrong track and omitting something obvious?)
Best algorithm this far
Just to give some background information, the best useful algorithm I have been able to figure out is as follows:
All samples in the font data for a single glyph are concatenated (flattened) into a one-dimensional array (vector, table).
Each sample has three possible states: 0, 255, and "something else".
This information is packed five consecutive samples at a time into a 5-digit base-three number (0..3^5).
As there are some extra values available in an octet (2^8 = 256, 3^5 = 243), they are used to signify longer strings of 0's and 255's.
For each "something else" value the actual value (1..254) is stored in a separate vector.
This data is fast to decompress, as the base-3 values can be decoded into base-4 values by a smallish (243 x 3 = 729 octets) lookup table. The compression ratios are highly dependent on the font size, but with my typical data I can get around 1:2. As this is significantly worse than LZ variants (which get around 1:3), I would like to try the static dictionary approach.
Of course, the usual LZ variants use Huffman or arithmetic coding, which naturally makes the compressed data smaller. On the other hand, I have all the data available, and the compression speed is not an issue. This should make it possible to find much better dictionaries.
Due to the nature of the data I could be able to use a lossy algorithm, but in that case the most likely lossy algorithm would be reducing the number of quantization levels in the pixel data. That won't change the underlying compression problem much, and I would like to avoid the resulting bit-alignment hassle.
I do admit that this is a borderline case of being a good answer to my question, but as I have researched the problem somewhat, this answer both describes the approach I chose and gives some more information on the nature of the problem should someone bump into it.
"The right answer" a.k.a. final algorithm
What I ended up with is a variant of what I describe in the question. First, each glyph is split into trits 0, 1, and intermediate. This ternary information is then compressed with a 256-slot static dictionary. Each item in the dictionary (or look-up table) is a binary encoded string (0=0, 10=1, 11=intermediate) with a single 1 added to the most significant end.
The grayscale data (for the intermediate trits) is interspersed between the references to the look-up table. So, the data essentially looks like this:
<LUT reference><gray value><gray value><LUT reference>...
The number of gray scale values naturally depends on the number of intermediate trits in the ternary data looked up from the static dictionary.
Decompression code is very short and can easily be written as a state machine with only one pointer and one 32-bit variable giving the state. Something like this:
static uint32_t trits_to_decode;
static uint8_t *next_octet;
/* This should be called when starting to decode a glyph
data : pointer to the compressed glyph data */
void start_glyph(uint8_t *data)
{
next_octet = data; // set the pointer to the beginning of the glyph
trits_to_decode = 1; // this triggers reloading a new dictionary item
}
/* This function returns the next 8-bit pixel value */
uint8_t next_pixel(void)
{
uint8_t return_value;
// end sentinel only? if so, we are out of ternary data
if (trits_to_decode == 1)
// get the next ternary dictionary item
trits_to_decode = dictionary[*next_octet++];
// get the next pixel from the ternary word
// check the LSB bit(s)
if (trits_to_decode & 1)
{
trits_to_decode >>= 1;
// either full value or gray value, check the next bit
if (trits_to_decode & 1)
{
trits_to_decode >>= 1;
// grayscale value; get next from the buffer
return *next_octet++;
}
// if we are here, it is a full value
trits_to_decode >>= 1;
return 255;
}
// we have a zero, return it
trits_to_decode >>= 1;
return 0;
}
(The code has not been tested in exactly this form, so there may be typos or other stupid little errors.)
There is a lot of repetition with the shift operations. I am not too worried, as the compiler should be able to clean it up. (Actually, left shift could be even better, because then the carry bit could be used after shifting. But as there is no direct way to do that in C, I don't bother.)
One more optimization relates to the size of the dictionary (look-up table). There may be short and long items, and hence it can be built to support 32-bit, 16-bit, or 8-bit items. In that case the dictionary has to be ordered so that small numerical values refer to 32-bit items, middle values to 16-bit items and large values to 8-bit items to avoid alignment problems. Then the look-up code looks like this:
static uint8_t dictionary_lookup(uint8_t octet)
{
if (octet < NUMBER_OF_32_BIT_ITEMS)
return dictionary32[octet];
if (octet < NUMBER_OF_32_BIT_ITEMS + NUMBER_OF_16_BIT_ITEMS)
return dictionary16[octet - NUMBER_OF_32_BIT_ITEMS];
return dictionary8[octet - NUMBER_OF_16_BIT_ITEMS - NUMBER_OF_32_BIT_ITEMS];
}
Of course, if every font has its own dictionary, the constants will become variables looked up form the font information. Any half-decent compiler will inline that function, as it is called only once.
If the number of quantization levels is reduced, it can be handled, as well. The easiest case is with 4-bit gray levels (1..14). This requires one 8-bit state variable to hold the gray levels. Then the gray level branch will become:
// new state value
static uint8_t gray_value;
...
// new variable within the next_pixel() function
uint8_t return_value;
...
// there is no old gray value available?
if (gray_value == 0)
gray_value = *next_octet++;
// extract the low nibble
return_value = gray_value & 0x0f;
// shift the high nibble into low nibble
gray_value >>= 4;
return return_value;
This actually allows using 15 intermediate gray levels (a total of 17 levels), which maps very nicely into linear 255-value system.
Three- or five-bit data is easier to pack into a 16-bit halfword and set MSB always one. Then the same trick as with the ternary data can be used (shift until you get 1).
It should be noted that the compression ratio starts to deteriorate at some point. The amount of compression with the ternary data does not depend on the number of gray levels. The gray level data is uncompressed, and the number of octets scales (almost) linearly with the number of bits. For a typical font the gray level data at 8 bits is 1/2 .. 2/3 of the total, but this is highly dependent on the typeface and size.
So, reduction from 8 to 4 bits (which is visually quite imperceptible in most cases) reduces the compressed size typically by 1/4..1/3, whereas the further reduction offered by going down to three bits is significantly less. Two-bit data does not make sense with this compression algorithm.
How to build the dictionary?
If the decompression algorithm is very straightforward and fast, the real challenges are in the dictionary building. It is easy to prove that there is such thing as an optimal dictionary (dictionary giving the least number of compressed octets for a given font), but wiser people than me seem to have proven that the problem of finding such dictionary is NP-complete.
With my arguably rather lacking theoretical knowledge on the field I thought there would be great tools offering reasonably good approximations. There might be such tools, but I could not find any, so I rolled my own mickeymouse version. EDIT: the earlier algorithm was rather goofy; a simpler and more effective was found
start with a static dictionary of '0', g', '1' (where 'g' signifies an intermediate value)
split the ternary data for each glyph into a list of trits
find the most common consecutive combination of items (it will most probably be '0', '0' at the first iteration)
replace all occurrences of the combination with the combination and add the combination into the dictionary (e.g., data '0', '1', '0', '0', 'g' will become '0', '1', '00', 'g' if '0', '0' is replaced by '00')
remove any unused items in the dictionary (they may occur at least in theory)
repeat steps 3-5 until the dictionary is full (i.e. at least 253 rounds)
This is still a very simplistic approach and it probably gives a very sub-optimal result. Its only merit is that it works.
How well does it work?
One answer is well enough, but to elaborate that a bit, here are some numbers. This is a font with 864 glyphs, typical glyph size of 14x11 pixels, and 8 bits per pixel.
raw uncompressed size: 127101
number of intermediate values: 46697
Shannon entropies (octet-by-octet):
total: 528914 bits = 66115 octets
ternary data: 176405 bits = 22051 octets
intermediate values: 352509 bits = 44064 octets
simply compressed ternary data (0=0, 10=1, 11=intermediate) (127101 trits): 207505 bits = 25939 octets
dictionary compressed ternary data: 18492 octets
entropy: 136778 bits = 17097 octets
dictionary size: 647 octets
full compressed data: 647 + 18492 + 46697 = 65836 octets
compression: 48.2 %
The comparison with octet-by-octet entropy is quite revealing. The intermediate value data has high entropy, whereas the ternary data can be compressed. This can also be interpreted by the high number of values 0 and 255 in the raw data (as compared to any intermediate values).
We do not do anything to compress the intermediate values, as there do not seem to be any meaningful patterns. However, we beat entropy by a clear margin with ternary data, and even the total amount of data is below entropy limit. So, we could do worse.
Reducing the number of quantization levels to 17 would reduce the data size to approximately 42920 octets (compression over 66 %). The entropy is then 41717 octets, so the algorithm gets slightly worse as is expected.
In practice, smaller font sizes are difficult to compress. This should be no surprise, as larger fraction of the information is in the gray scale information. Very big font sizes compress efficiently with this algorithm, but there run-length compression is a much better candidate.
What would be better?
If I knew, I would use it! But I can still speculate.
Jubatian suggests there would be a lot of repetition in a font. This must be true with the diacritics, as aàäáâå have a lot in common in almost all fonts. However, it does not seem to be true with letters such as p and b in most fonts. While the basic shape is close, it is not enough. (Careful pixel-by-pixel typeface design is then another story.)
Unfortunately, this inevitable repetition is not very easy to exploit in smaller size fonts. I tried creating a dictionary of all possible scan lines and then only referencing to those. Unfortunately, the number of different scan lines is high, so that the overhead added by the references outweighs the benefits. The situation changes somewhat if the scan lines themselves can be compressed, but there the small number of octets per scan line makes efficient compression difficult. This problem is, of course, dependent on the font size.
My intuition tells me that this would still be the right way to go, if both longer and shorter runs than full scan lines are used. This combined with using 4-bit pixels would probably give very good results—only if there were a way to create that optimal dictionary.
One hint to this direction is that LZMA2 compressed file (with xz at the highest compression) of the complete font data (127101 octets) is only 36720 octets. Of course, this format fulfils none of the other requirements (fast to decompress, can be decompressed glyph-by-glyph, low RAM requirements), but it still shows there is more redundance in the data than what my cheap algorithm has been able to exploit.
Dictionary coding is typically combined with Huffman or arithmetic coding after the dictionary step. We cannot do it here, but if we could, it would save another 4000 octets.
You can consider using something already developed for a scenario similar to Yours
https://github.com/atomicobject/heatshrink
https://spin.atomicobject.com/2013/03/14/heatshrink-embedded-data-compression/
You could try lossy compression using a sparse representation with custom dictionary.
The output of each glyph is a superposition of 1-N blocks from the dictionary;
most cpu time spent in preprocessing
predetermined decoding time (max, average or constant N) additions per pixel
controllable compressed size (dictionary size + xyn codes per glyph)
It seems that the simplest lossy method would be to reduce the number of bits-per-pixel. With glyphs of that size, 16 levels are likely to be sufficient. That would halve the data immediately, then you might apply your existing algorithm in the values 0, 16 or "something else" to perhaps halve it again.
I would go for Clifford's answer, that is, converting the font to 4 bits per pixel first which is sufficient for this task.
Then, since this is a font, you have lots of row repetitions, that is when rows defining one character match those of another character. Take for example the letter 'p' and 'b', the middle part of these letters should be the same (you will have even more matches if the target language uses loads of diacritics). Your encoder then could first collect all distinct rows of the font, store these, and then each character image is formed by a list of pointers to the rows.
The efficiency depends on the font of course, depending on the source, you might need some preprocessing to get it compress better with this method.
If you want more, you might rather choose to go for 3 bits per pixel or even 2 bits per pixel, depending on your goals (and some will for hand-tuning the font images), these might still be satisfactory.
This method in overall of course works very well for real-time display (you only need to traverse a pointer to get the row data).
tl;dr
I recently started listening to a security podcast, and heard the following sentence (paraphrasing)
One of the good hallmarks of a cryptographically strong random number is its lack of compressibility
Which immediately got me thinking, can random data be lossless-ly compressed? I started reading, and found this wikipedia article. A quoted block is below
In particular, files of random data cannot be consistently compressed by any conceivable lossless data compression algorithm: indeed, this result is used to define the concept of randomness in algorithmic complexity theory.
I understand the pigeon hole principle, so I'm assuming I'm way wrong here somewhere, but what am I missing?
IDEA:
Assume you have an asymmetric variable-length encryption method by which you could convert any N bit into either a N-16 bit number or N+16 bit number. Is this possible?
IF we had an assymetric algorithm could either make the data say 16 bits bigger or 16 bits smaller, then I think I can come up with an algorithm for reliably producing lossless compression.
Lossless Compression Algorithm for Arbitrary Data
Break the initial data into chunks of a given size. Then use a "key" and attempt to compress each chunk as follows.
function compress(data)
compressedData = []
chunks = data.splitBy(chunkSize);
foreach chunk in chunks
encryptedChunk = encrypt(chunk, key)
if (encryptedChunk.Length <= chunk.Length - 16) // arbitrary amount
compressedData.append(0) // 1 bit, not an integer
compressedData.append(encryptedChunk)
else
compressedData.append(1) // 1 bit, not an integer
compressedData.append(chunk)
end foreach
return compressedData;
end function
And for de-compression, if you know the chunk-size, then each chunk that begins with 0 perform the asymmetric encryption and append the data to the on going array. If the chunk begins with a 0 simply append the data as-is. If the encryption method produces the 16-bit smaller value even 1/16 as often as the 16-bit larger value, then this will work right? Each chunk is either 1 bit bigger, or 15 bits smaller.
One other consideration is that the "key" used by the compression algorithm can be either fixed or perhaps appended to the beginning of the compressed data. Same consideration for the chunk size.
There are 2N−16 possible (N−16)-bit sequences, and 2N possible N-bit sequences. Consequently, no more than one in every 216 N-bit sequence can be losslessly compressed to N−16 bits. So it will happen a lot less frequently than 1/16 of the time. It will happen at most 1/65536 of the time.
As your reasoning indicates, the remaining N-bit sequences could be expanded to N+1 bits; there is no need to waste an additional 15 bits encoding them. All the same, the probability of a random N-bit sequence being in the set of (N−16)-bit compressible sequences is so small that the average compression (or expected compression) will continue to be 1.0 (at best).
I am not asking how Huffman coding is working, but instead, I want to know why it is good.
I have the following two questions:
Q1
I understand the ultimate purpose of Huffman coding is to give certain char a less bit number, so space is saved. What I don't understand is that why the decision of number of bits for a char can be related to the char's frequency?
Huffman Encoding Trees says
It is sometimes advantageous to use variable-length codes, in which
different symbols may be represented by different numbers of bits. For
example, Morse code does not use the same number of dots and dashes
for each letter of the alphabet. In particular, E, the most frequent
letter, is represented by a single dot.
So in Morse code, E can be represented by a single dot because it is the most frequent letter. But why? Why can it be a dot just because it is most frequent?
Q2
Why the probability / statistics of the chars are so important to Huffman coding?
What happen if the statistics table is wrong?
If you assign less number or bits or shorter code words for most frequently used symbols you will be saving a lot of storage space.
Suppose you want to assign 26 unique codes to English alphabet and want to store an english novel ( only letters ) in term of these code you will require less memory if you assign short length codes to most frequently occurring characters.
You might have observed that postal code and STD codes for important cities are usually shorter ( as they are used very often ). This is very fundamental concept in Information theory.
Huffman encoding gives prefix codes.
Construction of Huffman tree:
A greedy approach to construct Huffman tree for n characters is as follows:
places n characters in n sub-trees.
Starts by combining the two least weight nodes into a tree which is assigned the sum of the two leaf node weights as the weight for its root node.
Do this until you get a single tree.
For example consider below binary tree where E and T have high weights ( as very high occurrence )
It is a prefix tree. To get the Huffman code for any character, start from the node corresponding to the the character and backtrack till you get the root node.
Indeed, an E could be, say, three dashes followed by two dots. When you make your own encoding, you get to decide. If your goal is to encode a certain text so that the result is as short as possible, you should choose short codes for the most frequent characters. The Huffman algorithm ensures that we get the optimal codes for a specific text.
If the frequency table is somehow wrong, the Huffman algorithm will still give you a valid encoding, but the encoded text would be longer than it could have been if you had used a correct frequency table. This is usually not a problem, because we usually create the frequency table based on the actual text that is to be encoded, so the frequency table will be "perfect" for the text that we are going to encode.
well.. you want assign shorter codes to the symbols which appear more frequently... huffman encoding works just by this simple assumption.. :-)
you compute the frequency of all symbols, sort them all, and start assigning bit codes to each one.. the more frequent a symbol is, the shorter the code you'll assign to it.. simple as this.
the big question is: how large the window in which we compute such frequencies should be? should it be as large as the entire file? or should it be smaller? and if the latter apply, how large? Most huffman encoding have some sort of "test-run" in which they estimate the best window size a little bit like TCP/IP do with its windows frame sizes.
Huffman codes provide two benefits:
they are space efficient given some corpus
they are prefix codes
Given some set of documents for instance, encoding those documents as Huffman codes is the most space efficient way of encoding them, thus saving space. This however only applies to that set of documents as the codes you end up are dependent on the probability of the tokens/symbols in the original set of documents. The statistics are important because the symbols with the highest probability (frequency) are given the shortest codes. Thus the symbols most likely to be in your data use the least amount of bits in the encoding, making the coding efficient.
The prefix code part is useful because it means that no code is the prefix of another. In morse code for instance A = dot dash and J = dot dash dash dash, how do you know where to break reading the code. This increases the inefficiency of transmitting data using morse as you need a special symbol (pause) to signify the end of transmission of one code. Compare that to Huffman codes where each code is unique, as soon as you discover the encoding for a symbol in the input, you know that that is the transmitted symbol because it is guaranteed not to be the prefix of some other symbol.
It's the dual effect of having the most frequent characters using the shortest bit sequences that gives you the savings.
For a concrete example, let's say you have a piece of text that consists of 1024 e characters and 1024 of all other characters combined.
With 8 bits for code, that's a full 2048 bytes used in uncompressed form.
Now let's say we represent e as a single 1-bit and every other letter as a 0-bit followed by its original 8 bits (a very primitive form of Huffman).
You can see that half the characters have been expanded from 8 bits to 9, giving 9216 bits, or 1152 bytes. However, the e characters have been reduced from 8 bits to 1, meaning they take up 1024 bits, or 128 bytes.
The total bytes used is therefore 1152 + 128, or 1280 bytes, representing a compression ratio of 62.5%.
You can use a fixed encoding scheme based on the likely frequencies of characters (such as English text), or you can use adaptive Huffman encoding which changes the encoding scheme as characters are processed and frequencies are adjusted. While the former may be okay for input which has high probability of matching frequencies, the latter can adapt to any input.
Statistic table can't be wrong, because in general Huffman algorithm, analyze hole text at the beginning, and builds frequent-statistics of the given text, while Morse has a static symbol -code map.
Huffman algorithm uses the advantage of a given text. As an example, if E is most frequent letter in English in general, that doesn't mean that E is most frequent in a given text for a given author.
Another advantage of Huffman algorithm is that you can use it for any alphabet starting from [0, 1] finished Chinese hieroglyphs, while Morse is defined only for English letters
So in Morse code, "E" can be represented by a single dot, because it is the most frequent letter. But why? Why is it a dot because of its frequency?
"E" can be encoded to any unique code for a specific code dictionary, so it can be "0", we choose it to be short to save memory, so the average bytes used after encode is minimized.
Why is the probability / statistics of the chars so important to Huffman coding? What happens if the statistics table is wrong?
why do we encode? save space right? Space used after encode is freq(wordi)*Length(wordi), it is what we should try to minimize, so we choose to assign words with high prob short code greedly to save space.
If the statistics table is wrong, then the encoding is not the best way to save space.