Given a k-ary tree, i want to convert it into a min-heap with minimum number of changes. Change is defined as relabelling a node.
one solution i have found is that, i can try a dp solution of changing a nodes value or not changing. But its going to be exponential in time complexity ?
Any ideas, (preferable with optimality proofs).
Example : Say the tree is, 1-3, 3-2, 1-4, 4-5. where 1 is root. Then i can relabel node 3 to 1 or 2, that is in 1 change it becomes a min-heap.
If all you want to do is make sure that the tree satisfies the heap property (the key stored in each node is less than or equal to the keys stored in the node's children), then you should be able to use something like the build-heap algorithm, which operates in O(n).
Consider this tree:
8
-------------
| | |
15 6 19
/ \ | / | \
7 3 5 12 9 22
Now, working from the bottom up, you push each node down the tree as far as it can go. That is, if the node is larger than any of its children, you replace it with the smallest of its children, and you do so until you reach the leaf level, if necessary.
For example, you look at the node valued 15. It's larger than its smallest child, so you swap it, making the subtree:
3
/ \
7 15
Also, 6 swaps places with 5, and 19 swaps places with 9, giving you this tree:
8
-------------
| | |
3 5 9
/ \ | / | \
7 15 6 12 19 22
Note that at the next to leaf level, each node is smaller than its smallest child.
Now, the root. Since the rule is to swap the node with its smallest child, you swap 8 with 3, giving:
3
-------------
| | |
8 5 9
/ \ | / | \
7 15 6 12 19 22
But you're not done because 8 is greater than 7. You swap 8 with 7, and you get this tree, which meets your conditions:
3
-------------
| | |
7 5 9
/ \ | / | \
8 15 6 12 19 22
If the tree is balanced, the entire procedure has complexity O(n). If the tree is severely unbalanced, the complexity is O(n^2). There is a way to guarantee O(n), regardless of the tree's initial order, but it requires changing the shape of the tree.
I won't claim that the algorithm guarantees the "minimal number of changes" for any given tree. I can prove, however, that with a balanced tree the algorithm is O(n). See https://stackoverflow.com/a/9755805/56778, which explains it for binary heap. The explanation also applies to d-ary heap.
I am reading the Pairing heap.
It is quite simple, the only tricky part is the delete_min operation.
The only non-trivial fundamental operation is the deletion of the
minimum element from the heap. The standard strategy first merges the
subheaps in pairs (this is the step that gave this datastructure its
name) from left to right and then merges the resulting list of heaps
from right to left:
I don't think I need copy/paste the code here, as it is in the wiki link.
My questions are
why they do this two pass merging?
Why they first merge pairs? not directly merge them all?
also why after merging pairs, merge specifically from right to left?
With pairing heap, adding an item to the heap is an O(1) operation because all it does is add the node either as the new root (if it's smaller than the current root), or as the first child of the current root. So if you created a pairing heap and added the numbers 0 through 9 to it, in order, you would end up with:
0
|
-----------------
| | | | | | | | |
9 8 7 6 5 4 3 2 1
If you then do a delete-min, you then have to look at each child to determine the minimum item and build the new heap. If you use the naive left to right combining method, you end up with this tree:
1
|
---------------
| | | | | | | |
9 8 7 6 5 4 3 2
And the next time you do a delete-min you have to look at the 8 remaining children, etc. Using this technique, creating and then removing all items from the heap would be an O(n^2) operation.
The two-pass method of combining in pairs and then combining the pairs results in a much more efficient structure. Consider the first case. After deleting the minimum item, we're left with the nine children. They're combined in pairs from left to right to produce:
8 6 4 2 1
/ / / /
9 7 5 3
Then we combine the the pairs right to left. In steps:
8 6 4 1
/ / / /
9 7 5 2
/
3
8 6 1
/ / / \
9 7 2 4
/ /
3 5
8 1
/ |
9 ---------
6 4 2
/ / /
7 5 3
1
|
----------
8 6 4 2
/ / / /
9 7 5 3
Now, the next time we call delete-min, there are only four nodes to check, and the next time after that there will only be two. Using the two-pass combining method reduces the number of nodes at the child level by at least half. The arrangement I showed is the worst case. If the items were in ascending order, the first delete-min operation would result in a tree with only two child nodes below the root.
This is a particularly good example of the amortized complexity of pairing heap. insert is O(1), but the first delete-min after a bunch of insert operations is O(n), where n is the number of items that were inserted since the last delete-min. The beauty of the two-pass combining rule is that it quickly reorganizes the heap to reduce that O(n) complexity.
With this combining rule, the amortized complexity of delete-min is O(log n). With the strict left-to-right rule, it's O(n).
This question already has answers here:
Closed 11 years ago.
Possible Duplicate:
Given a 2d array sorted in increasing order from left to right and top to bottom, what is the best way to search for a target number?
The following was asked in a Google interview:
You are given a 2D array storing integers, sorted vertically and horizontally.
Write a method that takes as input an integer and outputs a bool saying whether or not the integer is in the array.
What is the best way to do this? And what is its time complexity?
Start at the Bottom-Left corner of the Matrix and follow the rules stated below to traverse the matrix:
The matrix traversal is based on these conditions:
If the input number is greater than current number: Move Right
If the input number is less than current number: Move Up.
If the input number is equal to current number: Return Success
If the input number is not equal to current number and no transition is possible: Return Fail
Time Complexity: (Thanks to Martinho Fernandes)
The time complexity is O(N+M). In the worst case, the element searched for is in the upper-left corner, meaning you'll go up N times, and left M times.
Example
Input matrix:
--------------
| 1 | 4 | 6 |
--------------
| 2 | 5 | 9 |
--------------
| *3* | 8 | 10 |
--------------
Number to search: 4
Step 1:
Start at the cell where you have 3 (Bottom-Left).
3 < 4: Move Right
| 1 | 4 | 6 |
--------------
| 2 | 5 | 9 |
--------------
| 3 | *8* | 10 |
--------------
Step 2:
8 > 4: Move Up
| 1 | 4 | 6 |
--------------
| 2 | *5* | 9 |
--------------
| 3 | 8 | 10 |
--------------
Step 3:
5 > 4: Move Up
| 1 | *4* | 6 |
--------------
| 2 | 5 | 9 |
--------------
| 3 | 8 | 10 |
--------------
Step 4:
4=4: Return the index of the number
I would start by asking details about what it means to be "sorted vertically and horizontally"
If the matrix is sorted in a way that the last element of each row is less than the first element of the next row, you can run a binary search on the first column to find out in what row that number is, and then run another binary search on the row. This algorithm will take O(log C + log R) time, where C and R are, respectively the number of rows and columns. Using a property of the logarithm, one can write that as O(log(C*R)), which is the same as O(log N), if N is the number of elements in the array. This is almost the same as treating the array as 1D and running a binary search on it.
But the matrix could be sorted in a way that the last element of each row is not less than the first element of the next row:
1 2 3 4 5 6 7 8 9
2 3 4 5 6 7 8 9 10
3 4 5 6 7 8 9 10 11
In this case, you could run some sort of horizontal an vertical binary search simultaneously:
Test the middle number of the first column. If it's less than the target, consider the lines above it. If it's greater, consider those below;
Test the middle number of the first considered line. If it's less, consider the columns left of it. If it's greater, consider those to the right;
Lathe, rinse, repeat until you find one, or you're left with no more elements to consider;
This method is also logarithmic on the number of elements.
The first method that comes to mind is a vertical binary search, followed by a horizontal one when you find the row it should be in. Complexity will be O(log NM) where N and M are the dimensions of the array.
Further explanation:
Consider just the first number of every row. When you perform a binary search of these first numbers for the specified number, the result will be either the specified number if you're lucky, otherwise it will be the position before or after where the specified number would go depending on the binary search implementation. Once you find the two of the first numbers that the specified number should go between, you know that the number is in that row, and a second binary search will find the number if it is in the row.
I have a very large matrix (100M rows by 100M columns) that has a lots of duplicate values right next to each other. For example:
8 8 8 8 8 8 8 8 8 8 8 8 8
8 4 8 8 1 1 1 1 1 8 8 8 8
8 4 8 8 1 1 1 1 1 8 8 8 8
8 4 8 8 1 1 1 1 1 8 8 8 8
8 4 8 8 1 1 1 1 1 8 8 8 8
8 4 8 8 1 1 1 1 1 8 8 8 8
8 8 8 8 8 8 8 8 8 8 8 8 8
8 8 3 3 3 3 3 3 3 3 3 3 3
I want a datastructure/algorithm to store matricies like these as compactly as possible. For instance, the matrix above should only take O(1) space (even if the matrix was stretched out arbitrarily big), because there is only a constant number of rectangular regions, where each region only has one value.
The repetition happens both across rows and down columns, so the simple approach of compressing the matrix row-by-row isn't good enough. (That would require a minimum of O(num_rows) space to store any matrix.)
The representation of the matrix also needs to accessible row-by-row, so that I can do a matrix multiplication to a column vector.
You could store the matrix as a quadtree with the leaves containing single values. Think of this as a two-dimensional "run" of values.
Now for my preferred method.
Ok, as I made mention in my previous answer rows with the same entries in each column in matrix A will multiply out to the same result in matrix AB. If we can maintain that relationship then we can theoretically speed up calculations significantly (a profiler is your friend).
In this method we maintain the row * column structure of the matrix.
Each row is compressed with whatever method can decompress fast enough not to affect the multiplication speed too much. RLE may be sufficient.
We now have a list of compressed rows.
We use an entropy encoding method (like Shannon-Fano, Huffman or arithmetic coding), but we don’t compress the data in the rows with this, we use it to compress the set of rows.
We use it to encode the relative frequency of the rows. I.e. we treat a row the same way standard entropy encoding would treat a character/byte.
In this example RLE compresses a row, and Huffman compresses the entire set of rows.
So, for example, given the following matrix (prefixed with row numbers, Huffman used for ease of explanation)
0 | 8 8 8 8 8 8 8 8 8 8 8 8 8 |
1 | 8 4 8 8 1 1 1 1 1 8 8 8 8 |
2 | 8 4 8 8 1 1 1 1 1 8 8 8 8 |
3 | 8 4 8 8 1 1 1 1 1 8 8 8 8 |
4 | 8 4 8 8 1 1 1 1 1 8 8 8 8 |
5 | 8 4 8 8 1 1 1 1 1 8 8 8 8 |
6 | 8 8 8 8 8 8 8 8 8 8 8 8 8 |
7 | 8 8 3 3 3 3 3 3 3 3 3 3 3 |
Run length encoded
0 | 8{13} |
1 | 8{1} 4{1} 8{2} 1{5} 8{4} |
2 | 8{1} 4{1} 8{2} 1{5} 8{4} |
3 | 8{1} 4{1} 8{2} 1{5} 8{4} |
4 | 8{1} 4{1} 8{2} 1{5} 8{4} |
5 | 8{1} 4{1} 8{2} 1{5} 8{4} |
6 | 8{13} |
7 | 8{2} 3{11} |
So, 0 and 6 appear twice and 1 – 5 appear 5 times. 7 only once.
Frequency table
A: 5 (1-5) | 8{1} 4{1} 8{2} 1{5} 8{4} |
B: 2 (0,6) | 8{13} |
C: 1 7 | 8{2} 3{11} |
Huffman tree
0|1
/ \
A 0|1
/ \
B C
So in this case it takes one bit (for each row) to encode rows 1 – 5, and 2 bits to encode rows 0, 6, and 7.
(If the runs are longer than a few bytes then do freq count on a hash that you build up as you do the RLE).
You store the Huffman tree, unique strings, and the row encoding bit stream.
The nice thing about Huffman is that it has a unique prefix property, so you always know when you are done. Thus, given the bit string 10000001011 you can rebuild the matrix A from the stored unique strings and the tree. The encoded bit stream tells you the order that the rows appear in.
You may want to look into adaptive Huffman encoding, or its arithmetic counterpart.
Seeing as rows in A with the same column entries multiply to the same result in AB over vector B you can cache the result and use it instead of calculating it again (it’s always good to avoid 100M*100M multiplications if you can).
Links to further info:
Arithmetic Coding + Statistical Modeling = Data Compression
Priority Queues and the STL
Arithmetic coding
Huffman coding
A Comparison
Uncompressed
0 1 2 3 4 5 6 7
=================================
0 | 3 3 3 3 3 3 3 3 |
|-------+ +-------|
1 | 4 4 | 3 3 3 3 | 4 4 |
| +-----------+---+ |
2 | 4 4 | 5 5 5 | 1 | 4 4 |
| | | | |
3 | 4 4 | 5 5 5 | 1 | 4 4 |
|---+---| | | |
4 | 5 | 0 | 5 5 5 | 1 | 4 4 |
| | +---+-------+---+-------|
5 | 5 | 0 0 | 2 2 2 2 2 |
| | | |
6 | 5 | 0 0 | 2 2 2 2 2 |
| | +-------------------|
7 | 5 | 0 0 0 0 0 0 0 |
=================================
= 64 bytes
Quadtree
0 1 2 3 4 5 6 7
=================================
0 | 3 | 3 | | | 3 | 3 |
|---+---| 3 | 3 |---+---|
1 | 4 | 4 | | | 4 | 4 |
|-------+-------|-------+-------|
2 | | | 5 | 1 | |
| 4 | 5 |---+---| 4 |
3 | | | 5 | 1 | |
|---------------+---------------|
4 | 5 | 0 | 5 | 5 | 5 | 1 | 4 | 4 |
|---+---|---+---|---+---|---+---|
5 | 5 | 0 | 0 | 2 | 2 | 2 | 2 | 2 |
|-------+-------|-------+-------|
6 | 5 | 0 | 0 | 2 | 2 | 2 | 2 | 2 |
|---+---+---+---|---+---+---+---|
7 | 5 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
=================================
0 +- 0 +- 0 -> 3
| +- 1 -> 3
| +- 2 -> 4
| +- 3 -> 4
+- 1 -> 3
+- 2 -> 4
+- 3 -> 5
1 +- 0 -> 3
+- 1 +- 0 -> 3
| +- 1 -> 3
| +- 2 -> 4
| +- 3 -> 4
+- 2 +- 0 -> 5
| +- 1 -> 1
| +- 2 -> 5
| +- 3 -> 1
+- 3 -> 4
2 +- 0 +- 0 -> 5
| +- 1 -> 0
| +- 2 -> 5
| +- 3 -> 0
+- 1 +- 0 -> 5
| +- 1 -> 5
| +- 2 -> 0
| +- 3 -> 2
+- 2 +- 0 -> 5
| +- 1 -> 0
| +- 2 -> 5
| +- 3 -> 0
+- 3 +- 0 -> 0
+- 1 -> 2
+- 2 -> 0
+- 3 -> 0
3 +- 0 +- 0 -> 5
| +- 1 -> 1
| +- 2 -> 2
| +- 3 -> 2
+- 1 +- 0 -> 4
| +- 1 -> 4
| +- 2 -> 2
| +- 3 -> 2
+- 2 +- 0 -> 2
| +- 1 -> 2
| +- 2 -> 0
| +- 3 -> 0
+- 3 +- 0 -> 2
+- 1 -> 2
+- 2 -> 0
+- 3 -> 0
((1*4) + 3) + ((2*4) + 2) + (4 * 8) = 49 leaf nodes
49 * (2 + 1) = 147 (2 * 8 bit indexer, 1 byte data)
+ 14 inner nodes -> 2 * 14 bytes (2 * 8 bit indexers)
= 175 Bytes
Region Hash
0 1 2 3 4 5 6 7
=================================
0 | 3 3 3 3 3 3 3 3 |
|-------+---------------+-------|
1 | 4 4 | 3 3 3 3 | 4 4 |
| +-----------+---+ |
2 | 4 4 | 5 5 5 | 1 | 4 4 |
| | | | |
3 | 4 4 | 5 5 5 | 1 | 4 4 |
|---+---| | | |
4 | 5 | 0 | 5 5 5 | 1 | 4 4 |
| + - +---+-------+---+-------|
5 | 5 | 0 0 | 2 2 2 2 2 |
| | | |
6 | 5 | 0 0 | 2 2 2 2 2 |
| +-------+-------------------|
7 | 5 | 0 0 0 0 0 0 0 |
=================================
0: (4,1; 4,1), (5,1; 6,2), (7,1; 7,7) | 3
1: (2,5; 4,5) | 1
2: (5,3; 6,7) | 1
3: (0,0; 0,7), (1,2; 1,5) | 2
4: (1,0; 3,1), (1,6; 4,7) | 2
5: (2,2; 4,4), (4,0; 7,0) | 2
Regions: (3 + 1 + 1 + 2 + 2 + 2) * 5
= 55 bytes {4 bytes rectangle, 1 byte data)
{Lookup table is a sorted array, so it does not need extra storage}.
Huffman encoded RLE
0 | 3 {8} | 1
1 | 4 {2} | 3 {4} | 4 {2} | 2
2,3 | 4 {2} | 5 {3} | 1 {1} | 4 {2} | 4
4 | 5 {1} | 0 {1} | 5 {3} | 1 {1} | 4 {2} | 5
5,6 | 5 {1} | 0 {2} | 2 {5} | 3
7 | 5 {1} | 0 {7} | 2
RLE Data: (1 + 3+ 4 + 5 + 3 + 2) * 2 = 36
Bit Stream: 20 bits packed into 3 bytes = 3
Huffman Tree: 10 nodes * 3 = 30
= 69 Bytes
One Giant RLE stream
3{8};4{2};3{4};4{4};5{3};1{1};4{4};5{3};1{1};4{2};5{1};0{1};
5{3};1{1};4{2};5{1};0{2};2{5};5{1};0{2};2{5};5{1};0{7}
= 2 * 23 = 46 Bytes
One Giant RLE stream encoded with common prefix folding
3{8};
4{2};3{4};
4{4};5{3};1{1};
4{4};5{3};
1{1};4{2};5{1};0{1};5{3};
1{1};4{2};5{1};0{2};2{5};
5{1};0{2};2{5};
5{1};0{7}
0 + 0 -> 3{8};4{2};3{4};
+ 1 -> 4{4};5{3};1{1};
1 + 0 -> 4{2};5{1} + 0 -> 0{1};5{3};1{1};
| + 1 -> 0{2}
|
+ 1 -> 2{5};5{1} + 0 -> 0{2};
+ 1 -> 0{7}
3{8};4{2};3{4} | 00
4{4};5{3};1{1} | 01
4{4};5{3};1{1} | 01
4{2};5{1};0{1};5{3};1{1} | 100
4{2};5{1};0{2} | 101
2{5};5{1};0{2} | 110
2{5};5{1};0{7} | 111
Bit stream: 000101100101110111
RLE Data: 16 * 2 = 32
Tree: : 5 * 2 = 10
Bit stream: 18 bits in 3 bytes = 3
= 45 bytes
If your data is really regular, you might benefit from storing it in a structured format; e.g. your example matrix might be stored as the following list of "fill-rectangle" instructions:
(0,0)-(13,7) = 8
(4,1)-(8,5) = 1
(Then to look up the value of a particular cell, you'd iterate backwards through the list until you found a rectangle that contained that cell)
As Ira Baxter suggested,
you could store the matrix as a quadtree with the leaves containing single values.
The simplest way to do this is for every node of the quadtree to cover an area 2^n x 2^n,
and each non-leaf node points to its 4 children of size 2^(n-1) x 2^(n-1).
You might get slightly better compression with an adaptive quadtree that allows irregular sub-division.
Then each non-leaf node stores the cut-point (B,G) and points to its 4 children.
For example, if some non-leaf node covers an area from (A,F) in the upper-left corner to (C,H) in the lower-right corner,
then its 4 children cover areas
(A,F) to (B-1, G-1)
(A,G) to (B-1, H)
(B,F) to (C,G-1)
(B,G) to (C,H).
You would try to pick the (B,G) cut-point for each non-leaf node such that it lines up with some real division in your data.
For example, say you have a matrix with a small square in the middle filled with nines and zero elsewhere.
With the simple powers-of-two quadtree, you'll end up with at least 21 nodes: 5 non-leaf nodes, 4 leaf nodes of nines, and 12 leaf nodes of zeros.
(You'll get even more nodes if the centered small square is not precisely some power-of-two distance from the left and top edges, and not itself some precise power-of-two).
With an adaptive quadtree, if you are smart enough to pick the cut-point for the root node at the upper-left corner of that square, then for the root's lower-right child you pick a cut-point at the lower-right corner of the square, you can representing the entire matrix in 9 nodes: 2 non-leaf nodes, 1 leaf node for the nines, and 6 leaf nodes for the zeros.
Do you know about.... interval trees ?
Interval trees are a way to store intervals efficiently, and then query them. A generalization is the Range Tree, which can be adapted to any dimension.
Here you could effectively describe your rectangles and attach a value to them. Of course the rectangles can overlap, that's what will make it efficient.
0,0-n,n --> 8
4,4-7,7 --> 1
8,8-8,n --> 3
Then when querying for a value in one particular spot, you are returned a list of several rectangles and need to determine the innermost one: this is the value in this spot.
The simplest approach is to use run-length encoding on one dimension and not worry about the other dimension.
(If the dataset weren't so incredibly huge, interpreting it as an image and using a standard lossless image compression method would be very simple also--but since you'd have to work on making the algorithm work on sparse matrices, it wouldn't end up being all that simple.)
Another simple approach is to try a rectangular flood fill--start at the top-right pixel and increase it into the largest rectangle you can (breadth-first); then mark all those pixels as "done" and take the top-right most remaining pixel, repeat until done. (You'd probably want to store these rectangles in some sort of BSP or quad-tree.)
A highly effective technique--not optimal, but probably good enough--is to use a binary space partitioning tree where "space" is measured not spatially but by number of changes. You'd recursively cut so that you have equal numbers of changes on the left and right (or top and bottom--presumably you'd want to keep things square) and, as your sizes got smaller, so that you would cut as many changes as possible. Eventually, you'll end up cutting two rectangles apart from each other, each of which has all the same number; then stop. (Encoding by RLE in x and y will quickly tell you where the change points are.)
Your description of O(1) space for a matrix of size 100M x 100M is confusing. When you have a finite matrix, then your size is a constant (unless the program that generates the matrix doesn't alter it). So the amount of space required to store is also a constant even if you multiply it with a scalar. Definitely the time to read and write the matrix is not going to be O(1).
Sparse matrix is what I could think of to reduce the amount of space required to store such a matrix. You can write this sparse matrix to a file and store it as a tar.gz which will further compress the data.
I do have a question what does M in 100M denote? Does it mean Megabyte/million? If yes, this matrix size will be 100 x 10^6 x 100 x 10^6 bytes = 10^16 / 10^6 MB = 10^10/10^6 TB = 10^4 TB!!! What kind of a machine are you using?
I'm not sure why this question was made Community Wiki, but so it goes.
I'll rely on the assumption that you have a linear algebra application, and that your matrix has a rectangular type of redundancy. If so, then you can do something much better than quadtrees, and cleaner than cutting the matrix into rectangles (which is generally the right idea).
Let M be your matrix, let v be the vector that you want to multiply by M, and let
A be the special matrix
A = [1 -1 0 0 0]
[0 1 -1 0 0]
[0 0 1 -1 0]
[0 0 0 1 -1]
[0 0 0 0 1]
You'll also need the inverse matrix to A, which I'll call B:
B = [1 1 1 1 1]
[0 1 1 1 1]
[0 0 1 1 1]
[0 0 0 1 1]
[0 0 0 0 1]
Multiplying a vector v by A is fast and easy: You just take differences of consecutive pairs of elements of v. Multiply a vector v by B is also fast and easy: The entries of Bv are partial sums of the elements of v. Then you want to use the equation
Mv = B AMA B v
The matrix AMA is sparse: In the middle, each entry is an alternating sum of 4 entries of M that make a 2 x 2 square. You have to be at a corner of one of the rectangles in M for this alternating sum to be non-zero. Since AMA is sparse, you can store its non-zero entries in an associative array and use sparse matrix multiplication to apply it to a vector.
I do not have a specific answer for the matrix you have shown. In finite element analysis (FEA), you have matrices with redundant data. In implementing a FEA package in my under grad project, I used skyline storage method.
Some links:
Intel page for sparse matrix storage
Wikipedia link
The first thing to try is always the existing libraries and solutions. It is a lot of work getting custom formats working with all the operations you're going to want in the end. Sparse matrices is an old problem, so make sure you read up on the existing stuff.
Assuming you don't find something suitable, I would recommend a row-based format. Don't try to be too fancy with super-compact representations, you will end up with lots of processing needed for every little operation and bugs in your code. Instead try to compress each row separately. You know you are going to have to scan through each row for the matrix-vector multiplication, make life easy for yourself.
I would start with run-length-encoding, see how that works first. Once that is working, try adding some tricks like references to sections of the previous row. So a row might be encoded as: 126 zeros, 8 ones, 1000 entries copied directly from row above, 32 zeros. That seems like it might be very efficient with your given example.
Many of the above solutions are fine.
If you are working with a file consider file oriented
compression tools like compress, bzip, zip, bzip2 and friends.
They work very well especially if the data contains redundant
ASCII characters. Using an external compression tool eliminates
problems and challenges inside your code and will compress
both binary and ASCII data.
In your example you are displaying one character numbers.
The numbers 0-9 can be represented by a smaller four bit
encoding pattern. You can use the additional bits in
a byte as a count. Four bits gives you extra codes to
escape to extras... But there is a caution which reaches
back to the old Y2K bugs where two characters were used
for a year. Byte encoding from an ofset would have given
255 years and the same two bytes would span all of written
history and then some.
You may want to take a look at GIF format and its compression algorithm. Just think about your matrix as a Bitmap...
Let me check my assumptions, if for no other reason than to guide my thinking about the problem:
The matrix is highly redundant, not necessarily sparse.
We want to minimize storage (on disk and RAM).
We want to be able to multiply A[m*n] by vector B[n*1] to get to AB[m*1] without first decompressing either (at least not more than required to do the calculations).
We don’t need random access to any A[i*j] entry --all operations are over the matrix.
The multiplication is done online (as needed), and so must be as efficient as possible.
The matrix is static.
One can try all kinds of clever schemes to detect rectangles or self similarity etc, but that is going to end up hurting performance when doing the multiplication. I propose 2 relatively simple solutions.
I am going to have to work backwards a bit, so please be patient with me.
If the data is predominantly biased towards horizontal repetition then the following may work well.
Think of the matrix flattened into an array (this is really the way it is stored in memory anyway). E.g.
A
| w0 w1 w2 |
| x0 x1 x2 |
| y0 y1 y2 |
| z0 z1 z2 |
becomes
A’
| w0 w1 w2 x0 x1 x2 y0 y1 y2 z0 z1 z2 |
We can use the fact that any index [i,j] = i * j.
So, when we do the multiplication we iterate over the “matrix” array A’ with k = [0..m*n-1] and index into the vector B using (k mod n) and into vector AB with (k div n). “div” being integer division.
So, for example, A[10] = z1. 10 mod 3 = 1 and 10 div 3 = 3 A[3,1] = z1.
Now, on to the compression.
We do normal run of the mill Run Length Encoding (RLE), but against the A’, not A. With the flat array there will be longer sequences of repetition, hence better compression. Then after encoding the runs we do another process where we extract common substrings. We can either do a form of dictionary compression, or process the run data into some form of space optimized graph like a radix tree/suffix tree or a device of your own creation that merges tops and tails. The graph should have a representation of all the unique strings in the data. You can pick any number of methods to break the stream into strings: matching prefixes, length, or something else (whatever suits your graph best) but do it on a run boundary, not bytes or your decoding will be made more complicated. The graph becomes a state machine when we decompress the stream.
I’m going to use a bit stream and Patricia trie as an example, because it is simplest, but you can use something else (more bits per state change better merging, etc. Look for papers by Stefan Nilsson).
To compress the run data we build a hash table against the graph. The table maps a string to a bit sequence. You can do this by walking the graph and encoding each left branch as 0 and right branch as 1 (arbitrary choice).
Process the run data and build up a bit string until you get a match in the hash table, output the bits and clear the string (the bits will not be on a byte boundary, so you may have to buffer until you get a sequence long enough to write out). Rinse and repeat until you have processed the complete run data stream. You store the graph and the bit stream. The bit stream encodes strings, not bytes.
If you reverse the process, using the bit stream to walk the graph until you reach a leaf/terminal node you get back the original run data, which you can decode on the fly to produce the stream of integers that you multiply against the vector B to get AB. Each time you run out of runs you read the next bit and lookup its corresponding string. We don’t care that we don’t have random access into A, because we only need it in B (B which can be range / interval compressed but doesn’t need to be).
So even though RLE is biased towards horizontal runs we still get good vertical compression because common strings are stored only once.
I will explain the other method in a separate answer as this is getting too long as it is, but that method can actually speed up calculation due to the fact that repeat rows in matrix A multiplies to the same result in AB.
ok you need a compression algorithm try RLE (Run Length Encoding) its work very good when the data is
highly-redundant .