I created a list of matrix, I want to be able to apply operations with they, one thing I want to do is combine them using operator like cbin and rbind among others, one way to do it it's the next, but i just want to save the matrixs in the list and apply operations without writing everyone.
'Combinedmatrix<-cbind(elementlist[[1]],...,elementlist[[n]])'
Is there a way to do the same instead writing each element of the list?
I tried the next
'(i in 1:length(list)){combinedmatrix<-cbind(list[[i]])}'
in this case it only takes the last element and do nothing more,
the other way I tried was:
'i<-1:length(list)'
'combinedmatrix<-cbind(list[[i]])}'
in this case appear
'Error in list[[i]] : recursive indexing failed at level 2'
You can use reduce() from the purrr package. Assuming mlist is your list of matrices:
library(purrr)
reduce(mlist,rbind)
With example data:
> mlist <- list(matrix(1:9,nrow=3),matrix(1:9,nrow=3),matrix(1:9,nrow=3))
> mlist
[[1]]
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
[[2]]
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
[[3]]
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
> reduce(mlist,rbind)
[,1] [,2] [,3]
[1,] 1 4 7
[2,] 2 5 8
[3,] 3 6 9
[4,] 1 4 7
[5,] 2 5 8
[6,] 3 6 9
[7,] 1 4 7
[8,] 2 5 8
[9,] 3 6 9
Related
Issue: When building a matrix out of single rows, Julia interprets them as columns instead.
a = [1 2 3; 4 5 6; 7 8 9]
3×3 Array{Int64,2}:
1 2 3
4 5 6
7 8 9
b = [a[1:2,:]; a[1:2,:]] # Rows duplicated correctly
4×3 Array{Int64,2}:
1 2 3
4 5 6
1 2 3
4 5 6
c = [a[1,:]; a[1,:]] # Rows converted to columns
6-element Array{Int64,1}:
1
2
3
1
2
3
How to fix this?
In addition to the range index, you can transpose vectors
julia> [a[1, :]'; a[1, :]']
2×3 Array{Int64,2}:
1 2 3
1 2 3
It looks likes this approach is somewhat more performant, than the range index, but it should be tested on larger matrices, also it is not consistent, if you have ranges and single columns
using BenchmarkTools
f1(a) = [a[1:1,:]; a[1:1,:]]
f2(a) = [a[1, :]'; a[1, :]']
julia> #btime f1($a)
122.440 ns (3 allocations: 352 bytes)
2×3 Array{Int64,2}:
1 2 3
1 2 3
julia> #btime f2($a)
107.480 ns (3 allocations: 352 bytes)
2×3 Array{Int64,2}:
1 2 3
1 2 3
Solution: Although it may feel a bit perplexing, its because the type has changed from a matrix into a vector. To keep the type the same you'll need to select from your desired row, to your desired row in a similar manner to the second line of code in your example.
c = [a[1:1,:]; a[1:1,:]]
2×3 Array{Int64,2}:
1 2 3
1 2 3
I have the following system
Which represent a system of 4 known inputs with 12 known outputs.
What methods can I use to find the transfer matrix, can I use the neural network or something like that or it is only possible with matrix algebra?
Any help would be appreciated
thanks in advance
No need to use neural network, matrix algebra is enough!
Your question can be formulated as an optimization problem, i.e., minimize f(T) = norm(y - T*x) given y and x. If you have sufficient data pairs (x,y), then you can solve T.
Another easy way is to use generalized inverse of matrix to solve the transfer matrix T, i.e., T = Y*ginv(X). Here I will show you an example in language R
library(MASS)
Y <- matrix(1:36,nrow = 9)
X <- matrix(1:16,nrow = 4)
T <- Y %*% ginv(X)
where
> X
[,1] [,2] [,3] [,4]
[1,] 1 5 9 13
[2,] 2 6 10 14
[3,] 3 7 11 15
[4,] 4 8 12 16
> Y
[,1] [,2] [,3] [,4]
[1,] 1 10 19 28
[2,] 2 11 20 29
[3,] 3 12 21 30
[4,] 4 13 22 31
[5,] 5 14 23 32
[6,] 6 15 24 33
[7,] 7 16 25 34
[8,] 8 17 26 35
[9,] 9 18 27 36
and the transfer T is solved as
> T
[,1] [,2] [,3] [,4]
[1,] 1.95 1.025 0.1 -0.825
[2,] 1.65 0.925 0.2 -0.525
[3,] 1.35 0.825 0.3 -0.225
[4,] 1.05 0.725 0.4 0.075
[5,] 0.75 0.625 0.5 0.375
[6,] 0.45 0.525 0.6 0.675
[7,] 0.15 0.425 0.7 0.975
[8,] -0.15 0.325 0.8 1.275
[9,] -0.45 0.225 0.9 1.575
To verify the obtained T, you can use
> norm(Y - T%*%X,"2")
[1] 1.178746e-13
which is close to 0, indicating that the obtained T is valid.
I've been trying to figure this out by reverse engineering a .png file I created in GIMP. It's 4x4 pixels. My goal is to decode the raw pixels out of the file with the intent of reversing this to encode.
Here is a full hex dump of the file:
89504E47 0D0A1A0A 0000000D 49484452 00000004 00000004
08020000 00269309 29000000 3F494441 54081D01 3400CBFF
01CC96B1 134FE120 C0CECDF1 5101FFA5 60000000 000000E0
403201DF E59286DF 6D000000 00000004 EDB11F00 2E007A21
93EDB11F 3063136F 4733525A 00000000 49454E44 AE426082
According to the spec, we start with the PNG signature which is the first 8 bytes.
89504E47 0D0A1A0A
We then have repeating "chunk" structures, this file has 3 "chunks", the header (IHDR), the image data (IDAT) and then the end "chunk" (IEND).
Each chunk is arranged into: the first 4 bytes for the length of the chunk data, the next 4 bytes for the type of data, then n-bytes for the actual data and then 4 bytes for the cyclic redundancy check (CRC) of the type of data and actual data sections.
Following this through...
0000000D
Is the chunk's data length (13 bytes).
49484452
Is the chunk's type (IHDR).
00000004 00000004 08020000 00
Is the chunk's data (4 bytes width, height; 1 byte bit depth, colour type, compression method, filter method, interlace method).
269309 29
Is the CRC of the data and type (managed to get the code to work this out from here.
000000 3F
Is the next chunk's data length (63 bytes).
494441 54
Is the chunk's type (IDAT).
081D01 3400CBFF 01CC96B1 134FE120 C0CECDF1 5101FFA5 60000000 000000E0 403201DF E59286DF 6D000000 00000004 EDB11F00 2E007A21 93EDB11F 3063136F
Is the chunk's actual data (the image data compressed and filtered).
So my actual question is how do I decode this last section into raw pixels?
According to the spec, I must first decompress the data (INFLATE?) and then unfilter it (??) to be left with scanlines of pixels (my goal).
If this could be explained in pseudocode that would be amazing! Otherwise I'm familiar with Swift and less so with C...
This book explains the process of decoding for programmers:
https://www.amazon.com/Compressed-Image-File-Formats-JPEG/dp/0201604434
The entire process is way to complicated to fit within the space of a SO answer. PNG uses two different methods of compression: LZ and Huffman encoding.
I think pngcheck will help you considerably:
pngcheck -vv result.png
Sample Output
File: result.png (334985 bytes)
chunk IHDR at offset 0x0000c, length 13
600 x 450 image, 24-bit RGB, non-interlaced
chunk IDAT at offset 0x00025, length 65536
zlib: deflated, 32K window, default compression
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4
1 4 4 1 4 1 4 1 4 4 4 4 4 4 1 1 1 1 1 1 1 1 1 1 1
1 1 1 1 1 1 1 1 1 4 4 1 1 1 1 1 1 1 4 4 1 4 1 1 4
4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 (96 out of 450)
chunk IDAT at offset 0x10031, length 65536
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
1 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 (188 out of 450)
chunk IDAT at offset 0x2003d, length 65536
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 1 4 4 4 4 4 4 4 4 (273 out of 450)
chunk IDAT at offset 0x30049, length 65536
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
4 4 4 4 4 4 4 4 4 2 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 4 4 4 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4
4 4 4 4 1 4 4 4 (356 out of 450)
chunk IDAT at offset 0x40055, length 65536
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
4 4 4 4 1 4 4 4 4 4 4 4 1 4 4 4 4 4 4 4 4 4 4 4 4
4 4 4 1 4 4 4 4 4 4 4 1 1 1 4 4 1 4 4 1 1 1 4 4 4
4 4 1 1 1 4 1 4 4 4 1 4 1 4 4 4 1 4 1 4 4 4 4 4 4
4 1 1 4 1 4 4 1 4 1 (441 out of 450)
chunk IDAT at offset 0x50061, length 7188
row filters (0 none, 1 sub, 2 up, 3 avg, 4 paeth):
4 1 1 4 4 4 4 1 4 (450 out of 450)
chunk IEND at offset 0x51c81, length 0
No errors detected in result.png (8 chunks, 58.6% compression).
Your question is way too broad for pseudocode -- or at least, actionable pseudocode - if we go high-level enough, you already know the basics: 1. uncompress, 2. unfilter, 3. profit.
Since you state in comments that you want to do everything yourself, I propose that you start by implementing the default decompression algorithm, ZLIB CM=8 "deflate", which is well-described in https://www.ietf.org/rfc/rfc1951.txt among others.
suppose we have a matrix , and we want to extract zero value from the matrix
the code will be some thing like:
A <- matrix( c(4, 0, 3, 1, 2, 0,5,6,8,9,3,2,3,4,5,6,7,8,9,0,12,34,3,4,5,0,4, 3, 1, 25, 4,0,6,0,4,12,2,3,4,5,6,7,12,7,0), nrow=5,ncol=9,byrow = TRUE)
w<- which(A == 0 , arr.ind = TRUE)
the indices of zero value are :
> w
row col
[1,] 1 2
[2,] 3 2
[3,] 4 5
[4,] 1 6
[5,] 4 7
[6,] 3 8
[7,] 5 9
my question is :how to group the result of indices into two subsets like
1 2 3 6 8
4 5 7 9
What about if the condition ( extract all the values >= 9)
the result should be two groups
1 2 3 4 9
5 7
I am reading the Joy Of Clojure and came across an implementation of quick sort that uses lazy sequences to achieve better than O(n log n) performance when only taking the first few cons cells from the lazy sequence. I am trying to get it to work for a 2d array of integers, but I can't quite seem to get it:
So far this is what I have:
(defn sort-parts2
"Lazy, tail-recursive, incremental quicksort. Works against
and creates partitions based on the pivot, defined as 'work'."
[work]
(lazy-seq
(loop [[part & parts] work]
(if-let [[pivot & xs] (seq part)]
(let [smaller? #(< % pivot)]
(recur (list*
(filter smaller? xs)
pivot
(remove smaller? xs)
parts)))
(when-let [[x & parts] parts]
(cons x (sort-parts2 parts)))))))
sort.joy=> a
[[5 9 0 6 5 1 4 4 8 7]
[2 6 8 2 6 7 0 1 8 1]
[8 8 0 5 9 9 7 7 1 6]
[9 8 5 3 0 2 0 6 9 3]
[8 8 5 8 9 8 2 3 8 5]
[7 9 2 8 5 6 6 8 3 4]
[4 8 2 4 6 6 7 4 4 1]
[8 5 1 7 3 0 2 4 2 3]
[9 1 2 9 0 1 0 2 2 9]
[4 0 2 6 8 5 6 1 7 7]]
what I want to get for an input of (sort-parts2 a 0) //index on the first element of the subvectors is:
sort.joy=> aSorted
[[2 6 8 2 6 7 0 1 8 1]
[4 0 2 6 8 5 6 1 7 7]
[4 8 2 4 6 6 7 4 4 1]
[5 9 0 6 5 1 4 4 8 7]
[7 9 2 8 5 6 6 8 3 4]
[8 5 1 7 3 0 2 4 2 3]
[8 8 0 5 9 9 7 7 1 6]
[8 8 5 8 9 8 2 3 8 5]
[9 1 2 9 0 1 0 2 2 9]
[9 8 5 3 0 2 0 6 9 3]]
as is, this is what I am getting right now:
sort.joy=> (sort-parts a)
(0
1
4
4
5
5
6
7
8
9
[2 6 8 2 6 7 0 1 8 1]
0
1
5
6
7
7
8
8
9
9
[9 8 5 3 0 2 0 6 9 3]
2
3
5
5
8
8
8
8
8
9
[7 9 2 8 5 6 6 8 3 4]
1
2
4
4
4
4
6
6
7
8
[8 5 1 7 3 0 2 4 2 3]
0
0
1
1
2
2
2
9
9
9
[4 0 2 6 8 5 6 1 7 7])
Could someone please help me figure out a way to stop the loop from fully destructuring the 2d vector, thus returning a lazy vector with the head being a row vector? I have the comparotors for the rows figured out, thank you!
edit More general formulation of the problem: I want to sort a 2d vector, seen as a bellow into a lazy-sequence, where the head is the next sub-vector ordered by an index. To do this non-lazily, I am using:
(defn sortVector
"sorts a 2d vector by the given index"
[index coll]
(sort-by #(nth % index) coll))
You can derive a sort-parts-by function from the JoC code if you start off with the original and replace the smaller? function with one which compares the values of some key function on pivot and another item, rather than the items directly:
(let [pivot-key (keyfn pivot)
smaller? #(.compare ^java.util.Comparator comp
(keyfn %1)
pivot-key)]
...)
The names keyfn and comp come from clojure.core/sort-by -- see its docstring / implementation for the intended meaning.
Note that sort-parts is not intended to be called on the actual sequence to be sorted -- you have to wrap it in some sort of seqable first (probably a vector); see the definition of qsort in JoC for an example. You'll probably want a qsort-by to go along with the function sketched above.