I'm attempting to create an array of matrices by multiplying a t = range(0,stop=2*pi,length=101) by a matrix [1, 0] as follows
A = t .* [1 ,0]
but this produces the error ERROR: LoadError: DimensionMismatch("arrays could not be broadcast to a common size"). I would like each scalar, or element of t to be multiplied elementwise (in terms of t) with the elements of the vector [1 , 0], essentially performing an elementwise scalar--matrix product.
The reason I'm doing this is because I would later like to be able to multiply another constant matrix M with each column vector found in A. How can this be done in Julia v1.1?
You have to wrap the element you do not want to be broadcasted over in a container. Here is a standard way to do it (I decreased length kwarg to 3 to make the example more clear):
julia> t = range(0,stop=2*pi,length=3)
0.0:3.141592653589793:6.283185307179586
julia> A = t .* Ref([1 ,0])
3-element Array{Array{Float64,1},1}:
[0.0, 0.0]
[3.141592653589793, 0.0]
[6.283185307179586, 0.0]
julia> Ref([1 2; 3 4]) .* A
3-element Array{Array{Float64,1},1}:
[0.0, 0.0]
[3.141592653589793, 9.42477796076938]
[6.283185307179586, 18.84955592153876]
Instead of Ref container you can also use a 1-element tuple or 1-element vector as wrappers:
julia> t .* ([1 ,0],)
3-element Array{Array{Float64,1},1}:
[0.0, 0.0]
[3.141592653589793, 0.0]
[6.283185307179586, 0.0]
julia> t .* [[1 ,0]]
3-element Array{Array{Float64,1},1}:
[0.0, 0.0]
[3.141592653589793, 0.0]
[6.283185307179586, 0.0]
The reason why Ref should be preferred is that it is 0-dimensional, so that it is the most neutral of these three methods (i.e. influences the output in the least way - retaining the broadcast style of the other argument). Here are some examples:
julia> f1(x) = x .* (2, )
f1 (generic function with 1 method)
julia> f2(x) = x .* [2]
f2 (generic function with 1 method)
julia> f3(x) = x .* Ref(2)
f3 (generic function with 1 method)
julia> f1(1)
(2,)
julia> f2(1)
1-element Array{Int64,1}:
2
julia> f3(1)
2
julia> f1((1,2))
(2, 4)
julia> f2((1,2))
2-element Array{Int64,1}:
2
4
julia> f3((1,2))
(2, 4)
Related
How to add a 2d tuple to a 2d matrix in Julia?
t1 = ((10,20),(30,40)); #2d immutable tuple
a = [1 2;3 4] #2d matrix
a .+ t1
throws an error:
MethodError: no method matching +(::Int64, ::Tuple{Int64, Int64})
Closest candidates are:
+(::Any, ::Any, ::Any, ::Any...) at operators.jl:560
+(::T, ::T) where T<:Union{Int128, Int16, Int32, Int64, Int8, UInt128, UInt16, UInt32, UInt64, UInt8} at int.jl:87
+(::Integer, ::Ptr) at pointer.jl:161
...
Stacktrace:
[1] _broadcast_getindex_evalf
# .\broadcast.jl:648 [inlined]
[2] _broadcast_getindex
# .\broadcast.jl:621 [inlined]
[3] getindex
# .\broadcast.jl:575 [inlined]
[4] copy
# .\broadcast.jl:922 [inlined]
[5] materialize(bc::Base.Broadcast.Broadcasted{Base.Broadcast.DefaultArrayStyle{2}, Nothing, typeof(+), Tuple{Matrix{Int64}, Tuple{Tuple{Int64, Int64}, Tuple{Int64, Int64}}}})
# Base.Broadcast .\broadcast.jl:883
[6] top-level scope
# REPL[15]:1
Is there a vector/matrix addition method exists?
Obviously I can use a for loop for element by element addition.
Ah, so the problem here is that, while you call t1 a "2d tuple", it is really not; it is a nested tuple, a tuple-of-tuples, and is thus not really comparable to your 2d array (which really is a two-dimensional object and not just an array-of-arrays).
If you want to add a two-dimensional array to an immutable object that can be stack-allocated like a Tuple while being truly two-dimensional, then you can use the StaticArrays.jl package, which provides the immutable SArray type:
julia> a = [1 2; 3 4]
2×2 Matrix{Int64}:
1 2
3 4
julia> using StaticArrays
julia> t1 = SA[10 20; 30 40]
2×2 SMatrix{2, 2, Int64, 4} with indices SOneTo(2)×SOneTo(2):
10 20
30 40
julia> a + t1
2×2 SMatrix{2, 2, Int64, 4} with indices SOneTo(2)×SOneTo(2):
11 22
33 44
However, if for any reason you really want to stick with nesting instead of multidimensionality, then let's make the array nested as well
julia> t1 = ((10,20),(30,40)) #tuple-of-tuples
((10, 20), (30, 40))
julia> a = [[1,2],[3,4]] #vector-of-vectors
2-element Vector{Vector{Int64}}:
[1, 2]
[3, 4]
in which case you can easily solve by recursion
add(a,b) = a + b
add(a::Vector, b::NTuple{N}) where N = [add(a[i],b[i]) for i in 1:N]
add(a::NTuple{N}, b::Vector) where N = [add(a[i],b[i]) for i in 1:N]
julia> add(a,t1)
2-element Vector{Vector{Int64}}:
[11, 22]
[33, 44]
This approach will also scale to arbitrarily deeply nested objects:
julia> a = [[[1,2],[3,4]],[[5,6],[7,8]]]
2-element Vector{Vector{Vector{Int64}}}:
[[1, 2], [3, 4]]
[[5, 6], [7, 8]]
julia> t1 = (((10,20),(30,40)),((50,60),(70,80)))
(((10, 20), (30, 40)), ((50, 60), (70, 80)))
julia> add(a,t1)
2-element Vector{Vector{Vector{Int64}}}:
[[11, 22], [33, 44]]
[[55, 66], [77, 88]]
Unfortunately, there is widespread semantic confusion between true multidimensional arrays and the common hack of using a vector-of-vectors in languages which do not support real multidimensional arrays.
#cbk has already explained how nested arrays are not equivalent to multidimensional structures. But you can of course write your addition as a nested broadcast if your structure is already nested. Since (I belive) Julia 1.6, there is syntax for standalone "broadcasted operators", with which this is possible:
julia> t1 = ((10,20),(30,40))
((10, 20), (30, 40))
julia> a = [[1,2],[3,4]]
2-element Vector{Vector{Int64}}:
[1, 2]
[3, 4]
julia> broadcast(.+, a, t1)
2-element Vector{Vector{Int64}}:
[11, 22]
[33, 44]
If a is a Matrix, you can do the same on eachrow:
julia> a = [1 2;3 4]
2×2 Matrix{Int64}:
1 2
3 4
julia> broadcast(.+, eachrow(a), t1)
2-element Vector{Vector{Int64}}:
[11, 22]
[33, 44]
(In this specific case, map is equivalent to broadcast.)
Let's suppose we have a function, that gives us following:
julia> ExampleFunction(Number1, Number2)
5-element Vector{Tuple{Int64, Int64}}:
(2, 2)
(2, 3)
(3, 3)
(3, 2)
(4, 2)
I would like to convert Vector{Tuple{Int64, Int64}}
into a matrix, or in my case I would like to convert it into a 5x2 Matrix.
It's not entirely clear from your question how the 5-element vector of tuples of length 2, which has 10 elements in total, should be converted to a 3x2 matrix which holds 6 elements, but assuming that you meant 5x2 here's one way of doing it:
julia> x = [(1,2), (2, 3), (3, 4)]
3-element Vector{Tuple{Int64, Int64}}:
(1, 2)
(2, 3)
(3, 4)
julia> hcat(first.(x), last.(x))
3×2 Matrix{Int64}:
1 2
2 3
3 4
EDIT: As Phips mentioned an alternative below, here's a quick benchmark on Julia 1.7beta3, Windows 10 - I've thrown in a loop version as well, as it always makes sense to try a straightforward loop in Julia:
julia> convert_to_tuple1(x) = hcat(first.(x), last.(x))
convert_to_tuple1 (generic function with 1 method)
julia> convert_to_tuple2(x) = PermutedDimsArray(reshape(foldl(append!, x, init = Int[]), 2, :), (2, 1))
convert_to_tuple2 (generic function with 1 method)
julia> function convert_to_tuple3(x)
out = Matrix{eltype(x[1])}(undef, length(x), length(x[1]))
for i ∈ 1:length(x)
for j ∈ 1:length(x[1])
out[i, j] = x[i][j]
end
end
out
end
convert_to_tuple3 (generic function with 1 method)
julia> xs = [(rand(1:10), rand(1:10)) for _ ∈ 1:1_000_000];
julia> using BenchmarkTools
julia> #btime convert_to_tuple1($xs);
15.789 ms (6 allocations: 30.52 MiB)
julia> #btime convert_to_tuple2($xs);
22.067 ms (21 allocations: 18.91 MiB)
julia> #btime convert_to_tuple3($xs);
7.286 ms (2 allocations: 15.26 MiB)
(further edited to add $ for interpolation of xs into the benchmark)
The fastest code would be a non-copying one. In example from the other post this would reduce times from milliseconds to nanoseconds:
julia> xs
1000000-element Vector{Tuple{Int64, Int64}}:
(5, 1)
(4, 3)
⋮
(1, 4)
(9, 2)
julia> #btime reshape(reinterpret(Int, $xs), (2,:))'
10.611 ns (0 allocations: 0 bytes)
1000000×2 adjoint(reshape(reinterpret(Int64, ::Vector{Tuple{Int64, Int64}}), 2, 1000000)) with eltype Int64:
5 1
4 3
⋮
1 4
9 2
And for the copying code the fastest is going to be:
function convert_to_tuple4(x)
out = Matrix{eltype(x[1])}(undef, length(x), length(x[1]))
for i ∈ 1:length(x)
#inbounds #simd for j ∈ 1:length(x[1])
out[i, j] = x[i][j]
end
end
out
end
Benchmarks:
julia> #btime convert_to_tuple3($xs);
3.488 ms (2 allocations: 15.26 MiB)
julia> #btime convert_to_tuple4($xs);
2.932 ms (2 allocations: 15.26 MiB)
So I have a size N in julia and I need an NxN sparse matrix with N ones in it, in random places. What would be the best way to go about this?
At first I thought about randomly generating indexes and then setting those numbers to 1 in a sparse matrix but I recently found the sprand functions however I don't understand how to use them correctly or apply them to my problem. I tried using it with my limited understanding and it keeps generating error messages. Help is of course always greatly appreciated :)
Inspired by #DanGetz comment above, the following solution is a one-line function using randperm. I deleted the original answer as it was not very helpful.
sparseN(N) = sparse(randperm(N), randperm(N), ones(N), N, N)
This is also incredibly fast:
#time sparseN(10_000);
0.000558 seconds (30 allocations: 782.563 KiB)
A sparse matrix of dimension (N rows)x(M columns) has at most NxM components that can be indexed using the K=[0,N*M) integer set. For any k in K you can retrieve element indices (i,j) thanks to a Euclidean division k = i + j*N (here column major layout).
To randomly sample n elements of K (without repetition), you can use Knuth algorithm "Algorithm S (Selection sampling technique)" 3.4.2, in its book Vol2., seminumerical-Algorithms
In Julia:
function random_select(n::Int64,K::Int64)
#assert 0<=n<=K
sample=Vector{Int64}(n)
t=Int64(0)
m=Int64(0)
while m<n
if (K-t)*rand()>=n-m
t+=1
else
m+=1
sample[m]=t
t+=1
end
end
sample
end
The next part simply retrieves the I,J indices to create the sparse matrix from its coordinate form:
function create_sparseMatrix(n::Int64,N::Int64,M::Int64)
#assert (0<=N)&&(0<=M)
#assert 0<=n<=N*M
nonZero = random_select(n,N*M)
# column major: k=i+j*N
I = map(k->mod(k,N),nonZero)
J = map(k->div(k,N),nonZero)
sparse(I+1,J+1,ones(n),N,M)
end
Usage example: a 4x5 sparse matrix with 3 nonzero (=1.0) at random positions:
julia> create_sparseMatrix(3,4,5)
4×5 SparseMatrixCSC{Float64,Int64} with 3 stored entries:
[4, 1] = 1.0
[3, 2] = 1.0
[3, 3] = 1.0
Border case tests:
julia> create_sparseMatrix(0,4,5)
4×5 SparseMatrixCSC{Float64,Int64} with 0 stored entries
julia> create_sparseMatrix(4*5,4,5)
4×5 SparseMatrixCSC{Float64,Int64} with 20 stored entries:
[1, 1] = 1.0
[2, 1] = 1.0
[3, 1] = 1.0
[4, 1] = 1.0
⋮
[4, 4] = 1.0
[1, 5] = 1.0
[2, 5] = 1.0
[3, 5] = 1.0
[4, 5] = 1.0
Insisting on a one-line-ish solution:
using StatsBase
sparseones(N,M,K) = sparse(
(x->(first.(x).+1,last.(x).+1))(divrem.(sample(0:N*M-1,K,replace=false),M))...,
ones(K),N,M
)
Giving:
julia> sparseones(3,4,5)
3×4 SparseMatrixCSC{Float64,Int64} with 5 stored entries:
[1, 1] = 1.0
[2, 1] = 1.0
[3, 3] = 1.0
[2, 4] = 1.0
[3, 4] = 1.0
This method is essentially the same as the earlier answer with the advantage of re-using existing sample and being much shorter. It is even faster on larger matrices.
Consider these two functions:
Function 1:
function testVec(t,v,k,n)
for i=1:n
t .= v .* 3 .+ k .* 4;
end
end
Function 2:
module params
r = 3;
end
function testVec2(t,v,k,n)
for i=1:n
t .= v .* params.r .+ k .* 4;
end
end
They have drastically different performance:
#time testVec([1 2 3 4], [2 3 4 5], [3 4 5 6], 1000)
0.000036 seconds (7 allocations: 496 bytes)
#time testVec2([1 2 3 4], [2 3 4 5], [3 4 5 6], 1000)
0.003180 seconds (4.01 k allocations: 141.109 KiB)
Why does including the parameter r in a module make the function perform worse?
If I export the module params and include r in testVec2 without using the prefix params, its performance immediately improves (same as testVec). Why?
r in params module is a non-const global, which makes its type unstable (because some function can assign r to something else, of a different type).
Replace r = 3 with const r = 3 and the timings will be the same. Also see first section of Performance Tips.
I am confused about how parametric methods work in Julia. I also hope that parametric is the right word to use here. I've read the docs on methods but it is still not clear why the following error occurs. If I define a function as below
function Bazinga{T<:Real}(mat ::Union{Array{T,2},SparseMatrixCSC})
mat^4
end
and run
Penny = sparse(randn(10,10))
Bazinga(Penny)
I get
ERROR: MethodError: `Bazinga` has no method matching Bazinga(::SparseMatrixCSC{Float64,Int64})
Closest candidates are:
Bazinga{T<:Real}(::Union{Array{T<:Real,2},SparseMatrixCSC{Tv,Ti<:Integer}})
But Int64<:Int is true so what is the problem here?
If I redefine the function as
function Bazinga(mat ::Union{Array{Real,2},SparseMatrixCSC})
mat^4
end
no error is produced and the function works. Also there are now two methods of the function
Bazinga(mat::Union{Array{Real,2},SparseMatrixCSC{Tv,Ti<:Integer}}) at none:2
Bazinga{T<:Real}(mat::Union{Array{T<:Real,2},SparseMatrixCSC{Tv,Ti<:Integer}}) at none:2
in which both SparseMatrixCSC{Tv,Ti<:Integer} is in, but only in the first case it causes an error.
Any input is wellcome!
Edit1: And why is Array{Float64,1} <:Array{Real,1} false?
Edit2: The question in the Edit1 is solved by Arrays break string types in Julia . But I don't think it explains what happens with the method. Or at least I don't see it.
I don't quite know the reason under the hood, but using a TypeVar may be a workaround for your use case:
T = TypeVar(:T, Union{}, Real, false)
function Bazinga(mat::Union{Array{T,2},SparseMatrixCSC})
mat^4
end
julia> Bazinga(sparse(randn(2,2)))
2x2 sparse matrix with 4 Float64 entries:
[1, 1] = 0.840151
[2, 1] = -0.503551
[1, 2] = -0.437787
[2, 2] = 1.28652
julia> Bazinga(randn(2,2))
2x2 Array{Float64,2}:
0.203916 -0.261589
0.0107211 -0.0137373
UPDATE:
After some investigation, I find the "canonical" way to solve this problem is to define a typealias:
typealias RealMatrix{T<:Real} Array{T, 2}
function Bazinga(mat::Union{RealMatrix, SparseMatrixCSC})
mat^4
end
I guess the reason is Julia can't complete method matching tasks without knowing what on earth all of the parametric types are. Here is another example:
Foo{R<:Real,C<:Complex}(x::Union{Array{R}, SparseMatrixCSC{C}}) = x
Neither Foo(sparse([1+im 2; 3 4])) nor Foo(rand(2,2)) will work. So the correct way to write a parametric-union method is to constrain those different parameters via typealias:
julia> typealias Bar{T<:Real} Array{T,2}
Array{T<:Real,2}
julia> typealias Baz{T<:Complex} SparseMatrixCSC{T}
SparseMatrixCSC{T<:Complex{T<:Real},Ti<:Integer}
julia> Foo(x::Union{Bar, Baz}) = x
Foo (generic function with 1 method)
julia> Foo(rand(2,2))
2x2 Array{Float64,2}:
0.000739447 0.713386
0.32024 0.705593
julia> Foo(sparse([1+im; 2im]))
2x1 sparse matrix with 2 Complex{Int64} entries:
[1, 1] = 1+1im
[2, 1] = 0+2im
If argument types of the union have the same type parameter T, we can also use typealias as follows:
julia> typealias MyUnion{T,N<:Integer} Union{Array{T,2}, SparseMatrixCSC{T,N}}
Union{Array{T,2},SparseMatrixCSC{T,N<:Integer}}
julia> Foo{T<:Real}(x::MyUnion{T}) = x
Foo (generic function with 1 method)
julia> Foo(rand(2,2))
2x2 Array{Float64,2}:
0.0172915 0.587518
0.234148 0.00543953
julia> Foo(rand(2))
ERROR: MethodError: `Foo` has no method matching Foo(::Array{Float64,1})
julia> Foo(sparse(rand(2)))
2x1 sparse matrix with 2 Float64 entries:
[1, 1] = 0.900029
[2, 1] = 0.0634815
julia> Foo(sparse(rand(2,2)))
2x2 sparse matrix with 4 Float64 entries:
[1, 1] = 0.592632
[2, 1] = 0.066563
[1, 2] = 0.805307
[2, 2] = 0.923422
julia> Foo(rand(2,2))
2x2 Array{Float64,2}:
0.167078 0.673194
0.681418 0.316017
UPDATE 2:
In Julia-v0.6+, defining the typealias is not necessary:
function Bazinga(mat::Union{Array{<:Real,2},SparseMatrixCSC})
mat^4
end
Please refer to this answer for further details.