I am trying to use CUSP as an external linear solver for Mathematica to use the power of the GPU.
Here is the CUSP Project webpage. I am asking for some suggestion how we can integrate CUSP with Mathematica. I am sure many of you here will be interested to discuss this. I think writing a input matrix and then feeding it to CUSP program is not the way to go. Using Mathematica's LibrarayFunctionLoad will be a better way to pipeline the input matrix to the GPU based solver on the fly. What will be the way to supply the matrix and the right hand side matrix directly from Mathematica?
Here is some CUSP code snippet.
#include <cusp/hyb_matrix.h>
#include <cusp/io/matrix_market.h>
#include <cusp/krylov/cg.h>
int main(void)
{
// create an empty sparse matrix structure (HYB format)
cusp::hyb_matrix<int, float, cusp::device_memory> A;
// load a matrix stored in MatrixMarket format
cusp::io::read_matrix_market_file(A, "5pt_10x10.mtx");
// allocate storage for solution (x) and right hand side (b)
cusp::array1d<float, cusp::device_memory> x(A.num_rows, 0);
cusp::array1d<float, cusp::device_memory> b(A.num_rows, 1);
// solve the linear system A * x = b with the Conjugate Gradient method
cusp::krylov::cg(A, x, b);
return 0;
}
This question gives us the possibility to discuss compilation capabilities of Mathematica 8. It is also possible to invoke the topic of mathlink interface of MMA. I hope people here find this problem worthy and interesting enough to ponder on.
BR
If you want to use LibraryLink (for which LibraryFunctionLoad is used to access a dynamic library function as a Mathematica downvalue) there's actually not much room for discussion, LibraryFunctions can receive Mathematica tensors of machine doubles or machine integers and you're done.
The Mathematica MTensor format is a dense array, just as you'd naturally use in C, so if CUSP uses some other format you will need to write some glue code to translate between representations.
Refer to the LibraryLink tutorial for full details.
You will want to especially note the section "Memory Management of MTensors" in the Interaction with Mathematica page, and choose the "Shared" mode to just pass a Mathematica tensor by reference.
Related
I used Rcpp (especially Rcpp Armadillo) to perform a method that returns as result several large matrix, for example of size 10000*10000. How can I save these matrix to use them in R environment. Assume that my code in Rcpp looks like:
list Output (20000);
for( int i(0);i<20000;++1 ){
...
...
// Suppose that the previous lines allow me to compute a matrix Gi of size 10000*10000
Output(i)=Gi;
}
return Output;
The way I programmed is very costly and need enough memory. But I need the 20000 matrix to compute an estimator in R environment. How can I save the matrix ? I do not know if bigmatrix package can help me.
Best,
I finally found a solution. I noticed that I will need 15TB to save the matrices. That is impossible. What I finally did is to save only some features of the matrices, as eigenvalues for example and others. See more details here
I am working on solving the linear algebraic equation Ax = b by using Eigen solvers through mex function of Matlab. Given a complex sparse matrix A and a sparse vector b from Matlab workspace, I want to map matrix A and vector b in Eigen sparse matrix format. After that, I need to use Eigen's linear equation solvers to solve it. At the end I need to transfer the results x to Matlab workspace.
However, since I am not good at C++ and not familiar with Eigen either. I am stuck at the first step, namely constructing the complex sparse matrix in Eigen accepted format.
I have found there is the following function in Eigen,
Eigen::MappedSparseMatrix<double,RowMajor> mat(rows, cols, nnz, row_ptr, col_index, values);
And I can use mxGetPr, mxGetPi, mxGetIr, mxGetJc, etc, these mex functions to get the info for the above "rows, cols, nnz, row_ptr, col_index, values". However, since in my case, matrix A is a complex sparse matrix, I am not sure whether "MappedSparseMatrix" can do that.
If it can, how the format of "MappedSparseMatrix" should be ? Is the following correct ?
Eigen::MappedSparseMatrix<std::complex<double>> mat(rows, cols, nnz, row_ptr, col_index, values_complex);
If so, how should I construct that values_complex ?
I have found about a relevant topic before. I can use the following codes to get a complex dense matrix.
MatrixXcd mat(m,n);
mat.real() = Map<MatrixXd>(realData,m,n);
mat.imag() = Map<MatrixXd>(imagData,m,n);
However, since my matrix A is a sparse matrix, it seems that it will produce errors if I define mat as a complex sparse matrix like the following:
SparseMatrix<std::complex<double> > mat;
mat.real() = Map<SparseMatrix>(rows, cols, nnz, row_ptr, col_index, realData);
mat.imag() = Map<SparseMatrix>(rows, cols, nnz, row_ptr, col_index, imagData);
So can anyone provide some advice for that?
MatlLab stores complex entries in two separate buffers: one for the real components and one for the imaginary components, whereas Eigen needs them to be interleaved:
value_ptr = [r0,i0,r1,i1,r2,i2,...]
so that it is compatible with std::complex<>. So in your case, you will have to create yourself a temporary buffer holding the values in that interleaved format to be passed to MappedSparseMatrix, or, if using Eigen 3.3, to Map<SparseMatrix<double,RowMajor> >.
Moreover, you will have to adjust the buffer of indices so that they are zero-based. To this end, decrement by one all entries of col_ptr and row_ptr before passing them to Eigen, and increment them by one afterward.
In trying to port over some Matlab code to F#, I'm trying to make sure the translations are accurate. As of now, there are cases where I'm not completely sure whether there are mistakes. Since a lot of the code is statistical in nature, it would be convenient to be able to seed the F# generators to the same state as Matlab's. It would also help with triangulating the exact equations that are wrong. Wanted to ask before I started dumping Matlab generated random numbers to csv files and solving this issue in a manual way.
This is not a definitive answer as probably implementing your own random number generator in matlab and F# should yield the most reliable results. You are also bound to bump into issues of thread safety in .NET, and the shapes of matrices in matlab. For example
In matlab:
rng(200,'twister')
rand(1,5)
ans =
0.9476 0.2265 0.5944 0.4283 0.7641
In F#:
open MathNet.Numerics.Random
let random1b = MersenneTwister(200)
random1b.NextDoubles(5)
val it : float [] = [|0.9476322592; 0.4941436297; 0.2265474238;
0.1485590497; 0.5944201448|]
The 1st, 3rd, and 5th random numbers do match.
Now it's possible you can replicate this somehow by playing around with different versions and/or F# and matlab array dimensions.
The MathNet Random Docs.
My question is Can I generate a random number in Uppaal?
I would like to generate a number from a range of values. Even more, I would like to generate not just integers I would like to generate double values as well.
for example: double [7.25,18.3]
I found this question that were talking about the same. I tried it.
However, I got this error: syntax error unexpected T_SELECT.
It doesn't work. I'm pretty new in Uppaal world, I would appreciate any help that you can provide me.
Regards,
This is a common and misunderstood question in Uppaal.
Simple answer:
double val; // declaration
val = random(18.3-7.25)+7.25; // use in update, works in SMC (Uppaal v4.1)
Verbose answer:
Uppaal supports symbolic analysis as well as statistical and the treatment and possibilities are radically different. So one has to decide first what kind of analysis is needed. Usually one starts with simple symbolic analysis and then augment with stochastic features, sometimes stochastic behavior needs also to be checked symbolically.
In symbolic analysis (queries A[], A<>, E<>, E[] etc), random is synonymous with non-deterministic, i.e. if the model contains some "random" behavior, then verification should check all of them any way. Therefore such behavior is modelled as non-deterministic choices between edges. It is easy to setup a set of edges over an integer range by using select statement on the edge where a temporary variable is declared and its value can be used in guards, synchronization and update. Symbolic analysis supports only integer data types (no floating point types like double) and continuous ranges over clocks (specified by constraints in guards and invariants).
Statistical analysis (via Monte-Carlo simulations, queries like Pr[...](<> p), E[...](max: var), simulate, etc) supports double types and floating point functions like sin, cos, sqrt, random(MAX) (uniform distribution over [0, MAX)), random_normal(mean, dev) etc. in addition to int data types. Clock variables can also be treated as floating point type, except that their derivative is set to 1 by default (can be changed in the invariants which allow ODEs -- ordinary differential equations).
It is possible to create models with floating point operations (including random) and still apply symbolic analysis provided that the floating point variables do not influence/constrain the model behavior, and act merely as a cost function over the state space. Here are systematic rules to achieve this:
a) the clocks used in ODEs must be declared of hybrid clock type.
b) hybrid clock and double type variables cannot appear in guard and invariant constraints. Only ODEs are allowed over the hybrid clocks in the invariant.
I know it sound weird, but here is my scenario:
I need to do a matrix-matrix multiplication (A(n*k)*B(k*n)), but I only needs the diagonal elements to be evaluated for the output matrix. I searched cublas library and didn't find any level 2 or 3 functions that can do that.
So, I decided to distribute each row of A and each column of B into CUDA threads. For each thread (idx), I need to calculate the dot product "A[idx,:]*B[:,idx]" and save it as the corresponding diagonal output. Now since this dot product also takes some time, and I wonder whether I could somehow call cublas function here (say cublasSdot) to achieve it.
If I missed some cublas function that can achieve my goal directly (only calculate the diagonal elements for a matrix-matrix multiplication), this question could be discarded.
Yes, it can (until (and excluding) version CUDA 10).
"The language interface and Device Runtime API available in CUDA C/C++ is a subset of the CUDA Runtime API available on the Host. The syntax and semantics of the CUDA Runtime API have been retained on the device in order to facilitate ease of code reuse for API routines that may run in either the host or device environments. A kernel can also call GPU libraries such as CUBLAS directly without needing to return to the CPU." Source
Here you can see and Matrix-Vector Multiplication using cuda and CUBLAS library function cublasSgemv.
Bear in mind, however that there is no longer a device CUBLAS capability in CUDA 10.. From Robert_Crovella one can cite:
The current recommendation would be to see if CUTLASS 2 will help (it
is mostly focused on GEMM related activities). If not, write your own
code to perform the function, or call cublas from host code.
Nonetheless, currently there are several implementation online of Matrix-Vector Multiplication, for instance 1, 2, among others.
Make sure you are using the device library to call the cublas. You can't use the same library that you used to call it from the host; details about using the cuda device library can be found on cuda toolkit: http://docs.nvidia.com/cuda/cublas/index.html#device-api
Look at the cuda 5 samples under 7_CUDALibraries/ .
Here has a code example for your problem, I think this code link could help you. Thanks the Github's author.
__global__ void invokeDeviceCublasSgemm(cublasStatus_t *returnValue,
int n,
const float *d_alpha,
const float *d_A,
const float *d_B,
const float *d_beta,
float *d_C)
{
cublasHandle_t cnpHandle;
cublasStatus_t status = cublasCreate(&cnpHandle);
if (status != CUBLAS_STATUS_SUCCESS){
*returnValue = status;
return;
}
/* Perform operation using cublas */
status = cublasSgemm(cnpHandle,
CUBLAS_OP_N, CUBLAS_OP_N,
n, n, n,
d_alpha,
d_A, n,
d_B, n,
d_beta,
d_C, n);
cublasDestroy(cnpHandle);
*returnValue = status;
}