The bigmemory package allows users to create
matrices that are external to R, stored either in RAM or on disk,
allowing them to be bigger than the system RAM, and allowing them to
be shared across R sessions.
While these objects are defined by the big.matrix class in R, they
are really just wrappers that point to external memory. The actual
objects are implemented in C++, thus can be easily manipulated from
Armadillo is a C++ library that provides fast linear algebra
functionality. It can be used in conjunction with Rcpp through
the RcppArmadillo package.
This example demonstrates how to use the functionality provided by
Armadillo on big.matrix objects using Rcpp. To learn how to
work with big.matrix objects in Rcpp without using RcppArmadillo
see this previous gallery post.
Utilising the functions provided by armadillo simply requires the
Rcpp objects and big.matrix objects to be case to the appropriate
types that armadillo knows how to work with. To demonstrate this,
we again implement the equivalent of the colSums function, but allow
the user to specify which columns they want to apply the function to
So lets see how this function performs in comparison to colSums,
Loading required package: methods
Loading required package: bigmemory.sri
Loading required package: BH
bigmemory >= 4.0 is a major revision since 3.1.2; please see packages
biganalytics and and bigtabulate and http://www.bigmemory.org for more information.
expr min lq median uq max neval
res1 <- colSums(m[, ridx]) 56.24 57.98 59.13 75.12 1068.41 100
res2 <- BigArmaColSums(bigm@address, ridx) 12.48 13.84 19.53 36.55 61.63 100