The Average Investor's Blog

A software developer view on the markets

Low-hanging R Optimizations on Ubuntu

Posted by The Average Investor on Jul 1, 2011

A friend of mine brought my attention recently to the fact that the default R install is way to generic and thus sub-optimal. While I didn’t go all the way rebuilding everything from scratch, I did find a few cheap steps one can do to help things a little.

Simply install the libatlas3gf-base package. That’s all, but it boosts the performance on some R calculations many fold. This package is an optimized BLAS library, which R can use out of the box.

My next step was to enable some SSE instructions when packages are compiled. To do that one needs to overwrite some compiler settings. First, I needed to find the R home path on my system: R.home() returned /usr/lib64/R on my Ubuntu 11.04. The I created a file called /usr/lib64/R/etc/Makevars.site with the following content:

CFLAGS = -std=gnu99 -O3 -pipe -msse4.2
CXXFLAGS = -O3 -pipe -msse4.2

FCFLAGS = -O3 -pipe -msse4.2
FFLAGS = -O3 -pipe -msse4.2

How did I figured out what to add? Well, I looked up the defaults for these settings in /usr/lib64/R/etc/Makeconf and combined them with what I had in mind (adding SSE4.2 by default). I also removed the default -g. Now, when a new package is installed and compiled, you should see the options. For existing packages, uninstall them using remove.packages and then install them back using install.packages. I start R as root (sudo R) for these operations.

Does your CPU support SSE and what version? Run grep -i sse /proc/cpuinfo.

Last I noticed these two variables in Makeconf:

LAPACK_LIBS = -llapack
BLAS_LIBS = -lblas

The next thing I may try when my current simulations finish is to install optimized development libraries for BLAS and LAPACK (libatlas-dev for instance) and then change the above definitions …

About these ads

6 Responses to “Low-hanging R Optimizations on Ubuntu”

  1. Not a bad post. You could also use per user Makeconf files in ~/.R/. But one thing you have plain wrong and which should be corrected. The Atlas packages *do* provide lapack and blas so you never change -llapack and -lblas — it will work with or without Atlas as Atlas provides compatible plug-inl replacements for unoptimised lapack and blas.

    If any of this is unclear, please come to the r-sig-debian list and ask.

    Dirk (aka your R maintainer for Debian and hence Ubuntu)

    • It’s not a problem though linking against the libraries explicitly (using llapack and lblas), is it? I was mostly thinking to try a GPU implementation – cublas?

      Thanks for the forum link …

  2. Joe said

    Do you have any reported speedups for any benchmarks? (including your own code?)

  3. Benchmark results for different BLAS implementations, as well as a framework for BLAS and GPU comparison are in my CRAN package ‘gcbd’ and its associated vignette. In short, you can a) still do a lot better than Atlas in Ubuntu and b) GPUs are no panacea as you need fairly large matrices to reap benefits of faster computation over increased communication.

  4. [...] neither of them. Then I concentrated on improving the overall R performance. After applying a few easy to do things I had to look for something more [...]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

 
Follow

Get every new post delivered to your Inbox.

%d bloggers like this: