and python installed. You then need to install
. PANDAS
that you download version 2.2.x but I have used 2.3.0 without any difficulties. There are multiple ways to do what we want so I will present multiple methods of accomplishing the same goals. The differences in the two will also give insights into how to optimize these methods for your own problems.
This is not a description of how to use R. This is presented for those that already know R and want to call it from within python to use the advanced PANDAs data manipulation tools.
Install rpy2 with pip
This sets the correct stable version. I had a few difficulties with version above 2.3.0 but those might be fixed in the future
Ok to begin we need to import all the necessary libraries.
from numpy import *
import scipy as sp
from pandas import *
from rpy2.robjects.packages import importr
import rpy2.robjects as ro
import pandas.rpy.common as com
Talking to the R session
A lot of the confusion that can arise is due to the fact that under the hood you can think of python as running its own process of R that you can pass commands to and grab variables from. We can pass commands to the R session as by putting the R commands in the ro.r() method as strings
ro.r('x=c()')
ro.r('x[1]=22')
ro.r('x[2]=44')
print(ro.r('x'))
[1] 22 44
print(ro.r['x'])
[1] 22 44
Note that the method of the objects returned by ro.r() are of types that are not python types..
In [11]: type(ro.r('x'))
Out[11]: rpy2.robjects.vectors.FloatVector
Method 1
First we will begin by passing some commands to the R instance by reading in some data from one of R's built in datasets. The way how we will do this is first use the ro.r() method to pass a command to the R environment:
Now we will read in the R data.frame into a PANDAS data frame with the following command
pydf = com.load_data('mtcars')
Now the variable
pydf variable is a PANDAS dataframe in python that can be manipulated like any other.
In [11]: df.describe()
Out[11]:
mpg cyl disp hp drat wt qsec \
count 32.000000 32.000000 32.000000 32.000000 32.000000 32.000000 32.000000
mean 20.090625 6.187500 230.721875 146.687500 3.596563 3.217250 17.848750
std 6.026948 1.785922 123.938694 68.562868 0.534679 0.978457 1.786943
min 10.400000 4.000000 71.100000 52.000000 2.760000 1.513000 14.500000
25% 15.425000 4.000000 120.825000 96.500000 3.080000 2.581250 16.892500
50% 19.200000 6.000000 196.300000 123.000000 3.695000 3.325000 17.710000
75% 22.800000 8.000000 326.000000 180.000000 3.920000 3.610000 18.900000
max 33.900000 8.000000 472.000000 335.000000 4.930000 5.424000 22.900000
vs am gear carb
count 32.000000 32.000000 32.000000 32.0000
mean 0.437500 0.406250 3.687500 2.8125
std 0.504016 0.498991 0.737804 1.6152
min 0.000000 0.000000 3.000000 1.0000
25% 0.000000 0.000000 3.000000 2.0000
50% 0.000000 0.000000 4.000000 2.0000
75% 1.000000 1.000000 4.000000 4.0000
max 1.000000 1.000000 5.000000 8.0000
For instance, let's double the mpg
Now you can view it and manipulate it however you want with PANDAS. You could then pass it back to the R instance by first converting
pydf to an R data frame
rdf = com.convert_to_r_dataframe(df)
In [28]: type(rdf)
Out[28]: pandas.core.frame.DataFrame
and passing it to R
ro.globalenv['mtcars'] = rdf
Now we can check that is has our manipulations
In [26]: print(ro.r('mean(mtcars$mpg)'))
In [28]: print(ro.r('mean(newmtcars$mpg)'))
[1] 40.18125
So we can perform a regression
ro.r('''fit=lm(mpg ~ wt + cyl, data=newmtcars)''')
and we can look at a summary of the fit with
print(ro.r('summary(fit)'))
Let's start with importing the same routines as before
from numpy import *
import scipy as sp
from pandas import *
from rpy2.robjects.packages import importr
import rpy2.robjects as ro
import pandas.rpy.common as com
but let's now import the importr() method
from rpy2.robjects.packages import importr
Now we begin by loading in the R packages
stats = importr('stats')
base = importr('base')
datasets = importr('datasets')
We use the datasets.data.fetch('mtcars') method to create an environment with the dataset mtcars in it. This is done because of the way how some of the R data sets can be constructed.
envr = datasets.data.fetch('mtcars')
We can load in the dataframe into a PANDAS data frame with the following command
df = com.load_data('mtcars')
As an aside, if we wanted the R data frame we could do
At this point, rdf0 is an R data frame and cannot be operated on like a PANDAS dataframe
So now df is a data frame which we can manipulate at will. After we are done, we can convert it back to an R data frame
rdf = com.convert_to_r_dataframe(df)
And now we can construct a forumla for our regression:
formula = 'mpg ~ wt + cyl'
We then pass the needed information to the fit
fit_full = stats.lm(formula, data=rdf)
print(base.summary(fit_full))
Residuals:
Min 1Q Median 3Q Max
-4.2893 -1.5512 -0.4684 1.5743 6.1004
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 39.6863 1.7150 23.141 < 2e-16 ***
wt -3.1910 0.7569 -4.216 0.000222 ***
cyl -1.5078 0.4147 -3.636 0.001064 **
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 2.568 on 29 degrees of freedom
Multiple R-squared: 0.8302, Adjusted R-squared: 0.8185
F-statistic: 70.91 on 2 and 29 DF, p-value: 6.809e-12
Now if you want to see something really cool do the following:
ro.r('plot(mtcars$mpg, mtcars$wt)')
print(ro.r('identify(mtcars$mpg, mtcars$wt)'))
and left click on some points. Right click when you are done. It will label the points in the scatter plot by index so that you can delve into troublesome points.
Just to Show Off
import math, datetime
import rpy2.robjects.lib.ggplot2 as ggplot2
import rpy2.robjects as ro
from rpy2.robjects.packages import importr
base = importr('base')
datasets = importr('datasets')
mtcars = datasets.data.fetch('mtcars')['mtcars']
pp = ggplot2.ggplot(mtcars) + \
ggplot2.aes_string(x='wt', y='mpg', col='factor(cyl)') + \
ggplot2.geom_point() + \
ggplot2.geom_smooth(ggplot2.aes_string(group = 'cyl'),
method = 'lm')
pp.plot()
from rpy2.robjects.packages import importr
graphics = importr('graphics')
grdevices = importr('grDevices')
base = importr('base')
stats = importr('stats')
import array
x = array.array('i', range(10))
y = stats.rnorm(10)
grdevices.X11()
graphics.par(mfrow = array.array('i', [2,2]))
graphics.plot(x, y, ylab = "foo/bar", col = "red")
kwargs = {'ylab':"foo/bar", 'type':"b", 'col':"blue", 'log':"x"}
graphics.plot(x, y, **kwargs)
m = base.matrix(stats.rnorm(100), ncol=5)
pca = stats.princomp(m)
graphics.plot(pca, main="Eigen values")
stats.biplot(pca, main="biplot")
