Skip to content

Clojure-based, R-like statistical computing and graphics environment for the JVM

Notifications You must be signed in to change notification settings

clojure-spark/incanter

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Incanter

Incanter

Overview and motivation

Incanter is a Clojure-based, R-like statistical computing and graphics environment for the JVM. At the core of Incanter are the Parallel Colt numerics library, a multithreaded version of Colt, and the JFreeChart charting library, as well as several other Java and Clojure libraries.

The motivation for creating Incanter is to provide a JVM-based statistical computing and graphics platform with R-like semantics and interactive-programming environment. Running on the JVM provides access to the large number of existing Java libraries for data access, data processing, and presentation. Clojure’s seamless integration with Java makes leveraging these libraries much simpler than is possible in R, and Incanter’s R-like semantics makes statistical programming much simpler than is possible in pure Java.

Motivation for a Lisp-based R-like statistical environment can be found in the paper Back to the Future: Lisp as a Base for a Statistical Computing System by Ihaka and Lang (2008). Incanter is also inspired by the now dormant Lisp-Stat (see the special volume in the Journal of Statistical Software on Lisp-Stat: Past, Present, and Future from 2005).

Motivation for a JVM-based Lisp can be found at the Clojure website, and screencasts of several excellent Clojure talks by the language’s creator, Rich Hickey, can be found at Clojure TV.

Getting started with Clojure

For a great introduction to programming in Clojure, read Clojure – Functional Programming for the JVM. by R. Mark Volkmann. For an even more extensive introduction, get one of the books on Clojure Programming Clojure, 2ed by Stuart Halloway and Aaron Bedra, “The Joy of Clojure” by Michael Fogus and Chris Houser, “Clojure in Action” by Amit Rathore, “Practical Clojure” by Luke VanderHart and Stuart Sierra.

Other Clojure resources

Getting started with Incanter

Include in Clojure project

Include all incanter modules at once in your project.clj:

:dependencies [[org.clojure/clojure "1.6.0"]
               [incanter "1.5.7"]]

Or only the modules you need:

:dependencies [[org.clojure/clojure "1.6.0"]
               [incanter/incanter-core "1.5.7"]
               [incanter/incanter-charts "1.5.7"]]

Start repl:

lein repl

Use standalone app

Start by visiting the Incanter website for an overview, checkout the documentation page for a listing of HOW-TOs and examples, and then download either an Incanter executable or a pre-built version of the latest build of Incanter, which includes all the necessary dependencies, and unpack the file (if you would like to build it from source, read Building Incanter). You also might need to install libgfortran3 library that is required for jblas that is powering matrix operations (see jblas wiki for more details).

Start the Clojure REPL (aka the shell) by double-clicking on the downloaded executable or, if you downloaded the pre-built distribution, running one of the scripts in the Incanter directory: script/repl or script\repl.bat on Windows.

Usage

From the Clojure REPL, load the Incanter libraries:

user=> (use '(incanter core stats charts io))

Try an example: sample 1,000 values from a standard-normal distribution and view a histogram:

user=> (view (histogram (sample-normal 1000)))

Try another simple example, a plot of the sine function over the range -4 to 4:

user=> (def my-plot (function-plot sin -10 10))
user=> (view my-plot)

You can save plots into a png file:

user=> (save my-plot "plot.png")

Incanter can save charts into pdf or svg files. Check incanter.pdf/save-pdf and incanter.svg/save-svg functions.

Let’s play with some data now. We’ll look at London weather for 2012:

; function that returns dataset containing weather in London for given month in 2012
(defn weather-for-month [month]
  (-> (format "https://www.wunderground.com/history/airport/EGLL/2012/%d/10/MonthlyHistory.html?format=1" month)
      (read-dataset :header true)))

; get weather data for each month in 2012 and build single dataset
(def data (->> (range 1 13) (map weather-for-month) (apply conj-rows)))

; view dataset in a table and view histogram of mean temperature
(view data)
; wunderground.com formats temperature depending on locale/location/whatever 
; so you might need to use "Mean TemperatureF" otherwise you'll get NullPointerException.
(view (histogram "Mean TemperatureC" :nbins 100 :data data))

; function that given month "2012-9-20" extracts month and returns 9
(defn month [date] (Integer/parseInt (second (.split date "-"))))

; dataset that contains 2 columns: month and mean temperature for that month
; don't forget to change to "Mean TemperatureF" if you did so few steps back.
(def grouped-by-month
  (->> (map (fn [date temp] {:month (month date) :temp temp})
            ($ "GMT" data) ($ "Mean TemperatureC" data))
       to-dataset
       ($rollup :mean :temp :month)
       ($order :month :asc)))

; view line chart that shows that August was the warmest month
(view (line-chart :month :temp :data grouped-by-month))

The online documentation for most Incanter functions contain usage examples. The documentation can be viewed using Clojure’s doc function. For example, to view the documentation and usage examples for the linear-model function, call (doc linear-model) from the Clojure shell. Use (find-doc "search term") to search the online documentation from the Clojure shell. The API documentation can also be found at http://incanter.github.io/incanter/.

The Clojure Data Analysis Cookbook (published by Packt Publishing) contains several chapters dedicated to Incanter, including work with datasets, charting, etc. You can read the sample chapter that describes Incanter’s datasets.

There is dedicated mailing list for discussions about Incater. It’s hosted on Google Groups.

More Incanter examples

Documentation

The following documentation covers the Incanter and Clojure APIs and the APIs of the underlying java libraries.

Incanter documentation

Related API documentation

Building Incanter

To build and test Incanter, you will need to have Leiningen and git installed:

1. Clone the repository with git: git clone git://github.com/incanter/incanter.git

2. Install Leiningen (version 2.x)
a. Download the lein script: wget https://github.com/technomancy/leiningen/raw/preview/bin/lein
(use lein.bat on Windows)
b. Place it on your path and chmod it to be executable: chmod +x lein
c. Run: lein self-install

3. Execute script/install staying in the incanter directory – this will download all necessary dependencies, compile & install all Incanter’s modules into local Maven repository. The use of separate script is required because Incanter consists of several modules, and development versions are usually not available in official repositories.

4. Start a REPL: lein repl (it also starts nRepl server), or start a Swank server: lein ritz 4005

Other tasks:

  • If you want to run the tests for each of Incanter’s modules, use script/test
  • Each of Incanter’s modules are independent Leiningen projects. Just cd into modules/incanter-* and use Leiningen to build each one as a stand-alone library.
  • script/install uses Leiningen to build all the modules and install them in your local ~/.m2 repository.

Incanter dependencies

About

Clojure-based, R-like statistical computing and graphics environment for the JVM

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Clojure 95.4%
  • Java 2.5%
  • HTML 1.8%
  • Other 0.3%