smallseo.info

incanter

Clojure-based, R-like statistical computing and graphics environment for the JVM Incanter: Statistical Computing and Graphics Environment for Clojure

Incanter - How can I use filter with column keywords instead of nth?

(require '[incanter.core :as icore])

;; Assume dataset "data" is already loaded by incanter.core/read-dataset

;; Let's examine the columns (note that Volume is the 5th column)
(icore/col-names data)
==> [:Date :Open :High :Low :Close :Volume]

;; We CAN use the :Volume keyword to look at just that column
(icore/sel data :cols Volume)
==> (11886469 9367474 12847099 9938230 11446219 12298336 15985045...)

;; But we CANNOT use the :Volume keyword with filters
;; (well, not without looking up the position in col-names first...)
(icore/sel data :filter #(> (#{:Volume} %) 1000000))

Obviously this is because the filter's anon function is looking at a LazySeq, which no longer has the column names as part of its structure, so the above code won't even compile. My question is this: Does Incanter have a way to perform this filtered query, still allowing me to use column keywords? For example, I can get this to work because I know that :Volume is the 5th column

(icore/sel data :filter #(> (nth % 5) 1000000))

Again, though, I'm looking to see if Incanter has a way of preserving the column keyword for this type of filtered query.


Source: (StackOverflow)

Transform Incanter matrix to nested vector

Consider a function that outputs an Incanter matrix.

Here is an example matrix containing output from the function:

A 6x4 matrix 
-4.77e-01 8.45e-01  1.39e-01 -9.83e-18
 8.55e-01 2.49e-01  1.33e-01  2.57e-17
-2.94e-03 6.60e-03 -9.63e-01  1.16e-16
...
 6.64e-09  2.55e-08  1.16e-07 -1.11e-16
-1.44e-01 -3.33e-01  1.32e-01 -7.07e-01
-1.44e-01 -3.33e-01  1.32e-01  7.07e-01

I'd like to continue analyzing the rows of the matrix, which represent points. The function that I want to feed the Incanter matrix to takes nested vectors as inputs.

So the function would need the above data in the form

[[-4.77e-01 8.45e-01  1.39e-01 -9.83e-18]  [8.55e-01 2.49e-01  1.33e-01  2.57e-17]
 [-2.94e-03 6.60e-03 -9.63e-01  1.16e-16]  [6.64e-09  2.55e-08  1.16e-07 -1.11e-16]
 [-1.44e-01 -3.33e-01  1.32e-01 -7.07e-01] [-1.44e-01 -3.33e-01  1.32e-01  7.07e-01]]

It is the transformation from the Incanter matrix representation to the nested vector structure that I am unsure how to perform. Is there a simple way to convert the data's representation?


Source: (StackOverflow)

Why do incanter, other than incanter itself, namespaces expecting clojure/core/matrix class or .clj?

In a .clj file I have (use '(incanter core io ...)). This error occurs when attempting to evaluate that code: FileNotFoundException Could not locate clojure/core/matrix__init.class or clojure/core/matrix.clj on classpath: clojure.lang.RT.load (RT.java:443). If I change the code to just this: (use '(incanter)), then all is OK except that none of the necessary namespaces are available, which is to be expected. I have been using jEdit with the clojure plugin quite happily for the past 18 months and closely verified that my classpath was OK. I tried (use '...math.combinatorics) and the csv & json jars (these all worked OK) to be reasonably sure it wasn't a jEdit config problem. I looked for 'matrix' in the clojure/core jar, but did not find it. Any help to solve this problem greatly appreciated.

After further investigation: I think I have found the source of my problem: in the clatrix-0.3.0 namespace declaration clojure.core.matrix is required. When I remove clatrix from the classpath and evaluate (use '(incanter core ...)) this error occurs: FileNotFoundException Could not locate clatrix/core__init.class or clatrix/core.clj on classpath: clojure.lang.RT.load (RT.java:443). When clatrix is added to the classpath then my original error occurs (i.e. can't find clojure.core.matrix). Incanter-core does have a Matrix.class file. It seems incanter depends on clatrix which depends on clojure.core. matrix which doesn't exist. How does one solve this problem or is there a work-around?


Source: (StackOverflow)

Using incanter in leiningen/ clojure

I am new to clojure, and coming from R, I would like to work with Incanter.

Basic question: How do I run it?

I am running Leiningen 2.3.4 on Java 1.7.0_65 OpenJDK 64-Bit Server VM on a 64 Debian machine.

The following dependencies were installed in the project.clj file [from Eric Rochester's book: Clojure Data Analysis Cookbook].

:dependencies [[org.clojure/clojure "1.5.1"]
    [incanter/incanter-core "1.4.1"]
    [incanter/incanter-io "1.4.1"]])

What the book does not say, but is explained here, I cd'd to the folder holding the project.clj file and ran "lein deps". This imported a bunch of files [I see indeed incanter-core 1.4.1 and incanter-io 1.4.1].

When I now run "lein repl" from the same directory where the project.clj file is, I get:

FileNotFoundException Could not locate incanter_core__init.class or incanter_core.clj on classpath:   clojure.lang.RT.load (RT.java:443)

This has probably something to do with namespaces, but I have no idea where to start.


Source: (StackOverflow)

Modified Bessel functions of order (n)

I'm using Incanter and Parallel Colt for a project, and need to have a function that returns the modified Bessel function of an order n for a value v.

The Colt library has two methods for order 0 and order 1, but beyond that, only a method that return the Bessel function of order n for a value v (cern.jet.math.tdouble.Bessel/jn).

I'm trying to build the R function, dskellam(x,lambda1, lambda2) for the Skellam distribution, in Clojure/Java

Is there something I can do with the return value of the Bessel method to convert it to a modified Bessel?


Source: (StackOverflow)

Vectorized approach to creating a matrix from the outer product of two vectors

New to Incanter, and was wondering what a vectorized solution to creating a matrix based on the results of the pair-wise product of two lists, would look like. To be clearer, I have two lists that I create with

(def x (pdf-poisson (range 4) :lambda 2.2))
(def y (pdf-poisson (range 4) :lambda 1.5)).

I would now like a 4x4 matrix M such that M(1,1) is the product of x(1) and y(1), M(1,2) is the product of x(1) and y(2) etc.

Taking the outer product in Octave is easy, so was hoping that Incanter supported this also.

I can easily hand code this by mapping a function across the vectors, but wanted an idiomatic or vectorized approach, if that is possible.

Thanks, JT


Source: (StackOverflow)

How to display an incanter graph in Jpanel

Does anyone know how to display an incanter chart to jpanel without reverting to jfreechart?


Source: (StackOverflow)

Generating Random Numbers in Incanter

How do I use the random number generators in Parallel Colt from incanter?

I've listed these dependencies in my project.clj file:

:dependencies [[org.clojure/clojure "1.2.0"] [org.clojure/clojure-contrib "1.2.0"] [incanter/core "1.2.3"] [incanter/parallelcolt "0.9.4"]]

And then I tried (import cern.jet.random.tdouble Normal) and I get a class java.lang.ClassNotFoundException.

What am I doing wrong here?


Source: (StackOverflow)

Incanter labelling a time-series-chart with multiple series

I have an incanter time-series chart, which I can't seem to generate with the correct series labels on. My data looks like this:

__________________________
| :Time | :Count | :Name |
| 12344 |    0   | "A"   |
| 12344 |    1   | "B"   |
| 12344 |    2   | "C"   |
| 12345 |    4   | "A"   |

I have tried setting the series label to a small set of strings, but only the first value is displayed on the chart for the first series. My (incorrect sample) code to plot the chart looks like this:

      (view (time-series-plot :Time :Count
               :x-label "Date"
               :y-label "Points"
               :title "My Cool Graph"
               :legend true
               :group-by :Name
               :series-label "A" "B" "C"
               :data data-to-graph
               :points true
               ))

Any pointers much appreciated.


Source: (StackOverflow)

Incanter sample mean and variance not close to distribution mean and variance

I answered a question regarding generating samples with a positive support and known mean and variance using the gamma distribution in NumPy. I thought I'd try the same in Incanter. But unlike the results I got with NumPy, I could not get a sample mean and variance close to the distribution's mean and variance.

(defproject incanter-repl "0.1.0-SNAPSHOT"
  :description "FIXME: write description"
  :url "http://example.com/FIXME"
  :license {:name "Eclipse Public License"
            :url "http://www.eclipse.org/legal/epl-v10.html"}
  :dependencies [[org.clojure/clojure "1.6.0"]
                 [incanter "1.5.4"]])

(require '[incanter 
           [core] 
           [distributions :refer [gamma-distribution mean variance draw]] 
           [stats :as stats]])

(def dist 
  (let [mean 0.71 
        variance 2.89 
        theta (/ variance mean) 
        k (/ mean theta) ] 
    (gamma-distribution k theta)))

Incanter calculates mean and variance of the distribution

(mean dist) ;=> 0.71
(variance dist) ;=> 2.89

I calculate a sample mean and variance based on draws from that distribution

(def samples (repeatedly 10000 #(draw dist)))

(stats/mean samples) ;=> 0.04595208774029654
(stats/variance samples) ;=> 0.01223348345651905

I expected these stats calculated on the sample to be much closer to the mean and variance of the distribution. What am I missing?

Answer

Incanter had a bug inherited from Parallel Colt. The treatment of parameters is not consistent across methods in Parallel Colt. See issue report https://github.com/incanter/incanter/issues/245.


Source: (StackOverflow)

Unable to create matrix in incanter 1.4.0

I am playing around with incanter as I am learning clojure and I keep getting an error message when I try to create a matrix in REPL under lein:

user=> (use 'incanter.io)
nil
user=> (use 'incanter.core)
nil
user=> (def A (matrix [[1 2 3] [4 5 6] [7 8 9]])) 
NoSuchMethodError edu.emory.mathcs.utils.ConcurrencyUtils.getThreads
BeginN_2D()Icern.colt.matrix.tdouble.impl.DenseColumnDoubleMatrix2D.assign (DenseColumnDoubleMatrix2D.java:661)

It looks like a dependency issue but yet not sure how to resolve this.


Source: (StackOverflow)

Clojure / Incanter Data Transformations Capabilities

I'm considering Clojure / Incanter as an alternative to R and just wondering if clojure / incanter have the capabilities to do the following:

  1. Import the result of a SQL statement as a data-set ( I do this in R using dbGetQuery ).
  2. Reshape the data-set - turning rows into columns also known as "pivot" / "unipivot"- I do this in R using the reshape, reshape2 packages ( in the R world it's called melting and casting data ).
  3. Save the reshaped data-set to a SQL table ( I do this in R using dbWriteTable function in RMySQL )

Cheers !


Source: (StackOverflow)

Idiomatic way to add error bars to plot in Incanter

I'm creating a plot of a robot's belief of its distance to a landmark. The x-axis is number of measurements, and the y-axis is distance to landmark, which should include error bars to indicate the confidence in this estimate.

I haven't been able to find an good way to add error bars to the plot based off a value for the variance. Currently I'm creating a box-plot at each measurement by generating sample data about the mean with my value for the variance. This is clearly not ideal, in that it is computationally inefficient and is an imprecise representation of the information I'm trying to display.

Any ideas for how to do this? Ideally it would be on an xy-plot, and it could be done without having to resort to JFreeChart commands.


Source: (StackOverflow)

Incanter dependency

I am trying to implement the following link http://data-sorcery.org/category/pca/ and found myself stuck trying to load the necessary Incanter libraries, i.e.

(use '(incanter core stats charts datasets))

The only dependency that I have for Incanter is [incanter "1.5.4"]. Is this enough to load the libraries, am I just missing something?

I am not really sure how to load the 4 highlighted libraries in the link. To note I have been able to use Incanter previously in the REPL.

Edit: My text editor has the following

(ns my-namespace.filename
(:use [incanter.core]
      [incanter.stats]
      [incanter.charts]
      [incanter.datasets]))


(def iris (to-matrix (get-dataset :iris))) 
(view iris)

which returns the error CompilerException javax.net.ssl.SSLProtocolException: handshake alert: unrecognized_name, compiling:(pca.clj:11:22)

The error seems to stem from the inner part, namely the get-dataset... which I am unsure how to fix.


Source: (StackOverflow)

Troubles Importing Clojure Libs in Paradise

I occasionally get this problem, and generally work around it, but it's rather frustrating.

I have all of Incanter (check it out if you don't know it: it's superb) on my classpath. I try to import it (through a Slime REPL) like this: user> (use 'incanter.core), but fail.

Doing this: user> (use 'clojure.contrib.def) works just fine, and this file is in the same place–on my classpath.

Regardless, the error isn't anything about classpath: it's this:

Don't know how to create ISeq from: clojure.lang.Symbol
  [Thrown class java.lang.IllegalArgumentException] 

You can see my entire terminal here (a screenshot.)

I don't know what's going on here, and it's really frustrating, as I really would like to use Incancter, and I can from the Incanter binary's REPL. I definitely don't want to develop from that–and this should work.

Any help would be much appreciated.

EDIT:

It appears as though Incanter requires Clojure 1.2, and lein swank gives me Clojure 1.1. This might be the cause of my problems: if so, is there a way to continue to use Swank & Lein with Clojure 1.2?

Thanks again!

EDIT:

Apparently if you start using Clojure-1.1 and lein swank, you're stuck with it unless you make a new project.

If future people have this problem, this article helped me out, but also, at least for me, you must start a new lein project if you had begun it using leink swank and Clojure-1.1. Simply changing your project.clj file and then lein swanking again doesn't work.


Source: (StackOverflow)