StarProbe Performance

We performed a number of experiments to test speed of StarProbe. The result indicates how StarProbe can work on large datasets.

System

The following system was used in the experiments;

  • Hardware: Sony Vaio Laptop
  • CPU: 1.6GHz Pentium 4
  • Memory: 512MB
  • OS: Windows XP
  • Java Runtime: JRE 1.4.2 (Sun Microsystems)
Data

The data used was as follows;

  • Records: 1 million records
  • Fields: 13 fields (6 categorical, 7 numerical)
Performance

The result of the experiments are as follows. Note that it does not include I/O since the data used was well compressed by StarProbe and OS was able to cache all of them!

Pie charts 2 seconds
Scatterplots 6 seconds
Decision trees
(depth: 20 levels)
150 seconds
Hotspot analysis
(depth:2 levels)
20 seconds
Clustering(1 epoch training)
(9x9 grids, 2 inputs)
7 seconds