Clustering Data -- How to Have Fun in n-Dimensions

*
Accepted Session
Short form
osb2009-0146
Scheduled: Wednesday, June 17, 2009 from 3:50 – 4:35pm in St. Johns

Excerpt

The amount of information freely available on the internet from sources like
Twitter and Github grows every day. This gives us new opportunities to leverage
the collective consciousness.

Clustering is a wonderful method for finding useful information in large
amounts of data. But it can be an intimidating topic for programmers without a
lot of academic background. In this talk I will introduce and explain some
practical techniques for clustering real-world data.

Description

I will introduce the theory and goals of clustering algorithms. The literature in statistical analysis is made up of dense mathematical equations; so I will translate equations into pseudocode to make the topic more accessible to programmers.

I will expand on the theoretical discussing by demonstrating a simple example of a clustering problem: how to group volcanos in Alaska by geographical proximity. I will move on to algorithms with real-world applications, such as how to group users with similar tastes given a database of user ratings.

I may touch on more advanced techniques to improve the accuracy of resulting clusters. I will also discuss current limitations of statistical analysis. As an example, Netflix’ ongoing competition for an algorithm that can predict whether or not a user will like the movie Napolean Dynamite.

The examples from the talk will be implemented using JavaScript and CouchDB. My hope is that people from many different language and environment backgrounds will have some experience with JavaScript. And the data-processing capabilities of CouchDB are well suited to clustering algorithms.

Tags

cluster analysis, statistical analysis, data mining, pattern recognition, CouchDB, javascript

Speaking experience

Speaker