Intro To Pydra - A Distributed Computing Framework For Python

*

Excerpt

Pydra provides an easy to use framework for writing, running and managing distributed computing jobs in python. Come hear an overview of Pydra project and how you can use it to deploy your own distributed computing cluster.

Description

Pydra is a distributed and parallel computing framework for Python being built by the Oregon State University Open Source Lab. Pydra is used by the Protein Geometry Database, a biochemistry research tool, and will be used by other projects developed at the OSL. The Pydra team is working hard towards an official release later this year.

Pydra will provide:

  • A Web interface for cluster management, monitoring and job scheduling
  • Strong security within the cluster
  • An easy to use, flexible job framework, including MapReduce support
  • Fault tolerance and automatic fail over.

This session will provide an overview of the Pydra project’s architecture, concepts, and roadmap. You’ll learn how to deploy and use Pydra within your own applications.

Speaking experience

Speaker

  • Headshot small

    Peter Krenesky

    Open Source Lab

    Biography

    Peter is the Lead Software Engineer for the Open Source Lab. During his six years at the lab, he’s worked on many projects to improve life at the lab and academic computing. Peter founded the Ganeti Web Manager project in September 2010 to make cluster management at the OSL easier and to power the Supercell testing cluster.

    Some of Peter’s current projects include the Protein Geometry Database, a tool aiding biochemistry researchers, and Pydra, a cluster computing solution for Python. Past projects have included software for the One Laptop Per Child project and Helix Media Player. In his spare time, he hacks on Android applications.

    He holds a B.A. in Computer Science from the Wentworth Institute of Technology.