Stick a fork() in It - Parallel and Distributed Perl



Tools and approaches for multi-threaded, parallel, and distributed Perl programming. How to redefine your programs to leverage multiple cores and nodes without going insane. Includes analysis of traditionally sequential problems and their parallel implementations. Presents strategies for simple bolt-on clustering and managing parallel tasks with a focus on clean, testable, maintainable code.


Parallelism appears throughout the history of computing, yet many programmers have been caught off-guard by the proliferation of multi-core systems. Luckily, Perl and Unix provide lots of tools for leveraging multi-core or multi-node environments. Let’s quit spoon-feeding the computer one byte at a time and get out our fork()s.

Appetizer: The basic models and concepts of parallelization via a brief history of parallel computing followed by a few fun facts about spaceships, Unix, and Ahmdal’s law.

Main course: Armed with these fundamentals, we’ll then study a few compact examples of turning sequential problems into parallel ones. We’ll find ways to bend or break the rules and see how parallelism changes some of the old assumptions.

Dessert: A quick tour through an open-source parallel log processing system which demonstrates these techniques in a real-life production program.

Speaking experience