Monoids, and Sketches, and CRDTs, oh my!*
A (hopefully) accessible introduction to some of the key mathematical concepts that make distributed and streaming computation possible.
Discussions of distributed systems can get pretty heavy on the jargon. HyperLogLog, CountMinSketch, PN-counters, ORSWOTs. What are these beasts and what was wrong with your basic lists, hash tables, and trees? I’ll explain why distributed computation and big data forces us to more advanced data structures; introducing key concepts and examples.
Monoids let you do portions of a computation on many machines and combine the pieces together. Sketches approximate more familiar data structures when your data sets get too big for exact answers. And CRDTs (conflict-free replicated data types) let multiple people modify data simultaneously without corruption.
This talk should be accessible to all math-curious people and only requires prior familiarity with common data structures like lists and hash maps.
I have spoken at OSCON, Open Source Bridge, Lambda Jam, CUFP, YAPC, several user groups, and given internal tech talks on a variety of topics such as "How to Serve a Billion Requests a Day with Perl", "Beautiful Concurrency with Erlang", “Enabling Microservice Architectures with Scala”, SBT, Continuous Deployment strategies, and more. Where available, slides and videos are linked from http://kevin.scaldeferri.com/dist/
This is a new talk.
Kevin Scaldeferri is a software developer specializing in scalable, high-performance server applications. He spent several years at Yahoo building ad serving systems, and at Gilt Groupe, maintaining the core e-commerce and order processing components, as well as development tools, infrastructure, and architecture. He currently works on distributed data storage and query engines at New Relic.
He has spoken at OSCON, Open Source Bridge, Lambda Jam, CUFP, YAPC, several user groups, and given internal tech talks on a variety of topics such as “How to Serve a Billion Requests a Day with Perl”, “Beautiful Concurrency with Erlang”, SBT, Continuous Deployment strategies, and more.
- Title: Overdoing Microservices: how small is too small?
- Track: Practice
- Room: B201
- Time: 2:30 – 3:15pm
All the cool kids are doing it, but is it possible to have too much of a good thing? I’ll present some thoughts about things you can actually measure to decide if you’ve gone off the deep end with microservices.
- Speakers: Kevin Scaldeferri