Sweet Streams are Made of These: Data Driven Development for Stream Processing
YOW! Data 2021
The strength of a powerful stream processing engine is in how fast, and how much data it can process. This naturally adds complexity to existing integration points and can lead to development overhead. Luckily, there is a set of data-driven development principles that are built to alleviate precisely these challenges. This talk will go over what these are and how to apply them at various points throughout the development process, using real-world successes (and failures!) as examples. Although the examples are for highly complex systems, this talk will be beginner-friendly and applicable to non-streaming use cases.
Caito is a Software-Engineer-turned-Developer-Advocate. She represents the US region for Ververica (formerly Data Artisans, creators of Apache Flink) and is based in Portland, Oregon. She has presented her work with stream processing and data analytics at various meetups and conferences (mainly in the US and Europe). Outside of tech, she loves running, dance, woodworking, and terrible puns.