Loading…
This event has ended. Create your own event → Check it out
This event has ended. Create your own
Register Now or Visit the Website for more Information 
View analytic
Monday, May 9 • 2:00pm - 2:50pm
Will It Scale? The Secrets Behind Scaling Stream-processing Applications - Navina Ramesh, LinkedIn

Sign up or log in to save this to your schedule and see who's attending!

Scaling stream processing applications is sometimes seen akin to scaling batch processing applications. You may re-partition your input stream to scale throughput, similar to re-sharding a batch. However, it becomes challenging for "stateful" applications to “stay realtime”, as they frequently require fault-tolerant state-management. Providing low-latency, fault-tolerant processing for high-volume input streams is fundamentally governed by the state-management primitives provided by the stream processing systems. In this talk, we will discuss how such stateful applications are supported in the open-source stream-processing systems, such as Apache Flink, Spark Streaming and Apache Samza. We will, then provide a deep-dive on Apache Samza’s approach for state-management and fault-tolerance and discuss how it can be effectively used to scale stateful applications.

Speakers
avatar for Navina Ramesh

Navina Ramesh

LinkedIn
Navina Ramesh started her career in Yahoo! India, where she contributed on scaling the Yahoo! Search clusters for 3 years. At LinkedIn, she has worked on developing the Feed Personalization pipeline and improved the caching and pagination models in the Feed Infrastructure. She has also initiated and led the effort to provide real-time improvements to Feed Relevance. Starting late-2014, she quickly became an experienced Samza developer at... Read More →



Monday May 9, 2016 2:00pm - 2:50pm
Plaza A