Loading…
DeveloperWeek 2020 has ended
Friday, February 14 • 10:00am - 10:25am
PRO SESSION: Linear Scaling - Monolith to Microservices and Dataflow FULL

Log in to save this to your schedule and see who's attending!

Feedback form is now closed.
Limited Capacity full
Adding this to your schedule will put you on the waitlist.

Over the last 12 years, Credit Karma has grown to over 85 million members, added many new products, and expanded to an international user-base. Our huge member base offers a tremendous opportunity to champion financial progress for everyone--but only if we can find ways to reach out to them at scale.

Over the course of our growth, we’ve had to maintain a system with a non-linear PHP runtime, with very data-intensive SQL queries, running on scores of physical servers, and taking 20+ hours every day to send emails at scale; it took two weeks to email all of our users, and our operational overhead costs were unsustainable.

The first part of this talk details how we optimized the algorithm that was at the heart of this process. Instead of evaluating every user for eligibility for certain criteria, we now evaluate all users that qualify for the given criteria--resulting in a linear algorithm. The new algorithm is implemented as a set of Apache Beam transforms and deployed as a Dataflow application with BigQuery as its data source. Multi-threading our Dataflow application further parallelized our pipeline, and increased the speed of our eligibility selection process by an order of magnitude, while substantially reducing the operational overhead and the number of site incidents.

The second part of this talk describes how we scaled our sending process. Our downstream micro-services initially sent hundreds of thousands of emails/hour. But on peak days, we wanted to send several orders of magnitude more emails, which would take two full days! When we dug into the metrics and quotas and limits, we realized we had neglected the latencies of the downstream services. By horizontally scaling them and adding more consumers, we were able to scale the delivery rate to eight times with even more room to grow.

Find out how our new modular platform gives us a 5x speedup delivering emails, with dramatically reduced operational overhead, easier tuning, and fewer site incidents.

DeveloperWeek Speakers
avatar for Shine Garg

Shine Garg

Senior Software Engineer, Credit Karma
Shine Garg is Senior Software Engineer at Credit Karma. She is one of the first team members who designed and wrote the notifications platform that sends millions of messages to Credit Karma members every day. She also takes a keen interest in mentoring her coworkers.Prior to Credit... Read More →


Friday February 14, 2020 10:00am - 10:25am
DeveloperWeek PRO Stage B (Grand Ballroom)
  • Talk Type PRO SESSION
  • about Shine Garg is Senior Software Engineer at Credit Karma. She is one of the first team members who designed and wrote the notifications platform that sends millions of messages to Credit Karma members every day. She also takes a keen interest in mentoring her coworkers.<br><br>Prior to Credit Karma, Shine worked on mobile technology to keep teenagers safe, and powered image and video search. She has an M.S. in Computer Science from Columbia University.<br><br>When not tech’ing, she practices mindfulness meditation, thinks deeply about sustainability and greater good, and enjoys reading, writing, cooking, and hiking.