Software Development Engineer, AWS EMR Serverless (Amazon Web Services)
Apr, 2021 - Present
Contributed towards launching AWS EMR Serverless, a new AWS EMR offering. Enables clients to setup an EMR cluster 75% quicker by avoiding the need to setup, maintain, or configure EC2 hosts or EKS clusters.
Developed one of EMR Serverless control plane components with >99% availability and >15 TPS at launch that provides nodes to client's distributed application (i.e. Spark and Hive)
Improved clients' applications' start-time by >85% by implementing 2 features:
Added an application configuration to clients that proactively created nodes before application submission.
Evaluated various optimization strategies to reduce OCI images pull times by 90%.
Designed a multi-tenant service that enables multiple clients to run applications while ensuring network and compute isolation between clients.
Led the implementation of metrics and creation of monitoring dashboards in parts of EMR Serverless to improve debugging and proactively identify issues.