Jeffrey Zhu

• - Spearheaded the development and growth of Bing's deep learning inference service to become Microsoft's largest AI inferencing workload, hosting hundreds of deep learning models serving millions of requests per second in single millisecond latencies.
• - Enabled multimodal search capabilities for consumers and enterprises by building a trillion-scale vector similarity search platform and partnering with Microsoft Research to develop novel vector search algorithms.
• - Managed the exploration and implementation of Bing's $50M+ AI hardware strategy across GPU and other ASICs to accelerate foundational deep learning models to Bing's search quality
• - Drove cross-organization AI efficiency initiatives to save millions in cost by proactively identifying model consolidation opportunities and optimizing ML data and model workflows.

• - Coordinated execution of hybrid onboarding capabilities to accelerate cloud adoption rates of the existing $2.5 billion on-premises SharePoint business, adding 1000s of new tenants to Office 365
• - Designed hybrid end-user experiences that seamlessly connect existing on-premises deployments with Office 365, bringing over 150,000 new users into Office 365.
• - Delivered a redesign of SharePoint Online to achieve compliance with data residency and sovereignty laws. This laid the foundation for 100s of large multi-national customers to onboard into Office 365.

Presents the system architecture of Deep Learning Inference Service, Bing's platform for deep neural network model inference. This paper introduces core concepts such as intelligent model placement, heterogeneous resource management, resource isolation, and efficient routing. Highlighted as one of Topbot's top 2019 AI advances in machine learning infrastructure.