Zheqing (Bill) Zhu
朱哲清
Founder and CEO, Pokee AI (pokee.ai)
ex-Senior Staff Research Lead Manager
ex-Head of Applied Reinforcement Learning
Meta AI
LinkedIn: https://www.linkedin.com/in/zheqingzhubill/
Twitter: https://twitter.com/ZheqingZhu
Meta AI Profile: https://ai.facebook.com/people/zheqing-bill-zhu
Contact:
billzhu@pokee.ai (For industry-related inquiries)
zheqzhu@alumni.stanford.edu (For academic-related inquiries)
Zheqing (Bill) Zhu is Founder and CEO at Pokee AI building the next-generation foundation AI agents. Pokee AI is building interactive, personalized and efficient AI agents that excel in reasoning, planning and tool usage.
Before Pokee AI, Bill is a Senior Staff Research Lead Manager at Meta AI, where he serves as the Head of Applied Reinforcement Learning. His main interest lies in bringing state-of-the-art reinforcement learning technologies to real-life and bridging the gap between theoretical reinforcement learning and real-world systems. Prior to serving as Head of Applied Reinforcement Learning, he was the engineering manager and tech lead for Meta’s Ads Growth Machine Learning team, where he built the team from scratch and enabled exponential growth in business market for Meta.
Bill earned his PhD degree in Reinforcement Learning at Stanford University, advised by Professor Benjamin Van Roy, while working full-time at Meta AI leading the Applied Reinforcement Learning team. His main research focus is to understand theoretical and practical gaps in existing reinforcement learning algorithms in a real-world context. He received Master of Science in Computer Science from Stanford University (also while full-time at Meta AI) and Bachelor of Science in Computer Science with a Minor in Finance, summa cum laude, from Duke University. He has been the recipient of the Alex Vasilos Memorial Award, the Highest Distinction Graduate Award from Duke University and Ericsson BUSS Shanghai Quarterly Technical Award.
Professional Experience
Senior Staff Research Lead Manager - Head of Applied Reinforcement Learning, Facebook (Meta) AI, 2021 - now
Engineering Manager / Tech Lead, Ads Growth Machine Learning, Facebook (Meta), 2018 - 2021
Machine Learning Engineer, Ads Growth Machine Learning, Facebook (Meta), 2017 - 2018
Selected Publicly Available Research
Pearl - A Production-ready Reinforcement Learning Agent
ArXiv Link, Website, Github Repo. JMLR
Zheqing Zhu, Rodrigo de Salvo Braz, Jalaj Bhandari, Daniel Jiang, Yi Wan, Yonathan Efroni, Ruiyang Xu, Liyuan Wang, Hongbo Guo, Alex Nikulkov, Dmytro Korenkevych, Urun Dogan, Frank Cheng, Zheng Wu, Wanqiao XuNon-Stationary Contextual Bandit Learning via Neural Predictive Ensemble Sampling
ArXiv Link, Submitted to ICLR 2024 and Also Presented at INFORMS 2023
Zheqing Zhu, Yueyang Liu, Xu Kuang, Benjamin Van RoyOffline Reinforcement Learning for Optimizing Production Bidding Policies
ArXiv Link, KDD 2024
Dmytro Korenkevych, Frank Cheng, Artsiom Balakir, Alex Nikulkov, Zhihao Cen, Zuobing Xu, Zheqing ZhuOptimizing Long-term Value for Auction-Based Recommender Systems via On-Policy Reinforcement Learning.
ArXiv Link, RecSys 2023 (Also presented at KDD Workshop 2023)
Ruiyang Xu*, Jalaj Bhandari*, Dmytro Korenkevych, Fan Liu, Yuchen He, Alex Nikulkov, Zheqing ZhuDeep Exploration for Recommendation Systems.
ArXiv Link, RecSys 2023
Zheqing Zhu, Benjamin Van RoyScalable Neural Contextual Bandit for Recommender Systems.
ArXiv Link, CIKM 2023 (also presented at KDD Workshop 2023)
Zheqing Zhu, Benjamin Van RoyLearning to Bid and Rank Together in Recommendation Systems.
(ArXiv Coming Soon), Springer Machine Learning Journal
Geng Ji, Wentao Jiang, Jiang Li, Fahmid Morshed Fahid, Zhengxing Chen, Yinghua Li, Jun Xiao, Chongxi Bao, Zheqing ZhuIQL-TD-MPC: Implicit Q-Learning for Hierarchical Model Predictive Control.
ArXiv Link, ICRA 2024 (also presented at ICML Workshop 2023)
Rohan Chitnis*, Yingchen Xu*, Bobak Hashemi, Lucas Lehnert, Urun Dogan, Zheqing Zhu, Olivier Delalleau
Evaluating Online Bandit Exploration In Large-Scale Recommender System.
ArXiv Link, KDD Workshop on Multi-Armed Bandits and Reinforcement Learning: Advancing Decision Making in E-Commerce and Beyond, 2023
Hongbo Guo, Ruben Naeff, Alex Nikulkov, Zheqing ZhuTwo-tiered Online Optimization of Region-wide Datacenter Resource Allocation via Deep Reinforcement Learning.
ArXiv Link, Submitted to CoNext, 2023
Chang-Lin Chen, Hanhan Zhou, Jiayu Chen, Mohammad Pedramfar, Vaneet Aggarwal, Tian Lan, Zheqing Zhu, Chi Zhou, Tim Gasser, Pol Mauri Ruiz, Vijay Menon, Neeraj Kumar, Hongbo DongMulti-Agent Safe Planning with Gaussian Processes.
ArXiv Link, IROS 2020
Zheqing Zhu, Erdem Biyik, Dorsa Sadigh
Education
PhD, Reinforcement Learning, Stanford University, Advisor: Benjamin Van Roy, 2023 (Completed while leading the Applied Reinforcement Learning team at Meta AI)
MS, Computer Science, Stanford University, 2020
BS, Computer Science, summa cum laude, Duke University, Advisor: Ronald Parr, 2017
Honors
CMO's Highlight Launch List, Facebook (Meta), 2021
Win of Month / Win of Quarter, Ads Growth, Facebook (Meta), 2017-2021 (Multi-time Winner)
Alex Vasilos Memorial Award, Duke University, 2017
Gradudate with Highest Distinction, Duke University, 2017
Ericsson BUSS Shanghai Quarterly Technical Award, 2015
Community Services
Workshop Chair of AAAI 2023 Reinforcement Learning Ready for Production Workshop
Reviewer: NeurIPS, AAAI, MLJ
Invited Talks
Reinforcement Learning for Recommender Systems, Neflix Research, 2023
Reinforcement Learning for Recommender Systems, DataFunSummit, 2023
Deep Exploration for Recommendation Systems, at University of Chinese Academy of Sciences, 2023
Deep Machine Learning Panel, at ML Summit San Francisco, 2019
Deep Reinforcement Learning Applications, at Shanshu.ai, 2019