Kaiyu Zheng

Profile

I am a researcher / roboticist at the Boston Dynamics AI Institute. My research interests are in robot decision-making under uncertainty and human-robot interaction. I’ve worked on a few topics in robotics, including learning for probabilistic semantic mapping (TopoNets, GraphSPN), navigation (ROS Navigation Tuning Guide), POMDP planning (pomdp-py), spatial language understanding (SLOOP) and object search (GenMOS, COS-POMDP, 3D-MOS). My PhD thesis, Generalized Object Search, was about enabling most robots with a movable viewport to search for objects in 3D environments (see this video). So far, I've mostly worked with mobile robots, and I'm gearing towards (mobile) manipulators, where major challenges remain.

I finished my PhD in computer science at Brown University as a member of the H2R Lab, advised by Stefanie Tellex. I received M.S. & B.S. in computer science from the University of Washington with a minor in math, advised by Andrzej Pronobis and Rajesh P. N. Rao. [CV] (郑开宇)

Recent News

    2023

    Generalized Object Search
    Kaiyu Zheng
    PhD Thesis, Brown University. February, 2023.
    [pdf (7.9MB)] [bibtex] [show abstract] [show talk]
    A System for Generalized 3D Multi-Object Search
    Kaiyu Zheng, Anirudha Paul, Stefanie Tellex
    International Conference on Robotics and Automation (ICRA), 2023
    [pdf] [bibtex] [code] [poster] [show abstract] [hide video]

    2022

    Towards Optimal Correlational Object Search
    Kaiyu Zheng, Rohan Chitnis, Yoonchang Sung, George Konidaris, Stefanie Tellex
    International Conference on Robotics and Automation (ICRA), 2022
    [pdf] [bibtex] [poster] [code] [show abstract] [show talk] [hide demo]
    Hierarchical Reinforcement Learning of Locomotion Policies in Response to Approaching Objects: A Preliminary Study
    Shangqun Yu, Sreehari Rammohan, Kaiyu Zheng, George Konidaris
    Multidisciplinary Conference on Reinforcement Learning and Decision Making (RLDM), 2022
    [pdf] [bibtex] [show abstract]

    2021

    Dialogue Object Search
    Monica Roy*, Kaiyu Zheng*, Jason Liu, Stefanie Tellex
    Robotics: Science and Systems (RSS) Workshop on Robotics for People (R4P): Perspectives on Interaction, Learning and Safety, 2021
    [pdf] [poster] [bibtex] [workshop] [show abstract]
    Parallelizing POMCP to Solve Complex POMDPs
    Semanti Basu, Sreshtaa Rajesh, Kaiyu Zheng, Stefanie Tellex, R. Iris Bahar
    Robotics: Science and Systems (RSS) Workshop on Software Tools for Real-Time Optimal Control, 2021.
    [pdf] [poster] [bibtex] [workshop] [show abstract]
    Spatial Language Understanding for Object Search in Partially Observed Cityscale Environments
    Kaiyu Zheng, Deniz Bayazit, Rebecca Mathew, Ellie Pavlick, Stefanie Tellex
    International Conference on Robot and Human Interactive Communication (RO-MAN), 2021
    [pdf] [bibtex] [website] [code] [show abstract] [show talk] [hide demo]
    Multi-Resolution POMDP Planning for Multi-Object Search in 3D
    Kaiyu Zheng, Yoonchang Sung, George Konidaris, Stefanie Tellex
    International Conference on Intelligent Robots and Systems (IROS), 2021
    IROS RoboCup Best Paper Award
    [pdf] [bibtex] [website] [code] [show abstract] [show talk] [hide demo]

    2020

    pomdp_py: A Framework to Build and Solve POMDPs
    Kaiyu Zheng, Stefanie Tellex
    International Conference on Automated Planning and Scheduling (ICAPS) Workshop on Planning and Robotics (PlanRob), 2020
    [pdf] [bibtex] [docs] [code] [workshop] [show abstract]

    2019

    From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping
    Kaiyu Zheng, Andrzej Pronobis
    International Conference on Intelligent Robots and Systems (IROS), 2019
    [pdf] [bibtex] [slides] [project] [code] [show abstract] [hide video]

    2018

    Learning Graph-Structured Sum-Product Networks for Probabilistic
    Semantic Maps

    Kaiyu Zheng, Andrzej Pronobis, Rajesh P. N. Rao
    AAAI Conference on Artificial Intelligence (AAAI), 2018. (oral presentation)
    [pdf] [bibtex] [slides] [project] [code] [dataset] [show abstract] [hide video]

    2017

    Learning Semantic Maps with Topological Spatial Relations using Graph-Structured Sum-Product Networks
    Kaiyu Zheng, Andrzej Pronobis, Rajesh P. N. Rao
    International Conference on Intelligent Robots and Systems (IROS) Workshop on Machine Learning Methods for High-Level Cognitive Capabilities in Robotics (ML-HLCR), 2017.
    [pdf] [workshop] [poster]
    Learning Large-Scale Topological Maps Using Sum-Product Networks
    Kaiyu Zheng
    Senior Thesis, University of Washington, 2017
    [pdf] [bibtex] [show abstract]
    ROS Navigation Tuning Guide
    Kaiyu Zheng
    Chapter in Robot Operating System (ROS) - The Complete Reference (Volume 6), 2021 (Publication of the Week, Weekly Robotics)
    on arxiv since 2017
    [arXiv] [pdf] [book] [bibtex] [ROS site] [show abstract] [hide video]
    CSCI 2951-F: Learning and Sequential Decision Making
    Brown University, Providence, RI
    Head Teaching Assistant. Instructor: Michael L. Littman (Fall 2021)
    [website] [show description]
    CSE 446: Machine Learning
    University of Washington, Seattle, WA
    Teaching Assistant. Instructor: Sham M. Kakade (Winter 2018)
    Teaching Assistant. Instructor: Emily B. Fox (Winter 2017)
    [website (2017)] [website (2018)] [show description]
    CSE 311: Foundation of Computing
    University of Washington, Seattle, WA
    Teaching Assistant. Instructors: Paul Beame, Kevin Zatlousal (Spring 2018)
    Teaching Assistant. Instructors: Paul Beame, Shayan Oveis Gharan (Fall 2016)
    [website (2016)] [website (2018)] [show description]
    CSE 373: Data Structures and Algorithms
    University of Washington, Seattle, WA
    Teaching Assistant. Instructor: Evan McCarty (Fall 2017)
    [website] [show description]

    Teaching Certificates


    Sheridan Teaching Certificate I, Brown University (12/2021).
    [website] [show description]

    Feel free to reach me through email:
    kzheng10 [at] cs.brown.edu

    I am on Github@zkytony.