Kaiyu Zheng


I'm a Ph.D. student at Brown University, mainly advised by Stefanie Tellex in the H2R Lab. I am also advised by George Konidaris and Michael Littman.

My research interests are in the intersection of robotics, computer vision and reinforcement learning. Currently, I focus on active perception problems (e.g. object search), using a POMDP framework.

Prior to PhD studies, I graduated from the University of Washington in June 2018, with both M.S. and B.S. in Computer Science, and minor in Mathematics. At UW, advised by Andrzej Pronobis and Rajesh Rao, I conducted research on mobile robot navigation, as well as proposing new methods to use Sum-Product Networks for graph modeling and large-scale semantic mapping. Prior to that, I worked on using Myo to control a robotic arm advised by Vikash Kumar and Emo Todorov.

  • Our paper has been accepted to IROS 2019!


New! From Pixels to Buildings: End-to-end Probabilistic Deep Networks for Large-scale Semantic Mapping
Kaiyu Zheng, Andrzej Pronobis
International Conference on Intelligent Robots and Systems (IROS 2019).
The code and dataset will be released soon. Checkout libspn and COLD.
[arxiv] [project] [show abstract] [hide video]
New! Learning Graph-Structured Sum-Product Networks for Probabilistic Semantic Maps
Kaiyu Zheng, Andrzej Pronobis, Rajesh P. N. Rao
AAAI Conference on Artificial Intelligence (AAAI 2018). oral presentation.
The code and dataset will be released soon. Checkout libspn and COLD.
[pdf] [bibtex] [slides] [project] [show abstract] [hide video]
Learning Large-Scale Topological Maps Using Sum-Product Networks
Kaiyu Zheng
Senior Thesis, 2017
[pdf] [bibtex] [show abstract]


Cognitive rObot Localization Database (COLD) in preperation.
Andrzej Pronobis, Kaiyu Zheng, Kousuke Ariga, Rajesh P. N. Rao.
https://coldb.org [doc (local)]
ROS Navigation Tuning Guide
Kaiyu Zheng, 2017.
[pdf] [bibtex] [ROS site] [hide video]


New! An Introduction to Semantic Mapping in Robotics
together with Eric Rosen
Brown Robotics
— on 3/22/2019 in Providence, RI, US.
[slides] [show abstract]
Probabilistic Semantic Mapping Using Graph-Structured Sum-Product Networks
Kaiyu Zheng, Andrzej Pronobis, Rajesh P. N. Rao
UW Computer Science & Engineering Industry Affiliates Research Day
— on 11/15/2017 in Seattle, WA, US.
[poster] [slides] [show abstract]

Past Projects

Some course and side projects in my college years.
Forcasting Solar Irradiance by Learning from Time Series Data
Course project for CSE 546 (grad-level Machine Learning). Partnered with Kousuke Ariga*
Power generation from solar panels at a Photovoltaic (PV) power plant is directly proportional to solar irradiance, which depends on weather conditions. The goal of our project is to develop and analyze different machine learning methods to predict solar irradiance, leveraging a solar irradiance dataset. See abstract and report below.
[pdf] [show abstract]
Storage and Retrieval of Robotic Laser Range Data in Database Systems
Course project for CSE 544 (grad-level Database Systems). Partnered with Joshua Fan*
The goal of our project is to enable storing images of laser-range observations into a relational database, and retrieving similar images to a query image efficiently. We build a pipeline to extract image features and store them into a relational database following the Bag-of-Visual-Words model, and we implemented FIDS ("Flexible Image Database System") to speed up image retrieval, as well as Locality Sensitive Hashing to store data in a way that allows for faster retrieval. See abstract and report below.
[pdf] [show abstract]
Koolio.io: Share, in a Flippable Fashion
Founder; Main developer. Worked with a team of college friends.
A social media platform where flippable cards are the medium of information, organized by decks; Below are iframe-embedded cards, for when I was making dumplings. Click to flip them! It's live

[show demo1] [show demo2] [visit] [my profile]
JsConways: Conway's Game of Life in JavaScript
Conway's Game of Life is a cellular automaton that simulates generations of population development under a given initial configuration. It is proposed by mathematician John Horton Conway. Implemented this as a practice while self-studying JavaScript in sophomore year.
[github] [show video] [try-it]
ATLAS Replica with LEGO
In freshman year, spring 2014, I was a member of the UW ATLAS group where I studied a paper on track fitting and helped build the ATLAS replica with LEGO. [show video]

More about Me

I grew up in Guangzhou, China. My side interests include painting, writing, hiking, sports, horse-riding, history, piano, and Go.

My Chinese name is 郑开宇 (simplified), or 鄭開宇 (traditional).


Email: kaiyu_zheng [at] brown [dot] edu

Other: [github] [koolio.io]