Picture of Ben Agro

Ben Agro

Welcome to my website! My interests are real-world robotics and understanding (machine) intelligence. I have been a high-level rock climber for the majority of my life.

Experience

Waabi Logo
I am a researcher at Waabi working on next-generation autonomy systems under Raquel Urtasun. I developed novel implicit perception and prediction models that surpass state-of-the-art methods. Details to come!
RVL Logo
I was a research intern at the Robotics Vision and Learning Lab supervised by Florian Shkurti. We were working on learning methods for task and motion planning, and I developed a new algorithm for PDDLStream that learned-task specific heuristics for expanding the space of possible robot actions.
ASRL Logo
I was a research intern at the Autonomous Space and Robotics Lab supervised by Tim Barfoot. We worked on self-supervised semantic LiDAR segmentation for autonomous navigation. I developed a simulation of a complex indoor environment complete with dynamic actors and an augmented navigation stack used to train and evaluate our method.

Publications

A cartoon describing certifiable optimization

Toward Globally Optimal State Estimation Using Automatically Tightened Semidefinite Relaxations

Frederike Dümbgen, Connor Holmes, Ben Agro, Timothy D. Barfoot.

Pre-print

During my undergraduate thesis, I spent much time trying to find redundant constraints to tighten an optimization problem such that it was globally optimal. Finding these redundant constraints was time-consuming and tedious. This paper presents an automated method for finding redundant constraints for optimization problems.

Implicit Occupancy Flow Fields for Perception and Prediction in Self-Driving

Ben Agro*, Quinlan Sykora*, Sergio Casas*, Raquel Urtasun

CVPR 2023 (Highlight)

A new approach to perception and motion-forcasting for self-driving vehicles using a neural network to implicitly represent future occupancy and flow directly from sensor data.
Picture of a Franka Panda stacking blocks

Learning to Search in Task and Motion Planning with Streams

Mohammed Khodeir*, Ben Agro*, Florian Shkurti

CoRR 2021

Presents a new algorithm for PDDLStream that uses a graph neural network to search for geometrically feasible plans in a "best first" manner.
Diagram of thet semantic segmentation pipeline

Self-Supervised Learning of Lidar Segmentation for Autonomous Indoor Navigation

Hugues Thomas, Ben Agro, Mona Gridseth, Jian Zhang, Timothy D. Barfoot

ICRA 2021

A self-supervised learning approach for semantic segmentation of LiDAR over repeated navigation sessions.

Climbing

I love bouldering, and my focus is on sending hard boulders outdoors. My current goal is to send V13. Here is some of my climbing related media:

Personal Projects

Stereo Localization Thumbnail

Towards Globally Optimal Stereo Localization (Undergrad Thesis)

UofT

This is my undergrad thesis for Engineering Science at UofT under Prof. Tim Barfoot. We investigated how to make the problem of stereo localization (determining the pose of a stereo camera with respect to observed landmarks) globally optimal.
Video of Captor on CT4

Captor (Autonomous Drone)

UofT

I built and programmed an autonomous drone with reliable onboard SLAM and vision-only obstacle avoidance. The simulator I built for this project is here.
Video of Geometry Boy

Geometry Boy

UofT

I programmed a version of the popular game Geometry Dash that runs on the original Gameboy hardware.

News

Blog