Pick and place in a real home with zero training (OK-Robot)

Wheeled, Hard

OK-Robot combines off-the-shelf navigation and manipulation models for zero-shot pick-and-drop in real homes; it was tested in 10 homes on 170+ objects.

View build on GitHub ↗Share on X

Scan your home with an iPhone, build an open-vocabulary semantic map, then ask the robot to move objects. No task-specific training.

Hardware & cost: Hello Robot Stretch, an iPhone with LiDAR, a GPU workstation.

Difficulty: Hard.

Agent install prompt (paste into Claude Code, Cursor or Codex): "Read https://github.com/ok-robot/ok-robot, set up the navigation and manipulation environments per the README, and process a sample home scan."

Source: ok-robot/ok-robot on GitHub — https://github.com/ok-robot/ok-robot (facts checked against the project README, Oct 2026; prices vary by region).

Quick answers

What do you need to pick and place in a real home with zero training (OK-Robot)?: Hello Robot Stretch, an iPhone with LiDAR, a GPU workstation.
How hard is it to pick and place in a real home with zero training (OK-Robot)?: Hard.

How your agent drives it

Copied from the ok-robot/ok-robot README

Shell

$ mamba activate ok-robot-env
cd ok-robot-manipulation/src
python demo.py --open_communication --debug

Any coding agent that can run a shell can run these. Read the README before you run anything on a robot that moves.

Ideas to try with it

Our ideas, not the builder's claims.

Scan a room with an iPhone and ask the robot to move a named object
Tidy a kids' room by moving toys into a bin
Test it in a friend's home it has never seen

1 more idea

Log which objects it fails to grasp and why

Source: ok-robot/ok-robot on GitHub
Last commit: 2024-03-04
Builder: ok-robot
Repo: ok-robot/ok-robot, 617 stars
Built with: Hello Robot Stretch

Guides: Robots you can build with AI

What to build next

Builds that share a board, robot or agent with this one.