Reality Editor

Reality Editor is a Hack the North 2026 project, built with Darrel Wihandi. You scan a room once with the LiDAR on an iPhone, then point at things and say what you want changed. The part that matters is that the model never generates an image of the result. It emits operations against a 3D scene graph, and a constraint solver checks each one against the geometry that was actually measured. So when it says it shifted a dresser 40cm left so the closet door still opens, that is a report of what the solver did, not a description of a picture.

1. 1

   A LiDAR scan captures walls, floors, openings, and the furniture already in the room. Anything the scan could not see is marked as inferred rather than quietly guessed at.

2. 2

   You point at something and talk. OpenAI Realtime runs the voice loop over WebRTC, and the model emits an operation as target, relation, and anchor. It never sends coordinates, because it cannot see the room.

3. 3

   A TypeScript spatial engine on the phone turns that intent into a pose, checking collisions, clearances, and door swings. It reports back what it did, including when it had to move something or refused outright.

4. 4

   Furniture comes from procedural templates, a textured catalog, or generation when nothing in the catalog fits. All three reduce to bounds in metres, so the solver cannot tell them apart.

5. 5

   Segment Anything and LaMa paint unwanted objects out of the camera view, and a finished room exports as a scaled PDF floor plan.