A shared memory for physical work · HackUTD 2026

Know what happened. Anytime.

Work changes hands. Small adjustments get forgotten. Tools move. Anytime turns what a team sees into a searchable record with the original evidence attached.

Observation / 0142Evidence saved
Worker 03 · Camera 01 · 11:42 PM

Wrench last seen near Workbench B.

The source frame stays attached so someone else can check the observation.

Illustrative exampleReview required ↗

Small changes become hard problems when nobody has the full story.

In robotics meetings, we often spend more time looking for something or figuring out what changed than solving the problem in front of us. The same gap appears in maintenance teams, factories, and data centers.

01 / Tools

Where did it go?

Find the last recorded sighting of a tool with its time, camera, and source frame.

02 / Changes

What changed?

Review an equipment timeline across people and shifts instead of reconstructing it from memory.

03 / Handoff

What should I know?

Bring the next worker up to speed with observations they can inspect and correct.

How it works

Cameras collect. Models interpret. People verify.

Anytime organizes short visual observations into a shared timeline. Every useful claim points back to the frame that produced it.

01 / Capture

Observe

Worker cameras sample the physical work as it happens.

02 / Identify

Recognize

Object detection tracks important tools; a vision model drafts descriptions of visible actions.

03 / Remember

Connect

A central server combines camera records into individual and team timelines.

04 / Retrieve

Ask

Workers search across shifts and inspect the evidence behind an answer.

Explore an example

Ask the workplace memory.

An answer is only useful if the team can see what it came from. These sample questions show how a shared record could help.

Anytime / Example response

The adjustable wrench was last observed on Workbench B beside the red toolbox.

11:42 PM
Worker 03 camera recorded the wrench on the bench.
11:45 PM
No later sighting appears in this example timeline.

Illustrative interaction · AI observations require review against the original frames.

The prototype

Built for real workflows, designed to adapt.

The current project combines an edge capture agent, visual reasoning, a tool labeling workflow, and a searchable team timeline. The same structure can describe a robotics workshop or an industrial site.

EdgeRaspberry Pi camera agent and optional Limelight AprilTag observations
VisionNVIDIA Nemotron for draft descriptions; a custom YOLO training workflow for team tools
MemoryFastAPI, timestamped observations, source frames, and local SQLite storage
InterfaceIndividual and shared timelines, evidence search, and a map for location estimates