Encord tests brain-wave tagging for robot training data

A warehouse in San Leandro is being used to make a kind of robotics training data that does not exist on the open internet: a worker’s first-person video paired with signals from his brain.

According to TechCrunch, Encord — best known for tools that help companies annotate and evaluate machine-vision data — is running a trial in which a “pilot” disassembles a Jenga tower while wearing a headset that records what he sees and measures brain waves. The headset was built by Zander Labs, a German neuroscience startup. The goal is to tag moments of intent, surprise, or error as they happen, then test whether that extra layer of labeling helps customers’ robotics models learn manipulation tasks.

The experiment sits inside a broader scramble for physical-world data. Large language models could be trained on vast quantities of text scraped from the internet; robot learning has no comparable public corpus of hands moving objects in messy environments, with reliable ground truth about what went right and wrong. Encord’s head of robot learning, Vineeth Velmurugan, told TechCrunch that the raw material for training robotic models “simply does not exist,” and estimated that breaking the bottleneck would require a dataset many times larger than YouTube’s video archive.

That shortage is turning data collection into an industry rather than a side effect of product use. Self-driving car companies built fleets to gather their own sensor streams, but that approach is expensive and hard to replicate for general-purpose manipulation. Video-only training is cheaper but loses detail that matters when a robot has to grip, pour, stack, or recover from a slip. Encord is already collecting “egocentric” footage from factories, and in its San Leandro facility it is also using leader-follower rigs — paired robotic arms where one follows a human-controlled twin — to capture demonstrations of tasks such as pouring coffee or stacking poker chips.

Brain-wave tagging is an attempt to add a missing dimension: not just what the human did, but when the human noticed something was off. A Zander neuroscientist supervising the work, Lucas Gehrke, described brain activity during tasks as a clue for model builders about when to deploy more computationally intensive approaches. If such signals can be made reliable, they could let developers mark the hard parts of a task without relying on after-the-fact human review, and decide where expensive “high-effort” models are worth running.

For now, TechCrunch describes the work as a trial run: Encord plans to build an initial brain-wave-tagged dataset, test it with customer models, and only then decide whether it scales. In the warehouse, the props are mundane — wooden blocks, mugs, poker chips — but the business case is straightforward. Whoever can supply the missing data for robots gets paid before the robots can earn their keep.