The human brain is amazingly good at many things, but the brain's communication with the outside world is severely restricted. Obtaining even a single bit of information directly from the brain, bypassing these restrictions, can revolutionize human-computer interaction and the creation of artificial intelligence using human feedback.

The value of obtaining information directly from the brain, by decoding even a single mental process, is uniquely demonstrated in Zander Labs' seminal grid paradigm project. In this project, we decoded one of the most fundamental processes in the human brain, its judgment function, and utilized it for implicit guidance.

In short, we tapped into the brain's inherent judgment or reward function and used this to guide the navigation of a cursor in two dimensions, significantly improving its ability to learn and locate a given target. This required no conscious effort or even awareness from the participants, as their judgments were obtained implicitly, directly from their brain, using our Passive Brain-Computer Interface (passive BCI) concept.

‍

About the grid paradigm

Imagine your brain automatically steering a cursor to where you want it to go, without you being aware of this happening. This is essentially what this project demonstrated.

The experiment involves a grid of dots with a cursor moving randomly from one dot to an adjacent one. One dot is selected as the cursor’s starting position, and another dot is marked as the target with a characteristic red circle.

Our brain knows where we want the cursor to go: we want it to go the target (in the case of the example here, to the left bottom corner). Before each movement, the brain subconsciously predicts that the cursor will move in this desired direction. Making these kinds of predictions is one of the fundamental abilities of the brain. In fact, our brain cannot be prevented from creating predictive models of our world, and using these models to interpret new information.

It is exactly these models, and the corresponding cognitive processes, that we made use of in this project. Because, when the cursor moves away from the target, the participant’s predictions are not met and the brain shoots a specific ‘surprise’ signal, reflecting this prediction error. We found that the intensity of the signal is proportional to the amount of ‘surprise’ generated. Using passive BCI, we can detect this signal in real time. The system can then learn from this signal which actions satisfy the participant's expectations and which don't. The result in this case is a smart cursor, aware of our preferences, knowing which movements are good and which are bad. In our experiment, the cursor used this signal to move toward the target, in significantly fewer steps than it would have taken without that implicit, brain-based feedback.

In essence, we re-purposed a fundamental, internal judgment signal of the brain to reveal human preferences and to serve as a reward function for the computer. The computer itself, unaware of the target location, used this signal to learn the desired location and improve its actions. This is how passive BCI can provide implicit information to improve human-computer interaction and machine learning.

‍

Next-level communication between computers and the human brain

The grid paradigm is a clear example of how passive BCI can make aspects of human cognition available in real time to computers, AI, and other intelligent systems. These systems can then dynamically adapt in a variety of ways, without requiring any active involvement from the user. In fact, because it is decoded directly from their naturally-occurring brain activity, the user need not even be aware that any information is being communicated. The result is a fast, flexible, implicit communication channel, enabling new forms of human-computer interaction that boosts usability while providing access to fully new types of input.

This experiment in particular shows how even a single mental process, when used in the right way, can open up a wealth of opportunities. The grid paradigm targeted a mental process related to prediction, but re-purposed it to essentially reveal a person’s preferences: their judgement with respect to how ‘good’ or ‘bad’ the cursor movement was. In this form, this is a powerful, highly informative mental process to have access to.

Not just human-computer interaction can be improved by providing implicit brain-based feedback to the system. AI can also benefit from this, as human cognition can provide interpretations, judgements, and preferences related to aspects of the real world that the AI otherwise would not have access to—and can provide this information in real time. This opens up a world of opportunities for AI to learn directly from human intelligence, directly from the brain.

‍