An open or closed process: Determining the global scheme of perception

Project Information

HOWPER

Grant agreement ID: 786949

DOI

10.3030/786949

Project closed

EC signature date 15 May 2018

Start date 1 June 2018

End date 31 May 2025

Funded under

• EXCELLENT SCIENCE - European Research Council (ERC)

Total cost

€ 2 493 441,25

EU contribution

€ 2 493 441,25

Coordinated by

WEIZMANN INSTITUTE OF SCIENCE
Israel

Periodic Reporting for period 5 - HOWPER (An open or closed process: Determining the global scheme of perception)

Reporting period: 2024-06-01 to 2025-05-31

Summary of the context and overall objectives of the project

The HOWPER project explored what it really means for a brain—any brain—to perceive something. It looked at the steps involved in perception and what controls what we notice and when. This is important for two main reasons:
1. Helping People with Sensory or Cognitive Impairments: Understanding how perception normally works can help create better ways to support people who are blind, deaf, or have other sensory or mental challenges.
2. Building Smarter Machines: This knowledge also helps in designing better artificial "brains" for robots or self-driving vehicles so they can understand their environment more like humans do.
The project's key goal was to figure out whether human perception works more like a closed-loop (where actions and sensations constantly influence each other) or an open-loop (where things happen in one direction without ongoing feedback). The clear conclusion: perception works as a closed-loop process.
Two major clues supported this conclusion:
• Our brain's movements while sensing (like moving the eyes or hands) depend on what we’re currently sensing.
• During perception, these sensing movements stabilize in a kind of balance with the sensory information.
Besides confirming the closed-loop model, the research also helped:
• Improve devices that turn visual information into touch for blind people.
• Create new brain-inspired algorithms for robots.
Unexpectedly, the research also led to a new idea about the mind-body problem: instead of just “mind vs. body,” the observations propose a more social view—a brain-brain and brain-world duality—suggesting that the mind is a social construct. This could influence, for example, how people refer to the differences between science and religion in their attempts to understand the universe.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

The theory of closed-loop perception (CLP) was further developed and extended to a dualistic theory of brain-world (BW) brain-brain (BB) duality, in which analog BW loops interact with digital BB loops in each individual brain.
We tested open- and closed-loop schemes of perception with a battery of experiments addressing human vision and touch. Using hand, head and eye tracking and gaze-contingent displays we have found that human vision and touch follow the predictions of closed-loop perception and not those of open-loop perception. We further found that human individuals employ idiosyncratic, physiology-dependent, tactile scanning strategies and that during memory recall humans employ gaze dynamics through which the brain gradually converges to memorized interactions with its environment.
Developing novel biomimetic artificial perceivers, we found that employing the principles of active sensing and event-based coding improves perception in challenging conditions while keeping power consumption low. These findings are consistent with CLP and can be instrumental in future developments of sustainable, low power high performance machines.
Analyzing perceptual behavior of freely moving rodents we have found that perceptual behavior is sensitive to the available degrees of freedom in moving the sensory organs. Thus, constraining head motion results in redirecting attention from explorative to defensive mode and compensating with larger whisker movements. Studying sensory processing in a rat-machine hybrid we have found that sophisticated sensory processing, including complex interactions between whisker motion and touch, occurs already at the rodents’ brainstem.
We investigated the efficiency and strategies of use of an active-sensing based sensory substitution device (ASenSub), whose operational principles are based on CLP. We have found that blind and blindfolded sighted participants learn and perceive through ASenSub significantly faster than previously reported with passive-sensing based devices. Their strategies resembled natural tactile and visual strategies, respectively, and their phenomenological reports provided significant insights regarding internal perceptual processes, which are typically unconscious in natural conditions.
Overall, these results provide clear insights both into how brains perceive their environments and into how to build efficient, low-power machines. So far, these results have been exploited in a limited manner, primarily in building initial prototypes of such machines. The dissemination of these insights has also been limited—HOWPER team members have presented the results at relevant scientific conferences and in a number of public meetings. Clearly, further dissemination would contribute to better exploitation of these findings.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

The HOWPER project has significantly advanced our understanding of perception by challenging and overturning the dominant open-loop paradigm. Prior to this work, many models—both biological and artificial—assumed perception to be a mostly feedforward process. HOWPER demonstrated, through multiple experimental paradigms, that perception in humans and animals operates in a closed-loop fashion, where sensory input and motor actions continuously influence each other in real time. This represents a fundamental conceptual shift with broad implications for neuroscience, robotics, and cognitive science.
Key advances beyond the state of the art include:
• Empirical validation of closed-loop perception (CLP) in both vision and touch using dynamic, gaze- and motion-contingent experiments. These studies established not only that sensing is active and closed-loop, but that it stabilizes toward individual-specific motor-sensory steady states—a feature absent from open-loop models.
• Discovery of matched motor-sensory strategies, such as idiosyncratic tactile scanning patterns tailored to individual sensory thresholds, revealing a personalized and adaptive nature of perception not accounted for in existing models.
• Introduction of brain-brain and brain-world duality, extending CLP into a novel theory of cognitive structure that integrates social (digital) and sensual (analog) perceptual loops. This social-cognitive extension of perception theory repositions the mind as a non-physical, interaction-based construct—offering a fresh lens on longstanding philosophical debates such as the mind-body problem.
• Design and testing of novel biomimetic artificial perceivers based on CLP principles. These systems successfully employed active sensing and event-based coding to achieve robust perception in challenging conditions while maintaining low power consumption—significantly advancing the frontier of efficient autonomous machines.
• Validation of CLP in sensory substitution, where active-sensing devices (e.g. ASenSub) led to faster learning and more natural perceptual strategies than passive devices. This demonstrated practical translational potential for rehabilitation and assistive technologies, especially for visually impaired individuals.
• Revealing subcortical complexity in perception, including the discovery that complex sensorimotor integration occurs already at the brainstem level in rodents, emphasizing that perception is a multi-layered and embodied process beginning at early neural stages.
Taken together, these advances mark a departure from traditional theories that separate sensing and acting. HOWPER's results establish a new baseline for understanding perception as an active, circular, and embodied process, opening new avenues for both scientific inquiry and real-world applications in medicine and robotics.

Dualistic perception via digital brain-brain and analog brain-world channels

Gaps between biological and artificial perceptual mechanisms