How action preparation shapes what we perceive: Spatiotemporal visual processing in the context of goal-directed eye and hand movements

Project Information

PremotorPerception

Grant agreement ID: 898520

DOI

10.3030/898520

Project closed

EC signature date 17 April 2020

Start date 1 March 2021

End date 29 February 2024

Funded under

• EXCELLENT SCIENCE - Marie Skłodowska-Curie Actions

Total cost

€ 246 669,12

EU contribution

€ 246 669,12

Coordinated by

HUMBOLDT-UNIVERSITAET ZU BERLIN
Germany

Periodic Reporting for period 2 - PremotorPerception (How action preparation shapes what we perceive: Spatiotemporal visual processing in the context of goal-directed eye and hand movements)

Reporting period: 2023-03-01 to 2024-02-29

Summary of the context and overall objectives of the project

How action preparation shapes what we perceive: Spatiotemporal visual processing in the context of goal-directed eye and hand movements


A closer look at goal-directed eye and hand movements

Humans continuously make saccadic eye movements to see their surroundings with the high precision of the fovea. Each saccade shifts the image on the retinas, such that the visual system needs to integrate blurry peripheral views of objects with their high-resolution equivalents once brought into the fovea. Understanding how this seamless integration is achieved is critical for theories of human perception. The EU-funded project PremotorPerception combines psychophysical protocols that enable a continuous assessment of visual sensitivity throughout the visual field with transcranial magnetic brain stimulation to investigate the role of specific brain areas underlying these modulations. We assess dynamic modulations of visual processing before saccades and goal-directed hand movements that enable continuous perception as the eyes move.


Overall objectives

When inspecting a visual scene, we make a succession of saccadic eye movements to analyze objects of interest with the high precision of the fovea. Each saccade drastically shifts the image on the retina. As the acuity of the fovea markedly decreases with eccentricity, to maintain a stable percept, the visual system needs to seamlessly integrate blurry peripheral representations with their high-resolution equivalents once brought into the fovea by the saccade. We hypothesize that this integration is achieved by predictive and selective tuning of feature sensitivity shortly before the eye movement, to assimilate pre and postsaccadic percept. Combining a new psychophysical protocol that enables a continuous assessment of visual sensitivity throughout the view field with a powerful reverse correlation approach, we will conduct a systematic investigation of human presaccadic orientation and frequency modulations across space and time. We will establish whether local feature information is predictively modulated shortly before a saccade to enable a stable, continuous perception across the eye movement, and contrast the observed presaccadic modulations to the dynamics elicited by the preparation of goal-directed hand movements. Furthermore, using transcranial magnetic stimulation (TMS) we will selectively disrupt the functioning of early and higher visual areas to reveal their temporal interplay and characterize their respective contribution to both presaccadic attention and covert attentional orienting in the absence of eye movements. Understanding how the human visual system seamlessly integrates foveal and peripheral feature representations is not only critical to further our understanding of healthy and impaired human perception, but also forms the basis for artificial vision. Our findings will help constrain computational models of perception and attention and could improve the design of safer human-machine interfaces, e.g. for driving and air traffic control.

Work performed from the beginning of the project to the end of the period covered by the report and main results achieved so far

- Presaccadic perceptual dynamics across time and space
Data analysis and interpretation for two psychophysical studies.
WP1 resulted in 2 publications
Hanning & Deubel, 2022, Journal of Neurophysiology
Hanning & Deubel, 2022, Behavior Research Methods

- Presaccadic attention around the visual field
Programming and piloting, as well as data collection, analysis, and interpretation for four psychophysical studies.
WP2 resulted in 3 manuscripts,
one published: Kwak, Hanning, & Carrasco, 2023, Scientific Reports
two preprints under review: Hanning, Himmelberg, & Carrasco, 2023, bioRxiv; Liu, Melcher, Carrasco, & Hanning, 2023, bioRxiv
Another manuscript is in preparation: Kwak, Zhao, Lu, Hanning, & Carrasco, in prep.

- Neural correlates of presaccadic perceptual dynamics
Programming and piloting, as well as data collection, analysis, and interpretation for three psychophys-TMS (Transcranial Magnetic Stimulation) studies.
WP3 resulted in 2 manuscripts,
Two published / in press: Fernández, Hanning, & Carrasco, 2023, PNAS; Hanning, Fernández, & Carrasco, in press, Nature Communications
Another manuscript is in preparation: Hanning, Chen, Fernández, & Carrasco, in prep.

Progress beyond the state of the art and expected potential impact (including the socio-economic impact and the wider societal implications of the project so far)

Our insights into how the visual system integrates information across eye movements and what brain regions are crucial for these processes have far-reaching implications. Beyond contributing to our fundamental understanding of human perception, the research has the potential to impact healthcare, technology, safety, and accessibility in various sectors. By shedding light on how our visual system combines high-resolution foveal information with blurry peripheral representations, the project paves the way for advancements in both scientific knowledge and practical applications that could improve various aspects of society.

Socio-Economic Impact:

- Advancing Scientific Understanding:
The project contributes to our understanding of how the human visual system processes and integrates information during eye movements. This foundational knowledge has the potential to advance multiple scientific disciplines, including neuroscience, psychology, and vision research.

- Healthcare and Rehabilitation:
Insights gained from studying how the visual system adapts and integrates information could benefit individuals with visual impairments or disorders. This research might inform rehabilitation strategies or the development of interventions to improve visual perception in those with impaired vision.

- Human-Machine Interfaces:
The findings could have direct implications for the design of human-machine interfaces. By better understanding how humans manage visual information during eye movements, researchers can develop more effective and user-friendly interfaces for tasks like driving, air traffic control, and other visually demanding activities.

- Artificial Vision:
The project's findings could influence the development of artificial vision systems, such as those used in robotics, autonomous vehicles, and virtual reality. Understanding how to seamlessly integrate visual information from different parts of the visual field is crucial for creating more efficient and realistic artificial vision systems.


Wider Societal Implications:
- Enhancing Safety in Transportation: Improved human-machine interfaces informed by this research can enhance safety in transportation contexts. Interfaces that account for how the human visual system naturally integrates information during eye movements can help reduce the cognitive load on operators, leading to safer driving and better decision-making in air traffic control.

- Accessibility and Inclusion:
If the project's findings lead to advancements in visual rehabilitation techniques, it could contribute to increased accessibility and inclusion for individuals with visual impairments. This could lead to improved quality of life and expanded opportunities for affected individuals.

- Cross-Disciplinary Collaboration:
The project's combination of psychophysics and neuroscience highlights the importance of cross-disciplinary collaboration. This approach encourages researchers from different fields to work together, fostering innovation and broadening perspectives.

FSN2_FIG