Core technologies for virtual worlds (RIA) (Virtual Worlds and Photonics Partnerships)

Expected Outcome

Proposals are expected to contribute to developing core technologies for Virtual Worlds, with a focus on better, more realistic and more performant interaction and immersion, at application and components levels. The following outcomes are expected:

• Area A: Better and more realistic immersion through multimodal interaction. Project results are expected to contribute to the following expected outcome: • Improvement of the multimodal immersion experience combining eXtended Reality with advanced and innovative technologies.

• Area B: Innovative photonics technologies for projection, sensing and perception in virtual worlds. Project results are expected to contribute to the following expected outcome: • Improved performance of microdisplays or sensing devices serving Virtual Worlds by using innovative Optics and Photonics technologies.

Proposals are expected to focus on either Area A or Area B. The area should be clearly identified within the proposal. Special attention will be given to proposals (whether Area A or Area B) including transdisciplinary research in order to deliver and enhance uptake of suitable, accurate, ethical and safe solutions.

Proposed applications should aim at increasing awareness, acceptance and adoption of virtual worlds applications across sectors.

Scope

Virtual worlds will impact the way people live, work, create and share content, the way public administrations interact with citizens as well as the way businesses operate, innovate, produce and interact with customers.

Multimodal interaction and immersion are key dimensions of Virtual Worlds and will be supported by innovative optics and photonics technologies to achieve the full potential of Virtual Worlds core technologies.

A broader adoption of Virtual Worlds will need better and more realistic immersion and interaction, mixing modalities, sensors and actuators for an ever-improved user experience: touch, smell, haptics, etc will be better stimulated, bringing users closer to real-world sensations, serving Virtual Worlds.

Many challenges remain to be addressed to realize optics and photonics technical solutions that offer high-performance, excellent visual quality, high-quality user experience, and sustainable services and devices, for head-mounted displays (HMD), autostereoscopic displays or sensing devices.

Coupled with display and sensing technologies, algorithms and metrics should be considered to exploit the underlying light (transportation) models given the utilized modality and address e.g. scene representation, content generation, compression, transmission, content reconstruction, content-to-display adaptation, or rendering. Moreover, the quality of integration, both from a user and technological perspective, will be essential for a broad adoption of Virtual Worlds.

The Area A proposals under this topic should develop and demonstrate novel applications for Virtual Worlds that combine several modalities and aim to provide seamless and more realistic immersive interactions. Proposals should investigate novel scientific approaches or push the limit of existing ones to improve the synchronization and integration of the different modalities.

Proposals should focus on an enhanced use of multimodal technologies, integrating at least two modalities within an extended reality application. Modalities include haptics and force-feedback, vision, touch, smell, speech, etc.

The Area B proposals should address the development and integration of advanced innovative and high-performance Optics and Photonics technologies for display and sensing devices serving Virtual Worlds.

• For displays further improve current high-end microdisplays regarding power consumption (<1 mW), device efficiency, resolution (8K and beyond, pixel densities > 10kppi), high dynamic range (HDR), colour gamut, contrast and refresh rate holding the promise for truly immersive experiences;
• For sensing devices improve accuracy in diverse lighting conditions and with different eye physiologies.

Special attention should be given to the design and fabrication of suited waveguide optics, holographic elements, diffractive optics, reflective light guides, freeform optics (including micro-optics), or meta-surfaces with optimal optical properties and behaviour; to support the required field of view (FoV), to be lightweight and safe for the user, and provide excellent colour uniformity and high brightness efficiency; to satisfy the use case requirements for both professional use and mass adoption, i.e., be scalable to large volumes and cost-effective; to integrate targeted systems seamlessly into mixed-reality devices without increasing bulk or reducing comfort.

For both Area A and Area B proposals, the quality of integration, both from a user and technological perspective, will be essential for a broad adoption of Virtual Worlds. The proposals should include a focus on human perception and experience, from subjective, objective, and functional perspectives, and take into account EU values such as inclusivity, privacy, security and safety of users and the protection of their personal data, as well as diversity. Solutions should guarantee the privacy and rights of individuals and companies and ensure secure and trustworthy interactions to deliver and enhance uptake of suitable, accurate, ethical and safe solutions.

The Consortium should pay attention to developing solutions that are reliable, robust and interoperable. Proposals should leverage existing open standards and technologies in the domain of eXtended Reality, while contributing to ongoing standardisation work. Applications should be tested and anchored in real world environments and aimed at least one application domain.

We consider that proposals with an overall duration of typically 36 months would allow these outcomes to be addressed appropriately. Nonetheless, this does not preclude submission and selection of proposals requesting other durations.

Proposals should involve the effective contribution of Social Sciences and Humanities (SSH) disciplines and SSH experts, in order to produce meaningful and significant effects enhancing the societal impact of the related research activities.

This topic is implemented through a joint effort from the co-programmed European Partnership for Virtual Worlds and the Photonics Partnership and all proposals are expected to allocate tasks for cohesion activities with both partnerships, including the CSA HORIZON-CL4-2025-03-HUMAN-17: Specific support for the Virtual Worlds Partnership and the Web 4.0 initiative.

null

Activities are expected to start at TRL 3 and achieve TRL 5 by the end of the project – see General Annex B.