3D Reconstruction and Feature Matching Benchmarks

Call: 30th Open Access Grant Competition; OPEN-30-13

Researcher: Assia Benbihi

Institution: Czech Technical University in Prague

Field: Applied Mathematics

3D reconstruction, or 3D mapping, is a computer vision task that produces digital 3D models of a scene. The 3D models can take various forms, the most common being 3D point clouds and meshes. They allow intelligent systems to perceive the physical which is a pre-requisite for the system to localize itself in an environment, understand what surrounds it, and adopt an appropriate course of action. 3D mapping is technology central to many applications including autonomous navigation, virtual and augmented reality applications, 3D content creation for movies and games, urban and environmental planning, and the documentation of cultural heritage.

Typically, the mapping process first scans the scene with various sensors, (cameras or laser scanners), then integrates the data within an optimization process to produce a 3D map that faithfully replicates the real world. A natural question arises: how to measure the faithfulness of the 3D map, in other words, how to measure the mapping quality?

Evaluation benchmarks answer this question by hosting accurate 3D models of scenes that act as ’pseudo-ground-truth’ digital twins. Any 3D mapping method can then be evaluated by how faithful it is to this digital twin. Such an evaluation is a reliable proxy but the generation of pseudo-ground-truth maps requires prohibitive computations, which makes 3D mapping benchmarks challenging to produce.

This project proposes a novel 3D mapping benchmark focused on large-scale urban scenes that include famous Prague landmarks such as Old Town Square or The Prague Castle. Access to supercomputers facilitates this computationally heavy project, in which complexity resides in the intensive optimization that generates pseudo-ground-truth maps, the evaluation of state-of-the-art methods that rely on graphic card advancements, and the large scale of the benchmark.

Part of the released data, mainly the geo-located images, is the outcome of a previous Karolina project, OPEN-28-60 “The Prague Visual Localisation Benchmark”, and the resulting benchmark will be a catalyst for research in 3D mapping. This project is part of and funded by the GACR EXPRO project “A Unified 3D Map Representation” (23-07973X), of which one outcome is the 3D mapping method `Tetra-NeRF’ showcased in the video and that was facilitated by the Karolina OPEN-24-6 project.

Impact of binary evolution on the outcome of common envelope evolution in massive stars

Call: 30th Open Access Grant Competition; OPEN-30-50

Researcher: Camille Landri

Institution: Charles University

Field: Astrophysics

Stars in close binary systems evolve differently than isolated single stars. They undergo various types of interactions, significantly impacting their structure and evolution. For instance, it is common for one of the stars to donate some of its mass to the other. These phases of mass transfer may later become unstable, leading to the companion plunging into the interior of the donor star, and starting a phase of so-called "common envelope evolution". As the companion spirals into the inner layer of the donor, these layers can be ejected, and the binary system survives on very short orbit, often becoming a source of bright astrophysical phenomena and/or gravitational waves. Common envelope evolution is therefore an important process that has been extensively studied, but the impact of previous phases of binary evolution on common envelope evolution has yet to be properly assessed.

The evolution of stars, both single and in binary systems, occurs on timescales of billions of years, thus we have to rely on simulations to constrain these processes. In this project, we combine state-of-the-art 1D and 3D hydrodynamics simulations to model the complete evolution of a close binary system. Our simulations, performed on the supercomputers at IT4Innovations, will enable us to accurately characterize how preceding phases of binary evolution affect the final stages of the life of binary systems, which is extremely significant for research on common envelope evolution, stellar mergers, and the formation of short-period binaries.

This research is supported by the Horizon 2020 ERC Starting Grant ‘Cat-In-hAT’ (grant agreement no. 803158).

Discovery of Economic Opinions and Causal Relationships from Textual Data

Call: 30th Open Access Grant Competition; OPEN-30-3

Researcher: Jennifer Za Nzambi

Institution: Czech Technical University in Prague

Field: Informatics

Social media platforms are akin to an untapped gold vein harbouring a reservoir of public opinions, attitudes, and sentiments which, if realised, could revolutionise how public opinions are gathered and interpreted. This project introduces a novel method for extracting opinions about economic indicators and factor impacting society from social media texts by fine-tuning large language models, on datasets comprising of social media posts, comments and more. Through fine-tuning, language models can acquire the ability to understand and mimic the economic discourses within posts published on social media. This project’s value is threefold. First, it amalgamates carefully curated datasets through which the model can effectively learn domain-specificities and economic understanding on an advanced level. Second, it devises metrics based on perplexity comparisons of opposing statements which validate the model’s comprehension of economic texts, thereby measuring the model’s alignment with datasets it was fine-tuned on, and finally applies said models to datasets garnering results indicating that the model-based approach can rival, and in some cases outperform, survey-based predictions and professional forecasts in predicting trends of economic indicators. Beyond the scope of this study, the methods and findings presented could pave the way for further applications of language model fine-tuning as a complement, or potential alternative, to traditional survey-based methods.
This research is supported by the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement No 101002898).

Towards Robust End-to-End Diarization and Source Separation

Call: 30th Open Access Grant Competition; OPEN-30-22

Researcher: Mireia Diez Sánchez

Institution: Brno University of Technology

Field: Informatics

Speaker diarization (SD) is the task of automatically determining speaker turns in conversational audio, commonly known as the task that answers "who spoke when?". SD has several practical applications: indexing of audiovisual resources with speaker labels (for example, coloring subtitles depending on the speaker), allowing structured search and access to resources, meeting indexing, among others.

Besides, it is used as a preprocessing technique for other speech processing tasks, for example to allow speaker adaptation for automatic speech recognition (ASR) systems, also known as speech to text systems. Despite the big advances with large pretrained models such as the popular ChatGPT on conversational text processing, conversational audio processing is yet an extremely challenging task for machine learning approaches.

The recent development of neural network based end-to-end diarization (EEND) systems has boosted research in the field and shifted the paradigm on how to handle conversational speech. However, as most neural network based systems, EEND systems are data-hungry and difficult to train. In this project we plan to move the field forward by finding methods to optimize the training strategies; by combining the benefits of well-founded generative models with the powerful end-to-end approaches and by exploiting the synergies of the related speech processing tasks to enhance SD performance.

This research is closely related to several of the running projects at the Speech@FIT research group at the Faculty of Information Technology of the Brno University of Technology: NTT corporation, ROZKAZ, from the Ministry of Interior of the Czech Republic, and Eloquence, funded by Horizon Europe programme.
https://eloquenceai.eu/

Optical and magnetic properties of MXene-based quantum dots

Call: 30th Open Access Grant Competition; OPEN-30-18

Researcher: Barbora Vénosová

Institution: University of Ostrava

Field: Material Sciences

MXenes have attracted much interest in 2D materials due to their unique properties. MXenes are composed of transition metal atoms (M = Ti, V, Sc, Mo or Nb), carbon or nitrogen atoms (X), and terminal surface groups (T = -O, -OH, -F, -Cl and others) and have the general formula M_n+1X_nT_x. Due to the variety of M and X elements and the differences in surface and edge terminations, thousands of different MXene structures with unique properties can be prepared. For this reason, MXenes have become a rapidly expanding family of 2D materials. MXenes, with their characteristic planar structure, exhibit exceptional structural stability and good electrical conductivity. Their surfaces are tunable, and they possess a host of unique chemical properties. These features make them versatile, finding uses across various applications. From biosensors and batteries to adsorption and catalysis, from energy storage to environmental research. 

Recent studies have shown that reducing 2D materials to a 0D structure (quantum dot, QD, less than 10 nanometres in size) can improve or even create new properties. Combining edge effects, surface, and quantum confinement can achieve this.

Our project aims to control the tuning of MXene quantum dots (MXQDs) properties by modeling their size and structure. The first results show that the electronic and magnetic properties strongly depend on the quantum dots' size. Based on these results, we want to investigate the optical and magnetic properties of different MXQDs in detail. First, we want to investigate the influence of surface functional groups on the optical properties using time-dependent density functional theory (TD-DFT). Subsequently, the different modeling options of MXQDs (various combinations of M, and X atoms with other functional groups) will be systematically studied to tune the electronic, optical, and magnetic properties. This could provide opportunities to prepare new MXQDs structures with desired properties in many application areas (e.g. photonics, optoelectronics, and/or photocatalysts).

This research is supported by the European Union under the LERCO project (number CZ.10.03.01/00/22_003/0000003) via the Operational Programme Just Transition.

Towards sub-chemical accuracy of ab initio atomistic simulations of fusion enthalpies for molecular crystals

Call: 30th Open Access Grant Competition; OPEN-30-57

Researcher: Ctirad Červinka

Institution: University of Chemistry and Technology, Prague

Field: Material Sciences

Organic molecular materials are ubiquitous, spanning diverse areas from pharmaceuticals, over fertilizers, explosives to semiconductors. Fabrication of final products in those fields often requires a sufficient solubility of relevant precursors, and subsequently, availability of a viable crystallization scheme of the target compound from solution. Understanding the melting and crystallization processes of such materials is important to design novel efficient fabrication techniques and to localize the limits of operation windows, e. g. temperatures and pressures at which a target material maintains its desired properties.

Computational modelling of crystal structures and their melting with a full resolution of individual atoms can significantly contribute to this understanding and it goes hand in hand with our capabilities of predicting material solubility, thermal stability and other important properties. Existing protocols based on classical molecular-dynamics simulations fail to reach the chemical accuracy of such predictions, and furthermore, they offer only a limited predictive power. Development of accurate and widely transferable quantum-chemical models of the melting parameters for molecular materials will thus significantly contribute to the material research.

Current project exploits the computational capacity of the IT4Innovations infrastructure to search for optimal ways of incorporating modern theoretical models to describe mutual interactions of molecules in bulk solid or liquid materials.

Our aim is to establish a high-throughput computational methodology, being sufficiently accurate at the same time, for an initial screening of material properties. In future, it should be performed at the initial stage of material research for a vast number of candidate materials, enabling to preselect the most promising targets exhibiting the properties desired for a particular application (e. g. solubility of drugs, or thermal stability of semiconductors). Only then, a detailed experimental refinement and verification narrowed only to the most promising targets could follow, which would render material research more efficient by circumventing large portions of experimental work that can be very costly and labor-demanding.