AMY – Autonomous mobile robotic monitoring system for large-scale structures (internal)
The goal of Amy is the development of an autonomous mobile mapping system, which will be used especially for the inspection and monitoring of infrastructure elements. Amy consists of a mobile platform (robot vehicle), various sensors (including cameras, laser scanners) and data analysis software. The system will be modular – freely adaptable and equipped with open interfaces. In the medium term, Amy will be interconnected with other systems – the cooperation with other robots (including drones) will be in the focus of development.
Project management: Prof. Dr. Alexander Reiterer
Project duration: since 2019
Funding: internal
ECOSENSE – Multi-scale quantification and modelling of spatio-temporal dynamics of ecosystem processes by smart autonomous sensor networks (DFG SFB 1537)
Project B2: Remote assessment of laser-induced ChlF as indicator for canopy vitality using UAVs with robust lightweight instrumentation
Droughts, floods and heat waves threaten forest ecosystems worldwide. Yet the impact of these threats on forests, with their complex processes and soil-plant-atmosphere interactions, remains largely unexplored. Within the framework of ECOSENSE, an autonomous, intelligent sensor network is being developed with the help of which the processes and interactions in forest ecosystems can be analysed and modelled.
Chlorophyll fluorescence measurements can be used to measure the photosynthetic activity of plants, a main indicator of forest ecosystem vitality. Within the framework of sub-project B2, a compact UAV-mounted sensor for laser-induced chlorophyll fluorescence measurement is being developed. The data obtained with this sensor will be used in the future to create 4D vitality maps.
FOURIER – Innovative Artificial Intelligence methodologies for monitoring and maintaining large scale complex infrastructures and obtaining greener, more resilient and smart societies (MSCA)
DC 5 – Super-resolution techniques to enhance low-resolution metering and Inspection data
The quality of monitoring data significantly affects the accuracy of AI-based defect detection algorithms. In visual inspections using drones and open-source spatial data (e.g., OpenStreetMap, satellite data), image and video quality is often limited. This research will use AI-based multi time-series data and video super-resolution techniques to reconstruct high-resolution images and videos from low-resolution inputs, maximizing information gain and minimizing uncertainty. This approach reduces the need for high-resolution cameras and sensors. The goal is to develop AI-based asset classification and information extraction for assessing mechanical health, identifying degradation, failures, and damage from malicious events (e.g., impacts, drone-delivered damage). The techniques will be developed for buildings and green infrastructure subsystems, validated using data from partners and municipalities in case studies.
KaSyTwin – Generating and using digital twins of sewer infrastructure to increase operational availability and resilience (BMDV mfund)
Sewer infrastructure is of enormous relevance to public services worldwide. However, existing sewer infrastructure is generally very old and requires a great deal of maintenance. Currently, the systems are maintained and repaired as required. Together with our project partners we develop a process chain for creating digital twins of existing sewer infrastructure for predictive maintenance through intelligent resilience predictions. Therefore, we are developing robust sensor platforms equipped with cameras and laser scanning systems for mapping the sewer infrastructure. The aim is to set up two platforms that communicate with each other and whose data will form the basis for creating the digital twin. In addition, damage is to be detected and localised in real time by the sensor platforms.
LeafH2O – A drone-based dual-wavelength LiDAR for determining leaf water content (DFG SPP 2433)
Global climate change is causing more frequent periods of intense drought in many regions, which has fatal consequences for plants. This reduces productivity and increases susceptibility to pests and pathogens. As a result, the challenges for ecology and economics are increasing. A key indicator for tracking the effects is the leaf water content of plants. Until now, this has been very difficult and inefficient to determine. Remote sensing methods offer great potential here, but existing methods tend to be estimation methods that only consider the canopy. In LeafH2O, we will therefore work with our partner to develop a 2-wavelength laser scanner to determine leaf water content.
MoCES – Modeling of civil engineering structures with particular attention to incomplete and uncertain measurement data by using explainable machine learning (DFG SPP 2388)
The condition of a civil engineering structure is characterised by in-creasingly rapid degradation as it ages. A preventive action against aging is more successful the earlier it is taken. To prolong the usability of complex structures, much more information is required at a much earlier stage than is common today. To move toward predictive maintenance, fundamental research is needed on the methods of collecting, fusing, and evaluating all geometry, material, stress, and aging data. Digiti-sation, regarding the generation of a digital twin, is taking on a com-pletely new significance in this context. It enables the combination and real-time evaluation of all data required for operation and maintenance.
The main goal of our proposal is to research and develop new methods and processes for the automated modelling of complex building struc-tures. The aim is to fuse a wide variety of data streams and to consider their uncertainty and incompleteness. The modelling, which will be reali-sed based on machine learning methods, will be extended by an expla-natory component so that conclusions in the sense of object modelling and reconstruction are reproducible.
RoboTUNN – Robot-assisted inventory of underground tunnels for generating digital twins and predictive maintenance (BMDV mfund)
Subways have been operating in Germany for a long time. To ensure the usability and safety of the tunnels, they must be regularly maintained. Regular inspections of the facilities and their interiors are necessary to plan maintenance. Currently, these inspections are carried out manually. The data basis is also very inconsistent. That is why we at RoboTUNN are working with our partners to develop a system for automated tunnel inspection and digital twins as the basis for predictive maintenance management.
Our goal in the project is to develop a measuring box carried by a quadruped robot to record the tunnel geometry and damage. To speed up the processes, the evaluation of the data for damage detection is to be implemented using real-time AI. The modelling of the tunnel is implemented through semantic enrichment and automated generation of the geometry.
Forests are constantly changing. Recognising these changes is the task of forest condition monitoring. For this purpose, so-called Level I and Level II areas are examined on a random basis by visual assessment. Within the framework of WaldVital, multispectral cameras, dual-wavelength laser scanners and handheld laser scanners are to be used to support forest condition monitoring. The data obtained from these sensors and inventory data serve as the basis for automated analysis for condition and vitality assessment. This data will in turn be used to make reforestation areas more sustainable and to monitor them. In cooperation with our project partners, recording and evaluation methods will be developed and a concept for a forest information system will be developed.
Project management: Prof. Dr. Alexander Reiterer
Project duration: 2024 – 2027
Funded by:
Former research projects
ErfASst – Advance of level of automation for evaluating the structural safety of bridges (Leistungszentrum Nachhaltigkeit)
Bridges and tunnels are critical elements of traffic infrastructures. The outage of these elements has huge consequences for life and limb as well as for the security of supply. The stress for bridges grows with the increase of goods traffic. Nowadays every three years an inspection of the bridges occurs with manual and expensive methods.
The inspection becomes semi-automated and cheaper with ErfASst. New sensors and algorithms shall accelerate the detection of cracks and the evaluation of its effect on stability and usability. The new methods are realised in a demonstrator.
To preserve, protect and sustainably manage forests, continuous forest monitoring is necessary. Currently, forest inventories for forest structure monitoring are carried out in the form of cost- and time-intensive terrestrial sampling on very small areas. In the FiVe3D project, UAV-mounted multispectral cameras and laser scanners are used to record forests over large areas. The camera and scanner data are then used to derive forest structure parameters, such as tree positions, tree heights, crown base heights, tree species distribution and distribution of standing deadwood, as well as crown shape and volume, with the help of AI methods.
Project management: Prof. Dr. Alexander Reiterer
Project duration: 2022 – 2024
mdfBIM+ – Partially automated creation of object-based inventory models using multi-data fusion of multimodal data streams and existing inventory data (BMDV mfund)
To make the transport infrastructure in Germany safe and effective, the more than 65,000 bridges must be regularly inspected, rehabilitated or even newly built if necessary. Both new construction and rehabilitation require accurate and meaningful as-built data. Existing plans are often incomplete or out of date. In the rarest cases, 3D models of the existing building are available. This is exactly where mdfBIM+ comes in. The aim of the project is to develop a semi-automated process and digital tools for data interaction and modification for the creation of georeferenced, object-based as-built models. With multimodal data acquisition of the as-is state by static and mobile laser scanning (LiDAR), drone-based as well as mobile terrestrial photogrammetry, a refined as-built model is to be created in combination with existing as-built data, e.g. analogue floor plans, CAD plans, building books, expert reports, SAP. Through extensive automation, this can be used efficiently for the existing infrastructure and in particular the large number of bridge structures.
The project part of the University of Freiburg (Chair for Monitoring of Large-Scale Structures) focuses on the analysis of the multimodally acquired data. 3D point clouds from the different sources are fused and semantically segmented before they are merged with the as-built plans.
RaVeNNA 4pi – 4pi – Digital platform with 4PI real-time endoimaging of endoscopic 3D-reconstruction, visualisation und post-rehabilitation support for patient with bladder cancer (BMBF)
Endoscopy gains in importance for surgery, because it could be performed minimal-invasive and gentle surgeries with endoscopy. 3D-models form endoscopic pictures enable a more precise and more efficient surgery preparation and post processing. The reconstruction of bladder from endoscopic picture is difficult due to different reasons.
RaVeNNA should solve the reconstruction problems with structure from motion. Inside the reconstructed 3D-model bladder cancer should be detected and monitored. The classification algorithm of the structures inside the bladder is based on deep learn, especially convultional neuronal networks (ConvNets) going to be used.
Project management: Prof. Dr. Alexander Reiterer
Project duration: 2018 – 2021
SPENSER – Understanding and Predicting the Spatial and Temporal Variability of Snow Processes Under Different Vegetation Covers Combining Laser Observations and Point Measurements (DFG Individual Research Grants)
A comprehensive knowledge about how various vegetation and forest structures change snow accumulation and ablation processes is especially important since forest covers are one of the most rapidly changing land cover types. Especially human activities such as tree harvesting, regardless if done as clear cutting or selective cutting, and subsequent reforestation change the forest structure suddenly and substantially. Furthermore, changing climatic conditions will also result in expanding forest and shrub areas above timber line and hence changing the properties of forest canopies. Improved simulation models of how these changes affect snow and all the associated, above mentioned, processes will be a valuable tool to assess any impact these forest and climate changes may have on snow processes.
Firstly, an innovative UAV mounted laser scanning device specifically aimed at providing spatially distributed information for snow cover observations will be designed, implemented and evaluated. Secondly, several different forest and vegetation plots will be selected and instrumented with existing sensors. These plots will then be observed during frequent flights with the newly designed laser scanning system. Data analysis and the deduction of improved research models are further indispensable steps to bring the project a successfully end.
Project management: Prof. Dr. Alexander Reiterer
Project duration: 2020 – 2025
STREEM – Full Scale Testing of Tree Streamlining in Wind (DFG Individual Research Grants)
Due to increasing global urbanisation, trees are becoming more and more important, especially for urban planning. However, trees, for all their positive attributes, pose a risk during storms. To minimise this risk, the behaviour of trees in tree-wind interactions must be known. This is currently being investigated with tree pull tests or in the wind tunnel.
Within the project STREEM, a multi-sensor system consisting of four cameras and a tree-motion system (MPU) is being developed to determine the deformation of trees in the wind. The camera system must provide 3D information about selected areas of the tree and their movement und it must be able to deal with difficult conditions such as solar radiation. The central challenge in the development will be the identification and linking of corresponding points in the image time series and the fusion of the camera data with the airflow and MPU data. In addition to synchronising the sensors and storing the data, another challenge is the local referencing of the cameras.
