Spaces:

---

ibm-esa-geospatial

/

challenge

like 25

Running

Introduction

Natural hazards (such as wildfires, landslides, floods) are among the greatest threats to societies and ecosystems. Predicting these risks remains a daunting challenge. Traditional models are often tailored to specific regions and struggle to scale globally.

Today’s risk maps are frequently outdated or static. They often fail to account for climate change, urban expansion, or shifts in land use and soil properties, factors that can radically alter exposure to hazards. As a result, they become obsolete quickly. Satellite data, however, provide continually refreshed signals of changing conditions, allowing risk assessments to remain relevant to the threats communities face today.

To address these challenges, our framework follows a three-step strategy:

1. Hazard Detection (current focus): Lightweight binary segmentation decoders identify hazard footprints directly from satellite imagery. Each decoder is hazard-specific but built on a shared frozen backbone encoder that converts raw imagery into rich embeddings of the land surface.
2. Hazard Prediction (future direction): Detection outputs serve as ground-truth for models trained on temporal windows of pre-event imagery, enabling forecasts of where hazards are likely to occur.
3. Risk Assessment (long-term vision): Predictions are integrated with exposure and vulnerability data, turning hazard probabilities into actionable risk intelligence.

Both hazard detection and prediction models use TerraMind embeddings and Thinking-in-Modalities (TiM). Our framework applies most naturally to frequent hazards that leave repeated, observable signatures. These events supply the abundant training data required for machine learning. Rare events, in contrast, lack sufficient examples for effective predictive modeling.

Trained Models for Hazard Detection

At the heart of our framework lies a simple but powerful concept: specialized U-Net decoders trained for individual hazard types, all built on top of TerraMind’s frozen backbone. The frozen backbone acts as a feature extractor, while decoders specialize in detecting footprints such as flooded areas, burn scars, or unstable slopes.

For this phase, we focused on three hazards:

• Floods: Water vs. non-water
• Wildfires: Burned vs. unburned areas
• Landslides: Recent landslide vs unchanged

All models were trained with TerraMind_v1_base. For the flood detection model, we additionally leveraged TiM (Thinking in Modalities). Training used Dice loss, ideal for imbalanced segmentation tasks, and AdamW for stable convergence.

Model Performance

Key Insights

1. Despite different modalities (radar, optical, DEMs), the backbone-decoder design remains robust.
2. Even with lightweight decoders, performance matches or surpasses existing benchmarks.
3. TiM contribution: Flood detection already benefited from TiM LULC features, and future work will further extend TiM integration to enhance wildfire and landslide models (including DEMs).

From Hazard Detection to Predictive Modeling

While current models identify past hazard footprints, their value extends far beyond event mapping [3]. Accurate segmentation enables the creation of large-scale databases of historical events. Running detection pipelines across broad regions produces a catalog of when and where hazards occurred, a foundation for predictive modeling.

The aim is not to predict exact event timing but to provide risk assessments: estimating the likelihood of hazards occurring within a defined temporal window. This is particularly valuable for insurance, disaster planning, and infrastructure resilience.

Predictive models ingest temporal sequences of pre-event imagery, learning patterns that consistently precede hazards. Recurrent U-Net variants or time-series convolutional models, fed with sliding windows of days to months of imagery, can estimate probabilities of hazard occurrence at pixel or regional scale.

Both detection and prediction remain modular, built on TerraMind’s frozen backbone. Detection decoders, temporal predictors, and future risk scorers all leverage the same foundational embeddings and are enhanced with Thinking-in-Modalities (TiM) to better exploit multimodal information. This creates a flexible “AI store” for geospatial risk intelligence where new models can be added seamlessly.

Hazard Dataset Generation

Scaling hazard detection globally requires systematic dataset generation. Below, we outline the steps that automate this process across different hazards and inputs.

Pipeline & Preprocessing

The pipeline begins with multimodal inputs: Sentinel-1 GRD, Sentinel-2 L1C/L2A, or DEMs undergoing cropping, normalization, and co-registration before embedding extraction. Each hazard has tailored preprocessing:

• Flood Mapping (Sen1Floods11): Complex preprocessing to align Sentinel-1 and Sentinel-2 at pixel level.
• Wildfires (Sentinel-2 L2A, HLS-equivalent): Resampled to 30 m resolution to match HLS training data.
• Landslides (S2 L2A + DEM): Required co-registration of DEM and optical layers.

This modularity ensures domain-specific accuracy while maintaining backbone consistency.

Deployment

Dataset generation is driven by a simple query interface:

• Inputs: region coordinates, temporal window, hazard type
• Process: automatic data retrieval (via AWS registry), preprocessing, temporal sequence construction
• Outputs: segmentation masks of flood extents, burn scars, or unstable slopes

These outputs serve two purposes: immediate event detection and population of the historical hazard archive that fuels predictive modeling.

Scalability and Generalization

A core strength of the architecture is scalability. New decoders can be added without retraining the entire system, allowing growth to new hazards or related geospatial tasks.

The framework also supports generalization. By relying on consistent embeddings across modalities, it adapts from detection (mapping past events) to prediction (anticipating the risk of future ones). This flexibility positions TerraMind as a data-driven compass for navigating risk, adaptable to emerging climate-driven threats.

Current Stage and Future Integration

Much of the workflow is already automated. Pipelines for floods, wildfires, and landslides are operational, enabling event mapping. Temporal sequence generation is streamlined, users specify only parameters, while the system manages acquisition, preprocessing, and decoding.

The risk prediction component has been conceptually designed but not yet trained. Immediate next steps are:

1. Scaling hazard detection to broader regions to build a comprehensive training database.
2. Improving training data fidelity by prioritizing higher-resolution satellite imagery. For example, in the burn scars model, we initially relied on harmonized Landsat–Sentinel (HLS, 30 m) data because it provided the ground-truth labels, but future work will instead use native Sentinel-2 L2A imagery (10 m). By superposing Sentinel-2 inputs onto the HLS-derived ground truth, we retain validated labels while benefiting from higher resolution.
3. Leveraging TiM (Thinking-in-Modalities): Flood detection already benefited from TiM LULC features, and future work will further extend TiM integration to enhance wildfire and landslide models.
4. Training and validating predictive models using the detection archive as ground truth.

Through this staged approach, TerraMind already delivers actionable hazard detection while evolving toward predictive, interactive risk intelligence.

Contact email

[email protected]

Models

donia-metaplanet/TerraMind-Blue-Sky-Challenge

References

1. TerraMind: Large-Scale Generative Multimodality for Earth Observation
2. The Outcome of the 2022 Landslide4Sense Competition: Advanced Landslide Detection from Multi-Source Satellite Imagery
3. Mapping Global Floods with 10 Years of Satellite Radar Data
4. A global multimodal flood event dataset with heterogeneous text and multi-source remote sensing images
5. A high‐resolution global flood hazard model - PMC
6. Precipitation-triggered landslide prediction in nepal using machine learning and deep learning
7. Refined burned-area mapping protocol using Sentinel-2 data increases estimate of 2019 Indonesian burning
8. Forest Fire Burn Scar Mapping Based on Modified Image Super-Resolution Reconstruction via Sparse Representation
9. Space-time modeling of cascading hazards: Chaining wildfires, rainfall and landslide events through machine learning
10. Floods in central-eastern Europe - September 2024 | Copernicus EMS