# ADRIZ -> INGV Etna -- Proof of Value & "How to verify us" **INTERNAL / PREVIEW until Mark approves a send. Nothing here has been sent to INGV or posted publicly.** Etna / Sicily context only. This document is the credibility backbone of the ADRIZ proof-of-value page (https://adr-etna-ingv.pages.dev/ -> "Proof of value" tab). Its purpose is simple: **an INGV academic should be able to reproduce every number on that page themselves, from public sources, with no access to anything of ours.** Each claim below names the exact public source, the exact method, and the honest caveat. - **Date compiled:** 2026-06-27 - **Machine-readable data behind the page:** `etna_dashboard/public/data/proof_of_value.json` (timeline + scoreboard + model proof + limits), `comparison.json` (the auto-grading), `ingv_truth.json` (362 INGV-OE weekly reports we ingested). - **Compute discipline:** this document and the page are light web + repo + dashboard work only. No DGX, no GPU. (The historical model/eval numbers it cites were produced earlier on RunPod pods, documented in their own reports.) --- ## 0. The public sources (everything is checkable without ADRIZ) | # | Source | Public URL | What you verify with it | |---|---|---|---| | S1 | **INGV-OE weekly bulletin via Smithsonian GVP**, Etna volcano **211060** | https://volcano.si.edu/volcano.cfm?vn=211060 | INGV-OE's own activity state / lava / ash / ACC for each week. GVP quotes "Sezione di Catania - Osservatorio Etneo (INGV)" verbatim in English, dated archive 2000->present. Navigate to the week label in any timeline row. | | S2 | **GVP Weekly Volcanic Activity Report (RSS)** | https://volcano.si.edu/news/WeeklyVolcanoRSS.xml | The live current-week feed (machine-readable) we ingest. | | S3 | **INGV-OE native weekly bulletin (PDF)** | https://www.ct.ingv.it/index.php/monitoraggio-e-sorveglianza/prodotti-del-monitoraggio/bollettini-settimanali-multidisciplinari | INGV's authoritative primary document (carries the quantitative SO2-flux + tremor-RMS tables; PDF/image, not auto-parsed). | | S4 | **INGV-OE VONA** (aviation notices) | https://www.ct.ingv.it/index.php/monitoraggio-e-sorveglianza/prodotti-del-monitoraggio/comunicati-vona | Per-event ash alerts + Aviation Colour Code + UTC timestamps. | | S5 | **NASA FIRMS** active-fire archive + map | https://firms.modaps.eosdis.nasa.gov/ (API: `https://firms.modaps.eosdis.nasa.gov/api/area/`) | Independent VIIRS/MODIS active-fire detections + FRP for any date/AOI. The third-party fire truth. | | S6 | **Element84 Earth Search STAC** (Sentinel-2 L2A) | https://earth-search.aws.element84.com/v1 (collection `sentinel-2-l2a`) | The S2 scenes behind our lava-extent (SWIR) and burn-scar (dNBR) products. Free, no login. | | S7 | **ASF / HyP3** (Sentinel-1 InSAR) | https://hyp3-api.asf.alaska.edu/ (Earthdata login) | The S1 `INSAR_GAMMA` products behind the deformation report. | | S8 | **Copernicus / CDSE** (Sentinel-3 SLSTR, Sentinel-5P) | https://dataspace.copernicus.eu/ | S5P TROPOMI SO2 and (recommended add) SLSTR MIR FRP. | | S9 | **NASA LP DAAC / Microsoft Planetary Computer** (ECOSTRESS, Landsat) | https://lpdaac.usgs.gov/ , https://planetarycomputer.microsoft.com/ | The independent thermal LST sensors in the fused-thermal product. | --- ## 1. Event timeline -- how to verify each row Every row pairs **INGV's own report** (link in column 2) with **what we independently produced**. Reproduce as follows. ### 1.1 2025-08-21 -- August 2025 effusion (verdict: added_info) - **INGV side (S1):** open GVP 211060, week **20-26 August 2025** -> INGV-OE reports effusive activity with lava + ash. - **ADRIZ side:** - *FIRMS (S5):* query FIRMS area API, VIIRS S-NPP + NOAA-20, bbox around 37.75 N 15.00 E, 2025-08-19..23 -> active fire on the edifice, FRP max ~94 MW. - *S2 lava-extent (S6):* pull S2 L2A tile **33SVB** for 2025-08-21; compute `NHI_SWIR=(B12-B11)/(B12+B11)`, active-lava core = `NHI_SWIR>0.20 AND B12>=4000`; summit window 14.94-15.06 E, 37.70-37.80 N -> 462 active-core px = **18.5 ha**, 424 SWIR-saturated. - Full method + artifacts: `reports/sentinel2_lava_burnscar_REPORT.md` (Product 1). - **Why it's "added_info":** the weekly bulletin narrative states *that* there was effusion; our product adds a **quantitative 20 m active-lava footprint** the narrative does not carry. ### 1.2 2025-02-11 -- February 2025 eruption (ACC Red) (verdict: added_info, honest SO2 caveat) - **INGV side (S1, S4):** GVP 211060 week **12-18 February 2025** -> effusive, lava + ash; VONA ACC raised to **Red** that week. - **ADRIZ side:** FIRMS on-edifice fire FRP max 43.9 MW (S5); S2 (2025-02-12) active-lava core 255 px = **10.2 ha**, 465 saturated (S6). - **Honest caveat (verify it):** our single S5P overpass that day showed **no** summit SO2 plume (-0.76 DU). This is a single-pass advection dropout, not a state error -- the rolling-window prior absorbs it. Reproduce from `hyperspectral_etna/s5p_per_scene.json` and S8. ### 1.3 2025-06-02 -- June 2025 paroxysm (verdict: MISSED -- honest) - **INGV side (S1):** GVP 211060 week **28 May - 3 June 2025** -> 2 June paroxysm (brief lava fountain). - **ADRIZ side:** S2 caught the cooling flow at **+2 days** (2025-06-04, 6 active-core px) and **missed it at +3 days** (2025-06-05, 0 active-core px -- surface cooled below the high-T SWIR threshold). Reproduce: S2 33SVB scenes 2025-06-04 and 2025-06-05, same NHI/B12 method (S6); see `reports/sentinel2_lava_burnscar_REPORT.md` (Product 1, the "one miss"). - **Why it's a miss:** a 5-day-revisit optical sensor catches an effusive/cooling flow for only ~2-3 days; a short fountain seen several days late is gone. Concrete proof S2 is a mapping layer, not an alert layer. ### 1.4 2024-08-15 -- August 2024 paroxysm (verdict: agreed) - **INGV side (S1):** GVP 211060 week **14-20 August 2024** -> paroxysm, lava + ash. - **ADRIZ side:** S5P SO2 summit plume **+5.08 DU** (2024-08-13, S8) AND FIRMS on-edifice fire FRP max 33.9 MW (S5) -- two independent positives in the same window. - **Honest note:** our per-overpass severity label read "strombolian" where the weekly peak was "paroxysm" -- a weekly-vs-daily granularity difference, not a sensing disagreement. ### 1.5 2023-11-20 -- the QA catch (verdict: added_info) - **INGV side (S1):** GVP 211060 week **22-28 November 2023** -> Strombolian, **lava overflowing the crater from 24 Nov**; paroxysm 1 Dec. - **ADRIZ side:** our internal activity-label table had mis-coded 2023-11-20 as *quiescent*. The S2 detector independently flagged **50 high-T + 33 SWIR-saturated px** at the summit -- i.e. it **caught our own labelling error** by agreeing with INGV's report. Reproduce: S2 33SVB 2023-11-20 (S6); cross-check against GVP week above (S1). See `sentinel2_lava_burnscar_REPORT.md` footnote on 2023-11-20. - **Why it matters:** demonstrates the detector is independent of (and can correct) our label set. ### 1.6 2026-06-14 -- ash emission / VONA (verdict: agreed + the latency finding) - **INGV side (S1, S4):** GVP 211060 week **11-17 June 2026** -> ash_emission; 14-Jun VONA raised ACC Yellow->Orange (diffuse ash). - **ADRIZ side:** our ingester classified the week `ash_emission` / ACC=`yellow`, matching the VONA. Reproduce: run `etna_dashboard/processor/ingest_ingv.py` -> `ingv_truth.json` record `2026-06-17`. - **The honest finding:** this is the **only** weekly Etna report in the last 120 days. Etna was quiet Feb->early-June 2026, so GVP/INGV issued no weekly report -- the activity-triggered cadence is itself the latency story (the weekly oracle goes silent during quiet stretches). ### 1.7 2025-07-22 -- Sicily interior wildfire (Caltanissetta) (verdict: wildfire) - **Fire truth (S5):** FIRMS active fire, 37.32 N 14.62 E, 2025-07-22, FRP max 63 MW. Verify on the FIRMS map: https://firms.modaps.eosdis.nasa.gov/map/#d:2025-07-22;@14.62,37.32,11z - **ADRIZ side:** S2 pre/post **dNBR** burn scar (S6): `NBR=(B8A-B12)/(B8A+B12)`, `dNBR=NBR_pre-NBR_post`; pre 2025-06-30, post 2025-08-09. Fire-pixel dNBR median **0.428** vs background **0.123**; **AUC(fire-vs-background) = 0.867 [0.842, 0.887]** (rank-based, bootstrap CI, FIRMS-co-located so it is a real scar, not seasonal ag change). One of **5/5** Sicily 2025 sites with a significant co-located scar. See `sentinel2_lava_burnscar_REPORT.md` (Product 2). --- ## 2. Continuous agreement scoreboard -- how the numbers are computed The scoreboard reads `comparison.json`, produced by `etna_dashboard/processor/build_comparison.py`. Method: each of our observation dates is aligned to the INGV **weekly** bulletin window that contains it (+-3 day slack). 33 of our dates carry a call; **19** fall inside an INGV weekly window and are graded. Proportions carry a **Wilson 95% CI**. | Dimension | Independent? | Result | n | 95% CI | How to reproduce | |---|---|---|---|---|---| | **Volcanic activity present** (our SO2-plume OR on-edifice FIRMS fire vs INGV non-quiescent week) | yes | **100% (19/19)** | 19 | **[83.2%, 100%]** | For each graded date, check INGV state (S1) is non-quiescent and at least one of {our S5P plume (S8), our FIRMS on-edifice fire (S5)} is positive. | | **SO2 plume vs INGV activity** (independent S5P) | yes | 60% (3/5) | 5 | [23.1%, 88.2%] | Our S5P summit-vs-bg enhancement >=1 DU (S8) vs INGV non-quiescent (S1). | | **FIRMS fire vs INGV thermal/effusive** | yes | 63.2% (12/19) | 19 | [41.0%, 80.9%] | Our FIRMS on-edifice fire (S5) vs INGV effusive/paroxysm/lava (S1). | | State within 1 severity level | **no (transcription)** | 47.4% (9/19) | 19 | [27.3%, 68.3%] | Our label is compiled *from* GVP/INGV -- measures weekly-vs-daily granularity, **not** sensing. Excluded from any accuracy claim. | **The recall-only caveat (must be read):** the 19 graded dates all fall in INGV **non-quiescent** weeks (effusive 6, strombolian 6, paroxysm 6, ash 1; **quiescent 0**). So "volcanic-activity-present 100%" is **recall only** -- the negative class (INGV-quiet weeks) is untested in this overlap, and we have **not** measured a false-positive rate against INGV-quiet weeks here. **Quote the lower CI bound (83.2%), not 100%.** The dashboard surfaces this as an explicit class-balance warning. Closing it is a data-access task (add INGV-quiescent weeks with a matched ADRIZ overpass), not an algorithm change. --- ## 3. Model proof -- volcanic false-alarm on INGV's OWN camera frames - **Number:** **9.7% volcanic false-alarm [4.5%, 19.5%], n=62** on the **143-frame REAL bulletin-confirmed INGV-OE camera held-out set** (visible held-out volcanic/false-situation frames). At operating conf 0.40-0.50 this drops to **4.8-6.5%** with no loss of wildfire recall (96.8% on the synthetic slice). - **Why it is defensible:** the held-out frames are **INGV's own camera frames, labelled from INGV's own bulletins**, split by date/session, **never trained on** (0 phash collisions vs training). Full method, ablations, and honest negatives: `reports/ingv_native_model_REPORT.md` (sections 4.1 and 4.1b) and the schema `reports/ingv_camera_monitoring_SPEC.md`. - **What it proves:** the model is **volcano-hardened** -- it learns lava/ash/degassing as their own classes so a flank wildfire is not confused with summit activity, and vice-versa. The wildfire-vs- volcanic discrimination INGV's own literature calls spectrally impossible is achieved by **geometry + off-vent independent corroboration**, not by spectrum alone. - **Live demonstration:** the system has been run on INGV's **own public live cameras** (garr.tv PeerTube `ingv_catania`, CC BY 4.0), correctly abstaining at night and labelling summit FIRMS hits VOLCANIC, not wildfire (`reports/live_camera_REPORT.md`). --- ## 4. Honest limits (the same panel as the page -- not buried) 1. **Where we miss.** FIRMS' satellite thermal floor does not catch summit-Strombolian intra-crater activity that INGV's ground/webcam sensors see (the 7 FIRMS-vs-thermal disagreements). S2 lava-extent misses a short paroxysm seen >2-3 days late (the Jun-2025 +3d miss). 2. **Recall-only volcanic metric.** 0 INGV-quiescent weeks in the graded overlap -> specificity untested; quote 83.2% (CI lower bound), not 100%. 3. **Single-site.** All volcanic evaluation is Etna (+Eolie cameras). Every rate is small-n (5-62); read the CI. 4. **Synthetic wildfire positives.** Real on-camera Etna wildfires are rare, so the camera model's wildfire positives are largely synthetic composites; only the real-only slice is the headline. 5. **Weekly-resolution oracle.** INGV's machine-readable product (GVP-mirrored weekly bulletin) is a weekly summary; it cannot resolve the sub-weekly timing our per-overpass FIRMS (~3-6 h) and S5P feeds do, so per-day vs weekly-window labels legitimately differ at episode boundaries. 6. **Image/PDF-locked INGV data.** Tremor-RMS and SO2-flux t/day series are published as plots/PDF, so we do not ingest those numbers; VONA per-event UTC and HOTSAT FRP have no stable public per-day endpoint we found. These are **data-access gaps, not algorithm gaps** -- and the natural place an INGV data agreement would unlock latency + magnitude grading we deliberately do not claim today. --- ## 5. Reproduce the whole page in one line of reasoning 1. Open the live page -> "Proof of value" tab. 2. For any volcanic row: click the INGV link (S1), read INGV-OE's own words for that week; then pull the same-date FIRMS (S5) / S2 (S6) / S5P (S8) yourself with the method in section 1. 3. For the scoreboard: re-run `build_comparison.py` (or read `comparison.json`) and check each graded date against S1 + S5/S8 as in section 2; apply a Wilson interval to the counts. 4. For the model number: evaluate against the public held-out camera set named in section 3. 5. Every honest miss in section 4 is reproducible the same way -- that's the point. **Bottom line:** the value is shown, not asserted. Where we agree with INGV we link their own report; where we add information (20 m lava footprint, dNBR severity, fused thermal, InSAR LOS, a QA catch) we name the public data; and where we miss (FIRMS summit-Strombolian floor, S2 short-paroxysm timing, recall-only coverage) we say so plainly, with the reproduction recipe attached.