When Satellite Imagery Goes Dark: New Tool Shows Damage in Iran and the Gulf

How it Works

The tool works by conducting a statistical test on Synthetic Aperture Radar (SAR) imagery captured by the Sentinel-1 satellite which is part of the Copernicus mission developed and operated by the European Space Agency. SAR sends pulses of microwaves at the earth’s surface and uses their echo to capture textural information about what it detects. 

The SAR data for the geographic area covered by the tool is put through the Pixel-Wise T-Test (PWTT) damage detection algorithm, which was also developed by Ollie Ballinger. It takes a reference period of one year’s worth of SAR imagery before the onset of the war and calculates a “normal” range within which 99% of the observations fall. It then conducts the same process for imagery in an inference period following the onset of the war, and compares it to the reference period. The core idea is that if a building has become damaged since the beginning of the war, then the “echo” (called backscatter) from that pixel will be consistently outside of the normal range of values for that particular area. Investigators can then further probe potential damage around this highlighted area.

The plot below shows how the process was applied to Gaza and several Syrian, Iraqi and Ukrainian cities. The bars represent the weekly total number of clashes in each place, sourced from the Armed Conflict Location Event (ACLED) dataset. The pre-war reference periods are shaded in blue, spanning one year before the onset of each conflict. The one month inference periods after the respective conflicts  began are shaded in orange. The blue and orange areas are what the tool compares. 

The plot below shows an area with a number of warehouses in Tehran’s southwest. Some of the buildings show clear damage in optical Sentinel-2 imagery (something that has to be accessed outside of the tool via the Copernicus Browser). 

Clicking on the map within the tool generates a chart displaying that pixel’s historical backscatter; the red dotted lines denote a range within which 99% of the pre-war backscatter values fall. In this example, we can see that from March 14 onwards, the backscatter values over this warehouse begin to consistently fall outside of their historical normal range. This could signal that damage has been detected in the area.

Two important aspects of this workflow are that it utilises free and fully open access satellite data, as opposed to commercial satellite services; the second is that it overcomes some key limitations of AI in this domain, the most serious of which is called overfitting. This is where a model trained in one area is deployed in a new unseen area, and fails to generalise. Because we’re only ever comparing each pixel against its own historical baseline, we don’t run into that problem. 

Accuracy

The PWTT has been published in a scientific journal after two years of review.  Its accuracy was  assessed using an original dataset of over two million building footprints labeled by the United Nations, spanning 30 cities across Gaza, Ukraine, Sudan, Syria, and Iraq. Despite being simple and lightweight, the algorithm has been recorded achieving building-level accuracy statistics (AUC=0.87 in the full sample) rivaling state of the art methods that use deep learning and high resolution imagery. The plot below compares building-level predictions from the PWTT against the UN damage annotations in Hostomel, Ukraine. True positives (PWTT and United Nations agree on damage) are shown in red, true negatives are shown in green, false positives in orange, and false negatives in purple. The graphic shows the accuracy of the tool, while also emphasising that further checks on what it highlights should be conducted to draw full conclusions.  

It is important to note that just because the tool may show a high probability of a building or buildings being damaged or destroyed, that doesn’t make it definite. 

It is best to check with any other available imagery — either open source photos and videos that’ve been geolocated by a group such as Geoconfirmed or Sentinel-2 as well as other commercial satellite imagery if it’s up-to-date for the area. At time of publication, Sentinel-2 satellite imagery still offers coverage over the area that the tool focuses on. Other commercial satellite imagery providers have limited their coverage.

What the tool excels at is highlighting and narrowing down areas so that further corroboration or further confirmation can be sought.

Testing the Tool

Using the Iran Conflict Damage Proxy Map, we can spot some of the larger areas of potential damage or destruction that have occurred since the Iran war started.

Starting from a zoomed-out view of Tehran, there are a few spots that appear with large clusters of high damage probability. Cross-referencing these locations with open source map data from platforms like OpenStreetMap or Wikimapia, we can start finding sites that would make for likely targets – such as military sites.

One example of a potentially damaged site visible in the tool is the Valiasr Barracks in central Tehran, which was struck in the first week of the war. By going to the Copernicus Browser and reviewing the area with optical Sentinel-2 imagery, we can see clear indications of damage at the barracks.