Earth+: On-Board Satellite Imagery Compression Leveraging Historical Earth Observations

Due to limited downlink (satellite-to-ground) capacity, over 90% of the images captured by the earth-observation satellites are not downloaded to the ground. To overcome the downlink limitation, we present Earth+, a new on-board satellite imagery compression system that identifies and downloads only changed areas in each image compared to latest on-board reference images of the same location. The key of Earth+ is that it obtains latest on-board reference images by letting the ground stations upload images recently captured by all satellites in the constellation. To our best knowledge, Earth+ is the first system that leverages images across an entire satellite constellation to enable more images to be downloaded to the ground (by better satellite imagery compression). Our evaluation shows that to download images of the same area, Earth+ can reduce the downlink usage by 3.3× compared to state-of-the-art on-board image compression techniques without sacrificing imagery quality or using more resources (downlink, computation or storage).

由于下行链路（卫星到地面）容量有限，地球观测卫星拍摄的影像中超过90%未能被下载至地面。

为克服这一瓶颈，我们提出了 Earth+，一个全新的星上卫星影像压缩系统。该系统 通过与星上存储的同一区域的最新参考影像进行比对，仅识别并下载发生变化的区域。

Earth+ 的关键在于，它通过地面站上传星座中所有卫星近期捕获的影像，来获取最新的星上参考影像。

据我们所知，Earth+ 是 首个利用整个卫星星座的影像 来实现更优卫星影像压缩，从而使更多影像能够被下载至地面的系统。我们的评估表明，在下载同一区域影像时，与当前最先进的星上影像压缩技术相比，Earth+ 可将下行链路使用量降低3.3倍，且不牺牲影像质量，也无需占用更多资源（如下行链路、计算或存储资源）。

Introduction

Fresh and high-quality satellite imagery is key to many applications, from digital agriculture [29, 56, 72, 73], environmental monitoring [6, 46, 68, 84, 85], to automatic road detection [31, 60, 86], and many more. As a result, large constellations of Low-Earth-Orbit (LEO) earth observation satellites have been deployed [48, 74, 82] to capture high-quality imagery for any location multiple times a day [48, 74, 82].

However, most satellite imagery data captured by these satellites are currently not received on the ground due to the limited downlink (satellite-to-ground) capacity. According to a recent estimate, only 2% of the total image data observed by each satellite can be downloaded to the ground [48]. Some mission-specific satellites handle the downlink-capacity limitation by filtering images onboard the satellite [49, 82] to focus only on mission-specific areas prepaid by the customer. However, this approach is not sufficient for general-purpose satellite constellations (e.g., Sentinel-2 [51], Doves [74]), whose goal is to capture and download satellite imagery over wide geographical regions to serve more applications.

新鲜、高质量的卫星影像对于数字农业、环境监测、道路自动检测等诸多应用至关重要。因此，大规模的低地球轨道（LEO）地球观测卫星星座已被部署，以便每天多次捕捉任何地点的高质量影像。

然而，由于下行链路（卫星到地面）容量有限，这些卫星捕获的大部分影像数据目前都未能被传回地面。根据最近的一项估算，每颗卫星观测到的总影像数据中，仅有2%能够被下载到地面。一些特定任务的卫星通过在轨影像筛选来应对下行链路容量的限制，仅关注客户预付费的任务特定区域。但这种方法对于通用型卫星星座（如Sentinel-2、Doves）来说并不足够，因为它们的目标是捕捉并下载广阔地理区域的卫星影像以服务于更多应用。

This paper aims to improve onboard compression for satellite imagery. 1 . We are inspired by the observation that the terrestrial content changes slowly between two consecutive satellite visits at the same location [78, 88]. Thus, to compress a new image, we can compare it with a recent image of the same region, called a reference image, to detect the geographic tiles (defined in §3) within the region that has changed and then only compress and download the changed tiles. Our measurement on Planet dataset [74] shows that without the interference of clouds, only 20% of the tiles in each image have changed in the previous five days on average, which ideally can save downlink usage by up-to 5× (§3).

Yet, realizing the reference-based encoding for onboard imagery compression can be challenging because the reference image should be as fresh and contain as little cloud as possible (§3). Typically, the last cloud-free image captured by the same satellite [74] can be over 50 days old on average (§3). With such a large time gap, the reference image and the new image may have substantial differences (more than 50% of the tiles will have significant changes as shown in §3), making reference-based encoding less effective.

We present Earth+, a constellation-wide reference-based encoding system, where the reference images can be selected from historical images of any satellites in the constellation. By broadening the set of potential reference images, Earth+ increases the probability of obtaining fresh and cloud-free reference images. For example, with images from an entire constellation [74], cloud-free images can be obtained every 4.21 days on average, instead of every 50 days with one satellite (§4.1).

本文旨在改进星上卫星影像压缩技术。我们的研究 受到一个观察的启发：地表内容在两次连续的卫星过境之间变化缓慢。因此，要压缩一幅新影像，我们可以将其与该区域近期的影像（称为参考影像）进行比较，检测出区域内已发生变化的地理图块（定义见§3），然后仅压缩和下载这些变化的图块。

我们对Planet数据集的测量显示，在没有云层干扰的情况下，平均每幅影像中仅有20%的图块在过去五天内发生了变化，这在理想情况下可将下行链路使用量节省高达5倍（§3）。

然而，在星上实现基于参考的编码颇具挑战，因为 参考影像应尽可能新鲜且无云（§3）。通常情况下，由 同一颗卫星 拍摄的上一张无云影像平均可能是50多天前的（§3）。 如此长的时间间隔，可能导致参考影像与新影像之间存在巨大差异（如§3中所示，超过50%的图块会发生显著变化），这使得基于参考的编码效果大打折扣。

我们提出了 Earth+，一个星座范围的基于参考的编码系统，其中 参考影像可以从星座中任意卫星的历史影像中选取。通过扩大潜在参考影像的集合，Earth+ 增加了获取新鲜且无云参考影像的概率。 例如，借助整个星座的影像，平均每4.21天就能获取一张无云影像，而单颗卫星则需要50天（§4.1）。

Earth+ then leverages the existing uplinks (ground-tosatellite) to upload reference images selected from the whole constellation to the target satellite, as illustrated in Figure 1. (§4.2 will discuss why Earth+ does not leverage inter-satellite links instead.) The key challenge of this design is to handle limited uplink capacity of existing earth observation satellites (e.g., 250kbps [50]).

Earth+ 利用现有的上行链路（地面到卫星）将从整个星座中选出的参考影像上传到目标卫星，如图1所示。（§4.2将讨论为何Earth+不采用星间链路。）该设计的关键挑战在于应对现有地球观测卫星有限的上行链路容量（例如250kbps）。

We present two techniques (§4.3) to reduce the uplink usage of Earth+ without sacrificing the savings on the downlink.

First, Earth+ uploads reference images at a low resolution while still allowing the satellites to detect the most changed tiles (§4.3). The rationale is that low-resolution images are sufficient to decide which tiles have changed, which is easier than quantifying how much each pixel in the tile has changed.

Second, Earth+ does not need to store those unchanged tiles when capturing new imagery, which frees up the storage space. We utilize this freed storage space to cache reference images locally on-board, which allows Earth+ to further reduce the uplink usage by only uploading tiles that have changed relative to the on-board cached reference images.

Besides the two aforementioned techniques, our implementation of Earth+ (§5) also entails techniques to handle satellite-specific issues, including cloud detection, on-board computation constraints, handling different bands of satellite imagery, and bandwidth variations.

我们提出了两种技术（§4.3）来降低Earth+的上行链路使用量，同时不影响其在下行链路上的节省效果。

首先，Earth+ 以低分辨率上传参考影像，同时仍能让卫星检测出变化最显著的图块（§4.3）。其原理是，判断哪些图块发生了变化，比量化图块中每个像素变化了多少要容易得多， 因此低分辨率影像已足够胜任。

其次，Earth+ 在捕获新影像时无需存储那些未发生变化的图块，从而释放了存储空间。我们利用此释放的存储空间在星上本地缓存参考影像，这使得Earth+ 只需上传相对于星上缓存的参考影像发生变化的图块 ，从而进一步减少了上行链路的使用。

除了上述两种技术，我们实现的Earth+系统（§5）还包含了一些技术来处理卫星特有的问题，包括云层检测、星上计算资源限制、处理不同波段的卫星影像以及带宽变化。

To put Earth+’s contribution into perspective, the idea of sharing imagery across satellites in the constellation is not new (e.g., multipath satellite imagery delivery [7, 66]). Earth+, however, is the first that leverages constellation-wide imagery sharing to enable more images to be downloaded to the ground (by better satellite imagery compression).

为了全面展示Earth+的贡献，需要指出的是，跨卫星共享影像的想法并不新鲜（例如，多路径卫星影像传输）。然而，Earth+ 是首个利用星座范围的影像共享来实现更优卫星影像压缩，从而使更多影像能够被下载到地面的系统。

We evaluate Earth+ on real-world satellite specifications (uplink and storage capacities) of the Doves constellation [25] from Planet Labs. We test Earth+’s compression efficiency on two datasets. The first dataset is collected from Sentinel-2 dataset [51], with 3.6 TB data covering 110 thousand km2 from Washington State. We use this dataset to test Earth+ under a wide range of contents (e.g., mountains, forests, and cities), seasons, and under multiple imagery bands (13 bands in total). Since Sentinel-2 only contains two satellites, we further test Earth+’s performance using the Planet dataset [74], from which we obtain images from 40 satellites for one sampled location (due to the download limit) of 64 km 2 in the U.S. for three months. Our evaluation shows that:

• Compared to the state-of-the-art onboard compression schemes, Earth+ reduces the downlink bandwidth usage by 1.3-3.3× without hurting the imagery quality on all bands. This can reduce the reaction delays of ground applications (e.g., forest-fire alerts) by upto 3×.

• These improvements are achieved without using more uplink bandwidth than currently available or more compute or storage resources than the baselines.

• With more satellites in a constellation, Earth+ can further reduce the amount of downlink bandwidth usage. That said, Earth+’s reference-based encoding is not a good fit for applications that require lossless satellite imagery (§8).

我们在Planet Labs的Doves星座的真实世界卫星规格（上行链路和存储容量）上评估了Earth+。我们在两个数据集上测试了Earth+的压缩效率。第一个数据集来自Sentinel-2，包含3.6 TB的数据，覆盖了华盛顿州11万平方公里的区域。我们使用该数据集在广泛的内容类型（如山脉、森林和城市）、季节以及多个影像波段（共13个）下测试Earth+。由于Sentinel-2仅包含两颗卫星，我们进一步使用Planet数据集测试了Earth+的性能，从中我们获取了美国一个采样地点（64平方公里，受下载限制）在三个月内由40颗卫星拍摄的影像。我们的评估表明：

• 与当前最先进的星上压缩方案相比，Earth+ 在所有波段上将下行链路带宽使用量减少了1.3至3.3倍，且不损害影像质量。这可将地面应用（如森林火灾警报）的响应延迟最多减少3倍
• 这些改进是在不超出当前可用上行链路带宽，也不比基准方案使用更多计算或存储资源的情况下实现的
• 随着星座中卫星数量的增加，Earth+可以进一步减少下行链路带宽的使用

尽管如此，Earth+的基于参考的编码方案不适用于需要无损卫星影像的应用（§8）

Motivation

We start with the background on satellite imagery and earth observation satellite constellations.

我们首先介绍卫星图像和地球观测卫星星座的背景。

2.1 Background

Many applications can benefit from frequently updated (e.g., daily) and high-resolution satellite imagery. For example, precision agriculture ideally needs daily access to satellite imagery with each pixel corresponding to a 5m × 5m area on Earth [14, 30] to help timely decisions on the distribution of fertilizers, pesticides, and water. Also, wildfire monitoring requires the imagery to be updated frequently with sufficient resolution to promptly detect and respond to fire outbreaks, mitigating potential damage [82].

To provide fresh, high-resolution satellite imagery, many LEO satellites (e.g., >100 satellites [74]) are deployed to form satellite constellations. Figure 2 shows an illustrative example of a LEO satellite constellation, where multiple satellites are located in a sun-synchronous orbit 2 and these satellites can potentially stream data to the ground when they are close enough to one of the ground stations (we only plot one ground station in the figure for simplicity).

We characterize two features of such LEO satellite constellations:

• High-resolution imagery: LEO satellites are close to the ground (due to their low earth orbits) and can capture imagery with low ground-sampling distance (GSD for short, lower GSD means higher resolution).

• Frequent revisit: With a large number of satellites, any location on the earth’s surface will be frequently revisited (e.g., daily [74]), while a single satellite can only revisit one location once every ten days [36].

Note that in the following text, we denote the ground as the ground stations that the constellation can potentially contact and the computation and networking infrastructures around these ground stations.

许多应用都能受益于频繁更新（例如，每日更新）的高分辨率卫星图像。例如，精准农业理想情况下需要每日获取像素对应地面 \(5m \times 5m\) 区域的卫星图像 [14, 30]，以帮助就肥料、杀虫剂和水的分配做出及时决策。此外，野火监测也需要频繁更新具有足够分辨率的图像，以迅速发现并响应火情爆发，从而减轻潜在损失 [82]。

为了提供新鲜的高分辨率卫星图像，许多低地球轨道（LEO）卫星（例如，超过100颗卫星 [74]）被部署以组成卫星星座。图2展示了一个LEO卫星星座的示意图，其中多颗卫星位于太阳同步轨道上。当这些卫星与某个地面站足够接近时，它们便可能向地面传输数据（为简化起见，图中仅展示一个地面站）。

我们概括此类LEO卫星星座的两个特点：

• 高分辨率图像：LEO卫星因其低地球轨道而接近地面，能够捕获具有较低地面采样距离（GSD）的图像（GSD越低意味着分辨率越高）。
• 高频重访：拥有大量卫星使得地球表面的任何位置都能被频繁重访（例如，每日 [74]），而单颗卫星重访同一位置可能需要十天 [36]。

请注意，在下文中，我们将“地面”指代星座可以接触的地面站，以及围绕这些地面站的计算和网络基础设施。

2.2 Downlink bottleneck and our objective

Downlink capacity gap: Despite more images being captured by the satellites, only a small fraction of data are downloaded to the ground due to the limited capacity of the downlink (satellite-to-ground) [48, 66, 82]. Specifically, we refer to downlink bandwidth as the average download speed from satellites to the ground during each ground contact. The exact gap between the downlink capacity and the imagery data varies with the constellation, and a recent study shows only about 2% of the images captured by satellites are actually downloaded to the ground [48].

Further, the downlink demand is constantly growing, with higher resolution (e.g., a GSD of 0.5m [32]) and more bands found to be useful (e.g., vegetarian red edge band and water vapor band [33]). In contrast, the downlink grows slowly due to the long deployment cycle of satellites. These trends suggest that the gap between the demand for downlink bandwidth and its actual capacity will likely persist if not increase.

下行链路容量差距： 尽管卫星捕获了更多图像，但由于下行链路（卫星到地面）的容量有限，只有一小部分数据被下载到地面 [48, 66, 82]。具体而言，我们将下行链路带宽定义为每次地面接触期间从卫星到地面的平均下载速度。下行链路容量与图像数据之间的确切差距因星座而异，但最近一项研究表明，卫星捕获的图像中实际只有约2%被下载到地面 [48]。

此外，随着更高分辨率（例如，0.5m的GSD [32]）和更多波段（例如，植被红边波段和水蒸气波段 [33]）被发现具有应用价值，下行链路的需求在持续增长。相比之下，由于卫星部署周期长，下行链路的增长缓慢。这些趋势表明，如果现状不改变，下行链路带宽的需求与其现有容量之间的差距可能会持续存在甚至扩大。

Optimization objective: We aim to address the downlink bottleneck of satellite constellations by better satellite image compression. More specifically, we aim to use much less bandwidth to download the same amount of satellite imagery, measured in the number of photoed locations and frequencies, without compromising image quality. To measure the quality of the downloaded images, we use Peak Signal-to-Noise Ratio (PSNR for short), which aligns with satellite imagery compression literature [52, 55, 57, 80].

优化目标： 我们旨在通过更优的卫星图像压缩方法来解决卫星星座的下行链路瓶颈问题。更具体地说，我们的目标是在不损害图像质量的前提下，使用远低于以往的带宽下载同等数量（以拍摄地点和频率衡量）的卫星图像。为了衡量下载图像的质量，我们使用峰值信噪比（PSNR），这与卫星图像压缩领域的文献 [52, 55, 57, 80] 保持一致。

On-board constraints: While optimizing for the image quality and reducing the downlink consumption, we stick to real-world on-board storage, computation, and uplink constraints. We describe the real-world satellite specification that we used for our evaluation in §6.

星上约束： 在优化图像质量和减少下行链路消耗的同时，我们严格遵守现实世界中的星上存储、计算和上行链路的约束。我们将在第6节中描述用于评估的真实卫星规格。

2.3 Existing solutions

There are several approaches to addressing the downlink bandwidth bottleneck.

Upgrading infrastructures: The first is to physically increase the total downlink capacity of the satellite constellation by upgrading the infrastructure (e.g., building more ground stations [74] or adding more satellites [48, 51, 74]). The costs of such infrastructure changes can be prohibitive, and they can be slow. For example, it takes tens of millions of dollars to build and send just one single satellite [11]. On-board filtering: An alternative is to filter the imagery onboard the satellite [48, 49, 82]. For the mission-specific constellations that focus on specific regions, this approach can filter out most of the imagery. For instance, the Biomass mission targets forest areas to monitor forest coverage changes [3], while the IceBridge mission observes polar ice to gauge climate change impacts [1]. However, they must exclude data useful for other applications. For example, the Biomass mission omits about 91% of the Earth’s surface [10, 13], such as city areas (which are useful for smart city applications) and agriculture areas (useful for digital agriculture).

In-space application processing: Instead of downloading the imagery to the ground, a wide range of systems process the application onboard the satellite and stream the application results back to the ground [2, 15, 16]. However, this approach cannot support many applications due to limited on-board compute, while downloading imagery to the ground allows all applications to perform analytics based on downloaded imagery.

Inter-satellite link for multi-path imagery delivery: Boosted by coherent optical communication [59, 77, 87], the inter-satellite link capacity is quickly growing and allows multipath satellite imagery delivery that can significantly reduce imagery delivery latency [61, 66]. However, this approach does not increase the total downlink capacity of the satellite constellation, or reduce the total amount of imagery data that need to be downloaded, so it is still bottlenecked by limited downlink capacity.

On-board satellite imagery compression: This work focuses on onboard imagery compression, which is complementary to the first three approaches. Existing solutions include augmenting single-image codecs [37–39, 58, 67, 75, 76, 93] and developing more expensive neural-based codecs such as autoencoders [40, 41, 47, 95, 96]. However, these techniques focus on compressing single imagery from a single satellite, so they fall short in leveraging the redundancies between images for higher compression efficiency.

解决下行链路带宽瓶颈的方法有多种。

升级基础设施： 第一种方法是通过 升级基础设施（例如，建设更多地面站 [74] 或增加更多卫星 [48, 51, 74]）来物理上增加卫星星座的总下行链路容量。 这类基础设施改造的成本可能高得令人望而却步，且实施缓慢。例如，建造并发射一颗卫星就需要数千万美元 [11]。

星上筛选： 另一种方法是 在卫星上对图像进行筛选 [48, 49, 82]。对于专注于特定区域的专用任务星座，这种方法可以过滤掉大部分图像。例如，Biomass任务旨在监测森林覆盖变化，因此主要关注森林区域 [3]，而IceBridge任务则观测极地冰层以评估气候变化的影响 [1]。然而，这些任务必须排除对其他应用有价值的数据。例如，Biomass任务忽略了地球表面约91%的区域 [10, 13]，如城市区域（对智慧城市应用有价值）和农业区域（对数字农业有价值）。

在轨应用处理： 一些系统不在地面下载图像，而是在 卫星上直接处理应用，并将应用结果传回地面 [2, 15, 16]。然而，由于星上计算能力有限，这种方法无法支持许多应用，而将图像下载到地面则允许所有应用基于下载的图像进行分析。

利用星间链路实现多路径图像传输： 在相干光通信技术 [59, 77, 87] 的推动下，星间链路容量正在迅速增长，这使得多路径卫星图像传输成为可能，从而显著减少图像传输延迟 [61, 66]。然而，这种方法并未增加卫星星座的总下行链路容量，也未减少需要下载的图像数据总量，因此仍然受到有限下行链路容量的制约。

星上卫星图像压缩： 本研究专注于星上图像压缩，这一方法与前三种方法是互补的。现有解决方案包括增强单图像编解码器 [37–39, 58, 67, 75, 76, 93]，以及开发成本更高的基于神经网络的编解码器，如自编码器 [40, 41, 47, 95, 96]。 然而，这些技术专注于压缩来自单颗卫星的单张图像，因此未能利用图像间的冗余来获得更高的压缩效率。

Reference-based encoding

Next, we introduce reference-based encoding, a seemingly promising idea that leverages a reference image to pinpoint and download only regions that have recently changed. As we will see, directly applying this approach to a satellite does not work well as images locally available to each satellite may not be recent enough or contain too much cloud to realize the benefit of reference-based encoding.

接下来，我们介绍基于参考的编码 (reference-based encoding) —— 一个看似很有前景的方法，它利用参考图像来精确定位并仅下载近期发生变化的区域。然而，我们将看到， 直接将此方法应用于卫星效果不佳，因为每颗卫星本地可用的图像可能不够新，或者云层覆盖过多， 从而无法实现基于参考的编码的优势。

Background on reference-based encoding: Referencebased encoding is commonly used to compress a sequence of images whose content changes slowly and gradually with respect to time[42, 71, 78, 81, 88, 89], such as video streams. Existing reference-based encoding systems (e.g., video codecs [42, 71, 81, 89]) typically select some of the images as the reference and encode the remaining images by encoding their difference concerning the reference images. As existing codecs encode the images at the granularity of tiles (a tile is a block of pixels, where we use a 64×64 pixel block as a tile by default), and the difference is separately calculated per tile.

Since the satellite imagery captured for the same location also changes slowly over time (as shown in prior work [78, 88]), there is some recent work to apply reference-based encoding in onboard satellite imagery compression [78, 88]. Given a new image, it compares the image with a reference image of the location from the past and pinpoints the changed tiles with a pixel difference greater than the threshold compared to the reference. It then encodes those changed tiles and downloads the tiles in their entirety. 3 Our work follows this approach when encoding changed tiles (§5).

基于参考的编码背景： 基于参考的编码通常用于压缩内容随时间缓慢渐变的图像序列 [42, 71, 78, 81, 88, 89]，例如视频流。现有的基于参考的编码系统（如视频编解码器 [42, 71, 81, 89]）通常选择部分图像作为参考，并通过编码其余图像与参考图像之间的差异来对它们进行压缩。由于现有编解码器以图块（tile，一个像素块，我们默认使用 \(64 \times 64\) 像素块作为一个图块）为粒度对图像进行编码，因此差异是按图块单独计算的。

由于为同一地点捕获的卫星图像随时间变化也较为缓慢（如先前工作 [78, 88] 所示），近期已有一些研究将 基于参考的编码应用于星上卫星图像压缩 [78, 88]:

1. 给定一张新图像，该方法会将其与该地点过去的一张参考图像进行比较
2. 并找出与参考图像相比像素差异大于阈值的变化图块
3. 然后，它会对这些变化的图块进行编码，并完整下载这些图块

我们的工作在编码变化图块时也遵循此方法（第5节）

Reference images need to be fresh: While reference-based encoding seems to be a good fit for imagery compression, it is only effective if the age of the reference image—the time gap between the reference image and the currently observed image—is as low as possible. Reference image with high age leads to more changed areas in the currently observed image, which must be downloaded to the ground. Figure 3 provides an illustrative example, where the amount of changes need to be downloaded at Day 30, if using high-age reference images from (Day 1), will be much more compared to using low-age reference image (Day 27). To make it more concrete, we use three months of cloud-free (explained shortly) images from the Planet dataset [51] on one randomly sampled location in the U.S. Here, we say a tile has changed if it has an average pixel differences greater than 0.01 after aligning the illumination (§5). 4 Figure 4 shows a steady increase in the percentage of changed areas with the age of the reference image: the percentage of changed tiles will increase by 3× if increasing the age of the reference image from 10 days to 50 days.

参考图像需要具有时效性： 尽管基于参考的编码看起来非常适合图像压缩，但它仅在参考图像的时效性 —— 即参考图像与当前观测图像之间的时间间隔 —— 尽可能短的情况下才有效。

时效性差的参考图像会导致当前观测图像中出现更多变化区域，而这些区域都必须被下载到地面。

图3提供了一个说明性示例：如果在第30天使用时效性差的参考图像（来自第1天），需要下载的变化量将远多于使用时效性好的参考图像（来自第27天）。为更具体地说明，我们使用了Planet数据集 [51] 中美国一个随机采样地点三个月的无云（下文将解释）图像。

在这里，我们定义，如果在对齐光照后（第5节），一个图块的平均像素差异大于0.01，则认为该图块发生了变化。

图4显示，变化区域的百分比随参考图像时效性的增加而稳步上升：如果将参考图像的时效性从10天增加到50天，变化图块的百分比将增加3倍。

Reference images should be cloud-free: If some tiles in the reference image are covered by clouds, they are not useful as a reference to detect changes. As a consequence, the corresponding tiles in the current image can only be deemed as changed and downloaded to the ground. This greatly compromises the benefit of reference-based encoding.

参考图像应为无云图像： 如果参考图像中的某些图块被云层覆盖，它们就无法作为有效的参考来检测变化。因此，当前图像中的相应图块只能被视为已变化并下载到地面。这极大地削弱了基于参考的编码所带来的优势。

Why reference-based encoding is challenging? In practice, however, there may not always exist a reference in the satellite’s history images that is both fresh and covered by little cloud. For example, existing work [78, 88] stores a fixed reference image on-board, which will get older over time and make most of the areas being counted as changed and downloaded to the ground, negating the benefit of referencebased encoding. Moreover, even if a satellite were able to choose the reference image from all of its historical images, the most recent reference image with less than 1% cloud coverage would still be tens of days old. For instance, Figure 5 shows the age distribution of the closest reference images that are covered by less than 1% cloud if the satellite chooses the reference image by itself (i.e., the “Satellite-local” curve in the figure). We note that the age of the most recent cloudfree reference image is 51 days on average. The reason for the high ages of recent cloud-free images is two-fold:

• A single satellite revisits the same location at a low frequency (once every 10-15 days [36]). This is because LEO satellites can only capture a small area on Earth at a time (since their size is small [82] and they are close to Earth), necessitating extended periods to complete a full scan of the Earth before revisiting the same locations.

• Since, on average, 2/3 of the earth is covered by clouds [12], so even if the most recent image of the same location is ten days old, it may likely be (partly) covered by cloud and are not ideal choice for reference images.

为何基于参考的编码充满挑战？

然而在实践中，卫星的历史图像中可能并不总是存在既具时效性又少云的参考图像。例如，现有工作 [78, 88] 在星上存储固定的参考图像，这张图像会随着时间推移而老化，导致大部分区域被计为变化区域并下载到地面，从而抵消了基于参考的编码带来的好处。

此外，即便一颗卫星能够从其所有历史图像中选择参考图像，云层覆盖率低于1%的最新参考图像的时效性仍可能达到数十天。例如，图5展示了 当卫星仅从自身历史数据中选择参考图像 时（即图中的“Satellite-local”曲线），云层覆盖率低于1%的最近参考图像的时效性分布。

我们注意到，最新的无云参考图像的平均时效性为51天。近期无云图像时效性差的原因有两方面：

• 单颗卫星对同一地点的重访频率低（每10-15天一次 [36]）
  • 这是因为LEO卫星一次只能捕获地球上的一个小区域（因其体积小 [82] 且靠近地球），因此需要很长时间才能完成对地球的全面扫描，然后才能重访相同地点
• 由于地球平均有三分之二的面积被云层覆盖 [12]
  • 因此即使同一地点的最新图像是十天前拍摄的，它也很有可能（部分）被云层覆盖，从而不是理想的参考图像选择

Earth+: Constellation-wide Reference-based encoding

To improve onboard satellite imagery compression, we present Earth+, a reference-based encoding system that obtains fresh and cloud-free reference images from images captured by any satellites in the whole constellation, rather than the history images of the same satellite. This section introduces the idea of constellation-wide reference sharing (§4.1) and an overview of Earth+ (§4.2). We then present the design of Earth+ that makes constellation-wide reference-based encoding practical (§4.3).

为了改进星上卫星图像的压缩，我们提出了 Earth+，一个基于参考的编码系统。该系统从整个星座中任意卫星捕获的图像里获取具有时效性的无云参考图像，而非仅仅依赖同一卫星的历史图像。本节将介绍星座级参考共享的思想（§4.1）和Earth+的概览（§4.2），然后阐述我们如何设计Earth+以使星座级基于参考的编码在实践中可行（§4.3）。

4.1 Constellation-wide reference selection

Compared to the prior work, which only refers to local images observed by the same satellite, Earth+ augments the set of reference images that reference-based encoding can choose from and thus potentially reduces the age of reference images, leading to fewer changes to be downloaded to the ground.

To illustrate the benefits and challenges of Earth+, we contrast two designs.

• Satellite-local reference: Pick the latest cloud-free image observed by the same satellite as the reference image.

• Constellation-wide reference: Pick the latest cloud-free image observed by any satellite in the whole constellation as the reference image.

Note that the latter is not practical because it needs a large amount of bandwidth to share the reference images, a challenge we will tackle soon in §4.3.

Figure 6 gives an illustrative example of this contrast with a constellation of three satellites (in different colors). The goal is to compress images taken by these satellites for the same location. To simplify the discussion, all images in this example are cloud-free. Each satellite takes a cloud-free image every 30 days, so the satellite-local reference (Figure 6(b)) will be 30 days old. Consequently, in the last three images (Day 31, 41, and 51), 45%-65% of tiles are deemed as changed and need to be downloaded.

In contrast, with constellation-wide reference Figure 6(c)), since the reference image can be from any satellite, the freshest reference is only ten days old rather than 30 days. As a result, two of the three last images do not have any changed tiles and one has only 45% changed tiles, i.e., only 15% are changed tiles on average. In short, the ability to pick reference images from any satellite in the constellation reduces the age of reference images by 3× (30 days to 10 days) compared to the satellite-local design, and this reduces the changed tiles to download by 3.6× (55% area to 15% area).

与先前仅参考同一卫星观测的本地图像的工作相比，Earth+ 扩展了基于参考的编码可选择的参考图像集，从而可能降低参考图像的“年龄”（时效性），进而减少需要下载到地面的变化量。

为阐明 Earth+ 的优势与挑战，我们对比了两种设计。

• 卫星本地参考 (Satellite-local reference)：选择由 同一卫星观测 到的最新的无云图像作为参考图像
• 星座级参考 (Constellation-wide reference)：选择由 整个星座中任意卫星观测 到的最新的无云图像作为参考图像

值得注意的是，后一种方案并不完全实用，因为它需要大量带宽来共享参考图像，这是我们将在 §4.3 中着手解决的一个挑战。

图6用一个包含三颗卫星（以不同颜色表示）的星座为例，说明了这两种设计的对比。目标是压缩这些卫星在同一地点拍摄的图像。为简化讨论，本例中的所有图像均为无云图像。每颗卫星每30天拍摄一张无云图像，因此，卫星本地参考（图6(b)）的“年龄”将是30天。其结果是，在最后三张图像（第31、41和51天）中，有45%-65%的图块被视为已发生变化，需要被下载:

相比之下，采用星座级参考（图6(c)），由于参考图像可以来自任何卫星，最新的参考图像“年龄”仅为10天，而非30天。结果，在最后三张图像中，有两张没有任何变化的图块，另一张也仅有45%的图块发生变化，即平均只有15%的图块发生了变化。简而言之，与卫星本地设计相比，从星座中任意卫星选取参考图像的能力将参考图像的“年龄”缩短了3倍（从30天降至10天），并将需要下载的变化图块减少了3.6倍（从55%的区域面积降至15%）。

4.2 Earth+ workflow

Earth+ is a concrete design of constellation-wide reference-based encoding. It answers two basic questions: (1) which reference images should be shared between different satellites, and (2) how to share these reference images using the existing infrastructure.

To answer the first question, Earth+ reuses the images downloaded to the ground from all satellites and selectively uploads these images as reference images to the satellites. Figure 1(b) illustrates this workflow.

• During previous ground contact, the ground station uploads latest cloud-free images (that can come from any satellite in the constellation) as reference images for the locations that the satellite will fly by before the next ground contact 6 .

• When passing over a location, the satellite captures the imagery, removes clouds, detects changes using the reference images, and encodes the changes.

• During the next ground contact, the satellite downloads the encoded changes to the ground.

Compared to the workflow of traditional satellite imagery processing pipelines, which capture images and download them to the ground (as depicted in Figure 1(a)), Earth+uploads the reference images from the ground to the satellite. We rely on ground stations as an “overlay” point to share images downloaded from each satellite with other satellites. The rationale is two-fold. First, the ground stations can access any historical image observed by the whole constellation, allowing Earth+ to select reference images constellationwide. Second, the ground station has sufficient computing resources to more accurately detect clouds and upload only cloud-free images to satellites as the reference (§3).

A seemingly promising alternative to enable constellationwide reference is to let satellites share data via inter-satellite links (ISL). Earth+ does not use ISL because it is currently not available for earth observation satellites [82]. Further, the scale of existing earth observation constellations (less than 200 satellites) is insufficient to guarantee a stable ISL connection between any two satellites, as providing such a guarantee typically requires thousands of satellites (e.g., Starlink [69]).

Earth+ 是星座级参考编码的一个具体设计。它回答了两个基本问题:

1. 应该在不同卫星间共享哪些参考图像
2. 如何利用现有基础设施共享这些图像

为回答第一个问题，Earth+ 重复利用从所有卫星下载到地面的图像，并选择性地将这些图像作为参考图像上传回卫星。图1(b)阐释了这一工作流程。

• 在前一次与地面站接触期间，地面站将最新的无云图像（可来自星座中的任何卫星）作为参考图像上传至卫星，这些参考图像对应卫星在下一次接触地面站之前将飞越的地点
• 当飞越某一地点时，卫星捕获影像，去除云层，使用参考图像检测变化，并对变化进行编码
• 在下一次与地面站接触期间，卫星将编码后的变化数据下载到地面

与传统的卫星影像处理流程（如图1(a)所示，捕获图像并将其下载到地面）相比，Earth+ 增加了将参考图像从地面上传到卫星的步骤。

我们依赖地面站作为“覆盖点”（overlay point），将从每颗卫星下载的图像共享给其他卫星。其基本原理有两点:

1. 地面站可以访问整个星座观测到的任何历史图像 ，这使得 Earth+ 能够实现星座级的参考图像选择
2. 地面站拥有充足的计算资源，可以更准确地检测云层，并只将无云图像上传给卫星作为参考（§3）

一个看似有前景的替代方案是让卫星通过星间链路（ISL）共享数据。Earth+ 没有使用星间链路，因为目前它在地球观测卫星上尚不可用[82]。此外，现有地球观测星座的规模（少于200颗卫星）不足以保证任意两颗卫星之间稳定的星间链路连接，因为提供这种保证通常需要数千颗卫星（例如，星链（Starlink）[69]）。

4.3 Tackling limited uplink bandwidth

However, using the uplink to upload reference images to the satellites is not without challenges—the uplink has limited bandwidth (e.g., only 250 Kbps in DOVEs constellation [50]). Earth+ tackles this challenge with three ideas. Put together, they allow enough reference images to be sent to the satellites under the limited uplink bandwidth while allowing Earth+ to realize sizable downlink savings.

Downsampling reference images: Earth+ compresses reference images by downsampling (i.e., lowering resolution) and detecting changed tiles at a lower resolution. For example, if the original image is 4000x4000 and the reference image is downsampled to 500x500, the satellite will also downsample the captured image to 500x500 before calculating pixel differences and detecting changes. We then mark the tiles with average pixel difference over a threshold 𝜃 (see §5 for details on how to pick 𝜃) as changed tiles and only encode and download these changed tiles.

Detecting changes with downsampled images is less accurate than with full-resolution images. However, we notice that it mainly triggers false negatives (i.e., changed tiles might be mis-detected as unchanged). This is because the downsampling essentially averages out the pixel changes in a tile, so the amount of changes are lower compared to without downsampling. As a result, Earth+ uses a lower threshold 𝜃 to recall those false negatives.

To evaluate the effect of reference image compression, we compress the reference images using different compression ratios, and lower 𝜃 properly to align the amount of changed tile between different compression ratios (so that the amount of data need to be downloaded is aligned). In Figure 7, we show that we can compress the reference image by 2600× while only missing 1.7% changed tiles.

Incrementally updating reference images: As Earth+ applies reference-based encoding, which does not encode the unchanged areas in the captured satellite imagery, this saves the on-board storage space used for storing captured imagery by about 80% (since 80% of the areas do not need to be encoded on average, as shown in §6) and enables Earth+ to use the following optimization to further reduce the usage of uplink. Concretely, Earth+ locally caches the reference images onboard the satellite for all locations the satellite will visit and only uploads changed areas when uploading a new reference image to the satellite. The overhead of such caching is marginal (about 5% compared to the existing storage space used to store observed satellite imagery 7 ), and thus fits into the storage space conserved by reference-based encoding. Also, caching reference images on-board allows Earth+ to handle occasional uplink disconnection (more details in §5).

Uploading only cloud-free images: Earth+ requires cloudfree reference images to detect terrestrial changes. However, accurately identifying cloud-free imagery on-board can be computationally expensive as it requires neural networks to accurately detect faint clouds [94] and thus not doable on-board. Earth+ thus uses the ground to check if the images are cloud-free retrospectively before uploading it to the satellites.

然而，使用上行链路向卫星上传参考图像并非没有挑战 —— 上行链路的带宽有限（例如，在DOVEs星座中仅为250 Kbps [50]）。Earth+ 通过三个思路来应对这一挑战。综合运用这些方法，可以在有限的上行带宽下向卫星发送足够的参考图像，同时为 Earth+ 实现可观的下行链路节省。

降采样参考图像 (Downsampling reference images)： Earth+ 通过降采样（即降低分辨率）来压缩参考图像，并在较低分辨率下检测变化的图块。例如，如果原始图像是4000x4000，而参考图像被降采样到500x500，那么卫星在计算像素差异和检测变化之前，也会将捕获的图像降采样到500x500。然后，我们将平均像素差异超过阈值 \(θ\) 的图块标记为变化图块，并仅对这些变化图块进行编码和下载（关于如何选择 \(θ\) 的细节见§5）。

使用降采样图像检测变化不如使用全分辨率图像准确。但是，我们注意到这主要引发假阴性（即变化的图块可能被错误地检测为未变化）。这是因为降采样本质上平滑了图块内的像素变化，因此与不进行降采样相比，变化的量级会降低。因此，Earth+ 使用一个更低的阈值 \(θ\) 来召回这些假阴性样本。

为了评估参考图像压缩的效果，我们使用不同的压缩比来压缩参考图像，并适当降低 \(θ\) 以使不同压缩比下检测到的变化图块数量保持一致（从而使需要下载的数据量保持一致）。在图7中，我们展示了可以将参考图像压缩2600倍，而仅漏掉1.7%的变化图块。

增量更新参考图像 (Incrementally updating reference images)： 由于 Earth+ 应用了基于参考的编码，不对捕获的卫星影像中未变化的区域进行编码，这节省了约80%的用于存储捕获影像的星上存储空间（因为如§6所示，平均有80%的区域无需编码）。这使得 Earth+ 能够利用以下优化来 进一步减少上行链路的使用。

具体来说，Earth+ 在卫星上为卫星将访问的所有地点本地缓存参考图像，并且 在向卫星上传新的参考图像时，仅上传变化的区域。 这种缓存的开销很小（约为现有用于存储观测卫星影像空间的5%），因此可以利用基于参考的编码所节省出的存储空间。

此外，星上缓存参考图像也使 Earth+ 能够处理偶然的上行链路中断（更多细节见§5）。

仅上传无云图像 (Uploading only cloud-free images)： Earth+ 需要无云的参考图像来检测地表变化。然而，在星上准确识别无云图像可能计算成本高昂 ，因为它需要神经网络来精确检测薄云[94]，因此在星上难以实现。为此， Earth+ 利用地面站回顾性地检查图像是否无云，然后再将其上传到卫星。

Implementation

Illumination and cloud: In satellite imagery, the time gap between two consecutively-captured images can be hours [74] or days [51]. As a result, two consecutive images in the image sequence can differ a lot in terms of pixel values due to different illumination condition and cloud condition (as illustrated in Figure 8), making the general-purpose change detector (e.g., [45, 63, 81, 89]) no longer suitable for satellite imagery compression.

Note that there are other potential sources (e.g. sensor noise, image misalignment) that can also trigger large pixel differences. Earth+ does not explicitly address them, as they only appear in raw data sensed by the satellite, which is not accessible in public datasets.

Filtering out the cloud: Previous work [48, 78] observes that a wide range of applications (e.g., autonomous road detection, precision agriculture) focus on the geographical content on the ground, allowing cloudy areas to be filtered out without impacting analytic results. Based on this observation, Earth+ runs an on-board cloud detector to identify and filter out clouds. However, as accurate cloud detector is too computationally expensive for on-board use (§4.3), Earth+ runs a lightweight decision-tree-based cloud detector instead, which is a widely used cheap cloud detection algorithm that can still detect and filter out most clouds except for faint clouds and haze [53, 79, 90]. These faint cloud and haze are downloaded to the ground by Earth+ (thus increasing the downlink usage of Earth+) but do not affect the analytic results of applications, as these applications will perform accurate cloud removal as an initial pre-processing step.

On-board change detector: Based on the aforementioned cloud filtering mechanism, Earth+ then uses the following workflow to detect changes. First, Earth+ filters out cloud by detecting highly cloudy areas in the satellite imagery using a decision tree classifier, and remove this part of the data. Second, Earth+ drops those images if more than 50% of the areas are filtered by the cloud filter. Third, Earth+ aligns the illumination between the reference image and the captured image on less-cloudy areas using standard linear regression (since the illumination condition affects the pixel value linearly [92]). At last, Earth+ detects, encodes and downloads changes (details in §4.3).

Encoding changed tiles: Earth+ encodes those changed tiles by selecting the changed tiles as region-of-interest and runs region-of-interest encoding on the whole image using an off-the-shelf JPEG-2000 encoder (Kakadu [19]). While encoding such images, Earth+ makes sure that the bit spent on each encoded tile is a constant 𝛾 by configuring the bitper-pixel parameter of the Kakadu encoder as 𝛾 times the percentage of tiles that are changed.

Choosing parameters for Earth+: Earth+ introduces two parameters: change detection threshold 𝜃 (§4.3) and bit-perpixel 𝛾 . Earth+ chooses 𝜃 by profiling last year’s data on one single location, and uses this parameter on this year’s data for all locations. Earth+ then varies 𝛾 to trade-off between downlink usage and imagery quality.

Handling different bands: Unlike traditional RGB images, satellite imagery typically has multiple bands and the amount of changes of different bands are different. For example, vegetation bands measure the concentration of chlorophyll (which is sensitive to temperature), while traditional RGB bands are less sensitive to temperature. To handle such heterogeneity between bands, Earth+ treats each band separately, which means that Earth+ detects changes band-byband and updates the reference images band-by-band, allowing Earth+ to mark different areas as changed and download different amounts of changes for different bands.

Handling bandwidth fluctuation: To handle uplink fluctuation, as Earth+ locally caches the reference images, Earth+ can randomly skip the updating of some reference images, and instead rely on the cached old reference images (at the cost of downloading more areas). To handle downlink fluctuation, Earth+ leverages the layered codec feature, which allows Earth+ to download less layers when downlink is limited (at the cost of degraded the image quality). The feature of layered codec is widely supported by existing imagery encoders on the satellite (e.g., JPEG-2000 encoders [9, 19]).

Updating reference images: Earth+ needs to constantly update its reference images. A naive design is to constantly patch the reference image with newly observed changes, similar to how a video encoder updates its reference images [81, 89]. However, we found that this approach will gradually degrade reference image quality, since each patch will introduce some artifact to the reference image (which is caused by Earth+’s imperfect illumination alignment due to low reference image resolution) and such artifact accumulates.

As a result, Earth+ instead acquires reference images by whole image downloading: for each location, Earth+ downloads the first cloud-free image observed by any satellite in the constellation. After this, Earth+ will stop whole image downloading for this location for a month. This operation will not introduce large overhead in large-scale LEO constellations, as the overhead of guaranteed downloading is fixed (at most 12 times a year) and will be evenly spread out by all satellites in the constellation.

Note that although Earth+ starts to find a new cloud-free image as reference one month after observing the last reference image, the extra time it takes to actually find such cloud-free image can be excessively long, which indicates that the actual downloading frequency can be much lower than once per month. In the extreme case, assuming that the constellation contains only one satellite, the frequency of whole image downloading will be once every 81 days in average — only around 4 times a year, where this 80-day estimate comes from the fact that Earth+ starts updating the reference after one-month wait, together with an additional wait time of 51 days in average (§3) to actually observe such cloud-free reference image.

Gemini概括一下:

简单来说，这部分内容详细解释了 Earth+ 系统是如何在卫星上智能地筛选、处理和压缩图像，从而只把最有价值、变化最明显的部分传回地球，以解决各种技术难题。

以下是它的几个关键做法:

1. 核心挑战：应对光照和云层干扰

   • 问题：卫星在不同时间拍摄同一地点，会因为太阳角度（光照）和天气（云层）的变化导致图像看起来差异巨大。传统的“找不同”算法会把这些都当成变化，这是错误的
   • Earth+ 的做法：它有一套专门的流程来区分“天气光照引起的变化”和“地物真实发生的变化”

2. 关键步骤：在卫星上高效过滤云层

   • 问题：云层遮挡了地面，这些图像没用。但非常精确的云检测算法在卫星上运行太慢、太耗电。
   • Earth+ 的做法：在卫星上用一个快速但不够完美的“轻量级”算法，先把大部分明显的云过滤掉。剩下的一些薄云和雾霾就直接传回地面，让地面上更强大的计算机去处理

3. 核心思想：只编码和下载“变化”的部分

   • 做法：在排除了光照和云层干扰后，系统会识别出真正变化的区域（比如新建了一栋楼），然后只使用 JPEG-2000 这种压缩技术，把这些“感兴趣区域”编码后下载，图像的其他部分则不下载，从而极大节省了下行带宽

4. 系统鲁棒性：适应不稳定的网络

   • 问题：卫星与地面的通信带宽会波动
   • Earth+ 的做法：
     • 如果上传带宽（地面->卫星）不足，它可以暂时不更新卫星上的参考图，先用旧的
     • 如果下载带宽（卫星->地面）不足，它可以只传一个低质量版本的图像回来，保证基本通信

5. 保证质量：智能更新“参考底图”

   • 问题：系统需要一张“参考底图”来对比变化。如果一直用小补丁（patch）的方式更新这张底图，时间长了图像质量会像“复印件的复印件”一样越来越差
   • Earth+ 的做法：它不打补丁，而是定期（大约每月一次）直接下载一张全新的、高质量的无云图像来替换旧的“参考底图”，确保对比的基准是清晰准确的

Evaluation

In this section, we pick two state-of-the-art satellite imagery compression systems as our baseline and evaluate Earth+ against on two satellite imagery datasets. The key takeaway of our evaluation is three-fold:

• Compared to the state-of-the-art onboard compression schemes, Earth+ reduces the downlink bandwidth usage by 1.3-3.3× without hurting the imagery quality on not only RGB bands but also other satellite imagery bands.

• These improvements are achieved without using more uplink bandwidth than currently available, or more compute or storage resources than the baselines.

• With more satellites in a constellation, Earth+ can further reduce the amount of downlink bandwidth usage.

6.1 Experimental setup

Dataset: We evaluate Earth+ on two datasets (Table 2 illustrates the details of these two datasets).

Rich-content dataset: We collect 1-year images on 11 geographical locations in Washington State (where each location is of size 1600 km 2 ) from Sentinel-2 dataset [51]. We sample images from Washington State as it contains a wide variety of geographical contexts, including fluvial landscapes, agricultural areas with varied irrigation systems, mountainous regions with large elevation changes, etc, as shown in Figure 9a-e. Since the total file size of this data is 3.6 TB, to handle the large volumn of this dataset, we downsample the images in this dataset by 4×, width and height, where we confirmed on one location that such downsampling does not affect the savings of Earth+.

However, Sentinel-2 dataset [51] only contains two satellites in its constellation. To further show the potential of Earth+’s constellation-wide change-based encoding, we incorporate another dataset with lower coverage but with more satellites available.

Large-constellation dataset: we use Planet dataset [74] that contains multiple satellites in its constellation to showcase the potential of Earth+’s constellation-wide change-based encoding. Due to the quota limit of the Planet dataset, we only sample images on one randomly sampled location in the U.S. (illustrated in Figure 9f), with cloud coverage smaller than 5%. Our sampled dataset contains 48 satellites in total.

Real-world satellite specifications: see Table 1. In this table, we use data from year 2017 to year 2018 as we found the most public satellite specification data during that time period. As a result, such table may not faithfully reflect the specifications of latest satellites.

Uplink and downlink: We use the uplink and downlink specifications from Doves constellation. Specifically:

• Uplink: we assume that the uplink is of 250 kbps [50] and the connection duration is 10 minutes [20, 44]. Here we assume that the uplink bandwidth is a constant, as the uplink leverages the S-band to communicate [8], which is of low frequency, and thus severe weather conditions do not significantly affect its bandwidth [83].

• Downlink: we assume that the ground contact duration is 10 minutes [20, 44] and calculate the average bandwidth required to download a fixed amount of images.

Imagery encoder: We use the off-the-shelf JPEG-2000 encoder called Kakadu [19], which can run on satellite CPU. We note that JPEG-2000 is a variation of JPEG that supports more imagery bands and bit depths and is widely adopted in LEO satellite constellations [51, 74].

Metrics: Earth+ aims to reduce the downlink demand without hurting the quality of downloaded images. We measure the required downlink bandwidth by dividing the amount of downloaded data during one ground contact by the ground contact time (10 minutes [20, 44]) and measure the image quality via Peak Signal-to-Noise Ratio (PSNR for short). This aligns with satellite compression literature [52, 55, 57, 80], and prior work shows that a higher PSNR typically leads to higher application-side performance [43, 62].

We also evaluate the accuracy of vegetation area segmentation on forest areas on one forest location in Sentinel dataset (other locations has low vegetation coverage). We segment the vegetation area by calculating NDVI index [17, 18, 21–24] and thresholding the NDVI index by 0.1 [17]. The accuracy is defined as the percentage of pixels that are correctly identified as vegetation area or non-vegetation area.

Baselines: We consider two state-of-the-art baselines for on-board satellite imagery compression:

• Kodan [48]: drop low-value cloud data and download remaining non-cloudy areas.

• SatRoI [78]: run reference-based encoding using the first image for each location in our dataset as the reference image (we make sure that the first image for each location is cloud-free in our dataset).

• Lossless compression: compress the satellite imagery using lossless compression. We use two codecs that are commonly used on-board: JPEG2K codec (through Kakadu encoder [19]) and CCSDS 122.0-B-1 codec (via TER encoder [34]). The reason that the SatRoI baseline does not update its reference images via uplink is two-fold. First, the current uplink capacity is insufficient to upload even one reference image (even after the default image compression) for each image that will be downloaded during the one-year period of our evaluation. Also, if choosing references from satelliteobserved images, SatRoI may frequently use cloudy images as references (since 2/3 of the earth is covered by clouds [12] and the on-board cloud detector may incorrectly identify cloudy images as cloud-free, as illustrated in §5), whereas we ensure that the reference images in SatRoI are always cloud-free. As a result, our SatRoI baseline performs strictly better than SatRoI that naively updates reference images.

Also, we evaluate Earth+ using the standard JPEG-2000 image encoder, commonly used by existing satellites [5, 28]. While better satellite imagery encoders exist [37, 40, 67, 75], Earth+ complements these works, as these works focus on how to download the imagery in a target area using less bits, and Earth+ focuses on adjusting the target areas so that those unchanged areas are not downloaded to the ground. We also use JPEG-2000 encoder for other baselines.

简要概括一下:

一、核心结论是什么？

作者通过实验，得出了三个关键结论来证明 Earth+ 的优越性：

1. 大幅节省流量：和当前最先进的方法相比，Earth+ 能节省 1.3 到 3.3 倍的下载流量（下行带宽），而且图像质量一点都没变差
2. 不增加额外负担：实现这个效果，并不需要比现有方案更多的上传流量、计算能力或存储空间。它很“经济实惠”
3. “人多力量大”：如果卫星星座里的卫星数量越多，Earth+ 的节省效果会越好

二、实验是怎么做的：

为了让上面的结论有说服力，作者设置了非常详细的实验环境：

1. 实验数据：用了两种真实的卫星数据集

   • 一个叫 Sentinel-2（内容丰富，但只有2颗卫星），用来模拟多样的地理环境
   • 另一个叫 Planet（有48颗卫星），用来证明“卫星越多，效果越好”的结论

2. 对比对象 (Baselines)：他们选了两个“业界高手”和一个“老实人”来作对比，确保能胜过它们

   • Kodan：一种只下载无云区域的智能方法
   • SatRoI：另一种利用“参考图”来压缩的方法
   • 无损压缩：一种最基础、保证图像信息完全不丢失的压缩方法

3. 衡量标准 (Metrics)：用两个核心指标来评判好坏

   • 所需下行带宽：就是看谁下载图像占用的流量更少
   • 图像质量 (PSNR)：一个专业评分，分数越高说明图像越清晰、失真越小

6.2 Experimental results

tldr

Related Work

Single-image compression: A wide range of prior work has focused on single-image compression by augmenting traditional image codecs like JPEG-2000 [37–39, 58, 67, 75, 76, 93] or developing neural-based codec such as autoencoders [40, 41, 47, 95, 96]. Earth+ complements these works, as these works focus on how to download the imagery in a target area using less bits, and Earth+ focuses on adjusting the target areas so that those unchanged areas are not downloaded to the ground.

单图像压缩 (Single-image compression)

大量先前的工作专注于单图像压缩，其方法是增强传统的图像编解码器（如 JPEG-2000 [37–39, 58, 67, 75, 76, 93]），或开发基于神经网络的编解码器（如自动编码器 [40, 41, 47, 95, 96]）。Earth+ 与这些工作是互补的，因为这些工作专注于如何用更少的比特数下载目标区域的影像，而 Earth+ 则专注于调整目标区域，从而使未变化的区域不被下载到地面。

Change-based encoding: A rich set of literature aims to further compress images by detecting changes between images. A line of work builds video-based codecs (e.g., H.264 [89], H.265 [81], VP8 [42], VP9 [71] and autoencoders [64, 65, 91]) to leverage such redundancy, with the assumption that two consecutive captures have similar pixel values. This is not true for satellite imagery due to varying cloud and illumination conditions. Another line of work [78, 88] develops change-based encoding that is robust to varying cloud and illumination conditions. Earth+ also falls into this category. However, existing work can only update the reference image using single-satellite information, while Earth+ allows updating the reference image using images from all the satellites in the same constellation, resulting in a fresher reference image and, thus, better change-based encoding quality.

基于变化的编码 (Change-based encoding)

大量文献旨在通过检测图像间的变化来进一步压缩图像。一类工作构建了基于视频的编解码器（例如 H.264 [89], H.265 [81], VP8 [42], VP9 [71] 和自动编码器 [64, 65, 91]）来利用这种冗余，其假设是连续捕获的图像具有相似的像素值。由于多变的云层和光照条件，这一点在卫星影像中并不成立。另一类工作 [78, 88] 开发了对变化的云层和光照条件具有鲁棒性的基于变化的编码方案。Earth+ 也属于这一类别。然而，现有工作只能利用单颗卫星的信息来更新参考图像，而 Earth+ 则允许使用同一星座中所有卫星的图像来更新参考图像，从而得到时效性更高（fresher）的参考图像，并因此获得更好的基于变化的编码质量。

In-orbit computing: An alternative way to reduce the total downlink capacity is to have a concrete application in mind and drop out images that are irrelevant to this application (e.g., [48, 49, 82]). However, this approach may drop out images that are crucial for other applications. In contrast, Earth+ only drops areas that are unchanged, allowing Earth+ to be used by a wider range of applications.

在轨计算 (In-orbit computing)

另一种减少总下行容量的方法是，针对一个具体应用，在轨丢弃与该应用无关的图像（例如 [48, 49, 82]）。然而，这种方法可能会丢弃对其他应用至关重要的图像。相比之下，Earth+ 只丢弃未变化的区域，这使得 Earth+ 能被更广泛的应用所使用。

Multipath imagery delivering: One can reduce the latency of obtaining newly-observed satellite imagery by enabling multiple satellites to download the same imagery using intersatellite links [61, 66]. However, this approach does not increase the amount of imagery downloaded to the ground, as it does not increase the total downlink capacity of the constellation, or reduce the total amount of imagery data need to be downloaded. In contrast, Earth+ allows more imagery to be downloaded to the ground.

多路径影像传输 (Multipath imagery delivering)

一种方法是通过启用多颗卫星利用星间链路 [61, 66] 下载同一影像，来降低获取新观测卫星影像的延迟。然而，这种方法并未增加下载到地面的影像总量，因为它既没有增加星座的总下行链路容量，也没有减少需要下载的影像数据总量。相比之下，Earth+ 能够让更多影像被下载到地面。

Limitation

While Earth+ improves satellite imagery compression, several concerns remain.

尽管 Earth+ 改进了卫星影像压缩，但仍存在一些问题。

Lossy compression: Earth+’s compression is lossy. While it allows downloading more images, lossy compression may not be applicable to applications that require lossless compression. To address this issue, future work can improve the image quality of Earth+ by augmenting uplink bandwidth. Also, one can repurpose Earth+ for lossless compression by performing lossless delta-based compression.

有损压缩 (Lossy compression)： Earth+ 的压缩是一种有损压缩。虽然它允许下载更多图像，但有损压缩可能不适用于需要无损压缩的应用。为解决此问题，未来的工作可以通过增补上行带宽来提升 Earth+ 的图像质量。此外，也可以通过执行无损的基于增量（delta-based）的压缩，将 Earth+ 重新用于无损压缩场景。

Evaluating on ground-processed imagery: Due to the lack of raw satellite imagery data (i.e., Level-0 imagery data) in public datasets, we evaluate Earth+ on public imagery that is post-processed by the ground, which did not faithfully reflect the impact of geographical misalignment and sensor noise on Earth+. That said, we believe this issue is not severe as Earth+ detects changes using low-resolution reference images, which is less sensitive to misalignment and noises compared to full-resolution reference images.

在经地面处理的影像上进行评估 (Evaluating on ground-processed imagery)： 由于公开数据集中缺乏原始卫星影像数据（即 Level-0 级影像数据），我们在经过地面站后处理的公开影像上对 Earth+ 进行了评估，这未能如实反映地理配准误差（geographical misalignment）和传感器噪声对 Earth+ 的影响。话虽如此，我们认为这个问题并不严重，因为 Earth+ 使用低分辨率参考图像来检测变化，与全分辨率参考图像相比，它对配准误差和噪声不那么敏感。

Control messages: Earth+ uses the uplink bandwidth that is reserved for control messages to upload reference images. That said, we believe this is not a serious practical concern as the bandwidth needed for ground-to-satellite control messages is low (e.g., 2.4 kbps [70]) and do not currently use much of the uplink bandwidth capacity (e.g., 250 kbps [50]). Generalization of results: Our evaluation of Earth+ focuses on a specific set of satellite specs and imagery datasets, but it does not show how effective Earth+ would be if it is used on other or future earth-observation satellites. We hope our work will inspire more research to examine Earth+ in other environments.

控制信令 (Control messages)： Earth+ 使用了为控制信令预留的上行带宽来上传参考图像。话虽如此，我们认为这不是一个严重的实际问题，因为地对空控制信令所需的带宽很低（例如 2.4 kbps [70]），且目前并未占用大部分上行带宽容量（例如 250 kbps [50]）。

Generalization of results: Our evaluation of Earth+ focuses on a specific set of satellite specs and imagery datasets, but it does not show how effective Earth+ would be if it is used on other or future earth-observation satellites. We hope our work will inspire more research to examine Earth+ in other environments.

结果的普适性 (Generalization of results)： 我们对 Earth+ 的评估集中在一组特定的卫星规格和影像数据集上，但并未展示 Earth+ 在其他或未来的地球观测卫星上使用时的效果如何。我们希望我们的工作能启发更多研究，以检验 Earth+ 在其他环境中的表现。

Deployment concerns: Though Earth+ only changes software, there may be complications in implementing Earth+ on existing satellites as Earth+ requires a software update on the satellite’s imagery encoding module onboard the satellite.

Stepping back, we acknowledge that Earth+ does increase the system complexity, especially on the ground stations, including sharing downloaded images across ground stations efficiently. However, we believe Earth+ takes the first step towards delivering more images to the ground by constellationwide imagery sharing.

部署方面的顾虑 (Deployment concerns)： 尽管 Earth+ 只改变软件，但在现有卫星上实现 Earth+ 可能会有复杂性，因为它要求对卫星上的影像编码模块进行软件更新。

退一步讲，我们承认 Earth+ 确实增加了系统复杂性，尤其是在地面站端，包括如何在地面站之间高效地共享已下载的图像。但我们相信，通过星座级的影像共享，Earth+ 为向地面传输更多图像迈出了第一步。

Conclusion

While satellite imagery is useful for a wide range of applications, most of the imagery observed by the satellites is not downloaded to the ground due to limited downlink capacity. This work presents Earth+, a new onboard satellite imagery compression system to reduce the downlink bandwidth usage. Earth+ is the first to leverage images across an entire satellite constellation to allow downloading more images to the ground. Earth+ further uses several techniques to judiciously select and upload reference images under limited uplink capacity. We show that Earth+ can compress the imagery by upto 3.3× without compromising imagery quality on all bands or using more computation and storage resources, while staying within real-world uplink constraints.

尽管卫星影像可用于广泛的应用，但由于下行链路容量有限，卫星观测到的大部分影像并未被下载到地面。本文提出了一种名为 Earth+ 的新型星上卫星影像压缩系统，旨在减少下行带宽的使用量。Earth+ 首次利用了整个卫星星座的影像，从而能够将更多影像下载到地面。此外，Earth+ 采用多种技术，在有限的上行链路容量下审慎地选择并上传参考图像。我们证明了，Earth+ 可将影像压缩高达 3.3倍，且在所有波段上均不损害影像质量，也不需要使用更多的计算和存储资源，同时满足了真实世界的上行链路限制。