跳转至

Resource-efficient In-orbit Detection of Earth Objects

With the rapid proliferation of large Low Earth Orbit (LEO) satellite constellations, a huge amount of in-orbit data is generated and needs to be transmitted to the ground for processing. However, traditional LEO satellite constellations, which downlink raw data to the ground, are significantly restricted in transmission capability. Orbital edge computing (OEC), which exploits the computation capacities of LEO satellites and processes the raw data in orbit, is envisioned as a promising solution to relieve the downlink burden. Yet, with OEC, the bottleneck is shifted to the inelastic computation capacities. The computational bottleneck arises from two primary challenges that existing satellite systems have not adequately addressed: the inability to process all captured images and the limited energy supply available for satellite operations. In this work, we seek to fully exploit the scarce satellite computation and communication resources to achieve satellite-ground collaboration and present a satellite-ground collaborative system named TargetFuse for onboard object detection. TargetFuse incorporates a combination of techniques to minimize detection errors under energy and bandwidth constraints. Extensive experiments show that TargetFuse can reduce detection errors by 3.4× on average, compared to onboard computing. TargetFuse achieves a 9.6× improvement in bandwidth efficiency compared to the vanilla baseline under the limited bandwidth budget constraint.

随着大型低地球轨道（LEO）卫星星座的迅速普及，在轨生成了海量数据，这些数据需传输至地面进行处理。然而，传统的LEO卫星星座将原始数据下行传回地面，其传输能力受到极大限制。轨道边缘计算（OEC）利用LEO卫星的计算能力在轨处理原始数据，被视为一种有望缓解下行链路负担的解决方案。

然而，采用OEC后，瓶颈则转移至缺乏弹性的计算能力上。这种 计算瓶颈 源于当前卫星系统未能充分解决的两大核心挑战：

一是无法处理所有捕获的图像

二是卫星运行的能源供应有限

在本项目中，我们致力于充分利用稀缺的卫星计算与通信资源以实现星地协同，并提出了一个名为 TargetFuse 的星地协同系统，用于星上目标检测。TargetFuse 融合了多种技术，旨在能量和带宽双重约束下最大限度地减少检测误差。大量实验表明，与纯星上计算方案相比，TargetFuse 平均可将检测误差降低3.4倍。在有限的带宽预算下，相较于原始baseline方法，TargetFuse 的带宽效率提升了9.6倍

Introduction

Earth-observation (EO) satellites collect multispectral images for geospatial analysis, providing valuable sensing and computational applications in the harsh space environment, characterized by highly constrained energy and network connectivity. Visual tasks, which detect vehicles along interstate highways to estimate traffic [1], count buildings from key areas to predict population [2], or monitor animals from the wildness to track their behaviors [3], etc, are among the key use cases for EO satellites. Advances in technology enable satellites to collect vast amounts of Earth images daily, often reaching tens of Terabytes [4], [5]. However, traditional satellites operate as “bent-pipe” and typically downlink all raw observations to the ground, which is significantly restricted in transmission capability due to the scarce bandwidth resources (e.g., tens of Mbps) and limited satellite-ground connection duration.

对地观测（EO）卫星采集多光谱图像用于地理空间分析，在能量和网络连接均受高度限制的恶劣空间环境中，提供了宝贵的传感与计算应用。对地观测卫星的关键用例包括多种视觉任务，例如：检测州际公路上的车辆以估计交通流量 [1]，统计关键区域的建筑物以预测人口 [2]，或监测野生动物以追踪其行为 [3] 等。技术的进步使卫星每天能够采集高达数十太字节（TB）的海量地球图像 [4], [5]。然而，传统卫星作为“弯管”（bent-pipe）运行，通常将所有原始观测数据下行传输至地面，但由于带宽资源稀缺（例如，仅数十 Mbps）且星地连接时长有限，其传输能力受到极大限制。

To overcome the limitations of the bent-pipe architecture, Orbital Edge Computing (OEC) has emerged as a promising solution, which mitigates the communication bottleneck by processing the observation data in space and sending the process results to the ground rather than directly downlinking the raw data [6]. However, in-orbit processing faces a computational bottleneck due to the inherent limitations of satellites, including both computational capacity and constrained energy. EO satellite images are large (i.e., hundreds of millions of pixels) and arrive at a high rate, far exceeding what embedded satellite hardware can process. The computational bottleneck arises not only from lower computing power hardware but also from the inability to process all images within the energy budget harvested from the less productive solar panel. For instance, Baoyun satellite can harvest up to 260KJ of energy daily, but not all of it is utilized for computing. When about 150KJ of energy is allocated for computing, satellite can only support the computation of about 22% of the observable high-resolution 3K satellite images. Therefore, as an effort to support analysis for EO, this work focuses on in-orbit detection of Earth objects, considering the inherent limitation in computational and energy. Fig. 1 shows that satellites utilize Deep Neural Network (DNN) models in orbit to detect objects. Detection results for images captured on each track are aggregated, and satellites transmit these aggregated counts.

为了克服“弯管”架构的局限性，轨道边缘计算（Orbital Edge Computing, OEC） 已成为一种极具前景的解决方案。它通过在太空处理观测数据并将处理结果发送回地面，而非直接下行传输原始数据，从而缓解了通信瓶颈 [6]。然而，在轨处理面临着计算瓶颈，这源于卫星固有的局限性，即计算能力和能量供给均受限。对地观测卫星图像尺寸巨大（即数亿像素）且生成速率高，远超星上嵌入式硬件的处理能力。计算瓶颈不仅源于计算能力较弱的硬件，也因为无法在从效率较低的太阳能电池板收集的能量预算内处理所有图像。例如，“宝云”卫星每日最多可收集260千焦（KJ）的能量，但并非所有能量都用于计算。当约150千焦的能量分配给计算任务时，卫星仅能支持处理约22%的可观测高分辨率3K卫星图像。因此，为支持对地观测分析，本工作聚焦于在轨地球目标检测，并充分考虑了计算和能量的内在限制。如图1所示，卫星在轨道上利用深度神经网络（DNN）模型进行目标检测，将每条轨迹上捕获图像的检测结果进行聚合，并传输这些聚合后的计数值。

Terrestrial applications typically divide large images into smaller images and process each image on DNN models. However, they focus only on improving application performance, ignoring the vital system factor of computational overhead [7]. Prior attempts to address the computational bottleneck focused on distributing in-orbit processing across a constellation aimed at a particular purpose [6], but the relatively high-cost solutions did not considered the energy budget of each satellite. In-orbit computing still cannot meet onboard application requirements due to energy constraints. Therefore, a collaborative satellite-ground system is needed, where embedded satellite hardware initially processes images under energy budget constraints and transmits crucial images to the ground despite facing limited bandwidth [8]. In situations where a downlink system encounters a scarce link, efficient transmission within a short-term contact time should be prioritized. However, today’s satellites transmit data indiscriminately [9]. Less attention has been paid to the scarce downlink bandwidth budgets. Therefore, this work centers on processing images onboard the satellite and selectively transmitting crucial images by utilizing the downlink capability.

地面应用通常将大图像分割为较小的图像块，并在DNN模型上分别处理，但它们只注重提升应用性能，而忽略了计算开销这一至关重要的系统因素 [7]。先前为解决计算瓶颈所做的尝试，主要集中于将特定任务的在轨处理分散到整个星座 [6]，但这些成本相对较高的方案并未考虑单颗卫星的能量预算。由于能量限制，在轨计算仍无法满足星上应用的需求。因此，一个星地协同系统是必要的，在该系统中，星上嵌入式硬件首先在能量预算约束下处理图像，并在带宽有限的情况下将关键图像传输至地面 [8]。在下行链路稀缺的情况下，应优先考虑在短暂的接触时间内进行高效传输。然而，当今的卫星无差别地传输数据 [9]，对稀缺的下行带宽预算关注甚少。因此，本工作的核心是 在卫星上处理图像，并利用下行链路能力选择性地传输关键图像。

这里很重要

我们一定要清楚 TargetFuse 和先前工作的异同:

  • OEC (ASPLOS20): 星座协同 仅仅为了 "one particular purpose". 不具备泛化性
  • Kodan (ASPLOS23): 仅仅在乎 application performance. 没考虑energy的问题

事实上当energy不够时, constellation 远远达不到其预设的性能

因此 TargetFuse 打上了这两个patch:

  1. 具备普适性
  2. 考虑了 energy

There are potential opportunities to deal with these computational and downlink bottlenecks in the collaborative satellite-ground system: (1) By tiling large images into smaller ones and resizing them to fit standard DNN models’ input size, there exists an optimal tile size that satisfies higher detection accuracy and lower computational overhead. Preliminary experiments conducted on widely-used datasets reveal that tile size affects both detection accuracy and computing overhead (execution time). Therefore, we propose finding the optimal tile size, balancing accuracy and computational overhead, for executing the onboard DNN model; (2) Optimal tile size also establishes the confidence thresholds for onboard DNN models, influencing the decision of whether to downlink corresponding tiles to the ground. To efficiently utilize the available bandwidth, we implement bandwidth-aware downlinking throttling to dynamically select and downlink crucial tiles within the given bandwidth budget. The selection process will be based on the confidence thresholds set by the onboard DNN models; (3) Satellite images contain some semantically geospatial feature contexts with a high degree of similarity, specifically for the captured images as the satellite passes over its ground track. To alleviate computational and downlink bottlenecks, we introduce a lightweight, clustering-based data deduplication technique that leverages geospatial feature contexts. This technique optimizes data processing by efficiently identifying and removing redundant data, thereby reducing computational and downlink burdens.

在星地协同系统中，存在应对这些计算和下行链路瓶颈的潜在机遇：

(1) 通过将大图像切分为小图块并调整其尺寸以适应标准DNN模型的输入，存在一个 最佳切片尺寸 ，能够同时满足更高的检测精度和更低的计算开销。在广泛使用的数据集上进行的初步实验表明，切片尺寸同时影响检测精度和计算开销（执行时间）。因此，我们建议寻找最佳切片尺寸，以平衡执行星上DNN模型的精度与计算开销。

(2) 最佳切片尺寸也为星上DNN模型建立了置信度阈值，这影响着是否将相应图块下行传输至地面的决策。为高效利用可用带宽，我们实现了一种 带宽感知的下行传输节流机制 ，在给定的带宽预算内动态选择并下行传输关键图块。选择过程将基于星上DNN模型设定的置信度阈值。

(3) 卫星图像包含一些具有高度相似性的语义地理空间特征上下文，特别是当卫星飞越其地面轨迹时捕获的图像。为缓解计算和下行链路瓶颈，我们引入了一种 轻量级的、基于聚类的数据去重技术 ，该技术利用地理空间特征上下文，通过有效识别和移除冗余数据来优化数据处理，从而减轻计算和下行链路的负担。

We present TargetFuse, a collaborative satellite-ground system under the inherent limited computational and downlink constraints. To ensure a realistic simulation, we collect new data, including satellite operation details such as the computing power of embedded hardware, based on tested inorbit satellites. We provide a comprehensive evaluation of TargetFuse across various energy budgets, embedded hardware setups, and bandwidth budgets. Extensive experiments show that TargetFuse can reduce detection errors by 3.4× on average, compared to onboard computing. TargetFuse showcases a remarkable 9.6× improvement in bandwidth efficiency compared to the vanilla baseline under the limited bandwidth budget constraint. Our contributions can be summarized as follows:

• We design and implement a satellite-ground collaboration system for object detection, a critical use case for EO satellite, aiming to minimize detection errors. To ensure the system’s realism, we collect and publish new data that contains satellite operation details * .

• We propose image tiling to balance detection accuracy and computational overhead, clustering-based data deduplication to alleviate the computational bottleneck, and bandwidth-aware down-linking throttling to address downlink bottlenecks.

• We conduct extensive experiments across various energy budgets, embedded hardware configurations, and bandwidth budgets, and demonstrate that TargetFuse’s superior performance against four baselines.

我们提出了 TargetFuse，一个在固有的计算和下行链路双重约束下的星地协同系统。为确保模拟的真实性，我们基于已在轨测试的卫星，收集了包括嵌入式硬件计算能力在内的卫星运行细节新数据。我们对TargetFuse在不同能量预算、嵌入式硬件配置和带宽预算下进行了全面评估。大量实验表明，与纯星上计算方案相比，TargetFuse 平均可将检测误差降低3.4倍。在有限的带宽预算约束下，相较于原始基线方法，TargetFuse 的带宽效率实现了显著的9.6倍提升。

我们的贡献可总结如下：

  • 我们针对对地观测卫星的关键用例——目标检测，设计并实现了一个旨在最小化检测误差的星地协同系统。为确保系统的真实性，我们收集并发布了包含卫星运行细节的新数据
  • 我们提出了图像切片技术以平衡检测精度与计算开销，基于聚类的数据去重技术以缓解计算瓶颈，以及 带宽感知的下行传输节流机制 以应对下行链路瓶颈
  • 我们在多种能量预算、嵌入式硬件配置和带宽预算下进行了大量实验，并证明了TargetFuse相较于四种基线方法的卓越性能