L2D2: Low Latency Distributed Downlink for Low Earth Orbit Satellites

Large constellations of Low Earth Orbit satellites promise to provide near real-time high-resolution Earth imagery. Yet, getting this large amount of data back to Earth is challenging because of their low orbits and fast motion through space. Centralized architectures with few multi-million dollar ground stations incur large hourlevel data download latency and are hard to scale. We propose a geographically distributed ground station design, L2D2, that uses low-cost commodity hardware to offer low latency robust downlink. L2D2 is the first system to use a hybrid ground station model, where only a subset of ground stations are uplink-capable. We design new algorithms for scheduling and rate adaptation that enable low latency and high robustness despite the limitations of the receive-only ground stations. We evaluate L2D2 through a combination of trace-driven simulations and real-world satelliteground station measurements. Our results demonstrate that L2D2’s geographically distributed design can reduce data downlink latency from 90 minutes to 21 minutes.

大规模低地球轨道（LEO）卫星星座有望提供近乎实时的高分辨率地球影像。然而，由于卫星轨道低且在太空中高速运动，将海量数据传回地球是一项艰巨的挑战。采用少数几个耗资数百万美元的地面站的集中式架构，会产生长达数小时的数据下载延迟，并且难以扩展。我们提出了一种名为 L2D2的地理分布式地面站设计 ，该设计使用低成本的商用硬件来提供低延迟、高鲁棒性的下行链路。L2D2是 首个采用混合地面站模型的系统，其中只有一部分地面站具备上行链路能力。 我们设计了新的调度和速率自适应算法，使得即便在仅接收地面站的功能限制下，也能实现低延迟和高鲁棒性。我们通过结合轨迹驱动的仿真和真实的星地测量数据对L2D2进行了评估。我们的结果表明，L2D2的地理分布式设计可以将数据下行链路延迟从90分钟减少到21分钟。

Introduction

Low Earth Orbit (LEO) satellites mark a new frontier in communications and sensing research. Multiple companies [7, 20, 21, 28] have committed to invest tens of billions of dollars to deploy constellations of hundreds of small satellites such as CubeSats. These satellites aim to serve two main objectives: communication and Earth observation. Communication satellites provide low-latency, high-bandwidth, and universal internet connectivity. Earth observation satellites continuously orbit the earth (see Fig. 1) and collect imagery (aka eyes in the sky). Unlike traditional Earth observation satellites, new deployments consist of large LEO constellations of cheap CubeSats. These satellites aim to build a near real-time map of the earth and use it for real-time analysis of agriculture [2, 35], geological systems [62], disease spreads [5], natural disasters [4, 11], and geopolitics [44, 45].

低地球轨道（LEO）卫星标志着通信与传感研究的新前沿。多家公司[7, 20, 21, 28]已承诺投资数百亿美元，部署由数百颗小型卫星（如立方体卫星）组成的星座。这些卫星旨在服务于两个主要目标：通信和地球观测。通信卫星提供低延迟、高带宽的全球互联网连接。地球观测卫星则持续环绕地球（见图1）并收集影像（即“空中之眼”）。与传统的地球观测卫星不同，新的部署方案包含由廉价立方体卫星组成的大型LEO星座。这些卫星旨在构建一个近乎实时的地球地图，并将其用于农业[2, 35]、地质系统[62]、疾病传播[5]、自然灾害[4, 11]和地缘政治[44, 45]的实时分析。

Today, about 45% of LEO satellites in orbit [59] are used for Earth observation. These satellites collect hundreds of Gigabytes of data every day and need to transmit this data [51] to Earth. This is challenging for LEO satellites because their low orbits mean that they move fast with respect to an observer on Earth. For any ground observer, the satellite is visible for around ten minutes and has four to five good contacts every day (see Fig. 2). As a result, satellite companies deploy a few highly specialized (multi-million US dollars) ground stations [34] that can download large quantities of data in a short period [18, 19]. This design for ground stations suffers from multiple shortcomings:

• Downlink Latency: While large constellations [7, 20, 30] promise to collect data every hour to few hours (in some cases, minutes [30]), this data must wait at the satellite before it comes in contact with a ground station. This adds latency of one to several hours, which can be crippling for time-sensitive applications like natural disaster management (e.g. forest fires, floods, etc.) and crop monitoring.

• Scaling: When the constellation size is small, the ground stations are under-utilized as they are used for a few minutes per satellite contact. As the promised large constellation sizes materialize, the ground stations are bottle-necked by bandwidth and contention, at which point satellite companies must deploy new ground stations. In addition to the high cost of designing and maintaining these ground stations, they also suffer from deployment delays and million-dollar costs due to regulatory requirements. For example, Amazon Ground Stations could not transmit data for at least a year after their public announcement due to licensing delays [27].

• Robustness: The centralized architecture relies on few ground stations that are prone to hardware failures and weather-related connectivity issues. At high frequencies used by the ground stations (8 GHz and above), the links are prone to attenuation of up to 10dB due to rain and clouds [6]. Some LEO satellites have reported up to 88% packet loss [17].

• High Cost of Entry: The cost of licensing and setting up a ground station is prohibitive for new entrants like academic research satellites. Given the reduced costs of satellites (tens of thousands USD), the ground station becomes the bottleneck.

如今，在轨的LEO卫星中约有45%[59]用于地球观测。这些卫星每天收集数百GB的数据，并需要将这些数据[51]传输到地球。对于LEO卫星而言，这是一项挑战，因为它们的低轨道意味着相对于地面观察者，它们的移动速度非常快。 对于任何地面观察者来说，一颗卫星的可见时间大约为十分钟，每天有四到五次良好的通信机会（见图2）。 因此，卫星公司部署了少数几个高度专业化（耗资数百万美元）的地面站[34]，以便在短时间内下载大量数据[18, 19]。这种地面站设计存在多个缺点：

• 下行链路延迟 (Downlink Latency): 尽管大型星座[7, 20, 30]承诺每小时甚至每几分钟[30]收集一次数据，但这些数据必须在卫星上等待，直到其与地面站建立联系。这增加了一到数小时的延迟，对于自然灾害管理（如森林火灾、洪水等）和作物监测等时间敏感型应用来说，这可能是致命的。

• 扩展性 (Scaling): 当星座规模较小时，地面站的利用率很低，因为它们每次卫星过境只使用几分钟。随着承诺的大型星座规模成为现实，地面站将因带宽和竞争而成为瓶颈，届时卫星公司必须部署新的地面站。除了设计和维护这些地面站的高昂成本外，它们还因监管要求而面临部署延迟和数百万美元的费用。例如，亚马逊地面站（Amazon Ground Stations）在公开发布后至少一年因许可证延迟而无法传输数据[27]。

• 鲁棒性 (Robustness): 集中式架构依赖于少数几个地面站，这些地面站容易受到硬件故障和天气相关的连接问题影响。在地面站使用的高频段（8 GHz及以上），链路信号容易因雨和云而衰减高达10dB[6]。据报道，一些LEO卫星的数据包丢失率高达88%[17]。

• 高准入门槛 (High Cost of Entry): 许可和建立地面站的成本对于学术研究卫星等新参与者来说是令人望而却步的。考虑到卫星成本的降低（数万美元），地面站反而成为了瓶颈。

In this paper, we present a new ground station architecture for LEO satellites: L2D2. L2D2 has two key characteristics: (a) it uses a large set of geographically distributed low-cost ground stations. It relies on commodity equipment (e.g. small antennas) that can be deployed at rooftops and backyards instead of specialized hardware. (b) We make the observation that the primary data mode for Earth observation satellites is downlink; the uplink is infrequently used for control traffic alone. In fact, ground stations today support Gbps downlink but only hundreds of Kbps uplink [18, 19, 34]. Therefore, a majority of the ground stations in L2D2 are receive-only, removing the regulatory burden of setting up new L2D2 stations. In practice, this design choice leads to three key challenges:

• Adaptive Downlink Scheduling: Given the large number of satellites and ground stations, we need to dynamically schedule satellite-ground station contacts while accounting for orbits, link quality, and weather conditions. In our dataset with 259 satellites, a ground station may see up to 100 satellites (median 14) at the same instant. However, in a typical setting, a ground station communicates with one satellite at a single point in time. Therefore, we need to identify the optimal satellite-ground station matching across time and space. Furthermore, it must account for switching delays, i.e. a satellite cannot communicate while switching from one ground station to another. We observe that this scheduling problem is a variant of the circuit scheduling problem studied in the context of datacenters [10, 24, 39, 42, 57], which is known to be NP-hard. We leverage this observation to design a new (approximate) greedy algorithm for this scheduling problem that supports multiple objective functions – throughput, mean latency, and peak latency.

• Rate Selection: The satellite downlink rate depends on the channel conditions at a given location. For instance, rain can attenuate the downlink signal by 10 to 20 dB in X, Ku, and Ka bands used for satellite downlink [6]. In typical wireless systems, the optimal rate is selected using feedback from the receiver. How does one perform rate selection in the absence of such feedback? To solve this problem, we build a new link quality predictor that leverages historical data, predicted weather conditions, and orbital dynamics to predict the ideal datarate. The design of this predictor is non-trivial because of the local multipath experienced by satellite signals (e.g. solar panels close to antennas). Unlike state-of-the-art models for satellite signal propagation, our model can account for such ground station and satellite-specific variations.

• Satellite Feedback: How does a satellite know if its data has been received at a receive-only ground station and is safe to delete from its storage without acknowledgments? L2D2 uses delayed acknowledgments that are relayed through transmitcapable stations (3-4 in L2D2’s network) to the satellite, when the satellite flies over these stations. These acknowledgments allow the satellite to delete data that has been received and re-transmit lost data.

在本文中，我们为LEO卫星提出了一种新的地面站架构：L2D2。L2D2具有两个关键特征：

(a) 它使用大量地理上分布式的低成本地面站。它依赖于可以部署在屋顶和后院的商用设备（例如小型天线），而非专业硬件

(b) 我们观察到， 地球观测卫星的主要数据模式是下行链路；上行链路仅不频繁地用于控制流量

事实上， 如今的地面站支持Gbps级的下行链路，但上行链路仅为数百Kbps[18, 19, 34]。因此，L2D2中的大多数地面站都是 仅接收（receive-only） 的 ，这消除了建立新L2D2站点的监管负担。在实践中，这一设计选择带来了三个关键挑战：

• 自适应下行链路调度 (Adaptive Downlink Scheduling): 鉴于卫星和地面站数量庞大，我们需要在考虑轨道、链路质量和天气条件的情况下，动态地调度星地通信。在我们的数据集中，有259颗卫星，一个地面站可能在同一时刻看到多达100颗卫星（中位数为14颗）。然而，在典型情况下，一个地面站一次只能与一颗卫星通信。因此，我们需要在时间和空间上确定最优的星地匹配方案。此外，该方案必须考虑切换延迟，即卫星在从一个地面站切换到另一个地面站时无法通信。我们发现这个调度问题是数据中心领域研究的电路调度问题[10, 24, 39, 42, 57]的一个变体，该问题是已知的NP难问题。我们利用这一发现，为此调度问题设计了一种新的（近似）贪心算法，该算法支持多种目标函数——吞吐量、平均延迟和峰值延迟。

• 速率选择 (Rate Selection): 卫星下行链路速率取决于特定位置的信道条件。例如，在用于卫星下行的X、Ku和Ka波段，雨水可以使下行信号衰减10到20dB[6]。在典型的无线系统中，最优速率是通过接收端的反馈来选择的。那么，在没有这种反馈的情况下如何进行速率选择呢？为了解决这个问题，我们构建了一个新的链路质量预测器，它利用历史数据、天气预报和轨道动力学来预测理想的数据速率。该预测器的设计并非易事，因为卫星信号会经历局部多径效应（例如，靠近天线的太阳能电池板）。与最先进的卫星信号传播模型不同，我们的模型能够解释这种特定于地面站和卫星的变化。

• 卫星反馈 (Satellite Feedback): 在没有确认信号（acknowledgments）的情况下，卫星如何知道其数据已被一个仅接收的地面站成功接收并可以安全地从其存储中删除？L2D2使用延迟确认（delayed acknowledgments）。当卫星飞越具备传输能力的站点（L2D2网络中有3-4个）时，这些确认信息会通过这些站点中继给卫星。这些确认信息允许卫星删除已接收的数据并重传丢失的数据。

Our design has many advantages. Its geographical distribution ensures that a satellite encounters ground stations more frequently and can offload latency-sensitive data sooner. Allowing the ground station to be distributed also allows L2D2 to relax the requirement of extremely high throughput on individual links, therefore enabling the use of commodity components. Moreover, L2D2 is more robust to failures and weather variations since the impact of individual failures can be minimized by re-routing the data. For example, downlink can be dynamically scheduled so that cloudy weather in one part of the world is offset by clear weather in the other. Finally, L2D2 can allow new entrants to schedule communication for their satellites by providing a software abstraction.

To evaluate L2D2, we use a four-step approach. First, we collect real-world link quality measurements at five ground stations from four satellites operating in the X-band and Ka-bands. We are releasing this new dataset to the community. We use this dataset to validate individual components of our design. Second, to evaluate the distributed design at scale, we leverage the open-source SatNOGS ground stations operated by amateur radio operators. We compile this data from the publicly-hosted SatNOGS database. We use a network of 259 satellites and 173 ground stations deployed by different entities. These ground stations listen to satellite beacons at lower frequencies and do not correctly model the satellite downlink behavior in X-band or Ka-bands. Therefore, in step three, we model the link quality behavior of each of these ground stations as if they were a randomly chosen ground station from our X-band stations in the first step. Finally, we ask, in simulation, how this network would operate if each satellite had 100 GB of data to download per day. Based on this analysis, we summarize our results below:

• Link Estimation: L2D2’s link estimation algorithm achieves a median error of 0.39 dB in predicting the link quality as compared to state-of-the-art ITU models that achieve 2.39 dB median error (90th percentile 1.74 dB vs 6.03 dB). This translates to a datarate loss of 6.5% for L2D2 while the ITU model suffers 40.8% loss.

• Latency: L2D2 reduces the mean latency of data download from 90 minutes to 21 minutes and the 90-th percentile from 323 minutes to 71 minutes as compared to a baseline method that uses high-end ground stations with 10X more link capacity than L2D2’s ground stations.

• Data Transfer and Backlog: L2D2 downloads over 250 TB of data in a day from 259 satellites. In an experiment with each satellite collecting 100 GB per day, L2D2 reduces the median backlog (data not delivered) for a satellite from 7.6 GB to 3.4 GB (90-th percentile: 26.5 GB to 7.2 GB).

我们的设计有许多优点。其地理分布确保了卫星能更频繁地遇到地面站，并能更快地卸载对延迟敏感的数据。允许地面站分布部署也使L2D2能够放宽对单个链路极高吞吐量的要求，从而可以使用商用组件。此外，L2D2对故障和天气变化更具鲁棒性，因为可以通过重新路由数据来最小化单个故障的影响。例如，可以通过动态调度下行链路，使世界某地区的阴雨天气被另一地区的晴朗天气所抵消。最后，L2D2可以通过提供软件抽象，让新参与者能够为他们的卫星安排通信。

为了评估L2D2，我们采用了一个四步法。首先，我们在五个地面站收集了来自四颗工作在X波段和Ka波段的卫星的真实链路质量测量数据。我们即将向社区发布这个新的数据集。我们使用这个数据集来验证我们设计的各个组件。其次，为了大规模评估分布式设计，我们利用了由业余无线电爱好者运营的开源SatNOGS地面站。我们从公开托管的SatNOGS数据库中编译了这些数据。我们使用了一个由259颗卫星和173个不同实体部署的地面站组成的网络。这些地面站以较低频率监听卫星信标，并不能正确模拟X波段或Ka波段的卫星下行链路行为。因此，在第三步中，我们将这些地面站的链路质量行为建模，就如同它们是从我们第一步的X波段站点中随机选择的一样。最后，我们通过仿真提问，如果每颗卫星每天有100GB的数据需要下载，这个网络将如何运作。基于此分析，我们总结我们的结果如下：

• 链路预测 (Link Estimation): L2D2的链路预测算法在预测链路质量方面实现了0.39 dB的中位误差，而最先进的ITU模型的中位误差为2.39 dB（90百分位误差分别为1.74 dB vs 6.03 dB）。这转化为L2D2的数据速率损失为6.5%，而ITU模型则遭受40.8%的损失。

• 延迟 (Latency): 与使用高端地面站（其链路容量是L2D2地面站的10倍）的基准方法相比，L2D2将平均数据下载延迟从90分钟减少到21分钟，90百分位延迟从323分钟减少到71分钟。

• 数据传输与积压 (Data Transfer and Backlog): L2D2在一天内从259颗卫星下载了超过250TB的数据。在一个每颗卫星每天收集100GB数据的实验中，L2D2将卫星的积压数据（未交付的数据）中位数从7.6GB减少到3.4GB（90百分位：从26.5GB减少到7.2GB）。

As the LEO satellite deployments increase, a distributed framework is essential to enable a scalable, performant, and robust ground station design. This work is inspired by the past shifts in computing from singular highly specialized hardware to distributed lowcomplexity components. In designing L2D2, we make the following contributions:

• We present a new distributed hybrid design that uses low-cost receive-only ground stations to ensure low latency data transfer from LEO satellites.

• We design a novel scheduler for satellite-ground station links that accounts for temporal variation & switching delay.

• We build the first data-driven blind rate adaptation algorithm for LEO satellites.

• We evaluate L2D2 using measurements and large-scale simulations performed using multiple satellites and ground stations.

随着LEO卫星部署的增加，一个分布式的框架对于实现可扩展、高性能和鲁棒的地面站设计至关重要。这项工作的灵感来自于过去计算领域从单一高度专业化的硬件向分布式低复杂度组件的转变。在设计L2D2的过程中，我们做出以下贡献：

• 我们提出了一种新的分布式混合设计，该设计使用低成本的仅接收地面站，以确保从LEO卫星进行低延迟数据传输。

• 我们为星地链路设计了一种新颖的调度器，该调度器考虑了时间变化和切换延迟。

• 我们为LEO卫星构建了首个数据驱动的盲速率自适应算法。

• 我们利用来自多颗卫星和多个地面站的测量数据和大规模仿真对L2D2进行了评估。