LEOCraft: Towards Designing Performant LEO Networks

Low Earth Orbit (LEO) satellite constellations have revolutionized Internet access for millions of users. OneWeb and SpaceX are already operating constellations of hundreds and thousands of satellites, offering Internet service directly from space across 100+ countries. These exceptionally large networks come at a cost – thousands of routers (satellites) need to fly at ∼22× the speed of sound, thus making network design a non-trivial challenge. While the systems research community with decades of deep networking expertise has a relatively short window to influence the design of these networks, there is a serious lack of the right tools to enable such efforts. To address this, we introduce LEOCraft – an LEO network design framework to help the community visualize and evaluate the performance of different choices. LEOCraft offers integrated optimization techniques tuned upon the domain knowledge acquired from thousands of LEO constellation design's performance evaluations to optimize a new constellation design ∼5× faster than other off-the-shelf black-box optimization techniques. LEOCraft scales up seamlessly, tested for 83K satellites across multiple shells (more than 2× SpaceX's longterm proposal) with 1K ground stations, thus making it feasible for the community to explore LEO trajectory and topology design for even the largest of mega-constellations.

低轨 (LEO) 卫星星座深刻地变革了数以百万计用户的互联网接入方式. OneWeb 与 SpaceX 目前已在运营由成百上千颗卫星组成的星座, 直接从太空向全球 100 多个国家/地区提供互联网服务. 然而, 构建此类超大规模网络需付出相应的代价——数以千计的 "空间路由器" (即卫星) 需以约 22 倍音速运行, 这使得网络设计成为一项极具挑战性的复杂任务. 尽管拥有数十年深厚网络领域专业知识的系统研究界正处于能够影响这些网络设计的短暂窗口期, 但现阶段严重缺乏合适的工具来支撑相关的研究工作.

为应对这一困境, 本文提出了 LEOCraft——一个 LEO 网络设计框架, 旨在协助学术界可视化并评估不同设计方案的性能表现. LEOCraft 提供了集成的优化技术, 这些技术基于从数千次 LEO 星座设计性能评估中提取的领域知识进行了调优; 与现有的现成黑盒 (black-box) 优化技术相比, 其优化全新星座设计的速度提升了约 5 倍. 此外, LEOCraft 具备出色的无缝扩展能力, 本文已针对分布于多个壳层 (Shells) 的 8.3 万颗卫星 (该规模逾 SpaceX 长期规划的两倍) 及 1000 个地面站进行了测试, 从而使研究社区能够切实探索即便是最大规模巨型星座 (mega-constellations) 的 LEO 轨迹与拓扑设计问题.

Introduction

In the past few years, thousands of satellites have been deployed in Low Earth Orbit (LEO) to serve worldwide high-speed Internet broadband service directly from space [44]. The early movers of this new networking landscape, SpaceX's Starlink and OneWeb, initially relied upon bent-pipe connectivity in the first generation of constellations [58]. Further technological advancements have introduced robust laser Inter-Satellite Links (ISLs), enabling space-borne mesh networks with high bandwidth and speed-of-light grade latency [55]. Such a LEO satellite network can carry long-haul Internet traffic over the satellite network, thus reducing the requirement for denser Ground Station (GS) deployment across the globe [2]. This development enabled Starlink and Amazon's Project Kuiper to propose a 2nd generation megaconstellation of 30,000 [3] and 3,236 satellites [1] respectively, to wrap the Earth with a dense mesh of LEO satellites. However, the FCC approved only 7,500 satellites of Starlink across three different altitudes [3]. SpaceX started launching these new generation satellites in December 2022 [61]. In contrast, Kuiper was recently in the testing phase of the optical mesh network in space [5]. Therefore, none of the operators has achieved the true scale of ISL-enabled megaconstellations yet, keeping the brief window open for the networking community to influence such design.

The secret of constellation design: The success of Starlink's early launches sparked researcher's interest in exploring various aspects of LEO networks. Such recent works [44–46,50, 54–56,58,64,67,68,76] have used Starlink, OneWeb, and a few other constellation designs in their experiments since details are publicly available in the FCC filings [1,3,8,12,21,31]. However, none of these existing works include the complete set of design parameters determining the satellite's trajectory as well as topology [45,72,92,93]. Most of these efforts are restricted to a single shell, considering the inter-shell connectivity and aggregated performance measurement of multiple independent shells as "an open problem" [93].

Furthermore, Starlink and OneWeb, the biggest constellation operators, rely on proprietary systems, making their LEO networks function like black boxes. Why they chose a particular design, and what traffic matrices it targets, is unknown to the community. Therefore, in this work, we aim to unveil the subtle crux of designing an optimized LEO network from scratch. Our effort will help networking researchers understand these 'new space' networks better.

Why constellation design is non-trivial?: Designing a satellite network is not a new problem. In the 90s, serious efforts [37,41,52,65,66,89,90] have been devoted in studying constellation design. However, the current scale and design objectives have significantly transformed over time, imposing new challenges. Optimizing a satellite constellation in the 90s emphasized coverage with a limited budget of a few 10s of satellites, ensuring availability worldwide [42]. In contrast, now, with the scale of a few thousand or more satellites, network performance has become comparable to the terrestrial networks [53]. Therefore, optimizing a constellation today is no longer a geometrical optimization of overlapping satellite coverage [51], but rather, an optimization of the satellite's orbital trajectories and their ISL topology for a given Internet traffic demand matrix. Therefore, given the number of satellites, the design problem is to decide upon several parameters describing ISL topology and each satellite's orbital trajectory at ∼22× the speed of sound, such that network performance is maximized. A network design is already a hard problem [62]; further joint optimization of orbital parameters with added dynamics of satellite motion makes it a non-trivial high-dimensional optimization problem. A naive brute-force grid search approach looking for maximum network performance measurement could take years due to the curse of dimensionality. A few previous efforts explored the aspect of improving constellation performance with better ISL topology [45, 72]; however, the approach [45] suffers from scalability issues hence could not be extended to the direction towards superior constellation design, i.e., optimization of satellite's trajectory at scale to enhance the network performance.

Missing the right tools: An in-depth study of trajectory and ISL topology optimization of the LEO network requires the execution of tens of thousands of simulations while varying one or more orbital/design parameters to measure how these choices impact the network performance. Therefore, an LEO network evaluation framework must offer the flexibility to (i) rapidly implement different ideas (topology, ISL connectivity, routing strategy, etc.), (ii) plug and play with various optimization strategies, (iii) evaluate the effectiveness of optimization strategies and further tuning these implementations. Existing platforms, i.e., Hypatia's [64] packet-level simulator, or StarryNet's [67] Docker container-based emulator, suffer from fundamental scalability issues [63]. Hence does not go well for large-scale (tens of thousands of satellites) trajectory simulation. Another LEO network emulation platform xeoverse [63] is built upon a lightweight process-based network emulator platform Mininet [20] and claims to outperform Hypatia and StarryNet by large margins; however, their source code is not public for community use. Therefore, the community lacks the right tools to approach large-scale trajectory and topology optimization problems.

Our contributions: In this paper, we address these gaps with LEOCraft – an LEO satellite constellation design exploration, optimization, and visualization framework. LEOCraft relies upon process-based parallelism [14], thus bypassing the Global Interpreter Lock, a major computational bottleneck [17] in the existing platforms [64,67]. It is designed to be a modular framework that allows rapid implementation of new design ideas with minimal code change (typically 100 lines of code), which helps to run new experiments quickly. It operates at the flow level, computes network measurements from the intuitive mathematical model of the LEO network instead of resource-intensive packet-level simulations. For instance, the performance evaluation of Starlink's Gen1 three shells of 3,888 satellites (S-1 to S-3 in Table 1) in LEOCraft takes ∼2.5 minutes on a standard Desktop PC. Whereas with Hypatia, it takes hours for only RTT measurement of Starlink's single shell of 1,584 satellites. We also tested LEOCraft on a hypothetical mega-constellation of 83+ satellites across multiple shells (as shown in Table 1). Thus, LEOCraft enables rapid evaluation of design choices at a large scale. The visualization module also generates interactive views of the constellation, showcasing topology evolution and end-to-end route changes to augment the intuition of LEO dynamics.

We use LEOCraft to rigorously study the constellation design for given traffic demand matrices across a set of GSes and illustrate how the choices of design parameters impact the performance of these LEO networks. Since Internet traffic demand is skewed towards specific regions of the Earth with higher populations, the performance measurements of constellation designs exhibit common trends irrespective of the satellite budget. We use this domain knowledge and predictable characteristics of network performance measurements to prune the search space drastically. This reduces the running time by ∼5× as compared to the naive off-the-shelf black-box optimization techniques. We also demonstrate how inter-shell ISL can improve the throughput of a constellation and highlight some shortcomings of such ISL usage. We made LEOCraft publicly available [35] for community use.

Paper outline: The remaining paper is organized as follows – §2-3 summarize our overall background and problem overview. Then §4-6 discusses the development, optimization, and system design strategies used in LEOCraft framework. §7-8 present the experimental evaluation. §9 summarizes the related works. The paper is concluded in §10.

近年来,数千颗卫星已被部署于低地球轨道(LEO),旨在直接从太空提供全球性的高速互联网宽带服务.作为这一新型网络领域的先行者,SpaceX 的 Starlink 和 OneWeb 在其第一代星座中初步依赖"弯管"(bent-pipe,即透明转发)连接技术.随着技术的进一步发展,具备高鲁棒性的激光星间链路(ISL)被引入,从而催生了具有高带宽和光速级延迟的天基网状网络.此类 LEO 卫星网络能够在卫星网络之上承载长途互联网流量,进而降低了全球范围内密集部署地面站(GS)的需求.这一发展促使 Starlink 与亚马逊的 Kuiper 项目分别提出了包含 30,000 颗 和 3,236 颗卫星 的第二代巨型星座规划,旨在用密集的 LEO 卫星网格覆盖全球.然而,美国联邦通信委员会(FCC)仅批准了 Starlink 分布于三个不同轨道高度的 7,500 颗卫星.SpaceX 已于 2022 年 12 月启动了这些新一代卫星的发射工作.相比之下,Kuiper 近期正处于其太空光学网状网络的测试阶段.因此,目前尚未有任何运营商实现基于 ISL 的巨型星座的真实规模部署,这就为网络研究界介入并影响此类设计保留了一个短暂的时间窗口.

星座设计的核心机密: Starlink 早期发射的成功,激发了研究人员对 LEO 网络各个层面进行探索的浓厚兴趣.近期的相关研究在其实验中采用了 Starlink,OneWeb 及少数其他星座设计,因为这些设计的具体细节已在 FCC 的公开文件中披露.然而,现有的研究中,没有一项工作涵盖了决定卫星轨迹与网络拓扑的完整设计参数集.多数现有的研究工作仅局限于单一壳层(shell),并将层间连接以及多个独立壳层的聚合性能测量视为一个"开放性问题".此外,作为最大的两家星座运营商,Starlink 与 OneWeb 均依赖于专有系统,导致其 LEO 网络如同黑盒一般运行.它们为何选择特定的设计方案,以及这些方案针对的是何种流量矩阵,对学术界而言皆属未知.因此,在本文中,我们旨在从零开始揭示设计优化型 LEO 网络的内在关键与难点.我们的工作将有助于网络领域的研究人员更深入地理解这些"新航天"网络.

为何星座设计绝非易事?: 设计卫星网络并非新问题.早在 20 世纪 90 年代,便有大量严谨的研究致力于探讨星座设计.然而,随着时间的推移,当前的星座规模与设计目标已发生了显著的转变,带来了全新的挑战. 90 年代的卫星星座优化主要强调在仅有数十颗卫星的有限预算下实现覆盖,以确保全球的可用性. 相比之下,如今在数千甚至更多卫星的规模下,卫星网络的性能已具备与地面网络相媲美的能力.因此, 如今的星座优化已不再是针对重叠卫星覆盖范围的简单几何优化,而是针对给定的互联网流量需求矩阵,对卫星的轨道轨迹及其 ISL 拓扑进行联合优化.

因此,在给定卫星数量的前提下,设计问题转化为:

如何确定描述 ISL 拓扑的各项参数,以及每颗卫星在约 22 倍音速下的轨道轨迹,从而使网络性能达到最大化?

网络设计本身已属于一个困难问题;在此基础上进一步对轨道参数与卫星运动的动态特性进行联合优化,使其成为一个极其复杂的高维优化问题.受限于"维度灾难",若采用朴素的暴力网格搜索方法来寻找最大化网络性能的最优解,可能需要耗费数年时间.此前已有少数研究探讨了通过改进 ISL 拓扑来提升星座性能;然而,这些方法面临可扩展性问题,难以延伸至更优星座设计的方向,即在大规模尺度上优化卫星轨迹以提升整体网络性能.

缺乏适用的工具: 对 LEO 网络轨迹与 ISL 拓扑优化进行深入研究,需要在改变一个或多个轨道/设计参数的同时执行数以万计的仿真,以衡量这些参数选择对网络性能的具体影响.因此,一个理想的 LEO 网络评估框架必须具备以下灵活性:(1)能够快速实现各种新思路(如拓扑结构,ISL 连接模式,路由策略等);(2)能够即插即用地使用各种优化策略;(3)能够评估各类优化策略的有效性并进一步微调这些实现方案.现有的仿真平台,例如 Hypatia 的数据包级仿真器,或是 StarryNet 基于 Docker 容器的模拟器,均存在根本性的可扩展性瓶颈.故而它们无法良好地适应大规模(数以万计的卫星)的轨迹仿真任务.另一个 LEO 网络模拟平台 xeoverse 是构建在轻量级,基于进程的网络模拟平台 Mininet 之上,据称其性能大幅超越了 Hypatia 和 StarryNet;然而,其源代码并未向研究社区公开.因此,研究社区目前极度缺乏处理大规模轨迹与拓扑优化问题的适用工具.

主要贡献: 本文中,我们提出了 LEOCraft——一个用于 LEO 卫星星座设计探索,优化及可视化的框架,以填补上述工具空白.LEOCraft 依赖于基于进程的并行计算技术,从而成功绕过了现有平台中存在的主要计算瓶颈——全局解释器锁(GIL).它的架构设计具有高度的模块化特征,使得研究人员仅需少量代码修改(通常在 100 行以内)即可快速实现全新的设计思路,从而极大地加速了新实验的运行.它在流级别运作,通过直观的 LEO 网络数学模型计算网络测量指标,而非依赖资源密集型的数据包级仿真.例如,在标准台式 PC 上,LEOCraft 评估包含 3,888 颗卫星的 Starlink 第一代三个壳层网络(如表 1 中的 S-1 至 S-3)的性能仅需约 2.5 分钟.相比之下,若使用 Hypatia,仅对 Starlink 包含 1,584 颗卫星的单层网络进行 RTT 测量便需耗费数小时.我们还在一个跨越多个壳层,包含超过 8.3 万颗卫星的假想巨型星座上对 LEOCraft 进行了测试.因此,LEOCraft 能够在大规模场景下对各种设计方案进行快速评估.其可视化模块还能生成交互式的星座视图,动态展示拓扑结构的演变与端到端路由的变化,以此增强对 LEO 动力学的直观理解.

我们利用 LEOCraft 在一组给定地面站的流量需求矩阵下,对星座设计进行了严谨的研究,并详细阐述了不同设计参数的选择如何影响这些 LEO 网络的性能.鉴于互联网流量需求向人口密集的特定地球区域倾斜,无论卫星预算如何,星座设计的性能测量结果均表现出了一些共通的趋势.我们利用这一领域知识以及网络性能测量的可预测特征,大幅度地对搜索空间进行了剪枝.相较于直接使用现成的朴素黑盒优化技术,该方法将运行时间减少了约 5 倍.我们同时验证了层间 ISL 如何提升星座的吞吐量,并指出了使用此类 ISL 可能存在的一些缺陷.我们已将 LEOCraft 开源供社区使用.

论文结构: 论文其余部分的组织如下——第 2-3 节概述了总体研究背景与问题定义.随后,第 4-6 节探讨了 LEOCraft 框架中所采用的开发,优化及系统设计策略.第 7-8 节展示了实验评估结果;第 9 节总结了相关工作;本文的结论包含在第 10 节中.