StarryNet: Empowering Researchers to Evaluate Futuristic Integrated Space and Terrestrial Networks

专为研究和评估未来 ISTN 而设计的 Emulation 框架

(1) 核心问题与动机

• 背景: 低轨卫星星座的快速发展带来了新的网络挑战(如高动态性, 频繁的拓扑变化)
• 现有工具的局限:
  • 实地测试(Live Testbeds): 成本高, 难以复现, 缺乏灵活性
  • 模拟器(Simulators, 如 NS-3): 缺乏真实系统栈的支持, 无法运行真实代码
  • 现有仿真器(Emulators, 如 Mininet): 难以模拟大规模星座的高动态性, 且扩展性有限

写作逻辑

1. 新网络 新环境 新挑战
2. 做网络仿真实验有利于 xxx
3. 现在还没有合适的网络仿真工具
4. 我们的 starrynet 闪亮登场

(2) StarryNet 的解决方案

StarryNet 采用"数据驱动"与"轻量级仿真"相结合的方法, 构建了一个高保真的数字孪生环境

它允许研究人员在地面服务器集群上模拟大规模, 高动态的卫星网络

(3) 系统架构

• 星座观测器 (Constellation Observer):
  • 负责收集真实的公开数据(如卫星轨道数据 TLE, 地面站分布, 监管信息等)
• 星座同步器 (Constellation Synchronizer):
  • 基于收集的数据和混合模型, 计算卫星的实时位置, 可见性, 链路通断和传播延迟, 确保时空一致性
• 星座编排器 (Constellation Orchestrator):
  • 利用容器技术(如 Docker)模拟卫星节点
  • 支持多机分布式部署, 通过虚拟局域网(VLAN)和流量隔离技术解决大规模星座的扩展性问题
  • 采用"预测-多线程事件记忆"机制来高效处理频繁的链路更新
• 统一抽象层 (Unified Abstraction/APIs):
  • 提供易用的 Python API, 让用户可以自定义网络拓扑, 路由协议和实验场景

(4) 关键特性

• 星座一致性 (Constellation Consistency): 精确复现真实星座的时空动态
• 系统真实性 (System Realism): 每个节点运行真实的 Linux 网络栈, 支持运行未修改的应用程序
• 灵活性与可扩展性: 支持从数百到数千颗卫星的规模(如 Starlink Phase I), 并支持 Hardware-in-the-loop 测试

Introduction

Thanks to the resurgence in the space industry [41, 46, 48], big competitors such as SpaceX and Amazon are actively planning and deploying hundreds or even thousands of broadband satellites in low earth orbits (LEO). Such emerging mega-constellations (e.g., Starlink [34], Kuiper [9]) can be integrated into existing terrestrial Internet, i.e., constructing an integrated space and terrestrial network (ISTN) to: (1) provide pervasive last-mile network access; (2) enable lowlatency and high-bandwidth Internet transit [40,52,56]; and (3) facilitate efficient acquisition and delivery for big data from space (e.g., earth observation images) [44,74,77,78].

得益于航天产业的复兴 [41, 46, 48], SpaceX 和 Amazon 等行业巨头正在积极规划并向低地球轨道(LEO)部署成百上千颗宽带卫星. 此类新兴的巨型星座(例如 Starlink [34], Kuiper [9])可与现有的地面互联网融合, 构建空天地一体化网络(ISTN), 旨在: (1) 提供无处不在的"最后一公里"网络接入; (2) 实现低延迟, 高带宽的互联网中继传输 [40,52,56]; 以及 (3) 促进空间大数据(如对地观测图像)的高效获取与分发 [44,74,77,78].

While holding great promise, several unique characteristics of LEO satellites (e.g., high LEO dynamics) impose new challenges at various layers of the ISTN networking stack, and open a door to many new research problems, such as: (1) how should LEO satellites and ground facilities be interconnected to provide low-latency and continuous network services? (2) how should satellite routers be integrated into existing terrestrial Internet routing? and (3) are current constellation and protocol designs resilient enough to satellite failures in complex and harsh space environments? With many unexplored problems facing the "NewSpace" industry, it is thus foreseen that in the near future, there will be a surge of new ideas on the system and networking research relevant to ISTNs. But, how can researchers build an experimental network environment (ENE) to test, evaluate and understand their new thoughts?

尽管前景广阔, 由于低轨卫星具有若干独特特性(如高动态性), 这给 ISTN 网络协议栈的各层 带来了新的挑战, 并引出了许多新的研究问题, 例如:

(1) 低轨卫星与地面设施应如何互联, 以提供低延迟且连续的网络服务?

(2) 卫星路由器应如何融入现有的地面互联网路由体系?

(3) 在复杂恶劣的空间环境中, 当前的星座与协议设计是否具备足够的弹性以应对卫星故障?

面对"新航天"产业中诸多未探索的问题, 可以预见, 在不久的将来, 将会涌现大量关于 ISTN 系统与网络研究的新思路. 然而, 研究人员应 如何构建一个实验网络环境(ENE)来测试, 评估并验证这些新想法?

Typically, existing approaches for creating an ENE can be classified into three categories: (1) live networks or platforms [7, 20, 34, 75, 81], which allow experiments in real deployments; (2) network simulation [60,61,76], which uses discrete events to model and replicate the behavior of a real network; and (3) network emulation [6, 55, 68, 69], which can test real applications/protocols in a virtual network. However, as will be illustrated in §3, all existing approaches have their limitations in creating a desired ENE for ISTNs: (1) the feasibility and flexibility of live satellite networks are technically and economically limited for normal researchers; (2) the abstraction level of simulation might be too high to capture low-level system effects, hiding practical issues such as the resource competition under heavy workload, energy drain or software errors; (3) existing emulators fail to characterize the high dynamicity of LEO satellites and thus are insufficient to build an experimental environment with acceptable fidelity.

通常, 构建 ENE 的现有方法可分为三类:

(1) 实地网络或平台 [7, 20, 34, 75, 81], 允许在真实部署中进行实验

(2) 网络模拟(Simulation) [60,61,76], 利用离散事件来建模并复现真实网络的行为

(3) 网络仿真(Emulation) [6, 55, 68, 69], 可在虚拟网络中测试真实的应用程序/协议

然而, 正如第 3 节所述, 所有现有方法在构建理想的 ISTN ENE 方面均存在局限性:

(1) 对于普通研究人员而言, 实地卫星网络的可行性与灵活性受到技术与经济条件的限制

(2) 模拟方法的抽象层级可能过高, 难以捕捉底层系统效应, 从而掩盖了高负载下的资源竞争, 能耗或软件错误等实际问题

(3) 现有的仿真器难以刻画低轨卫星的高动态特性, 因此不足以构建具有可接受保真度的实验环境

The key challenge of building an expected ENE for ISTN research is: it is difficult to simultaneously achieve realism and flexibility in the experimental environment. First, terrestrial devices inherently lack the ability to reasonably mimic the high dynamics, system and network behaviors of realistic satellites. Second, mega-constellations typically consist of thousands of satellites. Thus the network scale required by an ENE for mega-constellations might be far more than the extent supported by existing ENE methods (e.g., [54,55,69]). Third, as a large number of satellites simultaneously move at a high velocity, continuously mimicking such frequent variations at scale could involve significant system overhead on the ENE.

构建理想 ISTN ENE 的关键挑战 在于: 难以在实验环境中同时兼顾真实性与灵活性.

首先, 地面设备本质上缺乏合理模拟真实卫星的高动态, 系统及网络行为的能力

其次, 巨型星座通常包含数千颗卫星, 因此面向巨型星座的 ENE 所需的网络规模可能远超现有 ENE 方法所支持的范围(例如 [54,55,69])

第三, 由于大量卫星同时高速移动, 在大规模下持续模拟这种频繁的变化可能会给 ENE 带来巨大的系统开销

This paper presents STARRY NET, an integrated experimentation framework that empowers researchers to conveniently build ENEs with acceptable realism, flexibility and cost (e.g., requiring only a few number of local/cloud machines) to satisfy various experimental requirements of ISTNs. The design of STARRY NET is inspired by a key insight obtained from the satellite Internet ecosystem: many organizations or communities in this ecosystem have released and shared their constellation-relevant data, including regulatory information [2,73], satellite trajectories [16,38], ground station distribution [26,39] and measurements from user terminals [32,33], etc. Therefore, the key idea behind STARRY NET is to build an experimental digital twin, i.e., a virtual presentation of a physical ISTN, in terrestrial environments by: (1) leveraging terrestrial machines to virtualize a large number of lightweight virtual nodes to emulate satellites in mega-constellations; and (2) exploiting a crowdsourcing approach to collect, combine and use realistic constellation information to drive the emulation of spatial and temporal characteristics of ISTNs.

本文提出了 STARRYNET, 这是一种综合实验框架, 旨在赋能研究人员便捷地构建具有可接受的真实性, 灵活性及成本(例如仅需少量本地/云端机器)的 ENE, 以满足 ISTN 的各类实验需求.

STARRYNET 的设计灵感源自卫星互联网生态系统中的一个关键洞察: 该生态系统中的许多组织或社区已经发布并共享了其星座相关数据, 包括监管信息 [2,73], 卫星轨迹 [16,38], 地面站分布 [26,39] 以及用户终端的测量数据 [32,33] 等.

因此, STARRYNET 的核心思想是通过以下方式在地面环境中构建实验性数字孪生(即物理 ISTN 的虚拟呈现):

(1) 利用地面机器虚拟化大量轻量级虚拟节点, 以仿真巨型星座中的卫星

(2) 采用众包方法收集, 整合并利用真实的星座信息, 以驱动 ISTN 时空特性的仿真

To achieve acceptable realism, STARRY NET employs a constellation synchronizer based on realistic constellation-relevant information to make the virtual ENE as consistent as possible to a real ISTN, such as: (1) constellation consistency: the ENE is built with the same scale of a physical mega-constellation, where each node emulates a satellite, a ground station or a terrestrial host. The spatial and temporal characteristics, such as time-varying satellite locations and intervisibility, are also configured and updated in each node based on our data-driven model-based analysis; (2) system and networking stack consistency: the ENE can support the run of unmodified applications as in real deployments; and (3) capability consistency: network and computation capabilities in the ENE are configured based on real hardware specifications. Further, to flexibly support various ISTN experiments and mimic large-scale and highly-dynamic mega-constellations, STARRY NET adopts a constellation orchestrator that judiciously schedules and manages system resources on multiple machines to collaboratively construct ENE on demand.

为实现可接受的真实性, STARRYNET 采用星座同步器, 基于真实星座信息使虚拟 ENE 尽可能与真实 ISTN 保持一致, 具体包括:

1. 星座一致性:
   • ENE 按照物理巨型星座的同等规模构建, 其中每个节点仿真一颗卫星, 一个地面站或一台地面主机
   • 节点的时空特性(时变的卫星位置, etc.)也基于我们的数据驱动模型分析进行配置与更新
2. 系统与网络协议栈一致性:
   • ENE 支持像真实部署一样运行未修改的应用程序
3. 能力一致性:
   • ENE 中的网络与计算能力基于真实硬件规格进行配置

此外, 为了灵活支持各类 ISTN 实验并模拟大规模, 高动态的巨型星座, STARRYNET 采用星座编排器, 合理调度和管理多台机器上的系统资源, 按需协同构建 ENE

We evaluate the ability of STARRY NET based on real constellation information in two steps. First, we show the acceptable fidelity of STARRY NET by comparing the experiment results obtained by STARRY NET with live satellite networks and other state-of-the-art ISTN simulators. Second, facing futuristic ISTN scenarios, we demonstrate STARRY NET’s flexibility by conducting three case studies to: (1) explore the trade-space of various space-ground inter-networking mechanisms; (2) evaluate the resilience of routing protocols in various constellation designs; and (3) perform hardware-in-the-loop tests to measure system effects under various workloads.

Summarily, this paper makes the following key contributions: (1) we design STARRY NET, a data-driven, emulation-aided ISTN experimentation framework (§4); (2) we implement STARRY NET with a collection of open APIs for creating and manipulating user-defined ENEs (§5); (3) we evaluate and analyze STARRY NET’s experimentation overhead and fidelity (§6), and show STARRY NET’s flexibility (§7) by conducting various case studies driven by realistic constellation information. STARRY NET is now available at: https://github.com/SpaceNetLab/StarryNet.

我们分两步评估基于真实星座信息的 STARRYNET 的能力. 首先, 通过将 STARRYNET 的实验结果与实地卫星网络及其他最先进的 ISTN 模拟器进行比较, 展示其可接受的保真度. 其次, 面向未来的 ISTN 场景, 我们通过开展三个案例研究来展示 STARRYNET 的灵活性: (1) 探索各种天地互联机制的权衡空间; (2) 评估路由协议在不同星座设计中的弹性; (3) 执行硬件在环测试, 以测量不同工作负载下的系统效应.

综上所述, 本文做出了以下主要贡献:

(1) 设计了 STARRYNET, 一种数据驱动, 仿真辅助的 ISTN 实验框架

(2) 实现了 STARRYNET, 并提供了一组开放 API, 用于创建和操作用户定义的 ENE

(3) 评估并分析了 STARRYNET 的实验开销与保真度, 并通过基于真实星座信息的多种案例研究展示了其灵活性

STARRYNET 现已在 https://github.com/SpaceNetLab/StarryNet 开源

写作逻辑分析

可以尝试用递归的顺序来表达:

1. LEO 特别有前景
2. 前景的同时, 存在很多新问题与新挑战
3. 尝试解决这些问题/挑战, 需要一个实验网络仿真环境(ENE)
4. 现在的 ENE 方法都有各自的局限性
5. 构建理想的 ENE 有三个关键挑战

我们: 提出 StarryNet, 这是一个理想的 ENE

Preliminaries

Integrated space and terrestrial networks (ISTN). Recent satellite operators/organizations are actively developing their mega-constellations [4,8,9,25,34,36], with hundreds to thousands of low earth orbit (LEO) satellites working together as a system. These satellites can be equipped with high-speed intersatellite links (ISLs), and construct an LEO satellite network (LSN). An LSN can further be integrated into existing terrestrial Internet via globally distributed ground stations [3,26,39] and very-small-aperture terminals (VSAT) [31], constructing an integrated space and terrestrial network (ISTN) that promises to provide pervasive, low-latency, broadband Internet services [40,52,56,57] for terrestrial users globally.

空天地一体化网络: 近期的卫星运营商与组织正积极部署其巨型星座 [4,8,9,25,34,36], 该系统由成百上千颗协同运行的低地球轨道(LEO)卫星组成.

这些卫星可配备高速星间链路(ISLs), 从而构建低轨卫星网络(LSN). LSN 可进一步通过全球分布的地面站 [3,26,39] 和甚小孔径终端(VSAT) [31] 与现有地面互联网融合, 构建空天地一体化网络(ISTN), 旨在为全球地面用户提供泛在, 低延迟且高带宽的互联网服务 [40,52,56,57].

Unique characteristics of ISTNs, as well as the new challenges. Two critical characteristics differentiate LSNs from existing terrestrial networks, and involve new challenges on the integration of satellites and terrestrial Internet. First, LEO satellites are moving at a high-speed with the respect to the earth surface. An LEO satellite might be visible for a certain ground vantage point only within a few minutes in one pass. Such high dynamics could inevitably result in technical challenges (e.g., frequent connectivity disruptions and routing re-convergence) at the networking stack of ISTNs. Second, while evolved, resources (e.g., bandwidth, CPU, energy) are still limited and costly in space, as compared with terrestrial network systems. Resource-intensive technologies might not be doable for resource-constrained satellites to sustain good network performance (e.g., applying sophisticated network coding techniques for packet recovery in remote space).

ISTN 的独特特性及面临的新挑战: 两个关键特性使 LSN 区别于现有地面网络, 并给卫星与地面互联网的融合带来了新的挑战.

首先, 相对于地球表面, LEO 卫星处于高速运动状态. 对于特定的地面观测点, 一颗 LEO 卫星在单次过顶期间的可见时间通常仅为数分钟. 这种高动态性必然会在 ISTN 的网络协议栈层面引发技术挑战(例如频繁的连接中断与路由重收敛).

其次, 尽管技术已有进步, 但与地面网络系统相比, 空间环境中的资源(如带宽, 计算能力, 能源)依然受限且昂贵. 资源密集型技术可能不适用于资源受限的卫星以维持良好的网络性能(例如, 在遥远的空间环境中应用复杂的网络编码技术进行数据包恢复).

Call for new research for futuristic ISTNs. The above characteristics and challenges accordingly raise a series of unexplored research problems in ISTNs, such as: (1) topology: how should LEO satellites and ground facilities be interconnected under the high space-ground dynamicity? (2) routing: how should we integrate hundreds or thousands of LEO satellites into Internet routing and tackle the potential performance degradation due to intermittent connectivity? (3) system effect: how much energy would a new functionality consume in space under various workloads? It is foreseeable that in the near future, in parallel with the evolution of real mega-constellations, there would be a surge of new research focusing on emerging satellite network systems. But, how should researchers test, assess and understand their new thoughts? The community requires a technically and economically feasible approach to construct Experimental Network Environments (ENE) and advance futuristic ISTN research.

对未来 ISTN 研究的呼吁: 上述特性与挑战相应地引出了一系列 ISTN 领域尚未探索的研究问题, 例如:

(1) 拓扑: 在高动态的天地环境下, LEO 卫星与地面设施应如何互联?

(2) 路由: 如何将成百上千颗 LEO 卫星融入互联网路由体系, 并解决因间歇性连接导致的潜在性能下降问题?

(3) 系统效应: 在不同工作负载下, 新功能在空间环境中将消耗多少能源?

可以预见, 在不久的将来, 随着真实巨型星座的演进, 将涌现出大量聚焦于新兴卫星网络系统的研究.

然而, 研究人员应如何测试, 评估并验证他们的新思路?

学术界亟需一种在技术和经济上均可行的方法来构建实验网络环境(ENE), 从而推动面向未来的 ISTN 研究!!!

How Can Researchers Evaluate Their New Thoughts for ISTNs?

3.1 ENE Requirements

Ideally, an ENE built for ISTN research is expected to simultaneously accomplish acceptable realism and flexibility. We summarize four baseline requirements as follows.

• (1) Constellation-consistency. The ENE is expected to be spatially and temporally consistent to the characteristics of real mega-constellations. For example, the ENE is desired to mimic a large number of network nodes at the same scale of a real mega-constellation, and can characterize the high dynamicity of LEO satellites, as well as its impact on network conditions (e.g., connectivity, delay variations).

• (2) System-level and networking stack realism. The ENE is expected to run user-defined system codes and network functionalities like in a real system and networking stack.

• (3) Flexible and scalable environment. Emerging megaconstellations are evolving rapidly. As most state-of-the-art constellations are still not in their final stage, the ENE is expected to flexibly support various network topologies at different scales to meet diverse research requirements.

• (4) Open, low-cost and easy-to-use interface. Finally, it is expected that the ENE could be open to the community, and can provide low-cost and easy-to-use programmable interfaces for researchers to carry out various experiments.

理想情况下, 为空天地一体化网络(ISTN)研究构建的 ENE 应同时实现可接受的真实性和灵活性. 我们总结了以下四个基准要求:

(1) 星座一致性(Constellation-consistency)

ENE 应在时空特性上与真实巨型星座保持一致

例如, ENE 需模拟与真实巨型星座同等规模的大量网络节点, 并能刻画低轨卫星的高动态性, 以及该特性对网络状况(如连接性, 时延变化)的影响

(2) 系统级与网络协议栈的真实性(System-level and networking stack realism)

ENE 应能像在真实系统和网络协议栈中一样, 运行用户定义的系统代码和网络功能

(3) 灵活且可扩展的环境(Flexible and scalable environment)

新兴的巨型星座正在快速演进. 由于大多数最先进的星座仍未达到最终阶段, ENE 应能灵活支持不同规模的各种网络拓扑, 以满足多样化的研究需求

(4) 开放, 低成本且易用的接口(Open, low-cost and easy-to-use interface)

最后, 期望 ENE 能向社区开放, 并为研究人员开展各类实验提供低成本且易用的可编程接口

3.2 Why Existing ENEs are Insufficient?

Existing approaches for building an ENE can be classified into three categories, differing in their realism, flexibility and cost: (1)live LSNs/platforms, (2)simulators, and (3)emulators.

Live LSNs or platforms. A straightforward approach for ISTN experimentation is to construct an ENE based on live LSNs, e.g., recently SpaceX’s Starlink has started its initial services in certain regions. Although this approach guarantees good realism, directly manipulating and inspecting a live LSN might be technically and economically difficult for a common research group. Current live LSNs are also limited in their flexibility when they face diverse, exploratory research requirements. Realistic mega-constellations are still under-constructed and evolving rapidly, and their regulatory information in practice cannot be flexibly modified for what-if analysis. Further, the network community has many public experimentation platforms [7,20] that can be shared among researchers. However, these platforms are originally designed for tests in terrestrial networks, not for ISTNs, and thus cannot characterize the unique network behaviors under large-scale LEO dynamics.

Simulators and orbit analysis tools for ISTNs. Numerical or discrete-event-based simulation presents another extreme as compared with live LSNs and platforms. STK [35] and GMAT [11] are representative orbit analysis tools that can perform complex analysis of spacecrafts as well as ground stations. However, both STK and GMAT mainly focus on orbit and spacecraft analysis and have limited support for network simulation. More recently, SNS3 [76], Hypatia [60] and StarPerf [61] are emerging simulators for ISTNs. SNS3 is an extension to the ns-3 platform, and it models a full satellite network with a geostationary (GEO) satellite and bent-pipe payload. Hypatia is a framework for simulating and visualizing the network behavior of emerging mega-constellations. Similarly, StarPerf is a simulator that enables users to characterize, estimate and understand the achievable network performance under a variety of constellation options. Although the above simulators can flexibly simulate various satellite characteristics as well as the impact of high dynamics on network behaviors, a fundamental limitation of those simulators is that they can not support the run of system codes/functionalities and interactive network traffic as in real deployments. The abstraction-level of simulators might be too high to capture system-level effects, and could hide other practical issues (e.g., software overhead under heavy workloads) in real systems.

Network emulators, and their variations. Emulation is a hybrid approach that integrates real applications, protocols and operating systems in a synthetic network environment. Similar to live networks, emulators can load and run real codes with interactive network traffic. Similar to simulators, emulators can support controllable and diverse topologies and their virtual hardware requires fewer resources as compared with live networks. The community has many prior efforts focusing on emulated environments, e.g., VM- or containerbased emulation [6,45,54,55,68–70,72,79–81,84,85].

However, existing emulators suffer from two limitations when they are applied for generating ENEs for ISTN research. First, they are not constellation-consistent, since existing emulators inherently lack the ability of mimicking planet-wide LEO dynamics and time-varying network behaviors in ISTNs. Second, the network scale for mega-constellations could be significantly larger than that in prior experimentation. For example, authors in [54] use 10 physical machines (25 VMs on each) to support a networked cluster with 250 nodes. Etalon [69] is a container-based emulator and its local testbed uses four servers to emulate 48 hosts in a data center network. Different from prior scenarios, ISTN experiments require a much larger network environment: only the first shell of Starlink Phase-I includes about 1584 LEO satellites. Since both VMs and containers can involve software overhead on the physical host machine, it is difficult for existing emulators (e.g., [54, 55, 68]) to support such large-scale and dynamic emulations for mega-constellations.

Our motivation. Table 1 summarizes the landscape of existing experimentation approaches that can be used to build ENEs. Collectively, we find that none of existing approaches can simultaneously satisfy the four expected features. Limitations of existing approaches thus motivate us to seek for a constellation-consistent, credible, flexible, and low-cost methodology to advance the test and evaluation of new research for futuristic ISTNs. We present such a framework, namely STARRY NET, aiming at empowering researchers to build ENEs accomplishing the four goals as described in §3.1.

现有的 ENE 构建方法可根据其真实性, 灵活性和成本分为三类: (1) 实地低轨卫星网络(LSN)/平台, (2) 模拟器(Simulators), 以及 (3) 仿真器(Emulators).

(1) 实地 LSN 或平台:

进行 ISTN 实验的一个直接方法是基于实地 LSN 构建 ENE, 例如 SpaceX 的 Starlink 近期已在特定区域启动初始服务.

尽管该方法保证了良好的真实性, 但对于普通研究团队而言, 直接操作和检测实地 LSN 在技术和经济上都具有困难. 当前的实地 LSN 在面对多样化, 探索性的研究需求时, 其灵活性也受到限制. 现实中的巨型星座仍处于建设中且快速演进, 其实际的监管信息无法灵活修改以进行假设(what-if)分析. 此外, 网络社区拥有许多可在研究人员间共享的公共实验平台 [7,20].

然而, 这些平台最初是为地面网络测试而设计的, 而非针对 ISTN, 因此无法刻画大规模低轨卫星动态下的独特网络行为.

(2) ISTN 模拟器与轨道分析工具:

与实地 LSN 和平台相比, 数值模拟或基于离散事件的模拟代表了另一种极端.

STK [35] 和 GMAT [11] 是代表性的轨道分析工具, 可对航天器及地面站进行复杂分析. 然而, STK 和 GMAT 主要关注轨道和航天器分析, 对网络模拟的支持有限.

近期, SNS3 [76], Hypatia [60] 和 StarPerf [61] 成为了新兴的 ISTN 模拟器. SNS3 是 ns-3 平台的扩展, 它模拟了一个包含地球静止轨道(GEO)卫星和弯管转发(bent-pipe)载荷的完整卫星网络. Hypatia 是一个用于模拟和可视化新兴巨型星座网络行为的框架.

同样, StarPerf 是一个允许用户在多种星座选项下刻画, 估算并理解可达网络性能的模拟器.

尽管上述模拟器能灵活模拟各种卫星特性及高动态性对网络行为的影响, 但其 根本局限 在于:

无法像真实部署那样支持系统代码/功能运行及交互式网络流量

模拟器的抽象层级可能过高, 难以捕捉系统级效应, 并可能掩盖真实系统中的其他实际问题(如高负载下的软件开销)

(3) 网络仿真器及其变体:

仿真是一种混合方法, 它在合成的网络环境中集成真实的应用程序, 协议和操作系统.

与实地网络类似, 仿真器可以加载并运行带有交互式网络流量的真实代码. 与模拟器类似, 仿真器支持可控且多样的拓扑结构, 且其虚拟硬件所需的资源少于实地网络. 社区此前已有许多致力于仿真环境的工作, 例如基于虚拟机(VM)或容器的仿真 [6,45,54,55,68-70,72,79-81,84,85].

然而, 现有的仿真器在应用于生成 ISTN 研究的 ENE 时存在 两点局限

1. 它们不具备星座一致性, 因为现有仿真器本质上缺乏模拟 ISTN 中全球范围低轨卫星动态及随时间变化的网络行为的能力
2. 巨型星座的网络规模可能远大于以往实验的规模
   • 文献 [54] 的作者使用 10 台物理机(每台运行 25 个虚拟机)来支持一个包含 250 个节点的网络集群
   • Etalon [69] 是一个基于容器的仿真器, 其本地测试床使用 4 台服务器来仿真数据中心网络中的 48 台主机
   • 与以往的场景不同, ISTN 实验需要大得多的网络环境: 仅 Starlink 第一阶段的第一层就包含约 1584 颗低轨卫星
   • 由于虚拟机和容器都会给物理主机带来软件开销, 现有的仿真器难以支持如此大规模且动态的巨型星座仿真

(4) 我们的动机:

表 1 总结了可用于构建 ENE 的现有实验方法的概况. 总的来说, 我们发现现有方法均无法同时满足上述四个预期特性.

现有方法的局限性促使我们寻求一种具有星座一致性, 可信, 灵活且低成本的方法, 以推进面向未来的 ISTN 新研究的测试与评估. 我们提出了这样一个框架, 即 STARRYNET, 旨在赋能研究人员构建实现 3.1 节所述四个目标的 ENE.

STARRY NET Design

4.1 System Overview

Key idea. The design of STARRY NET is inspired by an important insight obtained from the satellite Internet ecosystem: many organizations (e.g., regulators and satellite operators) and end users have shared a collection of public data for the community, including constellation regulatory information [2,73], orbital data observed from realistic satellites [16, 38], ground station distributions [3, 26, 39], and performance results measured from terrestrial user terminals [32,33], etc. Based on this important fact, STARRY NET creates ISTN experimental environments on demand by judiciously combining: (1) crowd-sourced real data trace; (2) model-based orbit and network analysis; and (3) large-scale network system emulation, to construct a real-data-driven digital twin, i.e., a virtual presentation synchronized to a real physical ISTN in terrestrial environments. In particular, the key idea behind our STARRY NET design can be summarized as follows: (1) leveraging a crowd-sourcing approach to collect, combine and explore realistic constellation-relevant information to calculate the spatial and temporal characteristics consistent to real mega-constellations; then (2) driven by such realistic information, exploiting a large number of networked virtual nodes and links to flexibly emulate a customized experimental environment, which characterizes system-level effects and network behaviors consistent to a real ISTN.

System architecture. Figure 1 depicts STARRY NET’s architecture, including four core components as described below.

• A Constellation Observer (§4.2) that leverages a crowdsourcing approach to collect public constellation information, network performance, ground station distributions etc., from the satellite ecosystem. It maintains several databases to support, guide and drive the construction of ISTN experimental environments for various research requirements.

• A Constellation Synchronizer (§4.3) which exploits a collection of hybrid models to calculate the spatial and temporal characteristics of a specific mega-constellation, based on collected data as well as user-defined configurations. Specifically, such characteristics include constellation scale, visibility, connectivity, time-varying propagation delay, etc., which are further used to configure the network emulation.

• A Constellation Orchestrator (§4.4) for automating the management, coordination and allocation for resources used to build the experimental environment upon multiple physical machines. The orchestrator can also interact with realworld facilities (e.g., real satellite hardware) to support network experiments with interactive Internet traffic.

• A Unified Abstraction (§4.5) offering flexible and easyto-use APIs for researchers to create and manipulate the configurations of the ISTN experimental environment.

核心思想:

STARRYNET 的设计灵感源自卫星互联网生态系统中的一个重要洞察: 许多组织(如监管机构和卫星运营商)及终端用户已为社区共享了大量公开数据, 包括星座监管信息 [2,73], 从真实卫星观测到的轨道数据 [16, 38], 地面站分布 [3, 26, 39] 以及从地面用户终端测得的性能结果 [32,33] 等.

基于这一重要事实, STARRYNET 通过合理结合以下三点来按需创建 ISTN 实验环境:

(1) 众包的真实数据轨迹

(2) 基于模型的轨道与网络分析

(3) 大规模网络系统仿真, 从而在地面环境中构建一个数据驱动的数字孪生, 即与真实物理 ISTN 保持同步的虚拟呈现

具体而言, STARRYNET 设计背后的核心思想可概括为:

(1) 利用众包方法收集, 整合并探索真实的星座相关信息, 以计算与真实巨型星座一致的时空特性

(2) 在此真实信息的驱动下, 利用大量网络化的虚拟节点和链路灵活仿真定制的实验环境, 使其刻画出与真实 ISTN 一致的系统级效应和网络行为

系统架构:

图 1 描绘了 STARRYNET 的架构, 包含以下四个核心组件:

1. 星座观测器(Constellation Observer, §4.2):
   • 利用众包方法从卫星生态系统中收集公开的星座信息, 网络性能, 地面站分布等
   • 维护多个数据库, 以支持, 引导并驱动满足各类研究需求的 ISTN 实验环境的构建
2. 星座同步器(Constellation Synchronizer, §4.3):
   • 基于收集到的数据及用户定义的配置, 利用一组混合模型计算特定巨型星座的时空特性
   • 具体而言, 这些特性包括星座规模, 可见性, 连接性, 随时间变化的传播时延
   • 这些特性将进一步用于配置网络仿真
3. 星座编排器(Constellation Orchestrator, §4.4):
   • 用于自动化管理, 协调和分配多台物理机器上的资源, 以构建实验环境
   • 该编排器还可与现实世界的设施(如真实的卫星硬件)交互, 以支持带有交互式互联网流量的网络实验
4. 统一抽象层(Unified Abstraction, §4.5):
   • 为研究人员提供灵活且易用的 API, 用于创建和操作 ISTN 实验环境的配置

4.2 Constellation Observer

The constellation observer is designed as a collector for constellation-relevant information, and it maintains three databases related to satellite, ground station and user terminal information respectively. Specifically, the observer searches and collects the latest: (1) regulatory information (e.g., from [2,10,73]) which describes frequency and orbital coordination of mega-constellations; (2) operating satellites information (e.g., from [16,38,71]); (3) ground station distributions (e.g., from [3,26,39]); (4) Internet user statistics (e.g., from [59]) which can be used to generate the distribution of terrestrial users; and (5) network measurements from end users (e.g., from [32,33]). The constellation observer classifies the above information, and saves them in the databases, which then can be used to drive other components and build ENEs to flexibly support various research experiments.

星座观测器被设计为星座相关信息的收集器, 并维护三个分别与卫星, 地面站及用户终端信息相关的数据库

具体而言, 观测器负责搜索并收集最新的:

(1) 监管信息(例如来自 [2,10,73]), 描述巨型星座的频率和轨道协调情况

(2) 在轨卫星信息(例如来自 [16,38,71])

(3) 地面站分布(例如来自 [3,26,39])

(4) 互联网用户统计数据(例如来自 [59]), 可用于生成地面用户分布

(5) 来自终端用户的网络测量数据(例如来自 [32,33])

星座观测器对上述信息进行分类并存入数据库, 随后这些数据可用于驱动其他组件并构建 ENE, 以灵活支持各类研究实验

4.3 Constellation Synchronizer

STARRY NET’s synchronizer leverages a collection of models to calculate various constellation characteristics and network behaviors, and accordingly generate a virtual network presentation synchronized to a real ISTN. At runtime, values in these models are assigned primarily based on real ISTN information collected by the constellation observer. In addition, to improve the flexibility, STARRY NET also allows researchers to manually configure model values based on his or her customized experimental requirements.

4.3.1 Hybrid models

Constellation model. STARRY NET’s synchronizer characterizes a satellite system in both constellation-granularity and orbit-granularity descriptions. First, STARRY NET uses the Walker notation [82] to describe a constellation: N/P/p, where N is the number of satellites per plane, P is the number of planes, and p is the number of distinct phases of planes to control spacing offsets in planes. Second, the orbit-granularity description enables a fine-grain notation for orbits in a certain constellation via specifying several primary orbital elements, including: (1) inclination, which is the angle between an orbit and the Equator as satellites move; (2) altitude, which is the height above sea level and determines the orbital velocity; (3) phase offset [56], which is a factor between [0,1], describing the relative position between two satellites in adjacent orbits. Ground station model. STARRY NET leverages the following parameters to describe a ground station: (1) geographic location; (2) the number of available antennas for ground communication; (3) the elevation angle, which determines the light-of-insight (LoS) of the ground station and can affect the available duration of space-ground communication. Network model. The inter-satellite or ground-satellite network connectivity is mainly affected by following factors: (1) the visibility between two communication ends; (2) the amount of available ISLs or antennas in satellites or ground stations; and (3) the connectivity policy, which decides how to establish a connection between two communication ends. STARRY NET enables two prefabs of connectivity policies for inter-satellite connection: (1) +Grid [58], where each satellite connects to two adjacent satellites in the same orbit and to two in adjacent orbits; (2) Motif [40], which is a repetitive pattern where each satellite connects to multiple visible satellites and each satellite’s local view is the same as that of any other. For ground-satellite connectivity, STARRY NET offers multiple optional schemes that control a ground station to connect to a visible satellite, e.g., selecting the one with the shortest distance or the longest remaining visible time. Since the delay is typically determined by the network topology and speed of the light, STARRY NET calculates the propagation delay of each link based on the physical distance between two ends. As network capacity might be too speculative in practice, link capacity is set by user-specific configurations. Computation model. The computation capability of satellites varies greatly in different real-world deployments. Generally, conventional space-grade processors have limited capability [53,65] as compared with that used in terrestrial computer systems. For example, the operating frequency of spaceborne processors (e.g., BAE-RAD series [21,22]) ranges from 110 to 466MHz per core. Recently, satellite operators and researchers start to use commercial off-the-shelf (COTS) processors in space stations [14] or in LEO small satellites [23,44] to reduce the manufacturing cost. To support various experimental requirements, STARRY NET allows researchers to manually configure the CPU capability of each satellite node through approximating the frequency and number of available cores of each emulated node, by scaling download CPU frequency and enforcing a maximum time quota for each node.

4.3.2 Constellation consistency

Spatial consistency. Based on constellation-wide information, STARRY NET first determines the amount of required containers. Each container runs realistic system environments and networking stack to emulate a satellite in the constellation, a ground station or a terrestrial user terminal. Second, using orbit-granularity information, STARRY NET calculates the latitude, longitude and height (i.e., LLH information) of each satellite at any given time slot. Finally, exploiting the above LLH information and minimum elevation angles, S TAR RY N ET calculates the visibility between arbitrary two satellites, or between satellites and ground facilities.

Temporal consistency. Since emerging LEO satellites are inherently moving at a high velocity, the locations, intervisibility, and network conditions (e.g., connectivity and propagation delay between two nodes) of an ISTN are changing over time. To achieve realism, these states should be temporally consistent to real scenarios. STARRY NET splits time into slots, calculates LLH information in each slot, and follows the hybrid models to determine time-varying network conditions in different slots. Such dynamic states will further be used for driving the dynamic emulation operated by the orchestrator.

STARRYNET 的同步器利用一组模型来计算各种星座特性和网络行为, 并据此生成与真实 ISTN 同步的虚拟网络呈现. 在运行时, 这些模型中的数值主要基于星座观测器收集的真实 ISTN 信息进行赋值. 此外, 为提高灵活性, STARRYNET 也允许研究人员根据其定制的实验需求手动配置模型参数.

4.3.1 混合模型

星座模型. STARRYNET 的同步器通过星座粒度和轨道粒度两种描述来刻画卫星系统. 首先, STARRYNET 使用 Walker 符号 [82] 来描述星座: N/P/p, 其中 N 为每轨道平面的卫星数量, P 为轨道平面数量, p 为用于控制平面间相位偏移的独特相位数. 其次, 轨道粒度描述通过指定几个主要轨道要素, 实现了对特定星座中轨道的细粒度标记, 包括: (1) 倾角(inclination), 即卫星运行时轨道与赤道之间的夹角; (2) 高度(altitude), 即海拔高度, 决定了轨道速度; (3) 相位偏移(phase offset) [56], 是一个介于 [0,1] 之间的因子, 描述相邻轨道上两颗卫星之间的相对位置.

地面站模型. STARRYNET 利用以下参数描述地面站: (1) 地理位置; (2) 用于地面通信的可用天线数量; (3) 仰角(elevation angle), 该角度决定了地面站的视线(LoS), 并影响空地通信的可用时长.

网络模型. 星间或星地网络连接性主要受以下因素影响: (1) 通信两端之间的可见性; (2) 卫星或地面站中可用星间链路(ISL)或天线的数量; 以及 (3) 连接策略, 决定如何建立通信两端之间的连接. STARRYNET 为星间连接提供了两种预设连接策略: (1) +Grid [58], 即每颗卫星连接同一轨道上的两颗相邻卫星以及相邻轨道上的两颗卫星; (2) Motif [40], 这是一种重复模式, 即每颗卫星连接多颗可见卫星, 且每颗卫星的局部视图与其他卫星相同. 对于星地连接, STARRYNET 提供了多种可选方案来控制地面站连接至可见卫星, 例如选择距离最短或剩余可见时间最长的卫星. 由于时延通常由网络拓扑和光速决定, STARRYNET 基于两端间的物理距离计算每条链路的传播时延. 鉴于网络容量在实际中可能难以准确推测, 链路容量由用户特定配置设定.

计算模型. 不同现实部署中卫星的计算能力差异巨大. 通常, 传统的航天级处理器相较于地面计算机系统而言能力有限 [53,65]. 例如, 星载处理器(如 BAE-RAD 系列 [21,22])的主频范围为每核心 110 至 466MHz. 近年来, 卫星运营商和研究人员开始在空间站 [14] 或 LEO 小卫星 [23,44] 中使用商用现成(COTS)处理器以降低制造成本. 为支持各类实验需求, STARRYNET 允许研究人员通过缩放下载 CPU 频率并强制每个节点的最大时间配额, 来近似每个仿真节点的频率和可用核心数, 从而手动配置每个卫星节点的 CPU 能力.

4.3.2 星座一致性

空间一致性. 基于全星座信息, STARRYNET 首先确定所需的容器数量. 每个容器运行真实的系统环境和网络协议栈, 以仿真星座中的一颗卫星, 一个地面站或一个地面用户终端. 其次, 利用轨道粒度信息, STARRYNET 计算任意给定时间槽内每颗卫星的纬度, 经度和高度(即 LLH 信息). 最后, 利用上述 LLH 信息和最小仰角, STARRYNET 计算任意两颗卫星之间或卫星与地面设施之间的可见性.

时间一致性. 由于新兴的 LEO 卫星本质上以高速移动, ISTN 的位置, 互视性及网络状况(如两个节点间的连接性和传播时延)随时间不断变化. 为实现真实性, 这些状态应在时间上与真实场景保持一致. STARRYNET 将时间划分为若干时槽, 计算每个时槽内的 LLH 信息, 并遵循混合模型确定不同时槽内随时间变化的网络状况. 这些动态状态将进一步用于驱动由编排器操作的动态仿真.

4.4 Constellation Orchestrator

The orchestrator is designed for four goals. First, the orchestrator exploits container-based emulation to construct an emulated ISTN environment upon one or many physical machines (depending on the constellation size). Second, the orchestrator configures the computation and network capability of each emulated node, according to users’ configurations and the spatial and temporal results calculated by the constellation synchronizer. For example, a space-ground connectivity should be dynamically updated based on the time-varying visibility between its two ends. Third, the orchestrator can connect the emulated ISTN to real-world network facilities (if any, e.g., real satellite prototypes or terminals) and support interactive Internet traffic. Finally, at runtime the orchestrator leverages a measurement agent to monitor and report the runtime resource usage (e.g., CPU/memory/bandwidth usage) to the user as a feedback of the experiment for further analysis.

编排器的设计旨在实现四个目标:

首先, 编排器利用基于容器的仿真技术, 在一台或多台物理机器(取决于星座规模)上构建仿真的 ISTN 环境

其次, 编排器根据用户配置以及星座同步器计算的时空结果, 配置每个仿真节点的计算和网络能力. 例如, 星地连接应基于两端间随时间变化的可见性进行动态更新

第三, 编排器可将仿真的 ISTN 连接至现实世界的网络设施(若有, 例如真实的卫星原型或终端), 并支持交互式互联网流量

最后, 在运行时, 编排器利用测量代理监控并向用户报告运行时资源使用情况(如 CPU/内存/带宽使用率), 作为实验反馈以供进一步分析

4.4.1 Multi-machine support for constellation emulation

Since each emulated satellite consumes computation, network and storage resources in practice, it is challenging to support the emulation of large-scale constellations on a single machine, especially when additional user-defined payloads (e.g., a new satellite routing protocol) need to be loaded for experimentation. STARRY NET addresses this limitation by integrating resources on multiple machines to support largescale, time-varying constellation emulations. When deployed on multiple machines, STARRY NET’s orchestrator divides these machines into two parts. First, one machine, selected as the resource manager, manages, schedules and allocates resources upon all machines to jointly create and maintain the ENE. Second, other machines, working as worker clusters, receive and follow commands from the resource manager. For each node in the ISTN (e.g., a satellite or ground station), the orchestrator creates a container to emulate it, and creates a virtual bridge connecting two nodes to emulate a link. Topology creation on multiple machines. One big challenge made by extending an emulated mega-constellation to multiple machines is to ensure that the emulated constellation is topologically consistent to the real constellation. Figure 2 shows an example of emulating an LEO satellite constellation including two adjacent orbits on two physical machines. Specifically, Figure 2a plots two Starlink inclined orbits, each of them having 22 satellites evenly spaced with available ISLs. Assume that we use two machines for this emulation, and each machine creates 22 containers to emulate 22 satellites. In a real constellation, each satellite connects its two neighbors in the same orbit, and to another satellite in the adjacent orbit. Ideally, an emulated topology for the constellation in Figure 2a can be easily created if each machine has more than 22 physical network interfaces. However, in practice the number of available interfaces of a machine is likely to be limited. As shown in Figure 2b, if we directly bridge the emulated network interface of each satellite to the physical interface, due to the lack of traffic isolation, emulated satellites in two machines will establish an "all-to-all" topology which is inconsistent to the grid-like topology in Figure 2a .

STARRY NET addresses this inconsistency on multiple machines by creating multiple VLANs to emulate independent ISLs, interconnect satellites in two machines and isolate intersatellite traffic as that in a real constellation. As plotted in Figure 2c, we build a VLAN for each ISL crossing different machines (e.g., vlink A1-B1 interconnects emulated satellite A1 and B1), and thus STARRY NET obtains the correct virtual network topology consistent to the real constellation topology as illustrated in Figure 2a upon multiple machines. Topology update on multiple machines. Due to the high dynamics of ISTN, the network topology fluctuates over time. If a connectivity change occurs on a single machine, i.e., all affected nodes are located on the same machine, STARRY NET deletes the old virtual link, and creates a new link connecting corresponding nodes. Otherwise, if a connectivity change involves nodes on multiple machines, STARRY NET exploits a VLAN-based approach to limit the link update operation to a single machine. Figure 3 plots an example of the emulation of space-ground handovers upon multiple machines. Assume there are three satellites, two ground stations in two sequential time slots. In the first slot, satellite S2/S3 connects to ground station GS1/GS2 respectively. As satellites move, the space-ground connectivity changes, and S1/S2 connects to GS1/GS2 respectively in the second slot. STARRY NET emulates the ground-satellite links (GSLs) by two VLANbased virtual links, and performs the correct handover by properly adjusting the connectivity change between S1/S2/S3 and vlink-GS1/GS2, in different slots on machine A.

由于每个仿真卫星在实际中都会消耗计算, 网络和存储资源, 因此在单台机器上支持大规模星座仿真具有挑战性, 特别是当需要加载额外的用户定义载荷(如新的卫星路由协议)进行实验时. STARRYNET 通过整合多台机器上的资源来解决这一限制, 以支持大规模, 时变的星座仿真. 当部署在多台机器上时, STARRYNET 的编排器将这些机器分为两部分.

首先, 选定一台机器作为资源管理器, 负责管理, 调度和分配所有机器上的资源, 以协同创建并维护 ENE.

其次, 其他机器作为工作集群, 接收并执行来自资源管理器的指令.

对于 ISTN 中的每个节点(如卫星或地面站), 编排器创建一个容器对其进行仿真, 并创建一个连接两个节点的虚拟网桥来仿真链路.

多机拓扑创建:

将仿真的巨型星座扩展至多台机器的一大挑战在于确保仿真星座在拓扑上与真实星座保持一致. 图 2 展示了在两台物理机器上仿真包含两个相邻轨道的 LEO 卫星星座的示例:

具体而言, 图 2a 绘制了两个 Starlink 倾斜轨道, 每个轨道有 22 颗均匀分布且具有可用 ISL 的卫星. 假设我们使用两台机器进行此仿真, 每台机器创建 22 个容器来仿真 22 颗卫星.

在真实星座中, 每颗卫星连接其同一轨道上的两个邻居, 以及相邻轨道上的另一颗卫星.

理想情况下, 如果每台机器拥有超过 22 个物理网络接口, 图 2a 中的仿真拓扑可以很容易地创建. 然而, 实际上机器的可用接口数量往往有限.

如图 2b 所示, 如果我们直接将每颗卫星的仿真网络接口桥接到物理接口, 由于缺乏流量隔离, 两台机器上的仿真卫星将建立一种与图 2a 中的网格状拓扑不一致的"全对全"拓扑.

STARRYNET 通过创建多个 VLAN 来仿真独立的 ISL, 互联两台机器上的卫星并像真实星座那样隔离星间流量, 从而解决了多机环境下的这种不一致性. 如图 2c 所示, 我们为每个跨机器的 ISL 构建一个 VLAN(例如, vlink A1-B1 互联仿真卫星 A1 和 B1), 因此 STARRYNET 可以在多台机器上获得与图 2a 所示真实星座拓扑一致的正确虚拟网络拓扑.

多机拓扑更新:

由于 ISTN 的高动态性, 网络拓扑随时间波动. 如果连接变化发生在单台机器上, 即所有受影响的节点位于同一台机器, STARRYNET 删除旧的虚拟链路, 并创建连接相应节点的新链路.

否则, 如果连接变化涉及多台机器上的节点, STARRYNET 采用基于 VLAN 的方法将链路更新操作限制在单台机器上.

图 3 绘制了多机环境下星地切换仿真的示例:

假设有两个连续时间槽, 涉及三颗卫星和两个地面站

在第一个时槽中, 卫星 S2/S3 分别连接至地面站 GS1/GS2. 随着卫星移动, 星地连接发生变化;

在第二个时槽中 S1/S2 分别连接至 GS1/GS2;

STARRYNET 通过两条基于 VLAN 的虚拟链路仿真星地链路(GSL), 并通过在机器 A 上正确调整不同时槽内 S1/S2/S3 与 vlink-GS1/GS2 之间的连接变化来执行正确的切换

4.4.2 Efficient time synchronization and state update

Another challenge made by multi-machine extension is the time synchronization and link update overhead. Specifically, to achieve temporal consistency, STARRY NET’s constellation synchronizer assigns a sequence of update events to the orchestrator to inform each emulated satellite when an update (e.g., a location/connectivity change) should happen. To trigger such events, a baseline approach is to let the centralized manager send commands to all emulated satellites in every slot, and trigger corresponding update events. However, such a real-time approach has limited scalability as the amount of emulated satellites increases, since each event requires to update the state of a certain virtual interface/container, and continuously and simultaneously performing a large number of updates can overload the manager, result in delayed update, and invalidate the temporal consistency of the emulation.

To reduce the state update overhead caused by mimicking temporal dynamics in mega-constellations, STARRY NET leverages a prediction-based multi-thread event memorization approach. We define the synodic period as the amount of time that it takes for the constellation to reappear at the same projection upon the earth surface. In the emulated environment, at the beginning of each synodic period, the orchestrator pregenerates an event list for each satellite that includes all its update events during the current synodic period. At runtime, each emulated satellite adopts an independent event update thread to read the local event list in each slot, and triggers the corresponding event scheduled in the current slot. Time clocks upon all machines are synchronized by the NTP [17].

多机扩展带来的另一个挑战是时间同步与链路更新开销. 具体而言, 为实现时间一致性, STARRYNET 的星座同步器向编排器分配一系列更新事件, 以通知每个仿真卫星何时应发生更新(如位置/连接变化).

为触发此类事件, 一种基准方法是让中心化管理器在每个时槽向所有仿真卫星发送指令, 并触发相应的更新事件. 然而, 随着仿真卫星数量的增加, 这种实时方法的可扩展性有限, 因为每个事件都需要更新特定虚拟接口/容器的状态, 持续且同时执行大量更新可能会使管理器过载, 导致更新延迟, 并使仿真的时间一致性失效.

为降低模拟巨型星座时间动态引起的状态更新开销, STARRYNET 利用基于预测的多线程事件记忆方法. 我们将会合周期(synodic period)定义为星座在地球表面重新出现相同投影所需的时间量. 在仿真环境中, 在每个会合周期开始时, 编排器为每颗卫星预生成一个事件列表, 其中包含当前会合周期内的所有更新事件. 在运行时, 每颗仿真卫星采用独立的事件更新线程在每个时槽读取本地事件列表, 并触发当前时槽调度的相应事件.

所有机器上的时钟通过 NTP [17] 进行同步.

4.5 Open APIs for ISTN Experiments

Environment APIs. STARRY NET provides the environment APIs for a researcher to load trace from the database, and create/control/run an ENE upon one or multiple machines with user-defined ISTN configurations. Once constellation and GS information are completely loaded, these configurations are delivered to the synchronizer to calculate spatial and temporal characteristics, which are further used by the orchestrator to construct the ENE. The environment APIs also allow users to configure the interval of discrete time slot to adjust the dynamicity. Note that the ENE not only maintains the runtime of emulated constellations, but also needs to run the test workload deployed by the researcher. We design a resource threshold ∆ to control the percentage of CPU/memory used by the framework. In other words, at least (100%-∆) of the total CPU/memory should be left for the researcher’s test cases. Self-node APIs. STARRY NET’s self-node APIs are designed to be called by the researcher’s programs on each emulated satellite. These APIs expose underlying satellite-related information to user programs. Specifically, in each emulated satellite, user program can obtain the index of current satellite/orbit, sunlight state, time-varying geo-location information, current satellite velocity and the index of adjacent reachable satellites, etc. Such satellite-specific information can be used for developing new on-board capabilities in ISTNs.

环境 API:

STARRYNET 提供环境 API, 供研究人员从数据库加载轨迹, 并在单台或多台机器上使用用户定义的 ISTN 配置创建/控制/运行 ENE. 一旦星座和 GS 信息加载完毕, 这些配置将被传递给同步器以计算时空特性, 编排器将进一步利用这些特性构建 ENE. 环境 API 还允许用户配置离散时间槽的间隔以调整动态性

注意, ENE 不仅维护仿真星座的运行, 还需运行研究人员部署的测试工作负载. 我们设计了一个资源阈值 \(\Delta\) 来控制框架使用的 CPU/内存百分比. 换言之, 总 CPU/内存的至少 (100% - \(\Delta\)) 应留给研究人员的测试用例

自身节点 API:

STARRYNET 的自身节点 API 设计用于被每颗仿真卫星上的研究人员程序调用. 这些 API 向用户程序暴露底层的卫星相关信息

具体而言, 在每颗仿真卫星中, 用户程序可以获取当前卫星/轨道的索引, 日照状态, 随时间变化的地理位置信息, 当前卫星速度以及相邻可达卫星的索引等. 此类特定于卫星的信息可用于开发 ISTN 中新的星载功能

Implementation and Usage

We highlight the salient aspects of STARRY NET’s implementation and usage in this section.

Framework implementation. STARRY NET’s observer is implemented as a combination of a crawler based on Scrapy [27], together with a MySQL-based data store. We implement STARRY NET’s synchronizer based on SkyField [29], an astronomy library that supports the calculation of high precision research-grade positions for satellites. STARRY NET’s orchestrator is implemented upon Docker [6] and it spans the emulated constellation across multiple machines. We use OpenvSwitch [19] to emulate and configure links, and use tc [67] to dynamically set artificial network conditions according to the numeric results calculated by STARRY NET’s synchronizer. Specifically, we optimized the link management module in tc to satisfy the requirement of light-weight state update. To accomplish flexibility, S TAR RY N ET ’s abstraction is implemented as a combination of a lib-STARRY NET library and a collection of shell commands. Collectively, the core components of STARRY NET are implemented in about 6500 lines of Python codes and scripts.

框架实现:

STARRYNET 的观测器是基于 Scrapy [27] 的爬虫与基于 MySQL 的数据存储相结合实现的. 我们基于 SkyField [29] 实现了 STARRYNET 的同步器, 这是一个支持计算高精度科研级卫星位置的天文学库. STARRYNET 的编排器基于 Docker [6] 实现, 并将仿真的星座跨越多台机器进行部署. 我们使用 OpenvSwitch [19] 来仿真和配置链路, 并利用 tc [67] 根据 STARRYNET 同步器计算出的数值结果动态设置人工网络条件. 具体而言, 我们优化了 tc 中的链路管理模块, 以满足轻量级状态更新的需求. 为实现灵活性, STARRYNET 的抽象层由 lib-STARRYNET 库与一系列 Shell 命令组合实现. 总体而言, STARRYNET 的核心组件由约 6500 行 Python 代码及脚本实现.

Framework usage. We illustrate the usage of STARRY NET with a concrete example as plotted in Figure 4: a researcher wants to evaluate a novel geo-location-based routing mechanism based on [63], under the Starlink constellation. In particular, this experiment can be conducted with STARRY NET in three steps. First, leveraging STARRY NET’s APIs, the researcher writes a user-defined experimental program (Figure 4a) for test. In this example, we show a geo-routing policy similar to [63], which runs on each satellite, and forwards received packets to the adjacent satellite that is the geographically closest to the destination. Second, the researcher prepares a set of manifest files describing the constellation configurations, e.g., orbital information and ground station distribution (Figure 4b). Finally, the researcher runs a batch of shell commands exposed by STARRY NET to load manifest files (e.g., starlink.json and gs.json), create experimental environment (e.g., "sl_con") on multiple machines, configure network parameters (e.g., uplink/downlink capacity), and run the user-specific program on emulated satellites (Figure 4c).

框架使用:

我们通过图 4 所示的一个具体示例来说明 STARRYNET 的用法: 一位研究人员希望在 Starlink 星座下, 评估一种基于地理位置的新型路由机制(参考 [63]). 具体而言, 利用 STARRYNET 进行该实验可分为三个步骤.

首先, 利用 STARRYNET 的 API, 研究人员编写一个用户定义的实验程序(图 4a)用于测试. 在此示例中, 我们展示了一种类似于 [63] 的地理路由策略, 该策略运行于每颗卫星上, 并将接收到的数据包转发给地理位置上最接近目的地的相邻卫星.

其次, 研究人员准备一组描述星座配置的清单文件(manifest files), 例如轨道信息和地面站分布(图 4b).

最后, 研究人员运行一批由 STARRYNET 暴露的 Shell 命令, 以加载清单文件(如 starlink.json 和 gs.json), 在多台机器上创建实验环境(如"sl_con"), 配置网络参数(如上行/下行链路容量), 并在仿真卫星上运行用户特定的程序(图 4c).

积累实验场景:

Hardware-in-the-loop Testing:

In real satellite deployments, it is very important to accurately estimate how much energy a new system or network function will consume before the launch. STARRY NET enables researchers to conduct hardware-in-the-loop testing to accurately evaluate the low-level system effects under various workloads. As a case study to demonstrate STARRY NET’s ability, we connect a 3U CubeSat prototype, equipped with a low-power processor [23,24] running real routing protocols to the virtual satellite network emulated by STARRY NET, as illustrated in Figure 13. Collectively, the 3U CubeSat prototype together with the emulation creates a virtual constellation network with 1584 Starlink satellites. We follow the satellite traffic model proposed in [51] to inject traffic and use a power monitor to measure the satellite prototype. Table 4 summarizes the power consumption in different states (e.g., calculating route convergence and forwarding traffic in various data rates). Our hardware-in-the-loop test demonstrates STARRY NET’s ability to create a hybrid ENE and evaluate real system effects for user-defined functionalities.

在实际卫星部署中, 准确估算新系统或网络功能在发射前所需的能量至关重要. STARRYNET 使研究人员能够进行硬件在环测试, 从而准确评估各种工作负载下底层系统的影响. 为了展示 STARRYNET 的功能, 我们以一个案例研究为例:

将一个配备低功耗处理器 [23,24] 并运行实际路由协议的 3U 立方体卫星原型连接到 STARRYNET 模拟的虚拟卫星网络, 如图 13 所示.

该 3U 立方体卫星原型与模拟网络共同构建了一个包含 1584 颗星链卫星的虚拟星座网络. 我们遵循文献 [51] 中提出的卫星流量模型注入流量, 并使用功率监测器测量卫星原型的功耗.

表 4 总结了不同状态下的功耗(例如, 计算路由收敛和以不同数据速率转发流量)

我们的硬件在环测试证明了 STARRY NET 能够创建混合 ENE 并评估用户定义功能的实际系统效果.

Limitation and Future Work

Experimental scope and limitations. Our STARRY NET framework mainly targets at various network-level experiments for ISTNs, e.g., evaluating a new routing/transportlayer protocol, or assessing the network performance of a new topology design in a highly-dynamic, resource constrained virtual ISTN environment. The scale of the experiment supported by STARRY NET is closely related to the underlying resources provided by physical machines. In its present form, the key limitation of STARRY NET can be summarized as follows. First, STARRY NET is essentially a data-driven framework combining constellation-relevant modeling and network emulation. Thus its fidelity tightly depends on the availability and accuracy of the public information shared by the satellite ecosystem. For example, in practice, public TLE data may provide inaccurate orbit information, which can have errors up to 12 km, and such errors can affect the calculation of network performance (e.g., inter-satellite visibility and propagation delay). Second, some parameters are hard to obtain from a practical satellite system today, because most megaconstellations are still in their early stage with limited access. For example, it is difficult to obtain the real ISL-enabled Starlink performance right now, since Starlink’s laser ISLs are still under internal test. Thus, STARRY NET allows researchers to manually configure the ISL parameters (e.g., link capacity) and customize their experiments based on various experimental requirements. Third, our framework is primarily based on virtualization-based network-level emulation, and thus it has limited ability to emulate physical layer (PHY) characteristics that can be observed in a live network experiment, e.g., spectrum adaptation and multiplexing [42], or the time consumed by a real satellite dish to detect PHY connectivity changes.

Future work. Satellite Internet mega-constellations are still evolving rapidly. New constellation designs are constantly being proposed, and existing constellation schemes are constantly being updated. In our future work, we will follow the evolution and deployment of realistic satellite Internet constellations. In particular, we will track the latest constellation information to update STARRY NET’s open database, calibrate the constellation models and further improve the fidelity of the STARRY NET framework. Moreover, based on these implications obtained from our case studies in §7, we will explore new network techniques tailored for ISTNs, e.g., practical and resilient satellite routing protocols in the future. Our latest research progress on STARRY NET will be updated on the website: https://github.com/SpaceNetLab/StarryNet.

实验范围与局限性:

本文提出的 STARRYNET 框架主要针对空天地一体化网络(ISTN)的各类网络级实验, 例如评估新型路由/传输层协议, 或在高动态, 资源受限的虚拟 ISTN 环境中评估新拓扑设计的网络性能.

STARRYNET 所支持的实验规模与物理机器提供的底层资源密切相关. 在其当前形式下, STARRYNET 的主要局限性可总结如下:

首先, STARRYNET 本质上是一个结合了星座相关建模与网络仿真的数据驱动框架. 因此, 其保真度紧密依赖于卫星生态系统共享的公开信息的可用性与准确性. 例如, 在实践中, 公开的 TLE 数据可能提供不准确的轨道信息(误差可达 12 公里), 此类误差会影响网络性能的计算(如星间可见性和传播时延).

其次, 目前很难从实际卫星系统中获取某些参数, 因为大多数巨型星座仍处于早期阶段且访问权限有限. 例如, 由于 Starlink 的激光星间链路(ISL)仍处于内部测试阶段, 目前难以获取启用 ISL 的 Starlink 的真实性能. 因此, STARRYNET 允许研究人员手动配置 ISL 参数(如链路容量), 并根据各种实验需求定制实验.

第三, 本框架主要基于虚拟化的网络级仿真, 因此在仿真实地网络实验中可观测到的物理层(PHY)特性方面能力有限, 例如频谱自适应与复用 [42], 或真实卫星天线检测 PHY 连接变化所需的时间.

Danger

明确提及对于 PHY 特性的仿真局限性

积累, DTCLab 用得上

未来工作:

卫星互联网巨型星座仍在快速演进. 新的星座设计不断被提出, 现有的星座方案也在不断更新. 在未来的工作中, 我们将紧跟真实卫星互联网星座的演进与部署. 具体而言, 我们将追踪最新的星座信息以更新 STARRYNET 的开放数据库, 校准星座模型, 并进一步提高 STARRYNET 框架的保真度. 此外, 基于第 7 节案例研究中获得的启示, 我们将探索专为 ISTN 设计的新型网络技术, 例如未来实用且具有弹性的卫星路由协议.

我们关于 STARRYNET 的最新研究进展将在以下网站更新: https://github.com/SpaceNetLab/StarryNet

Related Work

Section 3.2 has discussed most existing efforts relevant to the method of building ENEs. In this section we briefly introduce other ISTN works related to our study in this paper.

The network community has many recent efforts studying on the topology design [40, 58], routing [47, 52, 56, 57, 83], transport-layer congestion control [60,64], new satellite applications [62] and security issues [51] for emerging ISTNs. For instance, Motif [40] is a recent topology design for LSNs, in which each satellite is dynamically connected to other visible satellites to achieve low latency under various traffic configurations. Works in [40,56,57] suggest the use of pre-calculated shortest-path-based routing and traffic engineering schemes for ISTNs. On one hand, these pioneering studies indeed outline the promising network potential of futuristic ISTNs. On the other hand, the above new thoughts are evaluated by simulations with a high-level abstraction. STARRY NET can stimulate new research and advance existing ISTN works by evaluating them in a more realistic ENE to obtain practical insights for further optimizations.

The rapid evolution of ISTNs also attracted the attention of the system community. Specifically, orbital edge computing (OEC) [43, 44, 66] is a new computation architecture which leverages computational satellites to pre-process earth observation (EO) data, and save the data download overhead. Studies in [49,50] explored the feasibility of applying deep neural networks to process on-board satellite data. Since STARRY NET creates real system runtime and networking stack in an experimental ISTN environment, it can also help to evaluate system-level effects of these new algorithms, implementations and programming models designed for ISTNs.

研究领域(Community)	关注主题(Focus Areas)	具体工作/示例(Specific Works/Examples)	现状与 StarryNet 的价值(Status & StarryNet's Value)
网络社区(Network Community)	• 拓扑设计 [40, 58] • 路由技术 [47, 52, 56, 57, 83] • 传输层拥塞控制 [60, 64] • 新型卫星应用 [62] • 安全问题 [51]	• Motif [40]: 一种 LSN 拓扑设计, 通过动态连接可见卫星在不同流量配置下实现低延迟. • 路由/流量工程 [40, 56, 57]: 建议在 ISTN 中使用基于预计算的最短路径路由和流量工程方案.	• 现状: 许多开创性研究勾勒了 ISTN 的潜力, 但多基于高层抽象的模拟(Simulation)进行评估. • StarryNet: 提供更逼真的实验网络环境(ENE), 通过评估这些新思路来获取实用见解, 推动进一步优化.
系统社区(System Community)	• 轨道边缘计算(OEC) [43, 44, 66] • 星上数据处理 [49, 50]	• OEC: 利用具备计算能力的卫星预处理对地观测(EO)数据, 从而节省数据下载开销. • 深度学习应用: 探索应用深度神经网络处理星上数据的可行性.	• 现状: 随着 ISTN 的快速演进, 系统社区开始关注如何利用卫星进行计算与数据处理. • StarryNet: 通过在实验环境中创建真实的系统运行时和网络协议栈, 能够评估针对 ISTN 设计的新算法, 实现及编程模型的系统级效应.