Scalable Real-Time Emulation of 5G Networks with Simu5G

核心痛点与目标

• 背景:

  • 5G 应用 (如自动驾驶, 工业 4.0, MEC 边缘计算) 对延迟敏感, 开发者需要在应用上市前在真实的 5G 环境下测试其性能
  • 但搭建真实的 5G 硬件测试床 (Testbed) 成本极高且难以获取

• 目标:

  • 利用 Simu5G (基于 OMNET++ 的仿真器) 作为一个实时仿真器 (Emulator)
  • 允许真实的应用程序通过仿真网络传输数据包, 获得与真实 5G 网络相同的延迟和丢包体验, 从而在普通台式机上低成本地完成原型设计和性能评估

核心技术: 如何实现大规模实时仿真?

为了在有限的计算资源 (如普通 PC) 上实现大规模网络 (数百用户, 数十个基站) 的实时仿真, 论文提出了分级建模的策略, 平衡精度与性能:

Tip

Motivation:

进行 RT 仿真的前提是, 模拟事件的处理速度必须快于真实时间. 这就要求对网络实体进行轻量化建模, 以降低计算开销

Methodology:

在一般的系统工作中, 解决性能瓶颈的方式有很多, 常见的一种是 parallelization / distribution. 它们可以将大规模的工作分成多个小工作, 从而减少单个工作的耗时, 并同时提高工作的并行性

但:

1. "works best when processing-heavy, low-interaction models are distributed across machines, which is not the case here"
2. "parallel emulation has its own challenges, such as maintaining real-time synchronization"

因此我们选择了另一种方式: 分级建模 (Scalable Modeling), 通过 "局部真" + "全局简" 的方式来降低整体的计算开销.

这一部分主要阐述了 Simu5G 如何通过分级建模 (Scalable Modeling) 来解决大规模网络实时仿真的性能瓶颈问题

其核心思想是: 根据仿真需求, 将网络实体分为"前景 (Foreground)"和"背景 (Background)"两类, 在保证关键数据准确性的同时, 大幅降低非关键节点的计算开销

1. Foreground 实体, FG:

   • 用途: 用于 承载真实被测流量 的 UE 和 BS
   • 特点: 拥有完整的协议栈 (从应用层到物理层), 执行精确的信道计算和报告, 确保被测应用的性能数据准确无误

2. Background 实体, BG:

   • 用途: 仅用于制造干扰 (Interference) 和争夺网络资源 (Resource Contention), 使场景更逼真
   • 特点: 使用轻量级模型
     • BG UE: 仅包含流量生成器和移动模型, 去除了复杂的上层协议栈开销
     • BG Cell: 不需要执行完整的下行调度, 仅需计算干扰
     • 概率性 CQI (Probabilistic CQI): BG UE 可以跳过复杂的物理层信噪比计算, 直接根据预设的概率分布生成信道质量指示 (CQI), 大幅降低计算量

Introduction

Fifth-generation (5G) cellular networks will bring significant changes to the wireless networking landscape. In fact, they will enable unprecedented scenarios, such as smart cities, autonomous vehicles, augmented reality and Industry 4.0. Most of these services will be composed of both communication and computation, leveraging the deployment of computing and storage capabilities in the cloud-edge continuum. An independent, but complementary innovation is in fact represented by Multi-access Edge Computing (MEC), which will endow the mobile network with cloud-computing capabilities, to allow mobile users to leverage the power of complex algorithms such as those based on artificial intelligence. While MEC is independent of the underlying technology (it can already coexist with the current 4G networks, in fact), it is foreseen that the progressive deployment of 5G will be an enabler for more powerful MEC capabilities [30].

Application developers have a pressing need for instruments for fast prototyping and credible performance evaluation. In fact, some of the services that they will be developing may have stringent latency constraints, such as autonomous driving or factory automation. For these, changes in the network configuration or deployment may have a drastic impact on their timing properties. During the application development phase (hence before the application is brought onto the market and run on the live network), developers would benefit from knowing what performance they should expect from a 5G network in terms of bandwidth and latency, at the very least, so that they can in turn adapt the behavior and/or requirements of the application. On the other hand, 5G infrastructure owners and operators will need to assess the performance of the services they are hosting in a controlled environment, so as to, e.g., evaluate alternative deployments or network functions partitioning. There is therefore a need for instruments that allow one to quickly setup a testbed, where applications exchange traffic through a 5G network. Unfortunately, 5G network testbeds are hard to come by, especially for developers.

We recently developed Simu5G [3], [4], a discrete-event simulation library for 5G New Radio networks based on OMNeT ++ , which evolves from the well-known SimuLTE simulator of LTE/LTE-A networks [2] and is backward compatible with it. Simu5G is already being used by a large community of researchers. It allows one to setup arbitrarily complex network scenarios involving 5G access, leveraging network models taken from the INET library [6], vehicular mobility taken from Veins [12] or Artery [13], etc. This makes Simu5G capable of simulating scenarios including the full Internet protocol stack in order to evaluate the performance of applications exploiting 5G connectivity, as well as how network-level metrics are affected by the traffic generated by applications. An application developer can use Simu5G in at least three different – and complementary – ways: first, by building a model of its application, to the required level of abstraction, and integrating the latter in Simu5G; second, by compiling the real codebase with Simu5G, substituting network calls with few localized modifications. These are both useful for functional testing and non-real-time performance evaluation (e.g., message counting). We show in this paper that a third way exists, i.e., to use Simu5G as network transport, having application endpoints exchange packets through it, in real time. In the literature (e.g., [31]–[33]), such approach is referred to as emulation – and we will do the same from now on – since packets exchanged by real applications with the simulator perceive the same impairments (e.g., delay and losses) as if they were running on the real network. This is useful to test the real-time performance of an application, e.g., when user input is required, or when closed-loop sensing and control applications are to be tested. These applications can be, for instance, the two counterparts of a MEC app, one running on a 5G User Equipment (UE) in mobility and the other on a MEC host connected to the 5G infrastructure. This allows application developers to test the performance of their software on a 5G network, under controlled conditions (e.g., as for load, channel quality, mobility, etc.) in a preproduction environment, so as to obtain confidence regarding their performance. The main benefit of using Simu5G as a real-time emulator is that users can rapidly define a custom, arbitrarily complex 5G network scenario and run their own application through it, in order to measure its performance or demonstrate it live to a third party, without buying expensive hardware to implement a real network testbed. For instance, developers of a multiplayer virtual reality game may be interested in testing whether the latency introduced by certain 5G network configurations could be detrimental for the Quality of Experience perceived by the users of their application. Simu5G allows them to assess such performance in a simple and cost-effective way. Note that using a real 5G network (e.g., a commercial one provided by a telecommunications operator) would allow users no control on the experimental conditions. Using Simu5G as an emulator can also be beneficial to network operators, e.g., to test the impact of real applications (instead of models thereof) on their network.

Real-time emulation with Simu5G is possible thanks to the combination of several concurring factors. First, the capabilities of the OMNeT ++ environment to run a real-time event scheduler, and of the INET library to exchange packets between a running simulation and the operating system. Second, but certainly no less important, a careful modeling of Simu5G functionalities and entities is required to make the software lightweight enough to be executed in real-time on a desktop machine. In fact, real-time event scheduling is only possible if simulator events are processed faster than wallclock time on the host machine. This means that, depending on the network model, there will be a maximum scale, in terms of number of network entities (e.g., User Equipments and Base Stations), carried traffic, etc., beyond which realtime emulation will be impossible on a given host machine.

This paper presents the architectural and modeling choices that allow Simu5G to run in real time up to fairly large scales (i.e., ten cells, a thousand UEs). A scale such as this is required to create scenarios where realistic intercell interference and resource contentions are emulated. Our solution is to provide multiple models of the main network elements (i.e., UEs and cells), representing different tradeoff points between the level of detail and the processing overhead. For instance, a standard UE model is endowed with the entire protocol stack up to the application layer, and it can be used as an application endpoint during emulation and/or to compute node-specific or end-to-end metrics. A background UE, instead, only needs to generate uplink (UL) interference and resource contention at the MAC layer, without recording any kind of UE-specific metric. This makes it possible to do away with the processing overhead of all the upper layers, while still enabling it to participate to uplink/downlink MAC scheduling, occupy resources, and generate interference. A similar approach is used for cells: a background cell generates downlink (DL) interference correctly without the overhead of simulating the full model of their associated UEs. Moreover, we report a thorough performance evaluation of Simu5G’s emulation capabilities, identifying the limits in all the relevant factors (number of simulated UEs and cells, application traffic, system bandwidth, number of carriers, numerologies). Our results show that Simu5G can run in real time, carrying traffic up to 1-2 Mbps between external endpoints, while emulating a scenario with ten 5G cells and a thousand users. To the best of our knowledge, this is the first work documenting a free, open-source simulator featuring 3GPP-compliant 5G New Radio implementation that can run as both a simulator and a real-time emulator with the same codebase, at that scale. Being able to use the same codebase for both simulations and live prototyping abates the developing time and makes results more reliable and easier to demonstrate. For instance, Simu5G is being used in the framework of the Hexa-X EU project [8], to support the development and demonstration of federated learning algorithms for Explainable Artificial Intelligence (XAI) models.

The rest of the paper is organized as follows: Section II introduces the necessary background, describing the basic functionalities of OMNeT ++ and Simu5G. Section III discusses the modeling choices that allow scalable real-time emulation. Section IV evaluates the emulation capabilities of Simu5G on a standard desktop computer, while Section V reviews the related work. Finally, Section VI draws conclusions and highlights directions for future work.

第五代 (5G) 蜂窝网络将为无线网络领域带来深刻变革. 事实上, 它们将赋能诸如智慧城市, 自动驾驶车辆, 增强现实及工业 4.0 等前所未有的应用场景. 绝大多数此类服务将兼具通信与计算属性, 通过利用"云-边连续体 (cloud-edge continuum)"中部署的计算与存储能力来实现. 事实上, 多接入边缘计算 (MEC) 代表了一项独立却互补的创新, 它赋予移动网络以云计算能力, 从而使移动用户能够利用诸如人工智能等复杂算法的算力. 尽管 MEC 独立于底层技术 (事实上, 它已能与当前的 4G 网络共存), 但可以预见, 5G 网络的逐步部署将赋能更强大的 MEC 能力 [30].

应用程序开发人员迫切需要能够进行快速原型设计和可信性能评估的工具. 事实上, 他们将开发的部分服务可能具有严格的延迟约束, 例如自动驾驶或工厂自动化.

对于这些服务而言, 网络配置或部署的变更可能会对其时序特性产生剧烈影响. 在应用开发阶段 (即应用上市并在现网运行之前), 如果开发人员能知晓其在 5G 网络中预期的带宽和延迟性能, 将大有裨益, 以便他们据此调整应用的行为和/或需求.

另一方面, 5G 基础设施的所有者和运营商需要在受控环境中评估其托管服务的性能, 以便评估替代部署方案或网络功能划分等.

因此, 需要一种能够快速搭建测试床 (testbed) 的工具, 使应用程序能够通过 5G 网络交换流量. 遗憾的是, 5G 网络测试床往往难以获取, 这对开发人员而言尤甚.

我们近期开发了 Simu5G [3], [4], 这是一个基于 OMNeT++ 的 5G 新空口 (New Radio) 网络离散事件仿真库, 它演进自著名的 LTE/LTE-A 网络仿真器 SimuLTE [2], 并与之保持向后兼容. Simu5G 已被广泛的研究群体所使用. 它允许用户利用来自 INET 库 [6] 的网络模型, 来自 Veins [12] 或 Artery [13] 的车辆移动性模型等, 搭建任意复杂的网络场景 (包含 5G 接入). 这使得 Simu5G 能够仿真包含完整互联网协议栈的场景, 从而评估利用 5G 连接的应用程序性能, 以及应用产生的流量如何影响网络层级指标.

应用程序开发人员至少可以通过三种互补的方式使用 Simu5G:

首先, 建立所需抽象级别的应用模型并将其集成到 Simu5G 中;

其次, 将真实代码库与 Simu5G 一同编译, 用少量的本地化修改替换网络调用.

这两者均适用于功能测试和非实时性能评估 (如消息计数).

本文将展示第三种方式, 即利用 Simu5G 作为网络传输层, 让应用端点通过它实时交换数据包. 在文献中 (如 [31]-[33]), 这种方法被称为仿真 (Emulation)——下文我们将沿用此称呼——因为真实应用程序与仿真器交换的数据包所感知到的损伤 (如延迟和丢包), 与在真实网络上运行时是一致的.

这对于测试应用程序的实时性能 (例如需要用户输入, 或测试闭环传感与控制应用时) 非常有用. 这些应用可以是, 例如, 一个 MEC 应用的两个对应部分: 一个运行在移动中的 5G 用户设备 (UE) 上, 另一个运行在连接到 5G 基础设施的 MEC 主机上. 这允许开发人员在预生产环境的受控条件下 (如负载, 信道质量, 移动性等) 测试其软件在 5G 网络上的性能, 从而获得对其性能的信心.

将 Simu5G 用作实时仿真器 (Emulator) 的主要优势在于, 用户可以快速定义自定义的, 任意复杂的 5G 网络场景, 并通过其运行自己的应用程序, 以测量性能或向第三方进行现场演示, 而无需购买昂贵的硬件来搭建真实的网络测试床. 例如, 多人虚拟现实游戏的开发者可能有兴趣测试特定 5G 网络配置引入的延迟是否会损害用户的体验质量 (QoE). Simu5G 允许他们以简单且经济高效的方式评估此类性能. 值得注意的是, 使用真实的 5G 网络 (如电信运营商提供的商用网络) 将使用户无法控制实验条件. 将 Simu5G 用作仿真器对网络运营商同样有益, 例如, 用于测试真实应用程序 (而非其模型) 对其网络的影响.

Simu5G 的实时仿真之所以成为可能, 得益于几个并发因素的结合.

首先是 OMNeT++ 环境运行实时事件调度程序的能力, 以及 INET 库在运行中的仿真与操作系统之间交换数据包的能力.

其次 (同样重要的是), 需要对 Simu5G 的功能和实体进行精细建模, 使软件足够轻量化, 以便在台式机上实时执行.

事实上, 只有当仿真器事件的处理速度快于主机上的挂钟时间 (wallclock time) 时, 实时事件调度才成为可能.

这意味着, 取决于网络模型, 在实体数量 (如用户设备和基站), 承载流量等方面将存在一个最大规模, 超过该规模, 在特定的主机上将无法进行实时仿真.

本文介绍了允许 Simu5G 在相当大的规模 (即 10 个小区, 1000 个 UE) 下实时运行的架构和建模选择. 此类规模对于创建能够模拟真实小区间干扰和资源争用的场景是必要的.

我们的解决方案是: 为主要网络元素 (即 UE 和小区) 提供多种模型, 代表细节水平与处理开销之间的不同权衡点.

例如:

• 标准 UE 模型配备了直至应用层的完整协议栈, 可用作仿真期间的应用端点和/或计算节点特定或端到端指标
• 背景 (Background) UE 仅需在 MAC 层生成上行链路 (UL) 干扰和资源争用, 无需记录任何 UE 特定指标
  • 这使得它可以省去所有上层协议的处理开销
  • 同时仍能参与上行/下行 MAC 调度, 占用资源并生成干扰
• 类似的方法也用于小区: 背景小区可以正确生成下行链路 (DL) 干扰, 而无需仿真其关联 UE 完整模型的开销

此外, 我们报告了对 Simu5G 仿真能力的全面性能评估, 确定了所有相关因素 (仿真 UE 和小区的数量, 应用流量, 系统带宽, 载波数量, 参数集) 的限制. 我们的结果表明, Simu5G 可以在实时运行的同时, 在外部端点之间传输高达 1-2 Mbps 的流量, 同时仿真包含 10 个 5G 小区和 1000 个用户的场景.

据我们所知, 这是首个记录在案的, 基于同一代码库既能作为仿真器 (simulator) 又能作为符合 3GPP 标准的 5G 新空口实时仿真器 (emulator) 运行的免费开源工具, 且能达到此种规模. 能够使用相同的代码库进行离线仿真和现场原型设计, 缩短了开发时间, 并使结果更可靠且更易于演示. 例如, Simu5G 正被用于 Hexa-X 欧盟项目 [8] 框架中, 以支持用于可解释人工智能 (XAI) 模型的联邦学习算法的开发和演示.

本文余下部分安排如下: 第二节介绍必要的背景知识, 描述 OMNeT++ 和 Simu5G 的基本功能. 第三节讨论允许可扩展实时仿真的建模选择. 第四节评估 Simu5G 在标准台式计算机上的仿真能力, 而第五节回顾相关工作. 最后, 第六节总结全文并指出未来的工作方向.

Background

This section introduces the necessary background. More specifically, we briefly describe the OMNeT ++ framework and the INET library, focusing on the features connected with real-time emulation, and then we present an overview of Simu5G.

A. The OMNeT ++ Framework and INET Model Library

OMNeT ++ is an open-source framework for discrete-event simulation. It is free for academic use, and it has a very large users’ community (including corporate ones). OMNeT ++ is built on the concept of module. OMNeT ++ modules exchange messages through connections linking their gates. A simulation model is represented by a network, which is a top-level compound module, with no external gates. Hierarchy must be preserved when connecting modules: with reference to FIGURE 1, simple module 1 can only connect to the compound module gate, and not to 2 directly. This forces a user to design models that are easier to verify and extend. Event handlers, called by the simulation kernel on receipt of messages, implement the behavior of a module. For instance, a module can schedule a timer by sending a message to itself, set to be handled at the expiration time. Simple modules have an initialization and finalization function, which are called at the start and the end of a simulation. The OMNeT ++ framework has built-in functionalities, such as event queueing or random number generation, thus allowing users to concentrate on writing their own simulation models.

OMNeT ++ maintains separate model’s behavior, description and parameter values. Object-oriented C ++ is used to code a model’s behavior. The description (i.e., gates, connections and parameter definition) is instead coded separately using the Network Description (NED) language. NED is a declarative language, which exploits inheritance and interfaces, and it is fully convertible into XML. Using NED; one can write parametric simulation scenarios, e.g., a hub-and-spoke network with a variable number of spokes. Last, but not least, parameter values are written in Initialization (INI) files. INI files are read at runtime, and initialize the model. Multiple values or intervals can be specified for a parameter, which makes it easy to run simulation studies where factors vary.

In OMNeT ++ , simulated time advances because events are processed: every event carries a firing time, and events are sorted by firing time into an event queue. When the next future event is extracted from the queue, the current simulated time is advanced to that event’s firing time. However, OMNeT ++ allows one to use (among others) a real-time event scheduler, according to which the flow of simulated time is slowed down to the pace of real (wall-clock) time. This is only possible if simulated time flows faster than the real time, i.e., if the density of events and their processing time are not such as to overload the system processing capacity. The above condition depends on the hardware/software system, on how a simulator is coded, but also on the scenario being run. Typically, there will be a scale in terms of number of UEs, gNBs, or traffic transmitted within a 5G network, beyond which a given simulation will not be able to run in real time.

INET 1 is a popular model library devised for OMNeT ++ . It includes a wealth of models of computer network elements, e.g., hosts, protocols, routers/switches, connections, etc.. Notably, INET models many protocols of the TCP/IP suite, such as TCP, UDP, IPv4, IPv6, OSPF, BGP, etc., as well as wired and wireless layer-2 protocols (Ethernet, PPP, IEEE 802.11, etc.). A user can incorporate the INET library into an OMNeT ++ simulation scenario, and then be able to run a simulation where an application, residing on a host, communicates with another endpoint through a network of IP routers, each having Ethernet cards and PPP WAN connections. The two endpoints themselves may be using different access technologies, e.g., 802.11 and 802.15, without this being a problem.

Moreover, the INET library comes with modules that interface the simulation environment with the host operating system (OS). They make packets received by the host OS from either real network interfaces or local applications appear in the simulation, and send packets generated within the simulation out to the OS, that will process them as any other network packet, i.e., forward them to either real network interfaces or local applications. To do this, two conditions must apply for the simulated modules that send/receive packets from/to the host OS. First, the IP addresses they are configured with in the simulation must be known to the host OS so that it can forward packets to them, i.e., they must be ‘‘routable’’ within the host OS. This is achieved by adding dedicated IP routes to the routing table of the latter. Second, such simulated modules must be equipped with a new submodule provided by the INET library, called ExtInterface. The INET library provides specialized implementations for the ExtInterface module, each of them allowing the simulation to exchange packets at different layers of the network protocol stack, e.g., sending/receiving IP datagrams or UDP segments. For the scope of this paper, we only describe the ExtLowerEthernetInterface, which exchanges Ethernet frames with the host OS using raw sockets. For incoming packets, an emulationenabled real-time scheduler is responsible for fetching such packets from the socket, to convert them into the equivalent C ++ object representation used in the simulation, and to add them to the event queue, where they are processed alongside the other events generated within the simulation. However, the real-time scheduler fetches new packets from the socket only when the simulation time is in line (henceforth, coherent) with wall-clock time. When the simulation time lags behind the wall-clock time, real packets stay in the socket buffer and accumulate delay, until coherence is resumed. Moreover, if real packets arrive faster than the rate at which the scheduler drains the socket buffer, the latter fills up and new packets are eventually discarded. When packets need to be sent outside the simulation environment, they are transmitted to the host OS using the same socket. The above features allow one to use Simu5G as a network emulator, that transports packets of real applications, delaying them the way a 5G network would.

OMNeT++ 是一个用于离散事件仿真的开源框架. 它对学术用途免费, 并拥有庞大的用户社区 (包括企业用户). OMNeT++ 构建于"模块 (module)"的概念之上. OMNeT++ 模块通过连接其"门 (gate)"的链路来交换消息. 仿真模型由一个"网络 (network)"表示, 这是一个没有外部门的顶层复合模块 (compound module). 在连接模块时必须保留层次结构: 参考图 1, 简单模块 1 只能连接到复合模块的门, 而不能直接连接到 2. 这强制用户设计更易于验证和扩展的模型:

当接收到消息时, 仿真内核会调用"事件处理程序 (event handler)"来实现模块的行为. 例如, 模块可以通过向自己发送一条设定在过期时间处理的消息来调度计时器. 简单模块 (simple module) 具有初始化和终结函数, 分别在仿真开始和结束时调用. OMNeT++ 框架具有内置功能, 如事件排队或随机数生成, 从而允许用户专注于编写自己的仿真模型.

OMNeT++ 将模型的行为, 描述和参数值分离开来. 面向对象的 C++ 用于编写模型的行为. 描述 (即门, 连接和参数定义) 则使用网络描述 (NED) 语言单独编写. NED 是一种声明式语言, 利用了继承和接口, 并且可以完全转换为 XML. 使用 NED, 用户可以编写参数化的仿真场景, 例如具有可变辐条数量的轮辐式 (hub-and-spoke) 网络. 最后但同样重要的是, 参数值编写在初始化 (INI) 文件中. INI 文件在运行时被读取以初始化模型. 可以为参数指定多个值或区间, 这使得运行因变量变化的仿真研究变得容易.

在 OMNeT++ 中, 仿真时间的推进是因为事件被处理:

每个事件都携带一个触发时间 (firing time), 事件按触发时间排序放入事件队列中. 当从队列中提取下一个未来事件时, 当前的仿真时间将推进到该事件的触发时间

然而, OMNeT++ 允许用户使用 (其中包括) 实时事件调度器, 根据该调度器, 仿真时间的流逝速度被减缓至真实 (挂钟, wall-clock) 时间的节奏

这只有在仿真时间流逝得比实时快的情况下才可能实现, 即事件的密度及其处理时间不足以使系统处理能力过载

上述条件取决于:

• 硬件/软件系统
• 仿真器的编码方式
• 正在运行的场景

通常, 在 5G 网络中传输的 UE, gNB 的数量或流量方面会存在一个规模上限, 超过该上限, 特定的仿真将无法实时运行.

INET [1] 是为 OMNeT++ 设计的一个流行模型库. 它包含了丰富的计算机网络元素模型, 例如主机, 协议, 路由器/交换机, 连接等. 值得注意的是, INET 对 TCP/IP 套件中的许多协议进行了建模, 如 TCP, UDP, IPv4, IPv6, OSPF, BGP 等, 以及有线和无线二层协议 (以太网, PPP, IEEE 802.11 等). 用户可以将 INET 库集成到 OMNeT++ 仿真场景中, 然后运行仿真, 其中驻留在主机上的应用程序通过由 IP 路由器组成的网络与另一个端点通信, 每个路由器都具有以太网卡和 PPP WAN 连接. 这两个端点本身可能使用不同的接入技术, 例如 802.11 和 802.15, 这不会构成问题.

此外, INET 库附带了将仿真环境与主机操作系统 (OS) 对接的模块. 它们使主机 OS 从真实网络接口或本地应用程序接收的数据包出现在仿真中, 并将仿真内生成的数据包发送给 OS, OS 将像处理任何其他网络数据包一样处理它们, 即将其转发到真实网络接口或本地应用程序. 为此, 对于向/从主机 OS 发送/接收数据包的仿真模块, 必须满足两个条件. 首先, 它们在仿真中配置的 IP 地址必须为主机 OS 所知, 以便能够转发数据包给它们, 即它们在主机 OS 中必须是"可路由的". 这是通过向后者的路由表添加专用 IP 路由来实现的. 其次, 此类仿真模块必须配备 INET 库提供的新子模块, 称为 ExtInterface. INET 库为 ExtInterface 模块提供了专门的实现, 每种实现都允许仿真在网络协议栈的不同层交换数据包, 例如发送/接收 IP 数据报或 UDP 数据段. 针对本文的范围, 我们仅描述 ExtLowerEthernetInterface, 它使用原始套接字 (raw sockets) 与主机 OS 交换以太网帧. 对于传入的数据包, 启用仿真的实时调度器负责从套接字获取这些数据包, 将其转换为仿真中使用的等效 C++ 对象表示, 并将其添加到事件队列中, 以便与仿真内生成的其他事件一起处理. 然而, 仅当仿真时间与挂钟时间保持一致 (以下称为"相干", coherent) 时, 实时调度器才会从套接字获取新数据包. 当仿真时间落后于挂钟时间时, 真实数据包将停留在套接字缓冲区中并累积延迟, 直到恢复一致性. 此外, 如果真实数据包的到达速度快于调度器排空套接字缓冲区的速度, 后者将被填满, 新数据包最终将被丢弃. 当数据包需要发送到仿真环境之外时, 它们使用相同的套接字传输到主机 OS. 上述功能允许人们将 Simu5G 用作网络仿真器 (emulator), 它可以传输真实应用程序的数据包, 并按照 5G 网络的方式对其进行延迟.

B. Overview of Simu5G

Simu5G [3], [4] is based on OMNeT ++ and it incorporates models from the INET library. It represents the evolution of the well-known SimuLTE 4G network simulator [2] towards 5G NewRadio access, and incorporates all SimuLTE’s functionalities, allowing one to evaluate legacy or mixed scenarios as well. Simu5G simulates the data plane of both the core and the radio access networks.

As far as the core network is concerned, it allows users to instantiate a User Plane Function (UPF) or Packet Data Network GateWay (PGW) and an arbitrary topology, where forwarding occurs using the GPRS tunneling protocol (GTP). As far as the radio access is concerned, it allows one to instantiate gNBs and UEs, which interact using a model of the New Radio protocol stack. gNBs can be connected to the core network directly, as shown in FIGURE 2 (left), in the so-called standalone deployment. Alternatively, a gNB can operate in an E-UTRA/NR Dual Connectivity (ENDC) deployment, shown in FIGURE 2 (right), where LTE and 5G coexist [9]. This last deployment is expected to be the most common in the early phases of 5G deployment. In this last configuration, the gNB works as a secondary node for an LTE eNB, which acts as master node connected to the core network. The eNB and the gNB are connected through the X2 interface and all NR traffic traverses the eNB first.

UEs and gNBs are modeled as compound OMNeT ++ modules. Their architecture is shown in FIGURE 3. UEs have all the protocol stack until the application layer, whereas gNBs only have layer-3 functionalities. Both include a NR Network Interface Card (NIC), which models the NR protocol stack. The structure of the NR NIC modules is shown in FIGURE 3, and is different for the two entities. Notably, UEs have a dual stack (both LTE and NR), which allows them to leverage double (4G/5G) connectivity in an ENDC scenario, in-sequence packet delivery being guaranteed by the Packet Data Convergence Protocol. Packet transmission entails top-down traversal of the NR protocol stack shown in FIGURE 3, with messages exchanged by neighboring modules. Conversely, packet reception entails bottom-up traversal. Note that OMNeT ++ messages are events: the price to pay for complete modeling of the layers within the NR protocol stack is that the transmission of a single IP packet via the NR interface requires Simu5G to handle a sizable number of events, in the order or few tens, among inter-layer communication, fragmentation/reassembly, timers, ACK/NACK sending, etc..

As far as the physical layer is concerned, Simu5G follows the approach already used by SimuLTE, i.e., to model the effects of propagation on the wireless channel at the receiver, without modelling symbol transmission and constellations. When a sender sends a MAC Protocol Data Unit (PDU) to a receiver, the PHY modules of the two entities exchange an OMNeT ++ message, whose propagation delay is set to the duration of one NR time slot. On receipt of said message the receiver performs a series of operations, summarized as follows:

• compute the reception power of the signal on each Resource Block (RB) x occupied by the MAC PDU, starting from the transmission power at the sender and applying a channel model to model pathloss, fading and shadowing;

• compute the interference by summing up the power received by all the other senders that interfere on the same RBs (using the same transformation as above);

• compute the SINR on each RB x, using obvious algebra;

• ∀x, compute P x = BLER(CQI, SINR x ), the error probability for that RB given the Channel Quality Indicator (CQI) used by the sender and the received SINR. This is done by using Block Error Rate (BLER) curves, which can be obtained from measurements performed using link-level simulators (e.g., [11]). With reference to FIGURE 4, BLER curves have the SINR as an abscissa, an error probability as an ordinate. There is one curve for each CQI, and the one corresponding to the transmission format used by the sender is selected. Then, error probability P x is found at the abscissa represented by SINR x ;

• compute P = 1 − ∏ x (1 − P x ), the error probability of the whole MAC PDU, extract a sample of a uniform random variable, and test its value against P to check if the reception was correct.

It is shown in [10] that the above modeling reduces the computational complexity of the decoding operation, hence the simulation running time, it improves evolvability, making it easy e.g., to add new modulations, and it still allows arbitrary channel models to be used.

Simu5G simulates radio access on multiple carriers, in both Frequency- and Time-division duplexing (FDD, TDD). Carrier Aggregation is supported, and different component carriers (CCs) can be configured with different numerologies and either FDD or configurable TDD slot format. Moreover, different CCs can have different channel models. Simu5G also incorporates functionalities already modelled in SimuLTE, e.g., UE handover and network-controlled device-to-device (D2D) communications, both one-to-one and one-to-many. Being based on OMNeT ++ , it allows one to incorporate models from other OMNeT ++ libraries, such as user mobility (e.g., through Veins [12] or LIMoSim [13]).

Simu5G [3], [4] 基于 OMNeT++ 并融合了 INET 库的模型. 它代表了著名的 LTE/LTE-A 网络仿真器 SimuLTE [2] 向 5G 新空口 (New Radio) 接入的演进, 并包含了 SimuLTE 的所有功能, 允许用户评估传统或混合场景. Simu5G 模拟核心网和无线接入网的数据平面.

就核心网而言, 它允许用户实例化用户平面功能 (UPF) 或分组数据网络网关 (PGW) 以及任意拓扑结构, 其中转发使用 GPRS 隧道协议 (GTP) 进行. 就无线接入而言, 它允许实例化 gNB 和 UE, 它们使用新空口协议栈模型进行交互. gNB 可以直接连接到核心网, 如图 2 (左) 所示, 即所谓的独立组网 (Standalone) 部署. 或者, gNB 可以在 E-UTRA/NR 双连接 (ENDC) 部署中运行, 如图 2 (右) 所示, 其中 LTE 和 5G 共存 [9]. 这种配置预计将是 5G 部署早期的最常见形式. 在此配置中, gNB 作为 LTE eNB 的辅节点 (secondary node), eNB 作为连接到核心网的主节点 (master node). eNB 和 gNB 通过 X2 接口连接, 所有 NR 流量首先经过 eNB.

UE 和 gNB 被建模为复合 OMNeT++ 模块. 其架构如图 3 所示.

UE 拥有直至应用层的完整协议栈, 而 gNB 仅具有第 3 层功能. 两者都包含一个 NR 网络接口卡 (NIC), 用于模拟 NR 协议栈. NR NIC 模块的结构如图 3 所示, 且两个实体的结构不同.

值得注意的是, UE 具有双协议栈 (LTE 和 NR), 这使其能够在 ENDC 场景中利用双重 (4G/5G) 连接, 分组数据汇聚协议 (PDCP) 保证了数据包的按序交付.

数据包传输涉及自顶向下遍历图 3 所示的 NR 协议栈, 相邻模块之间交换消息. 相反, 数据包接收涉及自底向上遍历. 请注意, OMNeT++ 消息即为事件: 对 NR 协议栈内各层进行完整建模的代价是, 通过 NR 接口传输单个 IP 数据包要求 Simu5G 处理数量可观的事件 (约为几十个), 其中包括层间通信, 分片/重组, 计时器, ACK/NACK 发送等.

就物理层而言, Simu5G 遵循 SimuLTE 已使用的方法, 即在接收端模拟无线信道上的传播效应, 而不模拟符号传输和星座图. 当发送方将 MAC 协议数据单元 (PDU) 发送给接收方时, 两个实体的 PHY 模块交换一条 OMNeT++ 消息, 其传播延迟设置为一个 NR 时隙的持续时间. 收到该消息后, 接收方执行一系列操作, 总结如下:

• 根据发送方的传输功率并应用信道模型来模拟路径损耗, 衰落和阴影效应, 计算 MAC PDU 占用的每个资源块 (RB) 上的信号接收功率
• 通过将所有在相同 RB 上产生干扰的其他发送方接收到的功率相加 (使用与上述相同的变换), 计算干扰
• 使用显而易见的代数方法计算每个 RB 上的 SINR
  • 这是通过使用误块率 (BLER) 曲线完成的, 该曲线可以通过使用链路级模拟器 (如 [11]) 进行的测量获得
  • 参考图 4, BLER 曲线以 SINR 为横坐标, 错误概率为纵坐标. 每个 CQI 对应一条曲线, 选择对应于发送方使用的传输格式的那条曲线. 然后, 在由 表示的横坐标处找到错误概率
  • 
• 计算整个 MAC PDU 的错误概率, 提取均匀随机变量的一个样本, 并将其值进行比较以检查接收是否正确

文献 [10] 表明, 上述建模降低了与解码操作相关的计算复杂度, 从而缩短了仿真运行时间

它提高了可演进性, 例如易于添加新的调制方式; 并且它仍然允许使用任意信道模型.

Simu5G 模拟频分和时分双工 (FDD, TDD) 模式下的多载波无线接入. 支持载波聚合, 不同的分量载波 (CC) 可以配置不同的参数集 (numerologies) 以及 FDD 或可配置的 TDD 时隙格式. 此外, 不同的 CC 可以具有不同的信道模型. Simu5G 还包含了 SimuLTE 中已建模的功能, 例如 UE 切换和网络控制的设备到设备 (D2D) 通信 (包括一对一和一对多). 由于基于 OMNeT++, 它允许用户集成来自其他 OMNeT++ 库的模型, 例如用户移动性 (如通过 Veins [12] 或 LIMoSim [13]).

Scalable Modeling of UEs and Cells in Simu5G

在一般的系统工作中, 解决性能瓶颈的方式有很多, 常见的一种是 parallelization / distribution. 它们可以将大规模的工作分成多个小工作, 从而减少单个工作的耗时, 并同时提高工作的并行性.

但:

1. "works best when processing-heavy, low-interaction models are distributed across machines, which is not the case here"
2. "parallel emulation has its own challenges, such as maintaining real-time synchronization"

因此我们选择了另一种方式: 分级建模 (Scalable Modeling), 通过 "局部真" + "全局简" 的方式来降低整体的计算开销.

这一部分主要阐述了 Simu5G 如何通过分级建模 (Scalable Modeling) 来解决大规模网络实时仿真的性能瓶颈问题

其核心思想是: 根据仿真需求, 将网络实体分为"前景 (Foreground)"和"背景 (Background)"两类, 在保证关键数据准确性的同时, 大幅降低非关键节点的计算开销

1. 动机与挑战 (Motivation)

   • 需求: 为了使仿真结果 (如延迟, 抖动) 真实可信, 必须引入大规模的干扰和资源争用, 这意味着场景中需要包含数十个基站和数百上千个用户
   • 瓶颈: 全协议栈仿真的计算量极大, 主要瓶颈在于 MAC 层调度 (排序复杂度 O(NlogN)), PHY 层信道报告 (涉及大量浮点运算和矩阵计算 O(B2·N)) 以及协议栈遍历 (每层都会产生大量事件)
   • 对策: 为了在单台 PC 上实现实时仿真, 必须引入轻量级模型来处理那些仅用于制造"背景噪音 (干扰)"的实体

2. UE (用户终端) 的分级建模

   • 前景 UE (Foreground UE, FG-UE)
     • 定义: 承载用户需要观测的真实应用流量或统计数据的 UE
     • 实现: 包含完整的协议栈 (从应用层到物理层), 启用精确的物理层报告
   • 背景 UE (Background UE, BG-UE)
     • 定义: 仅用于在 MAC 层制造资源争用 (Resource Contention) 和信号干扰的 UE
     • 实现: 去除绝大部分协议栈, 仅保留 Traffic Generator 和 Mobility Model
     • 特点: 结构极简, 可以是"满缓冲 (Full-buffer)"状态, 持续请求资源

3. Cell (基站/小区) 的分级建模

   • 前景 Cell (Foreground Cell, FG-Cell)
     • 定义: 至少服务于一个前景 UE 的基站
     • 实现: 执行完整的协议栈操作, 包括下行 (DL) 和上行 (UL) 的完整调度, 确保前景流量的延迟和物理特性被精确计算
     • 增强: 内部包含一个 BackgroundTrafficGenerator 模块, 用于统一管理该小区内的背景 UE
   • 背景 Cell (Background Cell, BG-Cell)
     • 定义: 仅服务于背景 UE 的基站, 主要作用是向邻区产生干扰
     • 下行链路 (DL): 不需要调度. 只需统计每个 TTI (传输时间间隔) 占用了多少资源块 (RB) 来计算对他人的干扰功率, 无需关心具体分给了哪个用户
     • 上行链路 (UL): 需要调度. 因为上行干扰取决于具体哪个 UE 在哪个位置发送, 所以需要计算具体分配

4. 灵活的 CQI 计算策略 (CQI Computation):

   • 为了进一步降低背景 UE 的计算开销, Simu5G 提供了两种信道质量指示 (CQI) 计算模式, 支持动态切换
   • 真实模式 (Realistic): 基于物理模型计算信号衰减, 干扰和 SINR. 计算量大, 适用于移动的背景 UE
   • 概率模式 (Probabilistic):
     • 原理: 不进行复杂的物理信道计算, 而是根据预设的均值和方差 (如正态分布) 直接随机抽取 CQI 值
     • 优势: 计算速度极快
     • 适用: 静止的用户或距离较远的用户
   • 动态切换: 系统支持在仿真过程中根据情境 (如用户发生切换 Handover 时) 在两种模式间动态切换, 以平衡精度和性能

总结: 通过这种设计, 用户可以在一个场景中混合使用,

• 少量的"前景实体". 保证测试对象的精度
• 大量的"背景实体". 低开销地模拟真实的拥塞和干扰环境

从而在普通 PC 上实现大规模 5G 网络的实时仿真

Related Work

To the best of our knowledge, there are few works that can compare to the one described in this paper. As we argue in [3], physical-layer simulators, such as the 5G K-Simulator [18] suite or the Vienna 5G SL simulator [19], are designed with different purposes, namely evaluating physical-layer quantities, such as the Signal to Interference-plus-Noise Ratio (SINR) or spectral efficiency, as a function of physicallayer designs, such as antenna layout, transmission schemes. They are often very accurate in the modeling of the physical layer, but they seldom, if ever, allow end-to-end applicationlevel communication through the whole protocol stack. On the other hand, end-to-end simulators, such as Simu5G, are designed exactly for the last purpose. As a consequence, they often model the physical layer in such a way that the effects of physical impairments on packet transmission are preserved, without necessarily modeling symbol transmission, constellations, receiver algorithms, etc. Among endto-end simulators of 5G networks, we find 5G-LENA [20] and the 5G-air-simulator [21]. The former includes a subset of Simu5G’s features (e.g., it lacks FDD, dual connectivity, D2D, MEC modeling, etc.). The second one lacks some of the above too, and does not incorporate models of layer4 protocols, which makes it impossible (e.g.) to simulate anything TCP-based with the necessary credibility. To the best of our knowledge, there is no mention of either of the above working in real time. We are aware that ns3, on which 5G-LENA is based, allows real application to interact with simulated models, and can also run in real time (see, e.g., [28], [29], which discuss LTE emulation). However, this does not imply that 5G-LENA is lightweight enough to run in real time, nor it sheds any light on the scale – if any – at which this would be possible. We are not aware of any work assessing feasibility and scalability of real-time emulation of 5G networks using 5G-LENA.

A previous work of ours [1] introduced the theme of real-time emulation with Simu5G. However, that work relies on outdated versions of Simu5G, INET and OMNeT ++ . Simu5G’s version in [1] relied on version 3.6.4 of the INET library, which did not exhibit the same performance as version 4.3, on which this work is based. The differences are many, and in fact involve the mechanism for interfacing with external applications. All the above translated to improved baseline performance: for instance, it is now possible – and it was not in [1] – to run emulation with FG UEs at high numerologies. Moreover, in this work we have added modeling of BG UEs/gNBs, which allows Simu5G emulation to scale up considerably.

A related effort is the OpenAirInterface (OAI) 5G-RAN project [26], which is currently under way as we write, with a schedule foreseen to complete in the second half of 2022. The project promises to deliver an open-source software implementation of a programmable 5G RAN. With respect to that project, Simu5G maintains the distinct advantage of allowing both simulation and emulation with the same codebase. This allows one, for instance, to modify elements in a whole (simulated/emulated) network scenario with few modifications of an INI file, rather than (e.g.) by recompiling code on several physical/virtual machines. Moreover, Simu5G allows full control over the simulated scenario, which is constructed by the user, whereas it is unlikely that OAI 5G-RAN will.

Two simulation software packages exist whose features are at least comparable to Simu5G’s, namely Riverbed Modeler [16] and Tetcos Netsim [17]. As far as we are aware, Riverbed Modeler does not include any 5G NR module (the website mentions LTE release 8 and 9), and there is no indication that it allows real-time emulation. Tetcos Netsim instead allows emulation, and it has 5G mm-wave NewRadio access. However, both are only included in the standard, non-academic version, which is not free. Finally, commercial emulation software packages exist, aimed at network testing, e.g., Simnovus Callbox [22], VIAVI TeraVM 5G Core Test [23], NetTest 5G Network Emulators [24], Keysight Technologies 5G products [25]. Their focus is mainly on equipment testing, hence is different from ours.

类别 (Category)	代表工具/项目 (Tools/Projects)	特点与局限性 (Characteristics & Limitations)	与 Simu5G 的对比/优势 (Comparison/Simu5G Advantages)
物理层仿真器 (Physical-layer Simulators)	- 5G K-Simulator [18] - Vienna 5G SL simulator [19]	- 侧重点: 评估物理层指标 (如 SINR, 频谱效率) 与天线/传输设计的关系. - 优点: 物理层建模非常精确. - 缺点: 极少支持通过完整协议栈进行端到端应用层通信.	Simu5G 专为端到端通信设计, 保留物理损伤对数据包的影响, 但简化了符号传输等细节以支持全栈仿真.
其他端到端仿真器 (End-to-end Simulators)	- 5G-LENA [20] - 5G-air-simulator [21]	- 5G-LENA: 缺少 FDD, 双连接, D2D, MEC 等功能. 虽基于支持实时的 ns-3, 但尚无工作证明其 5G 实时仿真的可行性与扩展性. - 5G-air-simulator: 缺少第 4 层 (传输层) 协议模型, 无法可信地仿真 TCP 应用; 不支持实时运行.	Simu5G 功能更全 (含 MEC, D2D 等), 且本文明确验证了其在标准 PC 上进行大规模实时仿真的可行性与扩展性.
早期 Simu5G 工作 (Previous Simu5G Work)	- Simu5G (Ref [1] version)	- 依赖: 基于过时的 INET 3.6.4 和旧版 OMNeT++. - 性能: 接口机制旧, 性能较低, 不支持前景 UE 使用高参数集 (Numerologies).	本工作 (Current Work): - 基于 INET 4.3, 性能大幅提升. - 引入背景实体 (BG UEs/gNBs) 建模, 显著提升了仿真的可扩展性.
开源实现项目 (Open-Source Implementation)	- OpenAirInterface (OAI) 5G-RAN [26]	- 目标: 提供可编程 5G RAN 的软件实现 (预计 2022 下半年完成). - 部署: 修改场景通常涉及重新编译代码或配置多个物理/虚拟机.	Simu5G: - 代码复用: 仿真 (Simulation) 和 实时仿真 (Emulation) 共用同一代码库. - 易用性: 通过修改 INI 文件即可调整整个网络场景, 且用户对场景构建拥有完全控制权.
其他仿真软件包 (Other Simulation Packages)	- Riverbed Modeler [16] - Tetcos Netsim [17]	- Riverbed: 无 5G NR 模块 (仅提及 LTE Rel 8/9), 无实时仿真迹象. - Tetcos Netsim: 支持 5G 毫米波和仿真功能, 但仅限付费的非学术版本, 不免费.	Simu5G 是免费且开源的, 且完整支持 5G NR 标准.
商业仿真测试软件 (Commercial Emulation Software)	- Simnovus Callbox [22] - VIAVI TeraVM [23] - NetTest [24], Keysight [25]	- 侧重点: 主要用于设备测试 (Equipment Testing). - 性质: 商业闭源软件.	Simu5G 侧重于应用原型设计和网络性能评估, 且为低成本/无成本的解决方案.