Serval’s Design

In this section, we present the problem setup, Serval’s approach to distributing compute across Earth and space, and Serval’s incorporation of auxiliary information. Finally, we describe Serval’s execution engine.

Problem Setup

Fig. 1 shows the three layers in a satellite networking system: (1) the LEO satellite constellation containing hundreds of orbiting satellites, (2) few ground stations across the world, each of which can communicate with satellites, and (3) the cloud consisting of one or more data centers. The satellites continuously image the Earth’s surface and send the captured images down whenever they make the next contact with a ground station. The ground stations eventually upload images to the cloud.

图1展示了卫星网络系统中的三个层次：

(1) 包含数百颗在轨卫星的LEO卫星 "星座"

(2) 分布于全球的少数 "地面站"，每个地面站都能与卫星通信

(3) 由一个或多个数据中心组成的 "云端"

"卫星" 持续对地球表面进行成像，并在下一次与 "地面站" 建立联系时将捕获的图像下传。地面站最终将图像上传至"云服务器"。

The satellites are energy-constrained, i.e., they have access to limited amounts of power, and only a fraction of the power is available for compute. The primary usage of the power is to maintain temperature, power attitude determination, and control systems, communicate with ground stations, and other critical satellite functionality. We assume each satellite has limited computational capability such as a Jetson TX2 or Jetson AGX Orin. This assumption is validated by recent proposals of incorporating compute in satellites in academia [15,22,23,44,45], and by recent launches that incorporate computational capabilities on satellites [4,28].

Ground stations have no power limits and are equipped with more (though not infinite) computational resources than satellites. This assumption is reasonable in both traditional ground station designs which were capital-intensive, as well as in modern ground station designs wherein cloud providers co-locate compute resources at the ground stations.

卫星受到能源限制，即它们可用的电力有限，并且只有一小部分电力可用于计算。电力的主要用途是维持温度、为姿态确定与控制系统供电、与地面站通信以及其他关键的卫星功能。我们假设每颗卫星具有有限的计算能力，例如搭载了Jetson TX2或Jetson AGX Orin。这一假设得到了学术界近期在卫星中集成计算能力提议 [15,22,23,44,45] 的验证，也得到了近期发射的已具备计算能力的卫星 [4,28] 的证实。

地面站没有电力限制，并配备比卫星更多（尽管不是无限）的计算资源。无论是在资本密集型的传统地面站设计中，还是在云服务提供商将计算资源与地面站并置的现代地面站设计中，这个假设都是合理的。

Graph representation: Application developers who are customers of the satellite constellation operator company submit queries that will run on the Earth observation imagery. For example, Planet Inc. has a product named “Analytics" which allows users to run user-defined analytic applications. The user can choose a specific object detection model to run, and limit the set of images to run the selected model on by defining additional filters based on image metadata, including date, time, geographic location, cloud cover ratio, etc. [1]. However, today such queries are executed offline on the cloud once all the images have been delivered to the cloud — we seek to execute them in real-time.

While some queries may not be sensitive to taking days to generate a response, several queries are latency-sensitive and require responses in minutes. The satellite operator company may charge a premium for such latency-sensitive queries. Examples include: (a) a forest department may design a query like ‘images of forest fires in California’; (b) a trading firm may be interested in a query like ‘number of ships at major ports across the world’; (c) a disaster response team might be interested in ‘images of, or number of, flood-damaged buildings in Florida’.

查询表示：作为卫星星座运营商公司客户的应用开发者会提交将在地球观测影像上运行的查询。例如，Planet Inc. 有一个名为“Analytics”的产品，允许用户运行用户自定义的分析应用。用户可以选择运行一个特定的目标检测模型，并通过定义基于图像元数据（包括日期、时间、地理位置、云量比例等）的附加过滤器来限制运行该模型的图像范围 [1]。然而，如今这类查询是在所有图像都已交付到云端后，在云上离线执行的 —— 我们寻求实时执行它们。

虽然某些查询可能对耗时数天生成响应不敏感，但有几类查询是延迟敏感型的，要求在数分钟内得到响应。卫星运营商公司可能会对此类延迟敏感型查询收取额外费用。例子包括：(a) 森林部门可能设计一个“加利福尼亚州森林火灾的图像”的查询；(b) 一家贸易公司可能对“全球主要港口的船只数量”这样的查询感兴趣；(c) 灾难响应团队可能对“佛罗里达州被洪水损毁建筑物的图像或数量”感兴趣。

Serval supports two types of queries: (I) image outputs: queries that require images to be outputted matching the query, e.g., forest fire images from California may be needed for a detailed inspection of the damage and to create a plan for action, (II) statistic outputs: queries that only require the inference to be delivered, e.g., a hedge fund may need the count of cars in different parking lots across Beijing, rather than the actual images, or, financial traders may need to know the count of ships at a port.

Serval支持两种类型的查询：(I) 图像输出：需要输出匹配查询结果的图像的查询，例如，可能需要加利福尼亚的森林火灾图像来详细检查损害情况并制定行动计划；(II) 统计数据输出：只需要交付推断结果的查询，例如，一家对冲基金可能需要北京各地停车场的汽车数量，而不是实际图像，或者，金融交易员可能需要知道某个港口的船只数量。

We represent each query as a sequence of filters, which is consistent with the current commodity products such as Planet Analytics [1]. For example, ‘forest fires in California’ can be represented as California→forest→cloud→fires. Similarly, ‘ships around ports’ is represented as Port→Cloud→Ship count. Note that each query contains a cloud filter to remove images that are occluded by clouds. The filter representation is depicted visually in Fig. 2.

Each filter is a computational block that takes an image as an input and outputs either a boolean value or a number. For example, the California filter performs a geographical check and returns True or False, i.e., do the geographical coordinates of the image overlap with the geographical boundaries of California. Similarly, the filters Vessel counter may be a neural network (a stock network, or one supplied by the application developer) that detects and counts the number of vessels in an image. Cloud, Forest, and Fire are other neural network-based filters in examples above.

Serval’s goal is to prioritize images that pass all the filters corresponding to at least one latency-sensitive query. We also note that the filter representation of our queries lends itself to cross-query optimization. For example, if two different queries rely on the same underlying filters, Serval does not need to perform the computation twice, e.g., if two queries both rely on forests in California, then we do not need to perform forest detection twice. Similarly, Cloud filter needs to be computed only once for an image even though the image may be relevant to multiple queries.

我们 将每个查询表示为一系列过滤器（a sequence of filters） ，这与当前如Planet Analytics [1]等商业产品的做法一致。例如，“加利福尼亚的森林火灾”可以表示为 加利福尼亚→森林→云→火灾。同样，“港口周围的船只”可以表示为 港口→云→船只计数。请注意，每个查询都包含一个云过滤器，以移除被云层遮挡的图像。这种过滤器的表示方式在图2中有直观展示。

每个过滤器都是一个计算模块，它接收一张图像作为输入，并输出一个布尔值或一个数字 。例如，“加利福尼亚”过滤器执行地理检查并返回真或假，即图像的地理坐标是否与加利福尼亚的地理边界重叠。同样，“船只计数器”过滤器可能是一个神经网络（一个现有的网络，或由应用开发者提供），用于检测和计数图像中的船只数量。在上述例子中，“云”、“森林”和“火灾”是其他基于神经网络的过滤器。

Serval的目标是优先处理那些 通过了至少一个延迟敏感型查询 所对应的所有过滤器的图像 。我们还注意到，我们查询的过滤器表示方式天然支持跨查询优化。例如，如果两个不同的查询依赖于相同的底层过滤器，Serval无需重复执行计算。例如，如果两个查询都依赖于加利福尼亚的森林，那么我们无需执行两次森林检测。同样，对于一张可能与多个查询相关的图像，云过滤器也只需要计算一次。

Note on inter-satellite links: While Inter-satellite Links (ISLs) have generated a lot of interest, none of the Earth observation constellations today are equipped with ISLs. Hence we do not consider ISLs. Starlink demonstrated feasibility of laser-based ISLs [41,57], and Planet and Telesat announced their plans to explore radio-frequency ISLs across orbits (e.g., from Low Earth Orbit to Middle Earth Orbit or Geostationary Orbits) [36]. Given the uncertainty over feasibility and type of ISLs, we choose to exclude them in our analysis. If ISLs mature in the future, Serval’s design can be generalized to accommodate and exploit ISLs.

关于星间链路（Inter-satellite Links, ISLs）的说明： 尽管星间链路引起了广泛关注，但目前没有任何地球观测星座配备了ISLs。因此，我们不考虑ISLs。Starlink展示了基于激光的ISLs的可行性 [41,57]，Planet和Telesat也宣布了他们探索跨轨道射频ISLs（例如，从低地球轨道到中地球轨道或地球静止轨道）的计划 [36]。鉴于ISLs的可行性和类型存在不确定性，我们选择在我们的分析中排除它们。如果ISLs未来技术成熟，Serval的设计可以进行扩展以适应和利用ISL。

Distributing Compute Across Earth and Space

Today’s delay between image capture at satellite and image delivery to the cloud/end-user ranges from several hours to many days. To enable near-realtime insights for the end user(s), Serval’s primary goal is to reduce this delay to minutes for latency-sensitive queries, while accommodating the constraints imposed by the satellite’s limited power and compute capabilities. Towards this goal, we first consider the placement of computation at the different compute units: satellite, ground station, and the cloud servers. There are two existing approaches to perform inference on Earth imagery:

当前，从卫星捕获图像到图像交付至云端/最终用户之间的延迟从数小时到数天不等。为了给最终用户提供近实时的洞察，Serval的首要目标是将延迟敏感型查询的这一延迟缩短至数分钟，同时满足卫星有限的电力和计算能力所施加的约束。为实现此目标，我们首先考虑在不同计算单元（卫星、地面站和云服务器）上的计算部署问题。目前存在两种在地球影像上执行推断的方法：

(i) In-order Delivery and Computation: In the traditional approach, images are delivered to the cloud in the same timestamp order as they were captured by the satellite, and then inference is run on these images in the cloud. This approach suffers from large delays because of networking bottlenecks. Specifically, the LEO satellites exhibit fast motion with respect to ground stations on the Earth [23, 61]. Therefore, any ground station-satellite contact is fleeting – less than ten minutes per contact, few contacts per satellite per day. This intermittent connectivity, in conjunction with the small number of ground stations, leads to large networking delays. Since images cannot be processed before they are received at the data center, the time-to-insight for this approach is very large.

(i) 顺序交付与计算 (In-order Delivery and Computation): 在传统方法中，图像按照其被卫星捕获时的时间戳顺序交付到云端，然后在云端对这些图像运行推断。 由于网络瓶颈，这种方法存在巨大的延迟。具体而言，LEO卫星相对于地面站表现出高速运动 [23, 61]。因此，任何地面站与卫星的接触都是短暂的——每次接触不足十分钟，每颗卫星每天的接触次数也很少。这种间歇性的连接，加上地面站数量稀少，导致了巨大的网络延迟。由于图像在到达数据中心之前无法被处理，这种方法的洞察时间非常长。

(ii) In-orbit Computation: Recently, there is a growing push to place computation on the satellites. The first wave of this push demonstrated the use of satellite computation to reject cloud-occluded images [28]. Orbital edge computing [23] broadens the scope of satellite computing to reject images that do not meet the application goals, e.g., if the goal is to select images containing buildings, the satellite runs a building detector and rejects images that contain no buildings. This approach places a high computational load on satellites.

(ii) 在轨计算 (In-orbit Computation): 近期，将计算部署在卫星上的呼声越来越高。第一波浪潮展示了利用卫星计算来剔除被云遮挡的图像 [28]。轨道边缘计算 [23] 将卫星计算的范围扩大到剔除不符合应用目标的图像，例如，如果目标是选择包含建筑物的图像，卫星会运行一个建筑物检测器并剔除不含建筑物的图像。这种方法给卫星带来了沉重的计算负载。

Specifically, such in-orbit computation techniques suffer from two problems: (a) as the number of applications increases or becomes more complex (such as the compositional filters discussed in Sec. 3.1), satellites cannot accommodate this compute with their limited resources, and (b) new applications emerge for historical data. Such applications may not be known a priori. If a satellite discards data that does not match current applications, new applications that emerge later cannot be served.

具体来说，这类在轨计算技术存在两个问题：

(a) 随着应用数量的增加或变得更加复杂（如第3.1节中讨论的组合式过滤器），卫星以其有限的资源无法承载这种计算

(b) 新的应用会针对历史数据而出现。这类应用可能无法事先知晓。如果卫星丢弃了与当前应用不匹配的数据，那么未来出现的新应用将无法得到服务

Serval takes the practical approach of distributing compute between satellites and ground stations. Our approach is centered around three properties of satellite imagery:

• Image locations are predictable: Satellites follow predictable orbital paths that can be estimated using their orbit descriptors, e.g., using Two Line Element (TLE) orbit descriptors published by observatories as well as many satellite operators. Therefore, even before an image is taken, we can predict what the geographical content of the image is.

• Content of most images is glacial: For most images taken at a given location, most of their content is glacial, i.e., stationary—it does not change rapidly within a few days. An image that contains buildings yesterday, will likely contain buildings today. Similarly, forests, desserts, farms, and other land types rarely change over the time span of a few days.

• Ground station compute capabilities are rising: Increasingly, ground stations are designed to include computational resources. These computers are more powerful and better resourced in terms of power and networking as compared to computational devices on the satellite.

Serval采取了在卫星和地面站之间分配计算的实用方法。我们的方法围绕卫星影像的三个特性构建：

• 图像位置是可预测的： 卫星遵循可预测的轨道路径，这些路径可以使用其轨道描述符进行估算，例如，使用天文台以及许多卫星运营商发布的两行轨道根数（Two Line Element, TLE）轨道描述符。因此，甚至在图像被拍摄之前，我们就能预测图像的地理内容。

• 大多数图像的内容是静态的 (glacial)： 对于在特定位置拍摄的大多数图像，其大部分内容是静态的，即固定的——在几天内不会迅速变化。一张昨天包含建筑物的图像，今天很可能仍然包含建筑物。同样，森林、沙漠、农田和其他土地类型在几天的时间跨度内很少发生变化。

• 地面站的计算能力正在提升： 越来越多的地面站被设计为包含计算资源。与卫星上的计算设备相比，这些计算机在功率和网络资源方面更强大、资源更丰富。

Based on these observations, Serval divides the filters for all the queries into two types: (a) fast-changing dynamic filters that need to be executed on the satellite at run-time, and (b) glacial slow-changing filters that can be executed using stale imagery on the ground stations. Serval executes the glacial filters at the ground station or the cloud before the image is captured and uses the ground stations to communicate the results to the satellite in advance. This allows the satellite to process a very small fraction of images on the satellite in the bottleneck execution path.

基于这些观察，Serval将所有查询的过滤器分为两类：

(a) 需要在运行时于卫星上执行的、快速变化的 动态过滤器

(b) 可以在地面站上使用过时影像 (stale imagery) 执行的、变化缓慢的 静态过滤器 (glacial filters)

Serval在图像被捕获之前，就在地面站或云端执行静态过滤器，并利用地面站提前将结果传达给卫星。这使得卫星在瓶颈执行路径上只需处理极小一部分的图像。

Pre-computing such glacial information can significantly reduce the compute requirements on the satellite. As satellites orbit rapidly around the earth, their footprint looks like thin belts that extend almost vertically from the south pole to the north pole. Therefore, ordinary user requests, such as forests in California or farm land in the U.S., generally only makes up a tiny portion of a satellite’s imagery.

Serval attaches a dynamic or glacial attribute to each filter to aid the execution engine. This can either be supplied by the developer, or estimated from data. Consider the query ‘forest fires in California’. We find that among the millions of images captured by a constellation, only 0.4% contain any land area in California. Two-thirds of California images contain forests, i.e., only 0.25% of all images captured by the constellation contain ‘forests in California’. Since we can estimate the exact set of images that will contain ‘forests in California’ using historical data, the satellite needs to run a fire detector on only this small set of images. In this case, California and Forest are glacial filters, and Fire is a dynamic filter. Only dynamic filters need to run on satellites using fresh data.

预先计算这类静态信息可以显著降低卫星上的计算需求 。当卫星快速绕地球运行时，它们的覆盖区（footprint）看起来像从南极几乎垂直延伸到北极的窄带。因此，普通的用户请求，如“加利福尼亚的森林”或“美国的农田”，通常只占一颗卫星影像的极小一部分。

Serval为每个过滤器附加一个动态或静态属性，以辅助执行引擎。这个属性可以由开发者提供，也可以从数据中估算。以“加利福尼亚的森林火灾”这个查询为例。我们发现，在一个星座捕获的数百万张图像中，只有0.4% 包含加利福尼亚州的任何陆地区域。在这些加州图像中，有三分之二包含森林，即星座捕获的所有图像中只有0.25% 包含“加利福尼亚的森林”。由于我们可以利用历史数据精确估算出将包含“加利福尼亚的森林”的图像集合，卫星只需对这一小部分图像运行火灾探测器。在这种情况下，“加利福尼亚”和“森林”是静态过滤器，而 “火灾”是动态过滤器 。 只有动态过滤器需要使用最新的数据在卫星上运行。

Incorporating Auxiliary Information Sources

Ground stations, unlike satellites, have continuous access to auxiliary information sources like weather forecasts. We ask if such information can be used by Serval to either improve inference quality or to skip computation on the satellite.

We first make the observation that clouds occlude many of the images taken by a satellite, and a cloud-occluded image is not useful for any query. In fact, there has been a lot of work in cloud detection for satellite imagery before [28,35, 47, 53], both using statistical methods [35, 53] and neural networks [28,47].

Given this observation, Serval’s key idea is to leverage weather forecasts in order to skip on-board processing of images over areas that are forecast to have high cloud cover. Concretely, if the probability of cloud cover is high, then Serval assigns a ‘cloudy’ tag to the image and exclude it from any other computation and de-prioritize its transfer to the ground station. Conversely if the probability of cloud cover is very low, then Serval assigns a ‘cloud-free’ tag to the image. In this case, the processing pipeline on the satellite can skip the Cloud filter and proceed to the next steps of computation. Finally, all other images that are not tagged in either of the above ways are processed through the Cloud filter on-board the satellite.

In practice, we use cloud probability thresholds of { 0.2,0.8 } for the above processing. If the cloud cover probability for an image is less than 0.2, Serval assumes the image to be cloud-free. If the probability is greater than 0.8, Serval de-prioritizes the image. If the probability is between these two values, Serval processes the image through the Cloud filter, which is a neural network model.

While the current version of Serval does not incorporate weather forecasts in other inferences, there are many other potential optimizations for future work. For applications like forest fire, weather forecasts also contain information regarding forest fire risks, which can be used to select high-likelihood images to run filters on. Also, cloud detectors are known to confuse smoke and cloud which can be improved by combining cloud detection results and weather forecasts. Finally, auxiliary information may contain other types of information such as information extracted from previous satellite images, e.g., if a different satellite observed forest fire in the same region a few hours ago.

地面站与卫星不同，能够持续接入如天气预报等辅助信息源。我们旨在探究Serval系统是否能利用此类信息来提升推理质量，或跳过在卫星上执行的计算。

我们首先观察到一个现象：云层会遮挡卫星拍摄的大量图像，而一张被云遮挡的图像对任何查询任务均无助益。事实上，学术界先前已在卫星影像的云检测方面开展了大量研究工作 [28, 35, 47, 53]，这些工作综合运用了统计学方法 [35, 53] 和神经网络 [28, 47]。

基于此观察，Serval的核心思想是利用天气预报，以跳过对预报将有高云量覆盖区域的图像所进行的星上处理。具体而言，如果云层覆盖的概率很高，Serval会为该图像分配一个“多云”标签，并将其排除在任何其他计算之外，同时降低其传输到地面站的优先级。反之，如果云层覆盖的概率非常低，Serval则会为该图像分配一个“无云”标签。在此情况下，卫星上的处理流程可以跳过云检测模块（Cloud filter），直接进入后续计算步骤。最后，所有未被标记为上述任一标签的图像，都将通过星上的云检测模块进行处理。

在实践中，我们采用 {0.2,0.8} 的云量概率阈值来进行上述处理。如果一幅图像的云覆盖概率低于 0.2，Serval便假定该图像为无云图像。如果概率高于 0.8，Serval则会降低该图像的优先级。若概率介于这两个值之间，Serval会通过作为神经网络模型的云检测模块来处理该图像。

尽管当前版本的Serval尚未将天气预报信息融合到其他推理任务中，但未来的工作存在许多潜在的优化方向。例如，在森林火灾等应用中，天气预报同样包含森林火灾风险的信息，这些信息可用于筛选出高可能性图像，以对其运行特定的滤波器。此外，众所周知云检测器容易混淆烟雾和云层，这一问题可以通过结合云检测结果与天气预报来改善。最后，辅助信息还可能包含其他类型的信息，例如从先前的卫星图像中提取的信息——譬如，数小时前是否有另一颗卫星在同一区域观测到了森林火灾。

Serval’s Execution Engine

We describe how Serval captures the insights discussed in Sections 3.1-3.3. We discuss how Serval’s execution engine works on a satellite, at a ground station, and in the cloud.

The Satellites

Fig. 4 shows that each satellite maintains three queues for images: (i) a low priority queue, (ii) a high priority queue, and (iii) a compute queue. The low priority queue contains images that do not match any latency-sensitive query. The high priority queue contains images that definitely match latency-sensitive queries. The compute queue contains images that need more computation to be performed on the satellite in order to ascertain their status.

图4展示了每颗卫星为图像数据维护三个队列：(i) 低优先级队列，(ii) 高优先级队列，以及 (iii) 计算队列。低优先级队列包含不匹配任何延迟敏感查询的图像。高优先级队列包含确定匹配延迟敏感查询的图像。计算队列则包含那些为确定其状态而需要在星上执行更多计算的图像。

Network delivery order: When the satellite comes into contact with the ground station, it will first downlink images in the high priority queue. If the high priority queue is emptied and ground station contact remains, it will downlink the images in compute queue. This is because even though computation hasn’t run on these images on the satellite, these images have a higher likelihood of being latency-sensitive than images in the low-priority queue. Finally, if the compute queue is emptied and ground station contact remains, it will downlink images from the low priority queue. This multi-queue architecture allows Serval to benefit from limited compute available on satellites, even when it may not be able to compute on all images.

网络传输顺序：

当卫星与地面站建立联系时，它将首先下传高优先级队列中的图像。若高优先级队列已清空且与地面站的联系仍然保持，系统将接着下传计算队列中的图像。这样做的原因是，尽管这些图像尚未在星上完成计算，但它们属于延迟敏感图像的可能性高于低优先级队列中的图像。最后，若计算队列也被清空且与地面站的联系仍然保持，系统将下传低优先级队列中的图像。这种多队列架构使得Serval能够有效利用卫星上有限的计算资源，即便在无法对所有图像都完成计算的情况下也能获益。

Sat->GS 下传顺序: high-priority > computation > low-priority

Image placement in queues: When a satellite comes in contact with a ground station, the ground station sends the following information to the satellite: (a) pre-computed values for glacial or slow-moving filters for each image that the satellite is expected to capture in its upcoming orbital path leveraging the observation from Sec. 3.2 that satellite orbits are predictable, and (b) weather predictions for the geographical location corresponding to each image the satellite is expected to capture. For each image captured by a satellite, Serval’s first goal is to identify whether the image meets the requirements for any (at least one) of the on-board queries. Serval does so by immediately applying all pre-computed filters. For a given query, if any of its pre-computed filters rejects the image, the image is considered rejected by that query. If an image is rejected by all queries, then the image is placed in the low priority queue. Note that in practice this entire filtering process is fast as it does not involve significant compute but is a quick classifier based on pre-computed inferences received from the ground station.

图像入列策略：

当卫星与地面站建立联系时，地面站会向卫星发送以下信息：

(a) 利用3.2节中提到的卫星轨道可预测这一特性，为卫星在其即将到来的轨道路径上预计捕获的每张图像，预先计算好其静态或缓变滤波器（glacial or slow-moving filters）的值；

(b) 卫星预计将要捕获的每张图像所对应地理位置的天气预报。对于卫星捕获的每张图像，Serval的首要目标是识别该图像是否满足至少一个星上查询的要求。

为此， Serval会立即应用所有预计算滤波器。对于一个给定的查询，若其任何一个预计算滤波器拒绝了该图像，则视作该查询已拒绝此图像。 若一张图像被所有查询拒绝，它将被放入低优先级队列。需要注意的是，在实践中，整个筛选过程非常迅速，因为它不涉及大量计算 ，而是一个基于从地面站接收的预计算推理结果的快速分类器。

To be placed in the compute queue, an image must meet two criteria: (a) all the pre-computed glacial filters in at least one query must select that image, and (b) there must be at least one dynamic filter in that query that needs computation. For example, for ‘forest fires in California’, if an image is selected by the California and Forest filters, it is placed in the compute queue to run the Cloud and Fire filters.

Any image that is selected by all the filters in at least one query is placed by Serval into the high priority queue. If a query has some dynamic filters, then an image moves to the high-priority queue from the compute queue. For example, in the example above, if the image in the compute queue is selected by both Cloud and Fire filters, it will move to the high priority queue. Note that, if all the filters in a query are glacial filter (precomputed on the ground station), then, some images can skip the compute queue and directly move to the high priority queue as well.

一张图像必须满足两个标准才能被放入计算队列：(a) 至少有一个查询中的所有预计算静态滤波器都必须选中该图像；(b) 且该查询中必须至少有一个需要进行计算的动态滤波器。例如，对于“加州的森林火灾”这一查询，如果一张图像被加州（California）和森林（Forest）这两个滤波器选中，它将被放入计算队列，以待运行云检测（Cloud）和火情检测（Fire）滤波器。

入 computation queue

1. 至少有一个查询中的 所有预计算静态滤波器 都必须选中该图像
2. 该查询中必须至少有一个需要进行计算的 动态滤波器

任何被至少一个查询中的所有滤波器选中的图像，Serval都会将其放入高优先级队列。如果一个查询包含动态滤波器，那么图像会从计算队列移至高优先级队列。例如，在上述例子中，如果计算队列中的图像被云检测和火情检测滤波器同时选中，它将被移至高优先级队列。值得注意的是，如果一个查询中的所有滤波器都是静态滤波器（已在地面站预计算），那么一些图像可以跳过计算队列，直接进入高优先级队列。

Finally, if the output of a query is an inference and not an image (e.g., Vessel Count), then Serval places the image in the low priority queue and creates an empty image with just the metadata information (e.g., number of cars in parking lots or number of vessels at a port). Serval places this (much) smaller metadata value in the high priority queue.

最后，如果一个查询的输出是推理结果而非图像本身（例如，“船只计数”），Serval会将原始图像放入低优先级队列，并创建一个仅包含元数据信息的空图像（例如，停车场中的汽车数量或港口中的船只数量）。Serval会将这个体积小得多的元数据值放入高优先级队列。

In situations where the satellite has sufficient surplus energy to compute on an image, Serval pulls an image from the compute queue and runs the dynamic filters corresponding to the query on the image. At any time, if the satellite determines that an image is not possibly high priority in the multi-stage filtering process, it will immediately put it into the low priority queue and start doing computation on the next image. On the other hand, if the satellite determines that an image is high-priority, it will put the image into the high priority queue without running additional filters on that image. In each queue, images are ordered by their capture timestamp.

在卫星有充足剩余能源可用于计算的情况下，Serval会从计算队列中取出一张图像，并对其运行查询所对应的动态滤波器。在多阶段筛选过程的任何时刻，如果卫星判断某图像不可能成为高优先级，它会立即将该图像放入低优先级队列，并开始处理下一张图像。反之，如果卫星确定某图像为高优先级，它会直接将该图像放入高优先级队列，而无需再对其运行额外的滤波器。 在每个队列内部，图像均按其捕获时间戳排序。

The Ground Stations

Recall that ground stations have much higher computation capability than the satellites, while having much fewer power constraints. In Serval, the ground station maintains two queues: a high priority queue and a low priority queue. When a ground station receives an image from the satellite, it first runs the uncompleted filters of the query and the places the image in the high priority if the image is selected by all the filters in at least one query, otherwise it places it in the low priority queue. Whenever the ground station has free computational resources, Serval opportunistically uses the downtime to compute the results of ‘glacial’ filters for upcoming satellite contacts. A ground station has a steady backhaul connection to the cloud, and uses this backhaul to constantly stream images to the cloud starting from the high-priority queue.

回顾之前的内容，地面站拥有比卫星高得多的计算能力，同时其能源约束也少得多。

在Serval系统中，地面站维护着两个队列：一个高优先级队列和一个低优先级队列。

当接收到来自卫星的图像后，地面站首先会运行该查询中尚未完成的滤波器：

• 如果该图像被至少一个查询中的所有滤波器选中，则将其放入高优先级队列
• 否则，将其放入低优先级队列

每当有空闲的计算资源时，地面站便会机会性地利用这段时间，为即将到来的卫星通信窗口预先计算 静态（‘glacial’）滤波器 的结果。地面站拥有连接到云端的稳定回传链路，并利用该链路从高优先级队列开始，持续地将图像流式传输至云端。

The Cloud

The cloud acts as a frontend to all users. The images and insights will be uploaded to the cloud and made available for users to download. The cloud also acts as the central coordinator of the entire Serval system and computes the future orbits, positions of satellites and other preemptively computed values for the images taken in the future. The cloud will also schedules satellite-ground station contacts and sends the pre-computed values to the appropriate ground station to be relayed to the satellites in the control plane. Fig. 5 demonstrates Serval’s placement of different filters for the example queries in Fig. 2.

云端作为面向所有用户的前端。图像和分析结果将被上传至云端，并提供给用户下载。云端还充当整个Serval系统的中央协调器，负责计算未来卫星的轨道、位置，以及为未来将要拍摄的图像预先计算其他所需数值。云端还将规划卫星与地面站的通信窗口，并将预计算值发送给相应的地面站，再由地面站通过控制平面（control plane）转发给卫星。图5展示了Serval系统针对图2中示例查询的各类滤波器的部署位置。

TL; DR

上面读完，可能还是有点懵懵的，索性去问ai进行总结归纳了:

---

梳理卫星上的三级队列逻辑

整个系统可以看作一个在卫星上运行的多级动态筛选流程，目标是尽快识别出高价值图像，并为它们优先分配计算和传输资源

1. 低优先级队列 (Low-Priority Queue)

可以把这个队列理解为 “基本确认无用或价值不高的图像存放处”

• A. 队列定义 这个队列存放的是那些基本可以确定不匹配任何一个“延迟敏感查询”（即有时效性要求的任务）的图像。

• B. 图像进入此队列的触发原因（如何被“降级”到这里） 一张图像会因为以下三种情况之一被放入低优先级队列：

  1. 在初始筛选阶段被拒绝：

     • 触发点：图像刚被卫星捕获后。
     • 原因：卫星会立即使用地面站预先计算并发送过来的“静态滤波器”（如地理位置、植被类型等）进行快速检查。如果这张图像被所有的查询任务都拒绝了（即至少有一个静态滤波器不匹配），那么系统判定它对当前所有任务都无用，直接放入低优先级队列。

  2. 在计算过程中被淘汰：

     • 触发点：图像位于“计算队列”中，正在等待或接受星上处理时。
     • 原因：一张图像最初可能通过了静态滤波器的筛选，进入了计算队列。但当卫星对其运行更复杂的“动态滤波器”（如云检测、火情检测）时，如果任何一个动态滤波器给出了“拒绝”的结果，系统就判断它最终不符合高优先级要求，于是立即将其从计算队列中移出，放入低优先级队列。

  3. 查询任务本身只需要元数据：

     • 触发点：图像满足了一个查询的所有条件，但该查询的最终目的不是图像本身。
     • 原因：例如，一个任务是“统计港口船只数量”。当Serval处理完图像并得出船只数量（比如50艘）后，这个数字（元数据）才是高价值信息，而原始图像本身的紧急性就大大降低了。因此，系统会将这个数字放入高优先级队列（因为它体积小、价值高），而将完整的原始图像放入低优先级队列，仅作存档之用。

• C. 此队列的下一步 该队列的传输优先级最低。只有当高优先级队列和计算队列的图像都已下传完毕，且卫星与地面站的通信窗口仍然有余时，才会开始下传这里的图像。

---

2. 计算队列 (Compute Queue)

可以把这个队列理解为 “有潜力成为高价值图像的等候处理区” (中间暂态, 待转移)

• A. 队列定义 这个队列存放的是那些通过了初步筛选，但还需要在卫星上进行更多复杂计算（运行动态滤波器）才能最终确定其价值的图像。它是一个临时的中间状态。

• B. 图像进入此队列的触发原因 一张图像要进入计算队列，必须同时满足以下两个严格的条件：

  1. 通过了至少一个查询的全部“静态滤波器”：图像在初步筛选中没有被立刻拒绝，至少有一个任务认为它“可能相关”。
  2. 且该查询任务包含至少一个“动态滤波器”：这意味着仅靠地面站预先计算的信息还不足以做最终判断，必须在卫星上调用计算资源（如神经网络模型）进行进一步分析（如检测云、火、烟雾等）。

  例如，“加州的森林火灾”查询。图像首先通过“位于加州”和“有森林”这两个静态滤波器的筛选，然后因为它还需要进行“云检测”和“火情检测”这两个动态计算，所以它被放入计算队列。

• C. 此队列的下一步（状态切换的十字路口） 进入此队列的图像面临两种命运，这是一个关键的决策点：

  • 向上升级 (Promotion)：卫星利用空闲算力处理这张图像，运行所有必需的动态滤波器。如果图像成功通过了所有动态滤波器的检测，它就被确认为高价值图像，并立即从计算队列移入高优先级队列。
  • 向下淘汰 (Demotion)：如果在运行动态滤波器的过程中，任何一个滤波器失败了（比如，云量过高），这张图像就会被确认为不符合要求，并立即从计算队列移入低优先级队列。

---

3. 高优先级队列 (High-Priority Queue)

可以把这个队列理解为 “已确认高价值图像的VIP快车道”

• A. 队列定义 这个队列存放的是那些已经完全通过了某个查询任务的所有筛选步骤，被最终确认为“确定匹配”的延迟敏感图像或高价值元数据。

• B. 图像进入此队列的触发原因（如何“晋级”到这里） 一张图像（或其元数据）可以通过以下两种路径进入最高优先级：

  1. 从计算队列晋升而来（标准路径）：

     • 触发点：图像位于计算队列中，并完成了星上处理。
     • 原因：如上所述，图像成功通过了所有静态和动态滤波器的考验，其价值得到最终确认，因此被从计算队列“提拔”到高优先级队列。这是最常见的路径。

  2. 直接进入（跳过计算队列的快速路径）：

     • 触发点：图像刚被卫星捕获并完成初始筛选后。
     • 原因：如果一个查询任务所包含的所有滤波器都是静态的（即所有判断都可以在地面站预先计算好，卫星上无需进行复杂的动态计算），并且该图像满足了所有这些静态滤波器的条件，那么它就可以跳过计算队列，直接被放入高优先级队列。这相当于一个“简单任务”，可以被立刻确认。

  （另外，别忘了之前提到的特殊情况：对于元数据查询，是那个小体积的元数据结果被直接放入高优先级队列。）

• C. 此队列的下一步 该队列拥有绝对的最高传输优先级。一旦卫星与地面站建立联系，第一件事就是下传这个队列里的所有内容。