System Design

EagleEye improves the number of high-resolution targets downlinked with a mixed-resolution, leader-follower constellation. The target identification module finds targets by processing low-resolution imagery. The actuation-aware scheduler first analyzes targets, leverages actuation constraints, and produces a schedule for each follower to capture a sequence of targets. Each follower then adheres to its actuation schedule to capture the sequence of targets it is assigned.

EagleEye 通过一个混合分辨率的主从式星座，提升了下行链路传输的高分辨率目标数量。其 目标识别模块 通过处理低分辨率图像来发现目标。 感知驱动能力的调度器 首先分析目标，利用驱动约束，并为每个跟随卫星生成一个捕获一系列目标的调度方案。然后， 每个跟随卫星遵循其驱动调度 ，以捕获分配给它的目标序列。

Target Identification

The leader’s target identification module leverages onboard ML inference. The input to the inference model is a lowresolution frame. For EagleEye, we assume that an object detection ML model identifies targets of interest. Thus, the model must be trained for the target application using previously collected and labeled orbital imagery. The targets of interest must be of appropriate size in the input tiles, and the total number of tiles per frame must not exceed to energy or time budget of the leader. The output of the model is a set of target bounding boxes associated with latitude/longitude GPS coordinates.

主导卫星的目标识别模块利用了星上ML推理技术。推理模型的输入是低分辨率的图像帧。对于EagleEye，我们假设一个目标检测 (object detection) ML模型能够识别感兴趣的目标。因此，该模型必须使用先前收集并标注过的轨道图像，针对特定目标应用进行训练。感兴趣的目标在输入图块中必须具有适当的尺寸，并且每帧的总图块数量不能超过主导卫星的时间或能量预算。模型的输出是一组与经纬度GPS坐标相关联的目标边界框 (bounding boxes)。

Frame tiling and scaling. As with prior work on orbital edge computing [28, 30], the target identification module first decomposes a (large) frame into tiles that fit the input dimension of the ML model, and then processes each tile. To identify small objects in low-resolution data with high accuracy, EagleEye decomposes a frame into the appropriate tile size (that may be smaller than the ML model’s input), and then scaling the tile to the ML input size [28, 29]. The tile size with optimal accuracy is application- and systemdependent [29, 30] and recent work shows execution time benefits to operating away from the empirically-optimal tiling for accuracy [28]. Tile size is an important parameter because a decrease in tile size increases tile count, which increases the number of invocations of the ML model required to process a frame, ultimately increasing frame processing time. Changes to tile size also change the size (in pixel count) of the features of interest when input into the ML network. To avoid missing some data, a leader must finish processing each frame before it sees an entirely new frame, which imposes a hard deadline on frame processing. The number of tiles per frame must also avoid exhausting the leader’s energy store. To validate EagleEye’s use of frame tiling, §6 shows that across a wide range of tile sizes, frame processing time remains below the energy constraint and processing deadline.

图像帧分块与缩放

与先前关于轨道边缘计算的工作 [28, 30] 一样，目标识别模块首先将一个（大）图像帧分解为适合ML模型输入尺寸的图块 (tiles)，然后逐一处理。为了在低分辨率数据中高精度地识别小型物体，EagleEye将图像帧分解为适当尺寸的图块（可能小于ML模型的输入尺寸），然后将该图块缩放至ML模型的输入尺寸 [28, 29]。

具有最优精度的图块尺寸是依赖于具体应用和系统的 [29, 30]，并且最近的研究表明，采用非经验最优精度的分块方式在执行时间上具有优势 [28]。

图块尺寸是一个重要参数，因为减小图块尺寸会增加图块数量，从而增加处理一帧图像所需的ML模型调用次数，最终增加帧处理时间。为避免丢失数据， 主导卫星必须在观测到一帧全新的图像之前完成对当前帧的处理 ，这为帧处理设定了硬性截止时间 (hard deadline)。每帧的图块数量也必须避免耗尽主导卫星的能量储备。为了验证EagleEye中图像分块的有效性，第6节将展示，在广泛的图块尺寸范围内，帧处理时间始终低于能量约束和处理截止时间。

Target clustering. After identifying targets, the leader optimizes the set of targets by clustering targets that are close together, enabling them to be captured in a single, high-resolution image instead of requiring multiple captures. Fig. 7 shows the clustering problem schematically, with the center points of identified targets enclosed by a single box representing a high-resolution image capture.

We formulate target clustering as a planar point cover problem: given several points on a 2D surface, find the minimal set of rectangles that cover all points, searching across all rectangle positions and dimensions. We simplify the problem by assuming that the sides of high-resolution images are in parallel with the sides of low-resolution images. Note that allowing off-parallel, high-resolution images may further reduce the total number of high-resolution images, but we leave this extension for future work.

We use Integer Linear Programming to solve the problem and the solution is a set of boxes enclosing clustered targets that can be imaged together in a single high-resolution image. We find that the solver could find the optimal rectangle cover solution in 1 ms for 500 targets (figure not shown). After clustering, the scheduler treats the priority score of a cluster’s high-resolution frame to be the sum of the priority scores of the targets enclosed by the frame.

目标聚类

在识别出目标后， 主导卫星通过聚类 (clustering) 相近的目标来优化目标集，使得它们可以在单张高分辨率图像中被捕获 ，而无需多次拍摄。图7示意性地展示了聚类问题，其中被单个方框（代表一次高分辨率图像捕获）包围的是已识别目标的中心点。

我们将目标聚类表述为一个平面点覆盖问题：在二维平面上给定若干点，寻找能够覆盖所有点的最小矩形集合，这一过程需要搜索所有可能的矩形位置和尺寸。我们假设高分辨率图像的边缘与低分辨率图像的边缘平行，从而简化了该问题。值得注意的是，允许非平行的、倾斜的高分辨率图像可能会进一步减少所需的高分辨率图像总数，但我们将此扩展留作未来工作。

我们使用整数线性规划 (Integer Linear Programming, ILP) 来解决此问题，其解是一组包围了聚类目标的方框，这些目标可以在单次高分辨率成像中被一并捕获。我们发现，对于500个目标，该求解器可以在1毫秒内找到最优的矩形覆盖方案（图未示出）。聚类后，调度器将一个聚类对应的高分辨率图像帧的优先级分数视为该帧所包含的所有目标优先级分数之和。

Computing Actuation Time and Target Time Window

EagleEye’s actuation-aware scheduler computes an actuation plan for each follower to point at and capture each target. The plan must consider the time to point from one target to the next, and must consider the optimal path along which to capture all targets. The scheduler computes the plan as the solution to an Integer Linear Programming (ILP), which includes constraints corresponding to the physics of pointing and orbital motion, as well as the cost of visting targets in a particular order.

Actuation scheduling

The scheduler solves the actuation scheduling problem to produce a sequence of pointing actuations for each follower. The leader adds constraints and optimization goals to the ILP formulation, framing it as a generalized traveling salesman problem that plans routes for multiple salesmen to cover multiple cities optimally. One set of ILP constraints define the actuation time as distance between targets. Another set of constraints relate these actuation times to the time window for each target, for each follower. The ILP optimization goal is to maximize the value of the sum of priority scores of captured targets. The solution to the ILP is a set of pointing actuations (i.e., rotations) for each follower satellite to perform to cover its assigned set of targets.

Alternative formulations. As a point of comparison, we construct another solution that uses a greedy algorithm to choose the nearest unimaged target. The greedy algorithm does not achieve an optimal result, with 4.3–14.4% less coverage compared to our ILP-based solution, as shown in §6.

We also consider the AB&B approach used in prior work [27]. This approach exhibits high runtime, which limits its applicability (§6).

调度器解决驱动调度问题，为每个跟随卫星生成一个指向驱动序列。主导卫星在ILP公式中添加约束和优化目标，将其构建为一个广义旅行商问题 (generalized traveling salesman problem)，该问题为多个“销售员”（即卫星）规划路径以最优地覆盖多个“城市”（即目标）。

一组ILP约束将驱动时间定义为目标之间的“距离”。另一组约束则将这些驱动时间与每个跟随卫星对每个目标的成像时间窗口关联起来。ILP的优化目标是最大化被捕获目标优先级分数的总和。ILP的解是为每颗跟随卫星设定的一组指向驱动（即旋转动作），以完成其分配的目标集。

替代方案

作为比较，我们构建了另一个使用贪心算法 (greedy algorithm) 的解决方案，该算法总是选择最近的未成像目标。如第6节所示，贪心算法无法达到最优结果，其覆盖率比我们基于ILP的解决方案低4.3%–14.4%。

我们还考虑了先前工作 [27] 中使用的 AB&B 方法。该方法表现出很高的运行时长，限制了其适用性（§6）

Number of followers

One configuration for EagleEye is the number of followers in a group. Fig. 5 shows several configurations of a constellation with 4 satellites. We can choose to have (i) 2 groups with one leader and one follower in each group (Fig. 5c) or (ii) 1 group with one leader and three followers in each group (Fig. 5d). With more groups, the total area covered by all leaders increases, detecting more targets. With more followers in a group, more targets in each low-resolution image would be captured by the followers.

An extreme configuration is to have both a high- and a lowresolution camera in the same satellite group (Fig. 5e). For a fixed number of satellites, this configuration would have the maximum number of groups. However, it has two limitations. First, the satellite might miss some targets due to computation delay. The satellite has only about 15 seconds to capture, process, point to take high-resolution images of targets, and point back to take another low-resolution image of the ground track. As shown in Fig. 9 bottom, the mix-camera satellite might already fly away from the target when the computation is completed. We show how the compute time would affect the coverage in §6. On the other hand, the computation delay does not affect the leader-follower configuration (Fig. 9 top), as long as the computation is finished before the next low-resolution image comes in, because the follower satellites are behind the leader satellite. Second, a cubesat is highly volume constrained [29], mostly due to the high-resolution camera’s need for long focal length. It is likely infeasible to add a second high-coverage camera without increasing volume (and hence launch cost).

EagleEye的一个配置参数是组内的跟随卫星数量。图5展示了一个包含4颗卫星的星座的几种配置。我们可以选择 (i) 2个组，每组一个主导卫星和一个跟随卫星（图5c），或者 (ii) 1个组，包含一个主导卫星和三个跟随卫星（图5d）。

• 拥有 更多的组 ，所有主导卫星覆盖的总面积会增加，从而 探测到更多目标
• 而在 一个组内拥有更多的跟随卫星 ，则意味着 每张低分辨率图像中的更多目标将被跟随卫星捕获

一个极端的配置是在同一颗卫星上同时安装高分辨率和低分辨率相机（图5e, Mix-Camera）。对于固定数量的卫星，这种配置将拥有最大数量的组。然而，它有两个局限性:

首先， 由于计算延迟，卫星可能会错过一些目标。卫星只有大约15秒的时间来完成捕获、处理、指向并拍摄目标的高分辨率图像 ，然后再次指向以拍摄下一张星下点轨迹的低分辨率图像。如图9（下图）所示，当计算完成时，混合相机卫星可能已经飞过了目标。我们将在第6节展示计算时间如何影响覆盖率。

另一方面， 计算延迟不影响主从式配置（图9，上图） ，只要计算在下一张低分辨率图像进来之前完成即可，因为跟随卫星位于主导卫星之后。其次，立方星的体积受到严格限制 [29]，这主要是因为高分辨率相机需要较长的焦距。在不增加体积（以及发射成本）的情况下增加第二个高覆盖率相机，很可能是不可行的。

ML accuracy

We consider how the accuracy (precision, recall) of the object detection model running in the leader would affect coverage. A false positive increases the complexity of scheduling and potentially wastes time and energy of followers that could be used to image real targets. A false negative target is omitted from the scheduling and remains unimaged by any followers, thereby resulting in reduced coverage.

我们考虑在主导卫星上运行的目标检测模型的准确率（精确率、召回率）将如何影响覆盖率。一个假正例 (false positive) 会增加调度的复杂性，并可能浪费跟随卫星的时间和能量，而这些资源本可用于拍摄真实目标。一个假反例 (false negative) 目标则会从调度中被遗漏，无法被任何跟随卫星成像，从而导致覆盖率降低。

Scheduling for Moving Targets

For a moving target (e.g., a ship or a plane), EagleEye needs that the target does not move out of the targeted image during the time interval between the leader and the follower taking the images. Consider that the satellite ground speed is 𝑉 𝑠𝑎𝑡 , the target speed is 𝑉 𝑡𝑎𝑟𝑔𝑒𝑡 , the high-resolution image swath is 𝑠𝑤𝑎𝑡ℎ, and the distance along the ground track between when the leader captures an image and when the follower does is 𝐷 (which we call the “lookahead distance”). 𝛾 is a slack parameter indicating the fraction of a high-resolution image swath over which a target may move and still be considered in the frame. We relate the lookahead distance to target and satellite velocity, considering the slack:

𝐷 𝑉 𝑠𝑎𝑡 ∗ 𝑉 𝑡𝑎𝑟𝑔𝑒𝑡 ≤ 𝛾 ∗ 𝑠𝑤𝑎𝑡ℎ. Fig. 10 shows the maximum look ahead distance 𝐷 for different target speeds, assuming a satellite at 500 km altitude with 𝑉 𝑠𝑎𝑡 = 7.5 km/s, 𝑠𝑤𝑎𝑡ℎ = 10 km, 𝛾 = 0.1. We show that the maximum lookahead distance is 500𝑘𝑚 for a ship with ground speed 14 m/s (50 km/h) and 28 km for a plane with ground speed 250 m/s (900 km/h). The data shows that a reasonable lookahead distance accommodates reasonable target speeds. Note that we assume 𝑉𝑡𝑎𝑟𝑔𝑒𝑡 represents the maximum target speed. EagleEye will have more time slack if the target moves slower than the expected target speed.

对于移动目标（如船只或飞机），EagleEye要求在主导卫星和跟随卫星拍摄图像的时间间隔内，目标不会移出被摄图像的范围 。设卫星的星下点速度为 V_sat，目标速度为 V_target，高分辨率图像的刈幅为 swath，主导卫星捕获图像与跟随卫星捕获图像时沿星下点轨迹的距离为 D（我们称之为“前瞻距离”）。gamma 是一个松弛参数，表示目标可以在高分辨率图像刈幅内移动而不被视为移出画面的比例。我们将前瞻距离与目标和卫星速度关联起来，并考虑松弛参数：

Vsat​D ​× Vtarget ​≤ γ × swath

图10展示了在不同目标速度下的最大前瞻距离 D，假设卫星位于500公里高度，V_sat=7.5textkm/s，swath=10textkm，gamma=0.1。我们展示了对于地面速度为14 m/s（50 km/h）的船只，最大前瞻距离为500公里；对于地面速度为250 m/s（900 km/h）的飞机，最大前瞻距离为28公里。数据表明，一个合理的前瞻距离可以适应合理的目标速度。注意，我们假设 V_target 代表最大目标速度。如果目标移动速度低于预期，EagleEye将有更多的时间余量。

Discuss

Heterogeneous configurations overhead There is a nonrecurring engineering (NRE) cost associated with supporting two camera types. Integrating different cameras does not require substantial changes to highly-coupled subsystems, like communications hardware or ADACS, and we expect the time cost to be surmountable.

Reliability There could be satellite failure or network partition between satellites. If a leader fails, the followers in the group could fall back and simply capture nadir high-resolution images. Alternatively, the followers might be reassigned to another group under a new leader. If a follower fails, the leader can adjust by scheduling only with operational followers.

Recapture One future work is to explore target recapture (re-identification) between satellites. If the leader satellite identifies a target that has already been imaged by another follower previously, the leader can deprioritize the target and schedule the follower to capture other targets first. Alternatively, if the leader satellite identifies a target that has changed over time (e.g., a ship has moved), the leader can prioritize the target when scheduling.

Orbit Design As the constellation grows in size, there will be more overlapping regions captured by different satellites in a short period. To further improve coverage, we need to strategically distribute the constellation to reduce these overlapping areas. This requires adjustments to the constellation’s orbital parameters, such as orbit inclination and the number of orbital planes. We consider this to be future work.

• 异构配置的开销：支持两种类型的相机会产生相关的一次性工程成本 (NRE cost)。集成不同的相机不需要对高度耦合的子系统（如通信硬件或ADACS）进行重大更改，我们预计时间成本是可控的

• 可靠性：卫星可能会发生故障或卫星间的网络出现分区

  • 如果主导卫星发生故障，其组内的跟随卫星可以回退到仅拍摄天底点高分辨率图像的模式。或者，这些跟随卫星可能被重新分配给另一个新主导卫星下的组
  • 如果某个跟随卫星发生故障，主导卫星可以通过仅调度仍在运行的跟随卫星来进行调整

• 重捕获：未来的一个工作方向是探索卫星间的目标重捕获（再识别）

  • 如果 主导卫星识别出一个先前已被另一颗跟随卫星成像的目标，它可以降低该目标的优先级，并优先调度跟随卫星去捕获其他目标
  • 反之，如果主导卫星识别出一个随时间变化的目标（例如，一艘移动了位置的船），它可以在调度时提高该目标的优先级

• 轨道设计：随着星座规模的扩大，不同卫星在短时间内捕获的重叠区域将会增多

  • 为了进一步提高覆盖率，我们需要战略性地分布星座以减少这些重叠区域
  • 这需要调整星座的轨道参数，例如轨道倾角和轨道平面数量。我们视此为未来工作