Generalizable Spacecraft Trajectory Generation via Multimodal Learning with Transformers

Davide Celestini1, Amirhossein Afsharrad2, Daniele Gammelli2, Tommaso Guffanti2, Gioele Zardini3, Sanjay Lall2, Elisa Capello1, Simone D'Amico2, Marco Pavone2,
1Politecnico di Torino, 2Stanford University, 3Massachusetts Institute of Technology
Paper arXiv Code Data

Multimodal learning allows Transformer-based networks to generalize across different optimization problem specifications and effectively warm-start Sequential Convex Programming solvers.

Abstract

Effective trajectory generation is essential for reliable on-board spacecraft autonomy. Among other approaches, learning-based warm-starting represents an appealing paradigm for solving the trajectory generation problem, effectively combining the benefits of optimization- and data-driven methods. Current approaches for learning-based trajectory generation often focus on fixed, single-scenario environments, where key scene characteristics, such as obstacle positions or final-time requirements, remain constant across problem instances.

However, practical trajectory generation requires the scenario to be frequently reconfigured, making the single-scenario approach a potentially impractical solution. To address this challenge, we present a novel trajectory generation framework that generalizes across diverse problem configurations, by leveraging high-capacity transformer neural networks capable of learning from multimodal data sources. Specifically, our approach integrates transformer-based neural network models into the trajectory optimization process, encoding both scene-level information (e.g., obstacle locations, initial and goal states) and trajectory-level constraints (e.g., time bounds, fuel consumption targets) via multimodal representations.

The transformer network then generates near-optimal initial guesses for non-convex optimization problems, significantly enhancing convergence speed and performance. The framework is validated through extensive simulations and real-world experiments on a free-flyer platform, achieving up to 30% cost improvement and 80% reduction in infeasible cases with respect to traditional approaches, and demonstrating robust generalization across diverse scenario variations.

Numerical results

The generalization capabilities of the Autonomous Rendezvous Transformer (ART) are evaluated in simulations and on a Free-Flyer testbed:

  1. By varying the final time constraint of the optimization problem,
  2. By varying the operative scenario and the obstacles' configration.

Ablation studies are conducted to assess the impact of diversity in the training dataset on the learning process of ART.

Results relative to ART generalization capabilities with novel target time acquisition constraints.

Results relative to ART generalization capabilities with novel obstacles' configurations and avoidance constraints.

While dataset diversity generally improves generalization performance, excessive diversity may have negative effects depending on the specific application. Specifically, for the final time generalization, it results in an imprecise warm-start and a higher number of SCP iterations to converge to the final solution.

Experimental results

Experimental tests on a real-world Free Flyer testbed, confirm the benfit of multimodal learning for generalization capabilities. ART effectively generates high-quality warm-starting trajectories of different length while leveraging advantageous features of the scenario, even when working with novel obstacles' configurations (left) or with a limited context (right).

The velocity profile of the maneuver is correctly modulated to adapt to the target acquisition time.