PyTorch Conference 2024

I would like to share my research in training ML models on-board of satellites. Our project was selected to be deployed in space on the D-Orbit's ION SCV004 satellite. In what started as a demonstrator mission, we successfully tested our efficient, small foundational and feature encoding model RaVAEn and also achieved the world's first training of machine learning model using PyTorch on-board of a satellite. We were able to train a tiny classification model directly on the satellite, using few-shot learning with a small annotated dataset of cloudy tiles. While on-board inference often uses frozen model inference and libraries such as ONNX or TensorRT, for training we interestingly needed at least part of the model to be kept in the train mode which allows for weights changes - we leveraged PyTorch for that. We were able to encode an entire dataset of remote sensing tiles using a frozen encoder model on the Myriad VPU in 638ms, and later used the device's CPU with PyTorch to train a cloud classification model in just 910ms. These results open exciting new directions in adaptability to sensor degradation on communication-constrained devices.