PyTorch Conference 2024

As large-scale training datasets proved essential for generalization in AI models—be it for language, image, or video tasks—many internet-scraped datasets emerged. In domains where data at such a scale is unavailable, synthetic data stepped in to fill the gap. However, in video understanding tasks, particularly those involving human body action, synthetic data generation has not been plausible. Tasks such as sign language translation, assistive gesture recognition, and human understanding in autonomous driving have thus been unable to exploit the full potential of the recent data boom in AI. We propose a toolkit for generating synthetic data of human body actions based on recent advancements in pose transfer and human body animation. Through a study involving ASL users, we show that this toolkit generates highly realistic animations that convey the desired semantics. Moreover, we show that adding this synthetic data can significantly improve the performance of AI models involving human body actions. We make this toolkit easily integrated into existing PyTorch pipelines. In the future, we plan to extend our toolkit to other action recognition challenges.