Google Deepmind argues that video generators already contain the world models computer vision has been missing. The company has developed GenCeption, a model that repurposes a pre-trained video generator for classic computer vision tasks like depth estimation and segmentation. This approach leverages existing capabilities in video generation to perform specialized vision tasks without requiring extensive retraining.

The system builds on an open-source video model from Alibaba, demonstrating how pre-trained models can be adapted for different applications. GenCeption uses text prompts to guide its processing and delivers results in a single forward pass. This method significantly reduces the computational resources needed for training and inference.

GenCeption achieves state-of-the-art performance in depth estimation, segmentation, and 3D pose estimation while needing very little training data. It trained on just a small set of synthetic videos, highlighting the potential of repurposing existing models for new tasks. This approach could streamline the development of vision systems by reducing the need for large-scale data collection and training.

The implications of this approach are significant for the field of computer vision. By repurposing existing video generators, developers can reduce costs associated with training new models from scratch. However, this also raises questions about vendor lock-in and the governance of models that are adapted from third-party sources. Market reactions may vary depending on how accessible and customizable these models are for different applications.

Despite the promising results, the system is still in development and requires further validation. The claim that video generators already contain the world models computer vision has been missing is a bold one, and more research is needed to fully understand the scope and limitations of this approach. The field remains dynamic, with ongoing efforts to refine and expand the capabilities of vision systems.