Traditionally, building machine learning models in different domains requires collecting and labeling large amounts of domain-specific data to train individual models for each task. This approach is resource-intensive and inflexible to adapt to new tasks and domains. However, the foundation model approach develops a single model on massive amounts of diverse and unlabeled data, allowing it to learn general language understanding capabilities that can be adapted to various domains with a much smaller labeled data requirement.

The domain adaptation process leverages the pre-trained foundation model's knowledge and fine-tunes it on task-specific data, resulting in faster development and deployment of high-performance models. This paradigm shift towards foundation models represents a significant step forward in natural language processing, enabling more efficient, scalable, and adaptable AI solutions.

The Foundation Models program is Phase 2 of the AI for Everyone Program. In Phase 1, we produced open-source tools and datasets, including eight models and four publicly available datasets. One of the models, WangchanBERTa, has already gained significant traction with 1.2 million downloads. Phase 1 concluded in April 2023, and we have launched Phase 2 of the program in May 2023. The Foundation Models program applies state-of-the-art natural language processing techniques to various business domains, including finance, healthcare, retail, and legal. The program aims to develop a small set of core foundation models and toolsets to adapt them to domain-specific use cases. We also plan to facilitate the development of proof-of-concept applications.