PolyTask Orchestrator
A multi-task learning framework built on the modernBERT architecture that trains several NLP tasks together in one shared network, with dynamic loss-weighting and task scheduling intended to reduce GPU hours compared with fine-tuning a separate model per task.
PolyTask Orchestrator co-trains multiple natural-language tasks, such as intent classification, semantic similarity scoring and summarization, on a single modernBERT-based network with shared transformer layers and task-specific output heads, instead of fine-tuning and deploying a separate model for each task. Its task synchronization engine balances training updates across tasks so that one task does not dominate the loss landscape, and its loss weighting uses an uncertainty-weighted scheme, based on the method of Kendall et al., to let the model adjust each task's contribution automatically.
The project states that training multiple tasks in one session can reduce GPU hours by up to 40 percent compared with sequential fine-tuning, because backward passes share gradients across tasks.
It is built on the Hugging Face Transformers ecosystem and the modernBERT architecture.