Artem Grigorev, CTO of AI data company Toloka, joins SlatorPod to discuss the growing importance of human expertise in AI development, the shift toward smaller specialized models, and the challenges of evaluating AI performance.

Grigorev explains that while computing power and model capabilities attract significant attention, high-quality training data remains fundamental to AI progress. As models become more sophisticated, data creation has shifted from simple annotation tasks to complex, expert-driven workflows. He highlights growing demand for domain specialists, including PhD-level experts, as large language models increasingly automate simpler annotation tasks.

Grigorev also discusses Toloka's work on post-training smaller models for specific enterprise applications. By fine-tuning models on targeted datasets, businesses can potentially achieve better performance, lower costs, and faster response times than they would relying on large frontier models.

Turning to multilingual AI, he emphasizes the need to evaluate model performance across languages, rather than relying predominantly on English-language benchmarks.

Toloka Arena, launched earlier this year, is the company's private benchmarking initiative designed to address concerns around contaminated public benchmarks and assess AI agents against realistic business workflows. Grigorev explains how Arena tests agents across multiple attempts to assess consistency rather than a single successful run.

Grigorev also discusses the growing demand for robotics training data, including first-person videos of humans performing physical tasks, and the challenges of collecting such data at scale.

Looking ahead, he argues that human judgment will remain essential, even as AI becomes increasingly capable, particularly in determining whether AI-generated outputs meet human expectations and preferences.