EvoSkill: Automated Skill Discovery for Multi-Agent Systems

Salaheddin Alzubi, Noah Provenzano, Jaydon Bingham, Christian A Calvo, Weiyuan Chen, Tu Vu

Conference On Language Models 3 pre-proceedings (COLM 2026) Conference

BibTeX download is not available in the pre-proceeding

Coding agents are increasingly used as general-purpose problem solvers, but they often lack the domain expertise required for specialized tasks. Agent skills address this limitation through reusable modules that package instructions, scripts, and resources to augment agents with specialized capabilities. However, these skills are typically hand-crafted, which limits their scalability. We ask whether agents can instead acquire such skills automatically from experience. We introduce EvoSkill, a framework for automated skill acquisition. EvoSkill iteratively analyzes execution failures to identify capability gaps, proposes new skills or refinements to existing ones, and materializes them as structured skill folders. Candidate skills are selected based on held-out validation performance, while model parameters remain fixed. We evaluate EvoSkill on diverse benchmarks spanning financial-document reasoning, web search, multi-source reasoning, and coding (OfficeQA, SealQA, FRAMES, and LiveCodeBench), using models from the Claude, Gemini, Kimi, and gpt-oss families as executors. We show that EvoSkill can substantially improve agent performance across benchmarks, including gains of 15.7 points on SealQA and 11.4 points on OfficeQA. In a controlled comparison, skill evolution with EvoSkill substantially outperforms prompt evolution with GEPA. We further find evidence that evolved skills can transfer across settings: a search skill evolved on SealQA improves BrowseComp accuracy by 5.3 points without modification, while reuse across executor models improves performance in several model pairings, with outcomes depending on model and harness compatibility.