Szczegóły publikacji
Opis bibliograficzny
Could LLMs predict algorithm performance in automated design of metaheuristics? / Michal PLUHÁČEK, Paweł KOLENDO, Krzysztof TYLKA-SULEJA, Adam Viktorin // W: GECCO'26 Companion [Dokument elektroniczny] : proceedings of the 2026 Genetic and Evolutionary Computation Conference : San Jose, Costa Rica, July 13–17, 2026 / eds. Leonardo Trujillo, Ting Hu. — Wersja do Windows. — Dane tekstowe. — New York : Association for Computing Machinery, cop. 2026. — e-ISBN: 979-8-4007-2488-6. — S. 1416–1424. — Wymagania systemowe: Adobe Reader. — Bibliogr. s. 1422–1423, Abstr. — Publikacja dostępna online od: 2026-08-13
Autorzy (4)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 170295 |
|---|---|
| Data dodania do BaDAP | 2026-10-01 |
| Tekst źródłowy | URL |
| DOI | 10.1145/3795101.3814714 |
| Rok publikacji | 2026 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Creative Commons | |
| Wydawca | Association for Computing Machinery (ACM) |
| Konferencja | Genetic and Evolutionary Computations 2026 |
Abstract
This study investigates whether the current generation of frontier Large Language Models can predict the relative performance of metaheuristic optimization algorithms from source code alone, potentially reducing the computational cost of benchmarking during automated algorithm design. We task three models (GPT 5.4 Thinking, Gemini 3.1 Pro, and Claude Opus 4.6) with predicting improvement or regression between consecutive iterations of iteratively refined differential evolution, and with ranking independently designed algorithms on standard benchmarks. Initial results are promising (88% accuracy), but accuracy collapses on a second algorithm sequence (22-44%). A controlled comparison experiment provides evidence that the models’ predictions correlate with code complexity metrics rather than with actual algorithmic performance. Self-reported confidence scores are poorly calibrated across all models tested. These findings indicate that while LLM-based performance prediction is not yet a reliable substitute for benchmarking, it shows potential as a complementary pre-screening tool in iterative algorithm design, particularly in settings where code modifications follow predictable patterns.
