LLaMEA: Large Language Model Evolutionary Algorithm
A Python framework that couples large-language-model reasoning with an evolutionary loop to automatically invent, mutate and benchmark optimization algorithms, with an in-the-loop hyperparameter-optimization mode and niching strategies to keep a diverse population of solutions.
LLaMEA (Large Language Model Evolutionary Algorithm) is a framework that uses large language models such as GPT-4 to automatically generate and refine metaheuristic optimization algorithms. It evolves and optimizes algorithms iteratively based on performance metrics and runtime evaluation, without requiring extensive prior algorithmic knowledge, and integrates with IOHexperimenter for real-time performance feedback that guides the evolutionary process.
An extension, LLaMEA-HPO, adds an in-the-loop hyperparameter-optimization mechanism through SMAC, so numerical tuning is offloaded and the LLM's queries can focus on structural improvements to the algorithm instead. Fitness sharing and clearing strategies maintain a diverse population of solutions, a unified-diff mode evolves code through patch-based edits to save tokens, and a population-evaluation mode can score a whole generation in a single call to speed up expensive fitness functions.
It is extensible to other models and evaluation tools, and ships example scripts for black-box optimization, hyperparameter-optimization-assisted search, and full AutoML runs.