When Is Smaller Greener? Right-Sizing Language Models for Agentic AI

Smaller models are greener only up to a predictable task length. Because an AI agent succeeds only if every step succeeds, and failed attempts still consume resources, you should compare models on resources per successfully completed task rather than per call.

Practical examples

• At 20 steps, a 3-billion-parameter model used a tenth of the energy of a 27-billion one per completed task.
• At 100 steps, a 1.7-billion model used almost four times as much energy as an 8-billion one per completed task.

Limitations

• The predicted crossover fell within the observed range for 72 of the 88 observed crossovers.

Research · arXiv cs.AI · Publisher website: arxiv.org · Last verified 10 November 2026 · Catalog ID 133323