More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents
<p dir="ltr">LLM-powered coding agents, which operate in iterative loops (turns) to solve software engineering tasks, are becoming increasingly powerful. However, their practical deployment is hindered by significant and unpredictable costs. This challenge arises from a combination o...
Sábháilte in:
| Príomhchruthaitheoir: | |
|---|---|
| Rannpháirtithe: | |
| Foilsithe / Cruthaithe: |
2025
|
| Ábhair: | |
| Clibeanna: |
Cuir clib leis
Níl clibeanna ann, Bí ar an gcéad duine le clib a chur leis an taifead seo!
|
| _version_ | 1849927634633359360 |
|---|---|
| author | Chao Peng (22681619) |
| author2 | Pengfei Gao (13470673) |
| author2_role | author |
| author_facet | Chao Peng (22681619) Pengfei Gao (13470673) |
| author_role | author |
| dc.creator.none.fl_str_mv | Chao Peng (22681619) Pengfei Gao (13470673) |
| dc.date.none.fl_str_mv | 2025-11-25T12:25:13Z |
| dc.identifier.none.fl_str_mv | 10.6084/m9.figshare.30705524.v1 |
| dc.relation.none.fl_str_mv | https://figshare.com/articles/dataset/More_with_Less_An_Empirical_Study_of_Turn-Control_Strategies_for_Efficient_Coding_Agents/30705524 |
| dc.rights.none.fl_str_mv | CC BY 4.0 info:eu-repo/semantics/openAccess |
| dc.subject.none.fl_str_mv | Automated software engineering Empirical software engineering LLM Agent |
| dc.title.none.fl_str_mv | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| dc.type.none.fl_str_mv | Dataset info:eu-repo/semantics/publishedVersion dataset |
| description | <p dir="ltr">LLM-powered coding agents, which operate in iterative loops (turns) to solve software engineering tasks, are becoming increasingly powerful. However, their practical deployment is hindered by significant and unpredictable costs. This challenge arises from a combination of factors: quadratically growing token counts with each turn, the high price of state-of-the-art models, the large number of turns required for real-world tasks, and the tendency of agents to take inefficient or unnecessary actions. While existing research focuses on optimizing individual turns, the strategic control of the total number of turns remains an underexplored area for managing agent performance and cost. To address this gap, we conduct a comprehensive empirical study on the SWE-bench benchmark using three state-of-the-art models (Claude 4 Sonnet, Gemini 2.5 Pro, and GPT 4.1). We systematically evaluate the impact of three distinct turn-control strategies: an unrestricted baseline, a fixed-turn limit with reminders, and a novel dynamic-turn strategy that grants extensions on-demand. Our findings first reveal a fundamental trade-off in the unrestricted setting, where no single model excels across performance, cost, and turn efficiency. We then show that a fixed-turn limit, specifically at the 75th percentile of the baseline, serves as a "sweet spot", substantially reducing costs (by 24%-68%) with minimal impact on solve rates. Most significantly, our proposed dynamic-turn strategy consistently outperforms fixed-limit approaches, achieving comparable or better solve rates while further reducing costs by an additional 12%-24% by intelligently allocating resources only to tasks that need them. This work provides the first systematic analysis of turn-control strategies, offering simple yet effective guidelines for developers to balance cost and efficacy. We demonstrate that dynamic resource allocation is a superior, easy-to-implement approach for deploying powerful yet economically viable coding agents.</p> |
| eu_rights_str_mv | openAccess |
| id | Manara_f5a1121dee809444208d5c918e971f06 |
| identifier_str_mv | 10.6084/m9.figshare.30705524.v1 |
| network_acronym_str | Manara |
| network_name_str | ManaraRepo |
| oai_identifier_str | oai:figshare.com:article/30705524 |
| publishDate | 2025 |
| repository.mail.fl_str_mv | |
| repository.name.fl_str_mv | |
| repository_id_str | |
| rights_invalid_str_mv | CC BY 4.0 |
| spelling | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding AgentsChao Peng (22681619)Pengfei Gao (13470673)Automated software engineeringEmpirical software engineeringLLMAgent<p dir="ltr">LLM-powered coding agents, which operate in iterative loops (turns) to solve software engineering tasks, are becoming increasingly powerful. However, their practical deployment is hindered by significant and unpredictable costs. This challenge arises from a combination of factors: quadratically growing token counts with each turn, the high price of state-of-the-art models, the large number of turns required for real-world tasks, and the tendency of agents to take inefficient or unnecessary actions. While existing research focuses on optimizing individual turns, the strategic control of the total number of turns remains an underexplored area for managing agent performance and cost. To address this gap, we conduct a comprehensive empirical study on the SWE-bench benchmark using three state-of-the-art models (Claude 4 Sonnet, Gemini 2.5 Pro, and GPT 4.1). We systematically evaluate the impact of three distinct turn-control strategies: an unrestricted baseline, a fixed-turn limit with reminders, and a novel dynamic-turn strategy that grants extensions on-demand. Our findings first reveal a fundamental trade-off in the unrestricted setting, where no single model excels across performance, cost, and turn efficiency. We then show that a fixed-turn limit, specifically at the 75th percentile of the baseline, serves as a "sweet spot", substantially reducing costs (by 24%-68%) with minimal impact on solve rates. Most significantly, our proposed dynamic-turn strategy consistently outperforms fixed-limit approaches, achieving comparable or better solve rates while further reducing costs by an additional 12%-24% by intelligently allocating resources only to tasks that need them. This work provides the first systematic analysis of turn-control strategies, offering simple yet effective guidelines for developers to balance cost and efficacy. We demonstrate that dynamic resource allocation is a superior, easy-to-implement approach for deploying powerful yet economically viable coding agents.</p>2025-11-25T12:25:13ZDatasetinfo:eu-repo/semantics/publishedVersiondataset10.6084/m9.figshare.30705524.v1https://figshare.com/articles/dataset/More_with_Less_An_Empirical_Study_of_Turn-Control_Strategies_for_Efficient_Coding_Agents/30705524CC BY 4.0info:eu-repo/semantics/openAccessoai:figshare.com:article/307055242025-11-25T12:25:13Z |
| spellingShingle | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents Chao Peng (22681619) Automated software engineering Empirical software engineering LLM Agent |
| status_str | publishedVersion |
| title | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| title_full | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| title_fullStr | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| title_full_unstemmed | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| title_short | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| title_sort | More with Less: An Empirical Study of Turn-Control Strategies for Efficient Coding Agents |
| topic | Automated software engineering Empirical software engineering LLM Agent |