Analyzing LLM Instruction Optimization for Tabular Fact Verification

Instruction optimisation offers a model-agnostic way to improve reasoning without updating a language model’s parameters. This study compares its effect on tabular fact verification across direct prediction, chain-of-thought prompting, ReAct agents with SQL tools, and CodeAct agents with Python execution.

The experiments evaluate the DSPy optimisers COPRO, MiPROv2, and SIMBA over four benchmarks and three model families. Instruction optimisation consistently improves verification accuracy, although its benefits depend on the prompting and tool-use strategy.

MiPROv2 produces the most stable gains for chain-of-thought prompting, while SIMBA benefits ReAct agents most strongly at larger model scales. Behavioural analysis suggests that SIMBA promotes more direct numerical reasoning and discourages unnecessary tool calls.

Paper: https://aclanthology.org/2026.findings-eacl.161/