Program-Aided Language Models (PAL)

Program-Aided Language Models (PAL) is a reasoning framework that divides the problem-solving process between an LLM and a program interpreter. The LLM handles natural language understanding and problem decomposition, while a Python interpreter handles the actual computation. This hybrid approach eliminates arithmetic and logical errors that plague purely text-based reasoning methods like Chain-of-Thought prompting.¹⁾

PAL operates through a clear division of labor:

1. The LLM's role: Reads the natural language problem and generates intermediate reasoning steps as executable Python code.
2. The interpreter's role: Executes the generated code to compute the final answer.

This means the only “learning” task for the LLM is decomposing the problem into runnable steps. The actual solving is delegated to a deterministic program runtime, eliminating computational errors.

Given a math word problem like:

"Roger has 5 tennis balls. He buys 2 more cans of tennis balls.
 Each can has 3 tennis balls. How many tennis balls does he have now?"

A Chain-of-Thought approach would reason in natural language (potentially making arithmetic mistakes). PAL instead generates:

tennis_balls_initial = 5
cans_bought = 2
balls_per_can = 3
total = tennis_balls_initial + (cans_bought * balls_per_can)
print(total)  # 11

The Python interpreter executes this code and returns the correct answer with certainty.

PAL was evaluated across 13 mathematical, symbolic, and algorithmic reasoning tasks:²⁾

Notably, PAL using Codex outperformed the much larger PaLM-540B model using chain-of-thought prompting, demonstrating that the hybrid approach can compensate for smaller model size.

While PAL was initially developed with code-specialized models like Codex, it generalizes to general-purpose LLMs. Using text-davinci-002 and text-davinci-003, PAL still outperformed CoT, indicating the approach works effectively with models trained primarily on natural language rather than code.³⁾

• Requires code generation ability: The LLM must be capable of generating syntactically correct, executable code.
• Limited to computable tasks: Problems that cannot be expressed as programs (e.g., open-ended creative tasks) are not suitable.
• Interpreter dependency: Requires access to a code execution environment, which may pose security concerns.
• Decomposition errors persist: If the LLM misunderstands the problem, the generated code will implement the wrong logic.

• Prompt Engineering
• chain_of_thought_prompting
• Automatic Reasoning and Tool-Use (ART)
• Zero-Shot Prompting
• Few-Shot Prompting