Rethinking Transformer Efficiency: FORTH-Style Postfix Outperforms Prefix in LLM Benchmarks

The conventional top-down approach to problem-solving in large language models might be fundamentally mismatched with transformer architecture capabilities. New experimental evidence suggests that FORTH-inspired postfix notation—where operations follow operands—aligns better with how transformers process sequential information, yielding significantly higher accuracy than prefix approaches in computational tasks.

In a benchmark test comparing prefix and postfix notation for constructing parity trees, transformer models consistently demonstrated superior performance with postfix expressions. Parity trees are unbalanced binary structures where leaf nodes contain numbers and internal nodes represent whether the sum of their descendants is even or odd. This task requires maintaining computational state while generating output—a core challenge for autoregressive models.

The experiment tested two conditions across Anthropic's Haiku and Opus models:

1. Thinking mode: Allowed chain-of-thought reasoning
2. Direct generation: Output without intermediate steps

Results showed striking advantages for postfix notation:

Model	Thinking	Postfix Accuracy	Prefix Accuracy	Both Correct	Postfix Only	Prefix Only	Both Wrong
Haiku	Yes	88.3%	36.7%	110	155	0	35
Haiku	No	6.7%	4.3%	9	11	4	276
Opus	Yes	98.3%	81.3%	243	52	1	4
Opus	No	50.0%	9.7%	28	122	1	149

Three critical patterns emerged:

1. Postfix dominance: Across all configurations, postfix notation achieved substantially higher accuracy rates—by margins of up to 61.6 percentage points in Haiku without reasoning
2. Reasoning amplification: Chain-of-thought improved prefix accuracy dramatically (36.7%→81.3% for Opus) but couldn't close the gap with postfix performance
3. Model capability: Opus maintained respectable postfix accuracy even without reasoning (50%), suggesting more advanced architectures handle state management better

The advantage stems from how transformers generate tokens sequentially. Postfix notation's bottom-up construction—where operands appear before operators—mirrors the model's left-to-right processing flow. Each token generation step builds upon already established context, similar to FORTH's concatenative paradigm where "syntactic concatenation is semantic composition" (as Manfred von Thun observed). Prefix notation forces premature commitment to operators before operands exist in context, creating computational friction.

This has practical implications for AI-assisted programming:

• Domain-specific languages: Tools for database operations (like the proposed Sideways Passing Join technique) could optimize by adopting postfix structures
• Auto-completion systems: Code generators might produce more accurate suggestions when framing problems in concatenative patterns
• Model architecture: Future transformers could incorporate explicit stack-based mechanisms inspired by FORTH's design principles

While prefix notation dominates languages like Lisp, these results indicate that AI-assisted programming environments might benefit from reconsidering notation paradigms. The findings also validate the hypothesis that local, incremental transformations—where program state evolves through token-by-token concatenation—align more naturally with transformer capabilities than top-down decomposition.

Further research could explore applying these principles to optimization passes in database layers using finite automata transformations, potentially creating more efficient query execution plans. As transformer models evolve, deliberately adapting computational paradigms to their architectural strengths may unlock new efficiency frontiers.