Can sub-task handlers be swapped between neural and symbolic systems?

This explores whether the pieces that handle individual sub-tasks are interchangeable parts — whether a symbolic routine and a neural network can stand in for each other inside a larger pipeline. The corpus suggests the boundary is far more porous than the neural-vs-symbolic framing implies, and that the most interesting work treats sub-task handlers precisely as swappable slots.

The clearest version of this is the "LLM as plug-in handler" pattern. In LLM Programs Can algorithms control LLM reasoning better than LLMs alone?, an explicit symbolic algorithm owns the control flow and state, and each neural model call is just a step that receives only its step-specific context. The algorithm is the scaffold; the LLM is the replaceable component doing one bounded job. Agent Workflow Memory Can agents learn reusable sub-task routines from past experience? pushes this further, inducing reusable sub-task routines at finer-than-task granularity and recombining them hierarchically — routines that behave like callable symbolic procedures even though they're learned. And function calling itself decomposes cleanly into seven discrete subtasks Can breaking function calling into subtasks improve model generalization?, each of which can be trained, swapped, or routed independently. These all point the same way: if you define the interface of a sub-task narrowly enough, what fills it — code or network — becomes a design choice.

What makes the swap genuinely feasible is that neural networks already organize themselves modularly. Pruning experiments show networks implement compositional subroutines in isolated subnetworks, where ablating one affects only its corresponding function Do neural networks naturally learn modular compositional structure? — the neural side has discrete, addressable handlers, not one undifferentiated blob. Reasoning chains reinforce this: models internally rank tokens by functional importance and preferentially preserve the symbolic-computation tokens while discarding grammar and filler Which tokens in reasoning chains actually matter most?. So even inside a pure neural rollout, a quasi-symbolic core is doing the load-bearing work — which is exactly the part you'd want to be able to hand off to an explicit symbolic system.

There's a real tension here worth surfacing. One line of work argues you may not need the symbolic side at all: standard networks achieve compositional generalization through data and model scaling alone, with no architectural tricks, as long as training covers the combinations Can neural networks learn compositional skills without symbolic mechanisms? — and a single finite transformer is provably Turing-complete, programmable entirely through prompts Can a single transformer become universally programmable through prompts?. On that view, the neural substrate can simply absorb the symbolic handler's job. But the same corpus warns that standard training rarely produces such universally-programmable models in practice, which is why structured approaches like recursive subtask trees Can recursive subtask trees overcome context window limits? and hierarchical dual-recurrence Can recurrent hierarchies achieve reasoning that transformers cannot? impose explicit sub-task structure to get reliability that flat networks can't.

So the answer the collection points toward: yes, but the swap works best at a well-defined interface. When a sub-task has a clean contract — a function signature, a routine, a step in an algorithm — neural and symbolic handlers really are interchangeable, and the cutting-edge systems mix them freely. The thing you didn't know you wanted to know is that this isn't a one-way street: not only can symbolic scaffolds delegate steps to neural models, but neural models spontaneously grow symbolic-looking modular handlers inside themselves, which is what makes the boundary worth crossing in the first place.