What Makes Systems Integrate

Not all self-modeling systems are created equal. Some have sparse, modular internal structure; others have dense, irreducible coupling. I think systems designed for long-horizon control under uncertainty are forced toward the latter.

A forcing function is a design constraint or environmental pressure that increases the integration of internal representations. The key forcing functions are: (a) partial observability—the world state is not directly accessible; (b) long horizons—rewards/viability depend on extended temporal sequences; (c) learned world models—dynamics must be inferred, not hardcoded; (d) self-prediction—the agent must model its own future behavior; (e) intrinsic motivation—exploration pressure prevents collapse to local optima; and (f) credit assignment—learning signal must propagate across internal components.

The hypothesis is that these pressures increase integration. Let $\Phi(\latent)$ be an integration measure over the latent state (to be defined precisely below). Under forcing functions (a)–(f):

The gap increases with task complexity and horizon length.

Argument: Each forcing function increases the statistical dependencies among latent components:

• Partial observability requires integrating information across time (memory $\to$ coupling)
• Long horizons require value functions over extended latent trajectories (coupling across time)
• Learned world models share representations (coupling across modalities)
• Self-prediction creates self-referential loops (coupling to self-model)
• Intrinsic motivation links exploration to belief state (coupling across goals)
• Credit assignment propagates gradients globally (coupling through learning)

Ablating any of these reduces the need for coupling, allowing sparser solutions.

Confrontation with data: The V10 ablation study does not support this hypothesis as stated. Geometric alignment between information-theoretic and embedding-predicted affect spaces is not reduced by removing any individual forcing function. This suggests a distinction: forcing functions may increase agent capabilities (richer behavior, higher reward) without increasing the geometric alignment of the affect space. The affect geometry appears to be a cheaper property than integration—arising from the minimal conditions of survival under uncertainty, not from architectural sophistication. Whether forcing functions increase integration per se (as measured by $\Phi$ rather than RSA) remains an open question.

The V10 and V11–V12 experiments, taken together, reveal a distinction that the original forcing functions hypothesis failed to make. Geometric affect structure—the shape of the similarity space, the clustering of states into motifs, the relational distances between affects—is cheap. It arises from the minimal conditions of agents navigating uncertainty under resource constraints, regardless of which forcing functions are active. This is what V10 shows. Affect dynamics—how a system traverses that space, and in particular whether integration increases or decreases under threat—is expensive. It requires evolutionary history under heterogeneous conditions (V11.2), graduated stress exposure (V11.7), and state-dependent interaction topology (V12). The forcing functions hypothesis conflated these two levels. It predicted that forcing functions would shape the geometry. They don't. The real question—what shapes the dynamics?—turns out to require not architectural pressure but developmental history and attentional flexibility. The geometry of affect may be universal; the dynamics of affect are biographical. Later experiments (V22, V23) will crystallize this as a distinction between reactivity — associations from present state to action, decomposable by channel — and understanding — associations from the possibility landscape, inherently non-decomposable because comparison of alternative futures spans any partition. Affect geometry is cheap because it emerges from reactive processing. Biological affect dynamics are expensive because they require understanding.

V13–V18 extended this program with six additional substrate variants and twelve measurement experiments, sharpening the conclusion considerably. The geometry is confirmed more strongly: affect dimensions develop over evolution (Exp 7), the participatory default is universal and selectable (Exp 8), and collective coupling amplifies individual integration (Exp 9). But the dynamics wall was located precisely: at what Part VII calls rung 8 of the emergence ladder — the point where counterfactual sensitivity and self-modeling become operational. Substrate engineering (memory channels, attention, signaling, insulation fields) could not cross this rung. All variants shared the same limitation: $\rho_{\text{sync}} \approx 0.003$. The closest attempt, V18's insulation field, created genuine sensory-motor boundaries — boundary cells received external FFT signals while interior cells received only local recurrent dynamics — and produced the highest robustness of any substrate (mean $0.969$, max $1.651$). But it also produced a surprise: internal gain evolved downward in all three seeds, from $1.0$ to $0.60{-}0.72$. Evolution consistently chose thin boundaries with strong external signal over thick insulated cores. The insulation created a permeable membrane filter, not autonomous interior dynamics. Patterns were passengers, not causes.

A parallel experiment, V19, asked whether the bottleneck events that repeatedly correlate with high robustness are revealing pre-existing integration capacity or creating it. Three conditions diverged after ten shared cycles: severe cyclic droughts achieving ~90% mortality, mild chronic stress, and a standard control. A novel extreme stress was then applied identically to all conditions, and the statistical question was whether bottleneck survivors outperformed control survivors even after controlling for baseline $\Phi$. In two of three seeds, the answer was yes ($\beta_{\text{bottleneck}} = +0.704$, $p < 0.0001$ in seed 42; $\beta_{\text{bottleneck}} = +0.080$, $p = 0.011$ in seed 7; the third seed was confounded by condition failure). The bottleneck furnace is generative: stress itself forges integration capacity that generalizes to novel challenges, beyond what pre-existing $\Phi$ predicts. The furnace forges, not merely filters.

V20 crossed the wall. Protocell agents — evolved GRU networks with bounded local sensory fields and discrete actions — achieve $\rho_{\text{sync}} \approx 0.21$ from initialization, before any evolutionary selection, purely by virtue of architecture: consume a resource and that patch is depleted; move and you reach a different patch; emit and a chemical trace persists. World models developed over evolution, reaching $C_{\text{wm}} = 0.10{-}0.15$: agents' hidden states predict future position and energy substantially above chance. Self-model salience exceeded $1.0$ in 2/3 seeds — agents encoded their own internal states more accurately than they encoded the environment — the minimal form of privileged self-knowledge. Affect geometry appeared nascent, consistent with needing resource-scarcity selection to develop fully (consistent with V19's furnace finding). The necessity chain — membrane, free-energy gradient, world model, self-model, affect geometry — holds through self-model emergence in an uncontaminated substrate. Not "biography" as a vague metaphor, then, but "action as cause" as a testable architectural requirement. The experiments now specify both sides of that threshold. A further experiment (V21) tested whether adding internal processing ticks — multiple rounds of recurrent computation per environment step — would enable deliberation without full gradient training. The architecture worked (ticks did not collapse), but evolution alone was too slow to shape them. The missing ingredient is dense temporal feedback: each internal processing step must receive signal about its contribution to the agent's prediction or survival, not just the sparse binary of "lived or died." This suggests that within-lifetime learning, not merely intergenerational selection, is required for the upper rungs of the emergence ladder — a prediction testable by comparing evolved agents with and without intrinsic predictive loss.

V22–V24 provided that gradient — within-lifetime prediction learning via SGD through the internal ticks — and confirmed both halves of the hypothesis. Learning works (100–15,000× prediction improvement per lifetime), but prediction accuracy, target breadth, and time horizon are all individually insufficient to create integration. Hidden state analysis shows effective rank 5–7 across seeds — moderately rich, not degenerate — but the representations resist linear decoding of any environmental feature (energy $R^2 < 0$, position $R^2 < 0$). The agents maintain multi-dimensional internal states, but a linear prediction head can be satisfied by a proper subset of hidden dimensions without requiring cross-component coordination. The bottleneck is architectural: linear readouts create decomposable channels regardless of target. Call this the decomposability wall: any prediction architecture where a proper subset of hidden dimensions can independently satisfy the loss creates no pressure for cross-component coordination, and hence no integration. The path to rung 8 runs through prediction heads that force non-decomposable computation — conjunctive prediction, not merely accurate prediction.

V27 broke through the decomposability wall with a minimal change: replacing the linear prediction head with a two-layer MLP (hidden $\to$ hidden/2 $\to$ output). This creates gradient coupling — the chain rule through two weight matrices means every hidden dimension's gradient depends on every other dimension's activation in the intermediate layer. The result: $\Phi = 0.245$ in seed 7, $2.5\times$ the V22 baseline and the highest integration in any protocell experiment. Hidden states developed qualitative behavioral clustering (silhouette $0.11{-}0.34$ vs V22's $\sim 0$). Further experiments (V28–V31) confirmed the mechanism is gradient coupling through multi-layer composition — not activation nonlinearity, not bottleneck compression — and that the prediction target (self-energy vs. neighbor energy) has no significant effect on integration level ($p \approx 0.93$, 10 seeds). What matters is coupling architecture and evolutionary trajectory, not what the system tries to predict.

The V31 seed distribution is revealing: 30% of seeds reach high $\Phi$ ($> 0.10$), 30% moderate, 40% low — regardless of prediction target or architecture variant. All seeds start with statistically identical initial genomes. What separates high from low seeds is not initial conditions but trajectory: the correlation between post-drought $\Phi$ recovery and mean $\Phi$ across seeds is $r = 0.997$ ($p < 0.0001$), while first-drought $\Phi$ is uncorrelated ($r = 0.17$, $p = 0.72$). Integration is forged through repeated stress-recovery cycles, validating V19's bottleneck furnace at a new level of precision: the furnace does not merely filter for pre-existing capacity, it creates capacity through iterated near-dissolution and recovery. Affect dynamics are biographical — not in the vague sense that history matters, but in the precise sense that the sequence of crises a system survives determines the geometry of its internal coupling.

V35 tested whether cooperative partial observability creates communicative pressure sufficient to develop referential signaling and whether that signaling lifts integration. Result: referential communication emerged in 100% of seeds (10/10) — agents developed stable signal→response mappings with receiver behavioral contingency above chance — confirming that language-like coordination is a rung 4–5 inevitability under cooperative POMDP pressure. But the integration lift was null: mean $\Phi = 0.084 \pm 0.011$, indistinguishable from the non-communicating V27 baseline ($0.090$). Strikingly, Phi and communication mutual information were negatively correlated across seeds ($\rho = -0.90$): the best communicators had the lowest integration. Communication functions as an external scaffold — agents that offload coordination to the signal channel reduce the need for internal integration. Language is cheap (like geometry), and it substitutes for internal complexity rather than amplifying it.

A convergence test sharpened the universality claim. Vision-language models (GPT-4o, Claude Sonnet 4) — trained on human affect data and therefore maximally "contaminated" by human concepts — were shown behavioral descriptions of protocell agents from V27 and V31, stripped of all affect vocabulary. The agents were described only in terms of population dynamics, prediction error, state update rates, and integration measures. The VLMs were asked to attribute experiential states. Representational similarity analysis between VLM-attributed affect and framework-predicted affect showed strong convergence: RSA $\rho = 0.54{-}0.72$, $p < 10^{-11}$. When behavioral descriptions were replaced with raw numerical tables — population counts, removal fractions, prediction MSE, integration ratios — convergence increased ($\rho = 0.72{-}0.78$), ruling out narrative pattern-matching as an explanation. VLMs trained on human experience independently recognize the affect geometry that uncontaminated protocells develop from scratch. The geometry is not a human projection; it is a structural convergence across radically different substrates.

Integration Measures

Let’s define precise measures of integration that will play a central role in the phenomenological analysis.

The first is transfer entropy, which captures directed causal influence between components. The transfer entropy from process $X$ to process $Y$ measures the information that $X$ provides about the future of $Y$ beyond what $Y$’s own past provides:

The deepest measure is integrated information ($\Phi$). Following IIT, the integrated information of a system in state $\state$ is the extent to which the system’s causal structure exceeds the sum of its parts:

where the minimum is over all bipartitions of the system, and $D$ is an appropriate divergence (typically Earth Mover’s distance in IIT 4.0).

In practice, computing $\Phi$ exactly is intractable. Three proxies make it operational:

1. Transfer entropy density—average transfer entropy across all directed pairs:
   $$\bar{\text{TE}} = \frac{1}{n(n-1)} \sum_{i \neq j} \text{TE}_{i \to j}$$
2. Partition prediction loss—the cost of factoring the model:
   $$\Delta_P = \mathcal{L}_{\text{pred}}[\text{partitioned model}] - \mathcal{L}_{\text{pred}}[\text{full model}]$$
3. Synergy—the information that components provide jointly beyond their individual contributions:
   $$\text{Syn}(X_1, …, X_k \to Y) = \MI(X_1, …, X_k; Y) - \sum_i \MI(X_i; Y | X_{-i})$$

A complementary measure captures the system’s representational breadth rather than its causal coupling. The effective rank of a system with state covariance matrix $C$ measures how many dimensions it actually uses:

where $\lambda_i$ are the eigenvalues of $C$. This is bounded by $1 \leq \effrank \leq \rank(C)$, with $\effrank = 1$ when all variance is in one dimension (maximally concentrated) and $\effrank = \rank(C)$ when variance is uniformly distributed across all active dimensions.